Jamie and I had the pleasure of going out for dim sum with Greg Wilson the other day. You might remember Greg from such classics as Science 2.0 and Stack Overflow Dev Days Toronto. He was talking to us about his Software Carpentry course that he’s been running out of the University of Toronto and University of Alberta.

Software Carpentry is a two week course for science graduate students who need to do software development, but may not have had CS training. It covers things like scripting, version control, and unit testing. And I know that based on my experience in graduate school, this kind of course would have saved me a lot of time and pain.

Back in the dark ages (1990s) when I was in grad school, we coded in Fortran, didn’t write unit tests, and our version control was something like "Get my latest source out of this directory: /home/jim/calibrations/beware_of_the_jaguar_new_calibration_1997_06_09/". Needless to say, I learned the lessons taught in Greg’s course the hard way.

The course is great and desperately needed; however there is push-back from some in the academic community as to its necessity. The argument is something along the lines of "grad students are cheap, why should we pay more to train them outside of their field?" and "they can learn all the coding they need from books and more senior students."

Well, there’s a great counter-example to that in the news recently: ClimateGate!

The Climate Research Unit at the University of East Anglia has one of the most-cited climate change databases in the world. This database has been the basis for some of the UN’s predictions that climate change is real, and is driven by mankind (Anthropogenic Global Warming). Recently, either an external hacker or an internal whistleblower released to the internet a series of emails and source code that implies some non-transparent dealings by the scientists with respect to their data. Whether it’s merely academic infighting or actual fraud is up for debate.

Regardless of your feelings about anthropogenic global warming and the politics of academia, one thing is clear from the released source code: everyone working on that code base needs to attend Greg’s Software Carpentry course. One has to feel bad for Ian "Harry" Harris, who’s HARRY_README.TXT shows that the CRU source code has just about every mistake possible. And because of those mistakes, we can’t answer some very real questions about the conclusions coming out of the CRU.

The point here is not that "ClimateGate" has killed AGW. My point is simply that bad software development practices have contributed not to settling the AGW question, but have further muddied the waters. If the folks at CRU had been open about their practices, we could have an honest debate about AGW and what do to about it. Because, in part, of their bad development practices, we can’t have that discussion, and the anti-AGW folks now say there’s no point to talking about it at all — global warming is dead.

I wouldn’t go that far, but once you lose Rex Murphy, you’re in trouble.

So, to all the academic supervisors and grant approvers out there: get grad students trained up on software development (and statistics, but that’s another post). Every major bit of physical science from now on will involve programming. Training students to do software development is the same as teaching them calculus. It’s not a luxury, it’s a necessity.

What? They don’t learn calculus anymore?! Kids!

The talk was a preview of the keynote session that Damian will be giving at OSCON, and we were lucky enough to see it here in Toronto due to the diligence of Richard Dice, Mike Stok and the Toronto.PM PerlMongers group (with financial help from TorCamp).

I’ve never seen Damian talk, but he lived up to the hype. I don’t want to give too much away, so that if you have the chance to see this talk, you should do so. But it hit on some of my favourite topics: physics, Feynman diagrams, the origin of the “penguin diagram”, anti-particles, and of course, Perl. The end result was a Perl module that can have “positronic” variables that run backwards through your program, analogous to the simplified idea that a positron is an electron running backwards through time (although it isn’t, really). So you declare them at the top of the program, use their results immediately, and then only have to calculate the result at any later point in your program. Kinda like in “Bill and Ted’s Excellent Adventure”, when they kept saying that they, at some future point in time, would travel from the future back to the past and leave <insert totally useful device here> behind the couch for the present Bill and Ted to use. One recommendation, though; given the crazy verb tenses that Damian had to use to describe the behaviour of his positronic variables, I think he should pick up a copy of Dr Dan Streetmentioner’s “Time Traveller’s Handbook of 1001 Tense Formations” before giving the talk at OSCON.

My wife (and the rest of the blogosphere) is a big fan of Moo Cards. And now they’ve started printing business cards in the standard sizes. So I made up a set.

First I wanted a cool, physics-related image. This picture of an artistically enhanced event from the Big European Bubble Chamber has been around for a while and is quite beautiful. It’s under copyright by CERN itself, but they are very liberal in allowing people to use it. I filled out a web form asking for permission to put it on my contact cards and it was granted in two days.

Here’s my card:

The details are blocked out, not that that will stop anyone from figuring out how to call me.

In the first case, (the one that worked), my wife and I had to transition our iPhoto library. We had about 15,000 files on our underpowered iMac. Various online sources say may be too many, and we were seeing that our iPhoto was running very slowly. So we elected to split the library into multiple yearly libraries. To facilitate this split, we used iPhoto Library Manager to do the split. Fortunately, before I did the split, I backed up our photos to seven or so DVDs.

And it was a good thing that I did, because when we exported the photos from the main library to the yearly libraries, it failed to export the video clips that were stored. Instead it exported the thumbnail photos that represented the video clips.

I don’t think this is a problem with iPhoto Manager, as the internet lists the same problem with iPhoto itself. I suspect it’s an error in the iPhoto APIs that iPhoto Manager uses.

Fortunately, we had the videoclips stored on the DVDs and could recover them. My wife and I would have been extremely angry if we had lost all the video clips of our son from age zero to three, including him walking and talking for the first time.

The second case was one of me being dumb. I had moved personal files off my work machine onto my laptop, and I had also synchronized that to the home Linux server that I’m in the process of setting up. I then had a brain freeze, and thought that all the files on the iMac were synchronized too. I ended up deleting six years worth of GnuCash data files.

I had been backing up the iMac to an external drive using Retrospect Express. And I didn’t have it on a schedule, so it became a manual process that I would forget to do, and ended up pretty much stopping back in September. I can say good things about Retrospect; I managed to restore the files quite easily. Unfortunately, due to my own laziness, those restore files are three months out of date. But it does provide me a good opportunity to revamp my GnuCash file structure.

Scott Hanselman details what his backup strategy is, and it got me to thinking that I need a better home back up strategy. Probably three way sync between the iMac, the Linux box and my laptop. I’ll be investigating soon.

Updated: fixed typos.