Get 'em while they're young: Open Science for Junior Physics Lab

I've been wanting to write about an open science teaching experiment I've been doing at U. New Mexico that I think has been really successful. I've been thinking about presenting at a local teaching conference on campus (and I should still do that), and other venues, but I think blogging about it here will be an effective way of letting others know about it.

What it is in a nutshell

The course is Junior Lab (modern physics lab course) at the U. New Mexico. We run the course as close to "open science" as we can, with a course wiki on OpenWetWare as the foundation. The students post everything on the wiki: their primary lab notebooks on the wiki, analysis notes, Matlab code and Excel sheets, formal report rough drafts, and final drafts. Further, all of my instructor feedback is also posted on their work, in the margins. The only written things we don't put on the wiki are letter grades and a few confidential emails. (We also have no mechanism for putting video or audio conversations on the wiki...though that's an interesting and scary idea that just occurred to me.)

Does it work?

I say emphatically: YES! I have just finished teaching the course for the second time and I have been very pleased both times. I've also only received positive feedback about the open science style of the course--including many unsolicited emails from students saying the wiki was very helpful for them. Unfortunately, I only have anectdotal evidence...next fall I want to implement some kind of pre- and post-testing for assessment, as I talked about in yesterday's blog.

What have been some good outcomes?

Good science. When I decided to teach this course on OWW (OpenWetWare), I purposefully didn't put much planning into it (my style of time management). I also decided to just give it a whirl and "be bold" figuring the worst that could happen would probably not be much worse than an average lab course. Thus, I didn't really imagine all of the wonderful things that would happen as we got started. I think the thing that I have enjoyed the best is seeing the students reading the lab notebook of other students to get hints for how to set it up, how to do the analysis, etc...and then citing and linking the help they got!. I was seeing this happen and just so delighted to see them practicing science the way it should be practiced. For some of them, this was natural, for others, I could tell they felt like they were cheating or something, because it seemed too easy. I just constantly reminded them that this was science and too keep looking at previous work and to keep citing.

Open science training. I don't have evidence for this, but I feel like these students will be much more likely to practice open science later in their careers. I suspect they'll be required by at least one future instructor or advisor to go back to paper and pen and "science 1.0" ... and having been through this course, I also suspect they will rebel and lead changes wherever they are.

Much better instructor / student communication. One of the principles from "The One Minute Manger" (a cheesy little book that is very much worth reading) is to give feedback as close to instantaneously as possible. After reading that book, I think it's pretty obvious that early feedback is much more valuable than delayed. But in traditionally run lab courses, where labs are handed in on paper, the feedback is necessarily delayed quite a bit. Having the course in public on the wiki allows me to leave feedback and "grade" any time I have internet access. My goal is to give the feedback very quickly, but to be honest, I didn't do so hot this semester. Maybe for the first half of the course, I was able to provide written feedback within one week, but attending the New Faculty Workshop in November completely derailed me and I wasn't happy with being two to four weeks behind sometimes. Fortunately, I think the feedback is much more important early on in the course. Even with my failures, I still think this was a very positive aspect of the course compared to the paper alternatives.

What have been some of the challenges?

It takes a lot of time. Having 14 students in the lab, I'd say I spent at least 10 hours / week (a lot more at certain times) providing written feedback (aka "grading"), plus the 6 hours in lab, and about 3 hours / week preparing low-quality lectures. (We have one hour of lecture on statistical data analysis and other science topics, see agenda here.) Adding up those numbers doesn't seem like a lot, so I maybe am estimating incorrectly. In any case, it feels like a lot of time, and for sure the way I teach it is not at all scalable to more students. I feel like the one-on-one interactions (both real and virtual) are a critical aspect of the course--the students are apprentices. This is in contrast to the other course I teach (Conceptual Physics for >100 non-scientists), where I also value the personal interactions with the students, but not as essentially as in this course.

Technical difficulties. OpenWetWare is a fantastic resource, and I am enormously grateful for everything the OWW founders and Bill Flanagan have provided to help with this course. It is a very solid foundation and pretty much has everything we need for this course to work very well and to be far superior to the traditional version of a lab course. That said, there are very many technical improvements that could add a lot of value. Some of these were brought up at the Lab Notebook brainstorming session we had in October 2007. For example, integrated spreadsheeting would be great (but time-intensive to implement). Also, some kind of "auto-save" is necessary...a few students throughout the semester lost data due to either glitches or being logged out, or even their own mistakes. As we all know, losing data is a crushing blow, so it needs to be kept very much to a minimum. Two good students lost this battle and resorted to paper and pencil followed by uploading later, which of course is not a desirable outcome. (One idea I have to solve this with MediaWiki is to just install a "Save and Keep Editing" button, which would essentially be a "preview" button that actually saves the entry in the data base in addition to keeping the editing window open.) There are all kinds of other things, but I think I'm getting into a broader discussion of electronic lab notebooks in general.

What do I want to do differently next year?
I'm very happy with the way the course has worked so far. Most of the ideas I have for changing things next year are not related to the open science aspects of the course (for example, improving or adding new experiments). I would love to hear about any ideas you have, so please post them on the comments here! Also, if you would like to emulate open science aspects of this course at your institution, I would be really happy to help you get started.