Study finds on-line education beats classroom, but what does that mean?

We are not quite ready to abandon classroom learning in favor of on-line education.
We're not quite ready to abandon classroom learning in favor of on-line education.

A recent study for the Department of Education (NY Times piece; full 93-page PDF report) performed a meta-analysis of 99 students over the past 12 years, and found that students in on-line courses did slightly, but statistically significantly, better than those in traditional classrooms.

It’s an interesting study, and likely to spur a whole new slew of interest in on-line courses, but it’s really not clear what it means. I’m sure there are a zillion ways I could pull together data showing an education advantage of X over Y, for a zillion Xs and Ys, but one would have to be very careful in the interpretation. I’m willing to bet that most of my colleagues here at UMM would teach better in English than Chinese, and most faculty in China would teach more effectively in Chinese than English, but that hardly means one is a better teaching language than the other. Context is everything, and it’s not clear (at least in the survey study) what the contexts are.

A few possible issues:

  • 12 years is an eternity in the history of the web and web-based teaching. Are the studies from 12 years ago even talking about the same things as those now?
  • What are we actually comparing? Face-to-face courses have all kinds of variance, and their effectiveness changes with the instructor, the students in a particular running of the course, and external events. Presumably on-line courses will as well. Are we comparing the best to the best? The median to the median? I can easily imagine that an on-line course of a few dozen people can be a vastly better experience than a huge lecture hall of 800 students, even if the latter is still called “face-to-face” instruction. Similarly, one person struggling to manage 150 on-line students is not likely to look good compared to an energetic classroom discussion section of 12 people. The meta-survey doesn’t make it easy to see clearly what the comparisons are in the individual surveys, and I suspect that they probably vary widely, ranging from the pretty reasonable to apples-vs-kumquats.
  • How much of this is simply a function of novelty, both in faculty putting a lot of effort into a cool new thing, and students being impressed by the shiny new toy?

Etc., etc., etc.

I think the question isn’t, and can’t ever be, whether on-line is better than the classroom. In the end it’s about finding a way for a particular instructor and a particular student (or group of students) to work well together, and that’s going to depend on an awful lot of things and almost certainly change over time as teachers, students, and the world changes. On-line education and classroom education augmented with on-line components are clearly going to be an important (and probably increasing) part of that, but there will probably always be circumstances where a group of people are better served by some face time than by an on-line experience.

This study also looks at courses as isolated experiences. At a residential university like ours, the courses are crucial, but hardly the whole picture. Students learn a ton from simply living together, eating, doing laundry, volunteering, going to the movies, dating, being in clubs, and generally making all sorts of vital transitions as they move from 18 to 22 (give or take). Look at the important differences between someone’s who’s 16 and someone who’s 26, and an awful lot of that has nothing do to with courses. A good university experience can play a critical role both in and beyond the classroom, and a heck of a lot of that is tied up in physical presence.

Related posts

Cool discussion of Web 2.0 by ThoughtWorks crew

I spent a lot of time on the road to & from the Twin Cities in the last few weeks, so I used that chance to catch up on some old podcasts and explore some new ones. A really nifty discovery this weekend was a panel discussion on Web 2.0 by the smart folks at ThoughtWorks. The discussion is led by Martin Fowler. Fowler goes through Tim O’Reilly’s seminal “What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software” from 5 years ago, where O’Reilly lays out what he believes to be seven defining principles of Web 2.0. Fowler and the panel discusses each of these seven principles, looking at how they’ve held up over time. While the panelists didn’t think that all had held up equally well, in general O’Reilly had successfully identified many of the key trends. One might think this conversation is pretty esoteric, but I think it would be understandable and valuable to anyone looking to better understand what the web has become (and is still becoming). Definitely recommended!

It’s interesting to see in what ways various organizations do and don’t “get” these changes. Sadly, the U in general and the Morris campus in specific, for example, aren’t generally real on top of things when it comes to modern web technology. What’s particularly frustrating is the U’s unwillingness to work with and empower their users to help generate and manage content and value. Big Web 2.0 successes like Google and Amazon, Twitter and Flickr are all about leveraging user generated content. The U has its little fits in that direction (the U of M wiki, the UThink blogs), but they’re always peripheral to the life of the University, always in the back alleys instead of on the front page.

The ThoughtWorks discussion runs about an hour, and they divided it up into three chunks for podcasting. Unfortunately they haven’t released a new podcast since last July, so it appears that I’m late to the party and the party may be over. I look forward to listening to their other podcasts, and I certainly hope that they start making new episodes sometime soon.

If you’re interested you can find all their podcasts on the ThoughtWorks What We Say page through either RSS or iTunes.

Related posts