Education Metrics: Second Life Used to Teach Accounting Over 40% Effective (But Much Less When SL Isn't Mandatory)

Responding to my search last Friday for meaningful metrics on the efficacy of Second Life as an educational tool, University of Central Florida lecturer Steven Hornik Tweeted to me this very interesting series of student surveys, for the accounting class he teaches at the school. Hornik uses Second Life to create and convey 3D accounting equations that are part of the classwork, and in the surveys, 40% more of his students on average agreed that they were useful in teaching the underlying accounting concepts involved. "I would say that any tool in which 40% or more students agree helps them to understand accounting is well worth the effort," he concludes.Those are some promising numbers, I do agree. On the other side of the ledger, it's worth making two more observations:

In most of Hornik's classes, between 20-30% of students did not find Second Life effective. Even more concerning, during one semester when using Second Life to learn accounting concepts wasn't mandatory (Fall 2007), the average effectiveness rate plummeted: Only 17% found SL useful, while 17% did not -- and 55% did not even bother using Second Life. My takeaway: SL was relatively useful in this case, but only after it had been made mandatory, and only after Hornik and his class had endured the learning curve to get them into SL. As he writes:

I am always cognizant of the cost/benefit of using Second Life for my students and for ME (as it takes up most of my time, answering orientation questions, grading non-automated HW assignments, dealing with students whose laptops aren’t powerful enough, etc.) [Emph. mine]

Does the time and effort and mandatory requirement make Second Life effective enough for teaching in this case? Despite the excessive time burden, Hornik definitely thinks so, and makes a pretty good case, though I'm not fully convinced -- how about you?

Meanwhile, the search for more SL education metrics continues, please send me more!

Image credit: Steven Hornik.

Comments

[Doreen Garrigus]

They didn't put in the effort to learn SL when they weren't required to. This shouldn't be a surprise to anyone. When you don't know how to use SL, it is more a hindrance than a help.

[Ignatius Onomatopoeia]

Hornik's data support a concept that applies to most US undergrads: if it's not assigned and assessed, they WON'T do the work. Period.

We had a lot of trouble last term when a colleague made BOTH the House of Usher simulation in SL and Poe's story optional. I served as consultant for this assignment.

The student-actors who participated, only about 20% of the class, found the simulation worthwhile and, while working with them, it became clear that they HAD read the text. The other students watched, in class, as the faculty member showed the simulation on the wall.

My colleague naively believed that the novelty of SL would get more of the class interested. It did not.

[Paul]

What we need though, is not what amounts to an opinion survey ("I thought this was effective for me") but data on actual performance (grades etc) between classes that use SL and the same type of classes that don't.

[shaqq]

If you need clear evidence in favour of using SL in education you should check out the River City project at Harvard University.

[Nalates Urriah]

In general 45% of people are more visually oriented right-brain types and 45% more left-brained analytical and auditory. I think it interesting there is a close similarity in the numbers.

I would have thought SL would have a broader range of effect by providing visual, auditory, and hands on. I guess one would have to see the class to see if it is using all the learning channels.

[Ignatius Onomatopoeia]

@ Paul: in the two assessments of our first-year writing classes, my two randomly selected students outperformed writing by similar students picked from other sections that did not use SL.

And that does not mean diddly, because the two students who happened to be selected at random were already strong writers. The measurement was not for the effect of SL, anyhow; it did assess how the presence of a peer tutor could (or not) influence student work.

I'm bored silly by numbers. I won't do that sort of study, but someone should for a broader range of students. I'm sure that several journals would love to publish such a piece of research.

[Steven Hornik]

@paul I agree and disagree. We do need more data and I have a paper that is going to be published in Issues in Accounting Education in August 2010. In it I do examine grades (exam scores) vs student engagement from using SL. I find that when students are engaged via SL they outperform students who were less engaged. The paper goes into some possible reasons for students not being engaged and I hope to have a follow-up article that looks at that issue in more detail soon.

I disagree with your comment that the research needs to examine an SL class vs a non-SL class however. It's a common argument I see, but amounts (IMO) to comparing apples with oranges. You can do things in SL that you can't do in a traditional Face to Face class (otherwise why use SL) so by definition the two treatments are different.

Lastly, I do think this paper (when its finally published) represents one of maybe only a few actual empirical investigations of SL benefits related to student performance.

Hope that helps.

[Suzanne Aurilio]

Well-designed surveys of students assessing their learning gains are considered useful and valid (if still indirect) measures of an intervention's impact on student learning in classrooms. I'd recommend http://www.salgsite.org/ for such an instrument.

What's often lacking in classroom-based educational research is sound research design however. A chemist doesn't usually have educational research expertise. So any move towards using valid instruments is a great.