A Framework for Assessing Learning Outcomes in Online Business Simulations

Like the last entry, this entry is also derived from a paper I wrote for my master's work in the Technology, Innovation, and Education program at Harvard's Graduate School of Education. My advisor on this project was noted knowledge networks researcher Barry Fishman, visiting Harvard from the University of Michigan. Harvard Business School Publishing is about to launch the first two products in our new series of online business simulations and I wanted to investigate how we can ensure that this product line produces effective learning for our customers. This paper explores a framework for assessing effectiveness in online business simulations.
Introduction
Problem Statement and Paper Structure

Harvard Business School Publishing (HBSP) has created a new product line of online business simulations. We know from faculty customer surveys that they are highly concerned themselves with how easy the simulations are to use, administer, and debrief, and that they are equally concerned that their demanding, and increasingly technically-literate, students rate the learning experience as engaging and dynamic. Hence these eLearning products are designed to be engaging, interactive, usable and teachable.

We also know from faculty surveys that these types of learning tools are considered to be very effective. But what does the literature say about these kinds of claims? How can HBS Publishing know whether users of these simulations are truly learning? What counts as successful outcomes for the user? What is the relationship between instructional design and learning efficacy in these types of products? How can HBS Publishing evaluate these tools to gain confidence that they are designed properly to promote effective learning?

There is a standing and growing body of literature that