One framework that has become particularly popular amongst learning games designers is Evidence-Centered
Design (or ECD), because it offers a powerful conceptual design framework that can be used to collect
assessment data in many types of formats—including digital games. The ECD approach to constructing
educational assessments focuses on measurable evidence of a student’s learning. It was developed at the
Educational Testing Service (ETS) by Robert Mislevy, Russell Almond, and Janice Lukas, and has been built upon
extensively by many other researchers. Evidence-Centered Design offers a methodology for collecting and
analyzing evidence from tasks performed by the student.
1
A number of learning game designers have adopted and built upon the ECD model for creating game-based
assessments—including the work of Valerie Shute and her team at Florida State University on ‘stealth
assessments’
and the work of GlassLab Games.
2
Much of this work represents the cutting-edge of the intersection
of games and assessment. And though powerful, this model of game design can be very labor intensive.
Implementing the full ECD framework for designing true psychometrically valid assessments often takes many
months to develop the content model
3
behind the game, before the game design work even begins.
4
For the general learning game designer, this approach may not be necessary or even possible—and many are not
interested in developing psychometrically valid assessments in their games. Yet for those designers, to throw out
ECD entirely is a missed opportunity. The driving design questions and approach that lie at the heart of ECD offer
a powerful lens for dramatically improving the design and opportunities in learning games. As a result, there has
been a dramatic increase in the application of ECD across the learning game community.
5
In this guide, we will discuss how a design lens based on Evidence-Centered Design we call Balanced Design can
serve as a powerful tool and frame for the design of a learning game, as well as the impact and opportunities for
the designers as well as the educators and students if this approach is used.
Frameworks for structuring !
learning in games
1 Mislevy, R., Almond, R., & Lukas, J. (2003). A brief introduction to evidence-centered design. ETS Research Report Series, 2003(1), i-29.
2 Shute, V. J. & Ventura, M. (2013). Measuring and supporting learning in games: Stealth assessment. Cambridge, MA: The MIT Press. Mislevy, R., Oranje, A,
Bauer, M., von Davier, A., Hao, J., Corrigan, S., Hoffman, E., DiCerbo, K., & John, M. (2014). Psychometric Considerations In Game-Based Assessment. A
white paper by GlassLab Research and Institute of Play.
3 See figure on next page.
4 Shute, V. & Ventura, M. (2013). Stealth Assessment Measuring and Supporting Learning in Video Games. The John D. and Catherine T. MacArthur Foundation
Reports on Digital Media and Learning. Cambridge, MA: MIT Press.
5 e.g.: Shute, V. & Torres, R. (2011). Where streams converge: Using evidence-centered design to assess Quest to Learn. In M. Mayrath, J. Clarke-Midura, & D.
H. Robinson (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 91-124). Charlotte,
NC: Information Age Publishing.; Clarke-Midura, & D. H. Robinson (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical
implications from modern research (pp. 91-124). Charlotte, NC: Information Age Publishing.; Code, J., Clarke-Midura, J., Zap, N. & Dede, C. (2012). Virtual
performance assessment for serious games and virtual worlds. In H. Wang (Ed.), Interactivity in E-Learning: Cases and Frameworks. New York, NY: IGI
Publishing.; Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for
complex learning environments. The Journal of Technology, Learning and Assessment, 8(4).; Conrad, S., Clarke-Midura, J., & Klopfer, E. (2014). A Framework for
Structuring Learning%Assessment in an Educational Massively Multiplayer Online Educational Game –%Experiment Centered Design.!International Journal of
Game-Based Learning,!4(1), 37-59.
5