Transcript Document

Research Issues in Developing Games
for Learning and Assessment
Gregory K.W.K. Chung
California Educational Research Association (CERA)
San Francisco, CA – November 19, 2009
Overview
2
•
Project overview
•
Why study games for learning?
•
Tensions along the way
•
Some design variables
•
Study results
•
Conclusion and next steps
2/∞
Project Overview
•
Center for Advanced Technology in Schools (CATS)
• USC Game Innovation Lab
•
R&D focused on games and simulations for
learning and assessment
•
Content focus is pre-algebra (rational numbers,
solving equations, functions)
•
Target population is underprepared students
•
Systematic testing of features (instructional
variations, game-based) before full-scale
implementation
3/∞
Why Study Games for Learning?
•
If you build it, they will play (and learn) ...
• Given: Students choose to spend hours playing games
• Idea: Let’s put academic content in games
• Magic: Students will play the game, be engaged in the
game, and will learn the stuff
• fait accompli
• Recall scantron (1950s), word processors (1980s),
calculators (1980s), OPAC (1980s), Web (1990s) ...
• It’s going to happen with or without R&D, so let’s
figure out ways to shape the process
4/∞
Why Study Games for Learning?
•
Help determine the relationship among:
• Different instructional design variables AND
• Different game design variables AND
• Different types of learning outcomes AND
• Different types of students AND
• Different types of game outcomes
5/∞
Tensions: Games for Learning Math
•
•
•
•
•
•
•
•
6
game <–--> learning
fun <–--> math
play time <–--> efficiency
choose to play <–--> have to play
“pure” math <–--> “applied” math
basic skills <---> 21st century skills
simple tasks <–--> complex tasks
unobtrusive measures (embedded) <--->
obtrusive measures (external)
6/∞
The R&D Challenge
Math outcomes
Instruction
• Skills
• Conceptual
• Tutorial
• Feedback
Core mechanics
• Must use
math
Motivational
elements
understanding
?
Game outcomes
• Game level
• Gaminess
• Bling
7
7/∞
Game Design Variables
•
Feedback
•
•
•
•
•
Timing
Precision
Impasse-driven
In-game
Assessment
sensing
Instruction
• Game mechanics
• Conceptual
• Procedural
Type
• Scoring
• Performance
8
•
•
Core mechanics
• Part of game
•
Motivation
• Bling
8/∞
Outcome Variables
•
Math outcomes
• Skills
• Conceptual understanding
•
Game outcomes
• Student perception of “gaminess”
•
Flow
• Game level
9
9/∞
Prototype Gamelet
Game Design Requirements
• The Outcome
• Conceptual and computational fluency with rational numbers
(fractions)
•
The Math
• Idea of “unit” and fractional parts
• Additive operations
•
•
Denominator  no. of pieces in 1 unit
•
Numerator  no. of pieces
•
Equivalence
The Challenge: How to do math without killing the
11 / ∞
game
Prototype Game Design
•
Genre
• Puzzle—need to figure out how to navigate from start to
end points
•
Game and Learning Mechanics
• Jumping/bouncing from point to point
• Adding coils to go from point to point
• Only allowed to add pieces of the same fractional size
(i.e., common denominator)
• Need to convert among equivalent units (2/2 = 3/3 =
4/4)
12 / ∞
13 / ∞
14 / ∞
15 / ∞
Study
Research Study
•
Research Question
• To what extent do different kinds of feedback affect
understanding of fractions (i.e., unit), game
performance, and perception of game play?
•
Design
• 2 conditions that varied feedback
•
Gamey: Minimal math instruction
•
Mathy: Emphasized math concepts related to unit
17 / ∞
Sample
•
Sample
• N = 137
•
9th (30%); 10th (18%), 11th (31%), 12th (15%)
• Amount of weekly game play
•
0hr (21%); 1-2hr (40%); 3-6hr (19%); > 6hr (23%)
• Math achievement
•
•
Self-reported grades: A’s and B’s (55%), C’s (31%), D’s and
F’s (13%)
Math pretest: M = 6.34, SD = 3.39, Min. = 0, Max. = 11
18 / ∞
Measures
•
Math outcome
• Pretest, posttest
•
Game outcome
• Last level reached, perception of game
•
Game process measures
• Time, correct fraction additions, incorrect
fraction additions
•
Background
19 / ∞
Results
•
Did we build a game?
•
Did students learn from the game?
•
Was there an effect of type of feedback on:
• Learning?
• Game performance?
• Game perception?
20 / ∞
Did we build a game?
Yes
Results
23 / ∞
Results
24 / ∞
Results
25 / ∞
Did students learn from the game?
It depends
Did students learn from the game?
•
No overall effects of game play on math
posttest scores
• Not surprising—sample was composed of high and
low performers
•
However, our target group—low math
performers—appeared to profit from game
play
• Low performers’ posttest scores (M = 3.08, SD =
2.04) were significantly higher than their pretest
scores (M = 2.55, SD = 1.22). t (48) = 2.0, p = .05,
d = 0.32.
28 / ∞
Was there an effect of
type of feedback on learning?
No
Was there an effect of
type of feedback on game
performance?
Yes
Was there an effect of
type of feedback on game performance?
•
Students in the mathy condition (vs. the gamey
condition):
• Appear to have gone further in the game (p = .08, d = 0.31)
• Committed more correct additions (p = .003, d = 0.49)
• Committed fewer incorrect additions (p = .007, d = 0.48)
33 / ∞
Was there an effect of
type of feedback on game perception?
Probably
Was there an effect of
type of feedback on game performance?
•
Students in the mathy condition (vs. the
gamey condition):
• Perceived the game as more game-like (p = .08)
• Were more willing to use the game as part of
school work (p = .06)
• Agreed more with the statement that the game
helped them understand math (p = .003, d =
0.54)
36 / ∞
Summary
•
Did we build a game? (YES)
•
Did students learn from the game? (ONLY LOW
PERFORMERS)
•
Was there an effect of type of feedback on:
• Learning? (NO)
• Game performance? (YES)
• Game perception? (PROBABLY)
37 / ∞
Conclusion and Next Steps
•
Beginning to understand conditions under which
“mathification” may not hurt game play
• Speculate that math instruction helped students
progress in game
• Impasse-driven instruction
•
Results promising for the development of a
game that includes math content while
preserving game aspect
•
Need stronger instructional intervention
• Building tutorial, just-in-time feedback
38 / ∞
Backup