Transcript State of CS Education Research - Co Web Cc Gatech

A View on the State of
CS Education Research
Mark Guzdial
Professor, School of Interactive Computing
College of Computing
Georgia Institute of Technology
1
Story


Brief history of a 40 year old field:
From ESP to ICER
Themes:
◦ The Sorry State of CS1
◦ Prior Conceptions in/of CS
◦ How to Improve CS Ed, and the Research Needed

Future Directions sprinkled throughout
2
The Golden Age of CS Ed Research

Late 1960’s to 1990:
◦ Seymour Papert, Andrea diSessa,
Elliot Soloway, Ben Shneiderman, John Anderson,
Marian Petre, T.R.G. Green



Soloway & Spohrer “Studying the Novice Programmer”
Empirical Studies of Programmers and PPIG Workshops
Then NSF and ONR decided empirical research wasn’t
resulting in faster programmer development.
◦ All funding ceased. Nearly all US CS Ed research disappeared.
3
ACM International Computing
Education Research Workshop

Created as an outlet for the “Bootstrapping”
and “Scaffolding” efforts to re-create CS
Education Research in the United States.
◦ Funded by National Science Foundation (Andy
Bernat)
◦ Josh Tenenberg (U Wash) with Sally Fincher and
Marion Petre

Built around Multi-Institutional, Multi-National
(MIMN) studies.
◦ Why do things go wrong? Because of teacher?
School? Approach?
◦ Or maybe because learning CS is just hard?
All ICER Papers can be found at
http://www.acm.org/dl
Building from Teaching to Research
All ICER Papers can be found at
http://www.acm.org/dl
The Sorry State of CS1

Soloway’s Rainfall Problem
◦ 14% of CS1 students at Yale can solve it

McCracken study “Build a Calculator” (first MIMN
study)
◦ Average score: 21%

Lister (Leeds) study: Array and loops MCQ
◦ Average score: 60%
◦ 23% of students got less than 5 problems completed.
6
Allison Elliot Tew’s dissertation
• Allison Elliott Tew has
just completed the
first languageindependent validated
test of CS1
knowledge.
• 950 subjects, 3
schools, 2 countries.
• Average score on
pseudo-code: 33.78%
On native code:
48.61%
49
What’s so hard?
• Brian Dorn’s graphics designers (more
tomorrow) ranked “assignments” as one of
the hardest ideas to understand.
• Allison’s toughest problem: A=<x>, B=f(A),
what can we say about A and B?
• Middlesex University found that a test of
assignments and sequence was a nearly
perfect prediction of CS1 performance.
• ==> The simplest stuff is much harder than we
realize.
8
What do students know about computing
before they start? (Part 1)
What do students know about computing
before they start? (Part 2)
What do students know about computing
before they start? (Part 3)
Future work: Mechanisms and Efficacy

Research question: How do novices explain
the computing in their lives?
◦ How does Google work? How does a font get
rendered?

Research question: How much does learning
CS influence everyday use of computing?
12
Whether “Objects-first” really matters
There is no “first” in CS Education

Constructivism: People are not a blank slate.
◦ Students already “know” a lot about computation
entering CS1.
◦ Have to build on where they are, not where we want
them to be.


Changing prior conceptions is very hard.
Problem-based learning and contextualized
computing education work, because people can
learn partial and out-of-sequence information.
14
Current PhD Research: Mike Hewner

High school students cling to prior conceptions of
CS despite interventions.
◦ “Someone who can make magazine covers in
Photoshop is a really great computer scientist.”


Science educators address this problem, and
know what to look for.
What are students’ conceptualizations of CS?
◦ Productive vs. Potentially Problematic
Conceptualizations
15
Improving the state of CS Ed



•
People decide early that computer science is
the pits, and they never change their mind.
Yardi and Bruckman found this attitude in
pre-teens (ICER 2007)
Dorn (ICER 2010) found the same in adults.
P2: I went to a meeting for some kind of programmers, something or
other. And they were OLD, and they were nerdy, and they were boring!
And I'm like, this is not my personality. Like I can't work with people like
that.
16
Attitudes aren’t coming from CS Class
Can’t be. There
isn’t one.
17
Growing CS Ed in Secondary Schools

The US education system is locally controlled.
◦ Can’t fix things nationally.
Have to fix things 51 times over.



You can’t create a demand for CS teachers, without
CS classes to hire them into.
Students won’t take a CS class that doesn’t “count.”
Hard to retain and improve CS teachers without
certification.
18
The new AP CS and CS10K

The US NSF is funding a new Advanced Placement
exam “Computer Science: Principles”
◦ One way to change 50 states at once.

The College Board is non-profit, but has to break
even.
◦ For the new AP CS:P to be solvent, must have 20,000
test takers.

NSF’s CS10K Goal: To have 10,000 AP CS:P high
school teachers by 2015.
19
Current PhD Research: Lijun Ni

Teachers who develop teaching identities are retained
and improve.
◦ Much of teacher identity is based on certification.


In general, 46% of STEM teachers leave in first five
years.
How many of those 10K teachers will be left in 2020?
◦ How do CS teachers develop a sense of identity?
◦ Lessons learned: Merged identity not new, and
Community helps.
20
Future Research: Phonics of CS Ed

Students are confused about the details of CS Ed.
◦
◦
◦
◦

How assignments work.
How to read code.
How to sequence code.
The difference between IF and WHILE.
Objects, design, recursion, data structures:
All problems, but maybe smaller if we got the
simple stuff better.
◦ Context motivates. But we need to teach better.
21
Powerful idea we don’t use: Worked Examples

Sweller et al. taught algebra two ways:
◦ One group: Saw couple problems worked out, solved 7 problems.
◦ Second group: Saw couple problems worked out, saw other 7
problems also worked out.
 Take 1/3 as much time.
◦ On algebra test involving problem solving, groups perform the
same.

Pirolli used worked examples to teach recursion.
◦ Showed 8 problems for each of 7 LISP primitives.

Others (Atkinson, Derry et al., Catrambone et al.) show that
practice is critical to learning from examples.
22
Powerful idea we don’t use: SBF

Ashok Goel has been studying for 20 years the
knowledge designers have
◦ Developed Structure-Behavior-Function models.
◦ Structure: The Code.
◦ Function: What it achieves.
 Students tend to see structure, and know function.
◦ Behavior: What the code does to achieve function.

We don’t teach behavior.
23
Need a new medium for CS Ed



Textbooks can only show structure and
identify function.
Visualizations can show behavior, but we
know that it won’t be learned unless students
demonstrate it.
We in US need a new medium for learning CS
in a distance learning format, for CS10K.
24
Questions?