Transcript Evaluating Writing for Quantitative Reasoning

Evaluating Writing for
Quantitative Reasoning
Integrative Learning Project 2005 Summer Institute
Carleton College: Scott Bierman, Liz Ciner, Jackie Lauer-Glebov, Carol Rutz,
Mary Savina
Emails: [email protected]
[email protected]
[email protected]
[email protected]
[email protected]
Carleton’s ILP Project
► Cross-cutting
literacies and skills
► Tie in with faculty “workload” – improving
faculty members’ knowledge of what
everyone else is doing.
Today: One piece – Connecting quantitative
literacy and writing
Questions for you
► How
do you evaluate students’ writing on
your campus?
► How do you evaluate students’ quantitative
reasoning (or literacy or. . . ) on your
campus?
► How might you combine the two?
Outline
Quantitative Inquiry,
Reasoning, and Knowledge (QUIRK)
► Background:
initiative
► Background: Writing Portfolio initiative
► Writing about QUIRK: First-year seminars
► Developing criteria and reading student
writing
► Intersecting the cross-cutting literacies:
what the future may hold at Carleton
What is QuIRK?
►Inquiry
►Knowledge
►Reasoning
►http://www.go.carleton.edu/qui
rk>www.go.carleton.edu/quirk
Background: Writing Portfolio
Initiative
► May
2001 – Faculty vote to institute writing
portfolio requirement. Volunteers from class
of 2004 submit portfolios in May 2002.
► May 2003 – All sophomores in the class of
2005 submit writing portfolios.
► May 2004 – All sophomores in the class of
2006 submit writing portfolios.
The portfolio/QUIRK cycle
Carleton College 2004, FIPSE proposal
Background: History of QuIRK
►
►
►
►
►
►
►
►
2001- present– faculty meet informally to systematically discuss
concerns about students’ quantitative literacy.
May 2003 – Background white paper presented to Dean and
President
September 2004 – FIPSE grant approved .
September 2004 and continuing – First-year seminars offered,
course development funds for faculty, speakers, QuIRK faculty
workshops
June 2005 - QUANT squad formed, portfolio reading and rubric
revised
August 2005 – workshop featuring College of San Mateo
learning community – Tools for Thought (Jean Mach and
Mike Burke)
December 2005 – Writing workshop “Writing with Numbers”,
John Bean, facilitator
May 2006 – First writing examples from QuIRK first-year
seminars expected in writing portfolios
How can we help students
appreciate and strengthen QR?
► First-year
seminars
► QuIRK across the curriculum, examples:
 Improve QR in Biology lab reports.
 Develop a data project on the trans-Atlantic
slave trade for use in a History course.
 Improve a QR assignment in a Writing Course.
 Create a new Political Science course
emphasizing comparative electoral analyses.
► Faculty
development and campus events
First-year seminars
► IDSC
100, Measured Thinking: Principles of
Quantitative Reasoning
► POSC 100, Media and Electoral Politics
► SOAN 100, Myths of Crime
► ENTS 100, Geology and Human Health
Other QR assignments in first-year classes,
e.g. English 109 (Rutz, Shuffleton)
Evaluation of First-Year Seminars
(thanks to Jackie Lauer-Glebov, Carleton Office of Institutional
Research)
► 38
students in QR seminars and 45 control
students in other FYS seminars did not differ at
the pretest on seven QR-related questions on firstweek survey (e.g. “I have the skills to read and
understand a statistical analysis of data.”)..
► Compared to pretest scores, the posttest
responses to the QR questions by both QR and
control students were more positive.
► The QR seminar students had more positive
responses to the QR questions than the control
students.
Evaluating QuIRK in student writing
– May/June 2005
► Norming
sessions for “Quant Squad™”
► Quant Squad™ reads 281 portfolios (of
about 480 total), flagging 381 essays with
some QuIRK content from 102 courses, in
25 departments and programs.
► QuIRK taskforce revises criteria, articulates
goals, attends record-setting Minnesota
Twins game.
What we saw in student writing
► Students
are aware of the power of quantitative
claims and quantitative reasoning.
► Many student do not put numbers in context – “is
this number a big one?”
► Many students are less specific than they should
be and they overuse “most,” “many,” “few,”
“seldom.”
► Many students had problems interpreting numbers
and using them to advance arguments.
Some faculty development
implications
► Articulate
the importance of quantitative reasoning
to all faculty (choice of workshop topics).
► Rewrite assignments to encourage students to
report and interpret the quantitative content of
their sources.
► Demonstrate what we mean by “good
interpretation” of data (e.g. for gall fly papers).
► Highlight excellent assignments in the arts and
humanities that encourage students to use and
interpret numbers.
June 2005 goals articulation
► Goals:




Thinks quantitatively
Implements “quantitative analysis” competently
Interprets and evaluates thoughtfully
Communicates effectively
Revised criteria
►
►
►
►
►
►
►
►
►
States questions under consideration in
numerical/quantitative/measurable terms;
Identifies appropriate quantitative/numerical/empirical evidence to
address questions and issues;
Generates, collects, or accesses appropriate data;
Investigates questions and issues by selecting and carrying out
appropriate quantitative or numerical methods;
Uses quantitative methods correctly;
Presents and/or reports the quantitative data appropriately;
Focuses analysis appropriately on relevant data;
Interprets results to address questions and issues under consideration;
Assesses the limitations of the methods employed, if appropriate to the
task or assignment.
What’s next at Carleton? – Using
student writing to evaluate. . .
►
Visual literacy? (Working group formed January 2005
Visual representations of evidence should be governed by the principles of
reasoning about quantitative evidence. For information displays, design
reasoning must correspond to scientific reasoning. Clear and precise seeing
becomes as one with clear and precise thinking – Edward Tufte (1997), Visual
Explanations
►
►
►
►
►
Information literacy? (Mellon pilot project 2001-2004)
Group work? (ECC review in progress)
Integrative learning in individual courses? (HHMI/CISMI)
Ethical inquiry/reflection and civic engagement?
????
Thanks to . . .
► Neil
Lutsky, Sam Patterson, Jackie Lauer-Glebov
and the others who wrote the FIPSE grant
proposal
► Carol Rutz
► The QUANT Squad
► Corrine Taylor, Director of the Quantitative
Reasoning Program at Wellesley College
► Liz Ciner, John Ramsay and the others in the Dean
of the College office
► Lynn Steen, Randy Richardson and others outside
Carleton who’ve helped us