Transcript Issues of Assessment when Teaching through Problem

Penny Bidgood, Kingston University, UK
Assessment Matters – original assessment for
original student work, HEA York 2011

Why assess?

Who is being assessed/ assessing?

What should be assessed?

Where is the assessment taking place?

How is the assessment done?
HEA, York, 2011
Assessment –processes that appraise
knowledge, understanding, abilities or skills




Promoting learning by providing feedback
Evaluating knowledge, understanding, abilities, skills
Providing a grade
Enabling public and HE providers to know attainment
level
“Diversity of assessment practice between
and within different subjects is to be
expected and welcomed”
(http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/)
HEA, York, 2011

In implementing assessment policies consult subject
bench marks and professionals

Types of assessment should be appropriate for subject,
mode of learning and the student

Promote effective learning

Effective and appropriate measurement but avoid
excessive burden for students

Provide appropriate and timely feedback
HEA, York, 2011
Specialist statistics or Service modules
Similarity: analyse data appropriately and
report results effectively (exploratory
data analysis / statistical modelling)
Differences: class size
depth of mathematics
computer package used
HEA, York, 2011
Students have different strengths and
approaches to learning and may perform
differently in various types of assessment


Student Voice

A variety of assessment tools are required
HEA, York, 2011
Learning outcomes:“Communicate technical ideas in writing”
Lecturers: workloads
 Research takes priority
 Re-assessment issues
Design different types of assessment
HEA, York, 2011
Four Themes:1.
2.
3.
4.
Relating assessment to real world
problems
Assessing statistical thinking
Individualised assessment methods
Assessing problem solving
HEA, York, 2011
Statistics as “mathematics”
 Formal examinations and tests
Statistics as an “applied subject”
 Assignments and projects
Statistics in a consultancy
 Oral and written reports
 portfolios
HEA, York, 2011
Understand
 the purpose and logic of statistical
investigations
 the process of statistical investigations
 mathematical relationships
 probability and chance

Master procedural skills
HEA, York, 2011
Develop
 interpretive skills and statistical literacy
 ability to communicate statistically
 useful statistical dispositions


“The Assessment Challenge in Statistics Education” (1997)
Gal I and Garfield J B
“Assessment Methods in Statistical Education: An
International Perspective” (2010)
Bidgood P, Hunt N & Jolliffe F (eds)
HEA, York, 2011
1.
2.
3.
4.
5.
6.
Emphasise statistical literacy and develop
statistical thinking;
Use real data;
Stress conceptual understanding rather than
mere knowledge of procedures;
Foster active learning in the classroom;
Use technology for developing conceptual
understanding and analyzing data;
Use assessments to improve and evaluate
student learning.
HEA, York, 2011
Garfield (1994, 1995) stressed the need
for assessments that measure the
understanding of a problem solving
approach
 The Mathematics, Statistics and OR
Overview Report (2000) stated “Student
engagement and performance has often
been greatest when dealing with wellfocused problems of a practical nature”

HEA, York, 2011

Using real, or at least realistic, datasets in an
appropriate context of real problems.

Led by reforms in statistical education that
emphasises statistical thinking, reasoning
and conceptual understanding

Demands from employers – graduates with
technical skills and the ability to communicate
findings appropriately
HEA, York, 2011
English National Curriculum
HEA, York, 2011
HEA, York, 2011

Computer lab
◦ On-line testing from a large databank
◦ Using a computer package to analyse data
◦ Practical assessment

Formal examination

In the lecture room
◦ Often difficult to supervise adequately

In a seminar
HEA, York, 2011











Examination
Quizzes
Multiple choice test (question bank)
“Take-home” assignments
Coursework – apply specific techniques to particular problem
Analyse computer output
Group work assessment
Design and carry out a statistical investigation
Prepare a report and present it
Oral presentations
Case Studies/Projects
HEA, York, 2011



One week before students are given a
numerical question with a small amount of data
– just to illustrate the data and their layout.
In the examination proper students are given the
full set of data and access to SPSS.
They are required to explain their choice of test,
briefly describe the SPSS commands, report
summary statistics and draw appropriate
conclusions.
C. Dracup, Northumbria (PiSA project)
HEA, York, 2011
Which of the following statements about the
application of one-way ANOVA is false ?





The null hypothesis being tested is that all of the underlying means
for all groups are identical
The F-statistic compares the variability between the sample means
with the variability within samples
Large values of the F-statistic provide evidence of a difference
between the underlying true means
If the P-value is large, this implies that all the means are the same
A small P-value suggests that the data provides evidence of
differences in the underlying mean between some of the groups
Assessment on a Budget Wild et al in Gal and Garfield
HEA, York, 2011

Issue a published paper for students to
review in advance and answer question(s)
on it in a formal exam setting

Give students case studies throughout the
year, which they are free to discuss with
each other, but the assessment on the
case studies is in the form of a supervised
test
HEA, York 2011
The standard statistics assignment
◦ analyse of a set of data, using a suitable package, and
submit a written report
Plagiarism concerns can have a strong influence
on assessment strategies
Group or individual assessment?
Move away from “take-home” assignments,
particularly in large service modules.
Plagiarism in Statistics Assessment (PiSA) http://www.jiscpas.ac.uk/documents/pisa.pdf
HEA, York, 2011
The RSSCSE and the MSOR Network jointly
funded the PiSA project, which aimed to:



survey HE lecturers in Statistics to find out what
methods of assessment and strategies to deter
plagiarism are being employed currently;
to identify and synthesise elements of good
practice;
to disseminate findings widely.
HEA, York, 2011

Plagiarism is “To take and use as one’s own the
thoughts, writings or inventions of another”
(OED)

Collaboration is to work together for mutual
benefit
Collusion is to work together for mutual benefit
with the intention to deceive a third party

HEA, York 2011








Is coursework important in Statistics?
Are Statistics lecturers alert to plagiarism?
Is plagiarism causing a reduction in
coursework?
How are Statistics lecturers tackling plagiarism?
What good practice can we share?
Not prevalence
Not case history
Deterrence is the key
HEA, York, 2011
 Institutional
procedures
 Organisational measures
 Supervised assessments
 Individualised assessments
 Student-centred assessments
 Electronic submission
HEA, York 2011



Plagiarism, or more specifically collusion, is a
significant problem within large Statistics service
modules and all lecturers need to give serious
attention to anti-plagiarism assessment strategies.
The majority of Statistics lecturers are well aware
of plagiarism issues and are taking action,
however small, to combat it.
It is quite common for Statistics lecturers to fail to
apply institutional procedures in “minor” cases of
plagiarism. In contrast, some lecturers make
every effort to demonstrate how the regulations
and penalties might apply to Statistics
assessments, giving examples of cases detected
in previous years.
HEA, York, 2011




Plagiarism often goes undetected on large
service modules due to a multiplicity of
assessors. It is most likely to be detected
when one person assesses all the students.
There is much innovative work taking place in
the area of individualised assessment, but also
some duplication of effort.
Assessments that require students to collect
their own data, either individually or in small
groups, are widely employed.
Many lecturers have moved away from takehome assignments to in-class supervised
computer-based assessments.
HEA, York, 2011




In-class tests can be exposed to a high risk of
cheating by unsuitable accommodation,
inadequate invigilation, failure to check student
identities, and naïve organisation.
TURNITIN is being used increasingly.
In projects/case studies it is good practice to
include an element that assesses the student’s
working method and, ideally, an oral to check
that it is genuinely the student’s own work.
Online cheating companies openly offer an
easily accessible way for students to obtain
professional individual help with Statistics
assignments.
HEA, York, 2011

Randomise elements or parameters

Allocate different subset of a large dataset
e.g. ISCUS – Individualised Student Coursework
Using Spreadsheets. This is based on Excel and
allows you to use your own dataset.
(Developed by Neville Hunt, Coventry University)

Students allocated subset of same source data
based on ID number ABCDEFG
oe.g. delete rows C, D and E
oor calculate a 9G% confidence interval
HEA, York, 2011
Students find own data, from journals,
internet sources, about themselves




Allocate a particular periodical – data from any issue
in current year
Medical statistics – find own example of a medical
case study
Time series modules, find own data from
“Economagic”
Sports science – collect data on fellow students
e.g. heart rates
HEA, York, 2011


Report in form of poster, oral presentation,
written as for a newspaper article etc
Vary format of the submission
◦ Posters, typically produced by a group of
students, although each should be able to
“defend” the content
◦ Written report, but vary the “client” –
newspaper article, research paper, briefing
document for local MP etc
HEA,York, 2011




Assignment of sub tasks within a group
Degree of input from each participant
Quality of final product from group
Individual learning which has taken place
Peer Assessment
 Some element of marks might be how they rate
others’ contribution and their own
HEA,York, 2011



Need to include an element that assesses the
student’s working method
Ideally an oral exam or presentation
“4P model”
◦ Project log
◦ Project report
◦ Practical development
◦ Presentation
(Sue Starkings, reported in Gal and Garfield, 1997)
HEA, York, 2011
Aims

To make available real datasets and scenarios of
relevance to Business, Health and Psychology

To develop web-accessible statistics worksheets using
these datasets and various software (Excel, MINITAB,
SPSS)

To develop resources for producing individualised
datasets and assignments

Funded by the Higher Education Funding Council for England
October 2002-January 2006
HEA, York, 2011
ISI, Durban, 2009




Chatfield (2005) – difficult to teach using a problemsolving approach Problem Solving: A Statistician’s Guide
Rossman and Chance (2002) developed materials,
motivated with real data and scenarios, using various
problem-solving skills. ICOTS Proceedings
Jolliffe (2007) “Asking students to do real statistics on
real data and to report on the results, is now feasible in a
way that it was not in the past” (due to huge expansion
in technology) http://www.stat.auckland.ac.nz/~iase
Marriott et al (2009) developed assessment regimes that
correspond to the problem-solving approach in teaching
and learning www.amstat.org/publications/jse/v9n3/
HEA, York 2011





http://www.qaa.ac.uk/academicinfrastructure/
http://stars.ac.uk
http://app.gen.umn.edu/artist/
http://www.jiscpas.ac.uk/documents/pisa.pdf
http://www.amstat.org/education/gaise/GAISEcollege.htm
HEA, York 2011






Bidgood P, Hunt N & Jolliffe F (eds) (2010) “Assessment Methods in
Statistical Education: An International Perspective”
Gal I and Garfield J B (1997) “The Assessment Challenge in Statistics
Education”
o Starkings, S. Assessing Student Projects
o Wild et al Assessment on a Budget
Garfield J B (1994) “Beyond Testing and Grading: Using Assessment to
improve Student Learning” Journal of Statistics Education
Garfield JB (1995) “How Students Learn Statistics” International Statistical
Review 63 1
Holmes P (2004) “Assessment in Statistics: a two-edged sword” in
Assessment with a Purpose Conference
Hunt D N “Individualized Statistics Coursework Using Spreadsheets”
Teaching Statistics 29 2
MSOR Overview Report (2000) Quality Assurance Agency for HE
HEA, York, 2011
40
41
42
Generators for individualised datasets, assignments and
solutions

ISCUS – Individualised Student Coursework Using
Spreadsheets. This is based on Excel and allows you
to use your own dataset as well as those from STARS.
(Developed by Neville Hunt, Coventry University)

DRUID – Dynamic Resources Using Interesting Data
This is not tied to any statistics package but uses
specific datasets. (Developed at the RSS Centre for
Statistical Education, Plymouth University)
HEA, York, 2011
Q5.
Why is the clustered column chart unsuitable for the age data?
There are so many different ages they become cluttered rather
than clustered! The ages need to be grouped into intervals.

We are now going to draw histograms of the ages for each of the treatment
groups so that we can compare them

From the main menu select Graph > Histogram

Highlight Simple and click on OK

Enter Age in the Graph variables: box

Under Multiple Graphs > Multiple Variables choose In separate panels of the same graph and under Multiple
Graphs > By Variables enter Treatment group in the By variables with groups in separate panels: box

Click on OK

Under Scale> Y-Scale Type choose Percent

Click on OK

Enter an appropriate title as before

Click on OK

Click on OK again to produce the chart below.

HEA, York, 2011
Distribution of patients' ages
24
New drug
30
32
40
48
56
64
Placebo
Percent
25
20
15
10
5
0
24
32
40
48
56
64
Age
Panel variable: Treatment Group
Carlow, 2010
Q6.
Is there an evident difference in the distribution of patients’
ages between the two groups?
Both distributions appear quite similar although there is a wider spread of
ages for
thebaseline
Placeboweight
group. The most common age for the Placebo group
Height
and
is 40-44, whilst for the New drug group both 36 -40 and 52-56 have the
same frequencies.
Try replacing C2 (Age) by C4 (Height) in the analysis above.
Q7.
Does the distribution of patients’ heights differ between
groups?
Height has an almost uniform distribution in the Placebo group, but a more
uneven distribution in the New drug group.
Repeat the analysis using the baseline weights in C9.
Q8.
Does the distribution of patients’ weights differ between
groups?
Weight has a slightly negatively skewed distribution in the Placebo group,
while the distribution in the New drug group is bi-modal.
HEA, York, 2011