Diagnostic Testing : only the beginning A component in a longer process CDC Steele University of Manchester

University of Manchester

• Dates back as far as 1824 by several different routes but took its present form in 2004.

• 35 000 students • 29 000 Home/EU • 6 000 Overseas • 26 000 UG • 9 000 PG

University of Manchester

• 11 500 staff • 4 000 academic • 2 000 research • 341 Buildings

University of Manchester

• Four Faculties – Engineering and Physical Sciences – Life Sciences – Medical Sciences – Humanities

University of Manchester

• School of Mathematics – 90 academic staff – 400-500 mathematics students per year

Service courses to ~ 1000 students in various parts of the University.

– Aerospace Engineering – Civil Engineering – Mechanical Engineering – Chemical Engineering – Electrical Engineering – Chemistry – Optometry – Computer Science – Materials Science – Foundation Year

Entry Profiles of Maths Students

• A-levels – C1, C2, C3, C4 – 2 more modules from Mechanics (M1, M2 etc,) Statistics (S1, S2 etc), Decision (D1, D2 etc) – Many students students AS- or A-level in Further Maths – Generally require A+ in maths A-level

Entry Profiles of Service Students • A-levels – C1, C2, C3, C4 – 2 more modules from Mechanics (M1, M2 etc,) Statistics (S1, S2 etc), Decision (D1, D2 etc) – Some students AS- or A-level in Further Maths

Entry Profiles of Students

• A-levels. Most Engineering schools ask for Grade B in maths (part of ABB etc) • Other UK Qualifications • International or European Qualifications • University of Manchester Foundation Year

Degree Structure

• Most Students : 3 year degree. (BSc, BEng etc) • Some able students : 4 year undergraduate masters, M Math, M Eng etc • Each year : 2 semesters : Sept – Jan and Feb – June. 12 weeks teaching plus exams at end of semester.

The streamed system for Engineers : 1996-2009 • Engineers in several schools ‘streamed’ into P-stream, Q/M stream, R-stream.

P – stream. ‘Elite’ Q/M – stream. ‘Standard’

• Streaming follows diagnostic test but not only outcome of diagnostic test

R – stream. ‘At Risk’

Typical P-stream student

• A at A-level or equivalent • Strong on sections C1, C2, C3 • May be weak on some C4 material • May also be weak on a little earlier material.

Typical Q-stream student

• B,C at A-level or equivalent • Reasonable on sections C1, C2, • May be weak on some C3 material • Not terribly good on C4 material • May also be weak on a little earlier material.

Typical R-stream student

• May not have officially satisfied entry offer • May have large areas of maths which are causing difficulty.

The diagnostic and followup programmes • A means to measure weaknesses and concentrate efforts of students in the early weeks towards areas where they may have underachieved in the diagnostic test – Rusty – Not time for teaching – Genuine difficulty with topic.

– May not have been taught certain material

Pre-diagnostic mailshot

• Mock diagnostic posted to prospective students mid-August including – Mock test paper – Answers – Other possible questions – Links to paper and web-based resources describing the topics.

– Computerised Practice Questions

Diagnostic Test

• A test in Freshers’ week designed to measure student profiles across various mathematical topics.

Diagnostic Testing at UMIST/Manchester • Carried out since 1996 • 1997-2001 : 24 questions (6 sections) in 40 minutes • 2002-2009 : 48 questions (12 sections) in 80 minutes • Change following curriculum 2000.

• Total mark less important than series of ‘mini-marks’ for various sections.

Diagnostic Testing at UMIST/Manchester • Carried out since 1996 • 1996-2004 Short answers • 2005-2008 Multiple-choice (optical reader) • 2009 Multiple-choice (manual entry) • Carried out in Freshers’ week (week 0). Different times for different schools etc.

Questions and Answer box 1996 2004

Questions 2005-2009

Diagnostic Test : Sections

• A : Arithmetic and Algebra • B : Geometry and Trig • C : Sequences • D : Functions and Algebra • E : Polynomials • F : Logarithms and Exponentials • G : Simple Differentiation • H : Simple Integration • I : Rational Functions • J : Further Differentiation • K : Further Integration • L : Vectors A-H : C1, C2, C3 Material : I-L : C4 material

Diagnostic Followup Programme • For students on P- and Q- streams.

• R-stream could itself be considered as a followup • Allocates up to 2 sections from A-L for P stream, up to 2 sections from A-H for Q stream.

Diagnostic Followup Programme • Allocates up to two sections based on marks of 0/4, 1/4, or 2/4.

• Sections A (Arithmetic and Algebra) and D (Functions and Algebra) allocated preferentially, then in alphabetical order.

• Students told of relevant sections via web site towards end of week 1 of semester.

Diagnostic Followup Programme • Students informed of – Relevant section(s) – References to relevant materials – Includes HELM Resources – Pro-active and re-active sessions in Mathematics Resource Centre.

– Practice ‘tests’ on computer.

– Leading up to

Diagnostic Followup Programme • Computerised test in week 4 of semester on relevant topic(s) • Similar to practice tests.

• Exercise counts 20% towards coursework for relevant course unit (itself worth 20% of unit assessment).

Diagnostic Followup Programme • HELM Resources • HEFCE FDTL4 project 2002-2005 • Workbooks on various mathematical topics.

• Workbooks on mathematical topics

Example

Engineering Application

Diagnostic Followup Programme • Manchester Mathematics Resource Centre. – Support Centre for students to drop into with Mathematical Enquiries.

– Also runs pro-active sessions centring on material on Diagnostic Followup.

– Centre is within an Engineering Building but open to ALL students.

Diagnostic Followup Programme • Computer-assessment • Use of STACK, developed by Dr Chris Sangwin, University of Birmingham • Accepts mathematical input • Gives specific feedback • Varies coefficients sensibly

Diagnostic Followup Programme

Diagnostic Followup Programme

A B C D E F

Results 2007

Stud ents 75 114 184 163 41 24 Diag Test Follo wup 83 90 67 38 84 90 65 70 60 89 93 95 G I H J K L Stud ents 3 Diag Test Follo wup 89 53 12 18 24 69 49 49 29 69 83 88 87 87 92

A B C D E F

Results 2008

Stud ents 58 166 179 181 66 117 Diag Test Follo wup 89 78 70 65 84 87 82 78 64 89 80 86 G I H J K L Stud ents 62 Diag Test Follo wup 76 88 47 117 84 67 45 70 46 43 32 62 74 67 68 53 88

Diagnostic Results 2002-09

Longer Consequences of Diagnostic Test • Practice Tests, HELM Materials, Resource Centre all there for further use etc.

Purposes of Diagnostic Test

• Used to assign students to lecture course groupings (until now) • Used to assign students to tutorial groups • Used to inform staff of entry profiles of students • Used to assign each student an individualised programme of work

Diagnostic Followup for Maths Students • Same procedure up to diagnostic test i.e. mailshot, practice questions, • Followup made available to maths students but not feeding into coursework.

• Students told of weaker sections and encouraged to take part in followup on these.

• Can be used as ‘indicator’ of students who may or may not be engaging with the course.

Other Diagnostic Tests

University of Manchester Foundation Year • Students from countries with an educational system which stops slightly short of level for entry to 1 st year • Students desiring a change of direction • Students not quite reaching the level for entry to 1 st year

University of Manchester Foundation Year • Test B • 40 minute test.

• Three sections from Test A plus 3 sections on more basic work • Practice questions provided in advance plus opportunities to try questions electronically.

• Used to inform staff, to allocate students to lecture courses and tutorial groups etc.

Materials Science

• Cannot ask for Mathematics A-level or equivalent. Major variations in arrival mathematics profile

Materials Science

• Test D : 50 minutes. 8 sections i.e. six from Test B test plus two more from test A.

• Used to decide which students need to go in group for more intensive sessions.

Late arrivals

• Can be a problem with compulsory followup.

• Test session a week or two later etc.

• ‘Envelope’-based test system.

Conclusions

• Students arrive with potential weaknesses in some areas.

• Take Diagnostic test and weak areas are identified • Diagnostic Followup addresses areas of weakness • Students can continue to access materials after exercise officially finishes.