ACSME 2014 Not just another diagnostic test! Michael Jennings (UQ), Deb King (Melbourne), Adam Bridgeman (Sydney), Liz Johnson (Latrobe/Deakin) Michael Jennings.
Download ReportTranscript ACSME 2014 Not just another diagnostic test! Michael Jennings (UQ), Deb King (Melbourne), Adam Bridgeman (Sydney), Liz Johnson (Latrobe/Deakin) Michael Jennings.
Slide 1
ACSME 2014
Not just another diagnostic test!
Michael Jennings (UQ), Deb King (Melbourne),
Adam Bridgeman (Sydney), Liz Johnson
(Latrobe/Deakin)
Michael Jennings
Slide 2
Today’s talk…
…is brought to you by the Office of Learning
and Teaching of the Australian Government.
Michael Jennings
Slide 3
B.Engineering Numbers
Yr
’91
’01
’02
’03
’04
’05
’06
No.
400
600
560
530
520
570
750
Yr
’07
’09
’10
’11
‘12
’13
’14
No.
850
970
1000
1100
1200
1200
1200
Bachelor of Engineering first-year numbers (UQ enrolment data)
Prereqs:
1991 - English, Maths 1 (Maths B), Maths 2 (Maths C), Physics, Chemistry
1995 onwards - English, Intermediate Maths (B), Physics OR Chemistry
Approximately 70% of students come in with both Intermediate &
Advanced Mathematics (B & C)
Michael Jennings
Slide 4
History
Diagnostic Test
• Leo Howard Test 1972-1995
• Paper test – 2007
• Online – 2009 to present
Michael Jennings
Slide 5
Current test – 2009 to present
• UQ T&L grant – cross-discipline team of
Engineering & Science (inc. Maths)
• Background questions
• Learning approaches
• F-10 Maths & Senior Intermediate Maths
• Physics
• Chemistry
Michael Jennings
Slide 6
Current test – OLT grant
Led by USQ, involving UQ, UNE, UTS,
and UNewcastle
• http://getset.ceit.uq.edu.au/
20 maths questions
–18 MCQs, 2 enter the answer
– Can’t remember & never seen
before options
12 physics and 12 chemistry questions
Michael Jennings
Slide 7
Aim of the test
• Is NOT to tell students how much they don’t
know / forgotten
• Is to gauge current mathematical / science
understanding
• Is to tell students what maths/science is used in
their courses so they can choose the appropriate
courses
• Is to identify at-risk students
Michael Jennings
Slide 8
Results
• Consistent over the years
• Students who have done both Intermediate
and Advanced Mathematics (Maths B & C) at
school do better
• Senior topics not done as well as junior ones
• Differentiation and integration
• Correlation between test results, ATAR and
success in first-year engineering
Michael Jennings
Slide 9
Students’ thoughts
Benefit from doing quiz?
%
response
It gave me insight into the knowledge I need for 1st year
It flagged some things I have forgotten and need to
revise
60
I feel this is the beginning of my learning journey at uni
29
It made me feel that my needs were being assessed
No, it achieved none of the above
29
2
82
Michael Jennings
Slide 10
Are you glad you did it?
%
Yes, I have a better idea of what I know and don't
know
73
Yes, I have a better understanding of 1st yr knowledge
expectations
62
Yes, I feel more confident
26
I’m not glad I did it at all
4
What actions will you take as a result?
%
Revise my high school notes
59
Find a relevant text book
40
Ask someone who knows for help
25
Do nothing as I'll pick it up during semester
17
Do nothing because I did well
8
Michael Jennings
Slide 11
What did you dislike about doing the quiz?
%
I haven't disliked doing it at all
38
I forgot a lot of what I knew during the holidays
43
It has scared me
11
It was stressful
6
It didn't help
2
It was time consuming and boring
8
I was still in a holiday mood
35
Kavanagh, L., O’Moore, L, & Samuelowicz, K. (2009). Characterising the first year cohort
knowledge. Proceedings of the AAEE 2009 conference.
Burton, L., Dowling, D., Kavanagh, L., O’Moore, L, & Wilkes, J. (2012). Examining First
Year Students’ Preparedness for Studying Engineering. Proceedings of the AAEE 2012
conference.
Michael Jennings
Slide 12
Question No
2013
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Q13
Q14
Q15
Q16
Q17
Q18
Q19
Q20
Q21
Q22
Q23
Q24
Q25
Concept
Add two numerical fractions
Add two algebraic fractions
Solve an equation
Expand a factorised quadratic
Factorise a quadratic
Solve a quadratic
Solving simultaneous equations
x=
y=
Simplify an expression with powers
Understanding sine, cos, and tan
Straight line equations
What is the slope of the line?
What is the y intercept?
Simple derivative
Optimising a function
Visualising a derivative
Adding different base logarithms
Derivatives - Product rule
Derivatives - Chain rule
Simple integral
Definite integral
Calculate the area under graph
Composition of functions
Correct
Answers
%
97.54
77.05
92.87
89.47
91.95
83.76
70.04
71.98
76.46
84.89
88.99
78.52
92.72
77.67
65.22
64.1
49.38
73.45
89.66
64.81
76.6
64.62
Michael Jennings
Slide 13
Feedback matrix
• http://getset.ceit.uq.edu.au/
Michael Jennings
Slide 14
OLT Extension Grant
2014-2015
• Melbourne, Sydney, Latrobe science,
mathematics and business programmes
• UQ Science
• Macquarie
• Southern Cross, Charles Sturt, Griffith,
UWS
Michael Jennings
Slide 15
What maths is in your course(s)?
What do you want to know?
– Procedural vs conceptual understanding.
With what do you think students struggle?
When are you going to do the test?
What are you going to do with the info from the
test?
QS in Science – www.qsinscience.com.au
Michael Jennings
Slide 16
IT details
• Drupal 7
• Blackboard + other programs
Michael Jennings
Slide 17
Acknowledgements
The 2009 online diagnostic test was funded by a The University of
Queensland Teaching and Learning grant. The project team was
A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and
Information Technology), Dr Liza O’Moore (School of Civil
Engineering), Professor Peter Halley (School of Chemical
Engineering), Professor Paul Lant (School of Chemical Engineering),
Professor Peter Adams (Faculty of Science), Michael Jennings
(School of Mathematics and Physics), Professor Lawrie Gahan
(School of Chemistry & Molecular Biosciences) and Dr Anton Raynor
(School of Mathematics and Physics), .
The OLT funded project involved USQ, UQ, UNE, UTS and
UNewcastle. http://www.olt.gov.au/project-get-set-success-usingonline-self-assessments-motivate-first-year-engineering-studentsengag
The feedback matrix was designed by A/Prof Lydia Kavanagh (Faculty
of Engineering, Architecture and Information Technology), Dr Liza
O’Moore (School of Civil Engineering).
Michael Jennings
Slide 18
References
Burton, Lorelle, Dowling, David, Kavanagh, Lydia, O'Moore, Liza &
Wilkes, Janelle (2012). Examining first year students' preparedness
for studying engineering. In: Llewellyn Mann and Scott Daniel, AAEE
2012 Conference Proceedings. 23rd Annual Conference of the
Australasian Association of Engineering Education (AAEE 2012),
Melbourne, Australia, (1020-1028). 3 - 5 December 2012.
Jennings, M. (2011). The transition from high school to university: The
University of Queensland perspective. In: John Hannah, Mike
Thomas and Louise Sheryn, Te Ara Mokoroa: The Long Abiding
Path of Knowledge - Proceedings of Volcanic Delta 2011. Volcanic
Delta 2011: The Eighth Southern Hemisphere Conference on
Teaching and Learning Undergraduate Mathematics and Statistics,
Rotorua, New Zealand, (139-149). 27 November - 2 December
2011.
Kavanagh, L., O’Moore, L., & Samuelowicz, K. (2009). Characterising
the first year cohort knowledge. 2009 Conference Proceedings.
Annual Conference of the Australasian Association of Engineering
Education (AAEE 2009).
Michael Jennings
ACSME 2014
Not just another diagnostic test!
Michael Jennings (UQ), Deb King (Melbourne),
Adam Bridgeman (Sydney), Liz Johnson
(Latrobe/Deakin)
Michael Jennings
Slide 2
Today’s talk…
…is brought to you by the Office of Learning
and Teaching of the Australian Government.
Michael Jennings
Slide 3
B.Engineering Numbers
Yr
’91
’01
’02
’03
’04
’05
’06
No.
400
600
560
530
520
570
750
Yr
’07
’09
’10
’11
‘12
’13
’14
No.
850
970
1000
1100
1200
1200
1200
Bachelor of Engineering first-year numbers (UQ enrolment data)
Prereqs:
1991 - English, Maths 1 (Maths B), Maths 2 (Maths C), Physics, Chemistry
1995 onwards - English, Intermediate Maths (B), Physics OR Chemistry
Approximately 70% of students come in with both Intermediate &
Advanced Mathematics (B & C)
Michael Jennings
Slide 4
History
Diagnostic Test
• Leo Howard Test 1972-1995
• Paper test – 2007
• Online – 2009 to present
Michael Jennings
Slide 5
Current test – 2009 to present
• UQ T&L grant – cross-discipline team of
Engineering & Science (inc. Maths)
• Background questions
• Learning approaches
• F-10 Maths & Senior Intermediate Maths
• Physics
• Chemistry
Michael Jennings
Slide 6
Current test – OLT grant
Led by USQ, involving UQ, UNE, UTS,
and UNewcastle
• http://getset.ceit.uq.edu.au/
20 maths questions
–18 MCQs, 2 enter the answer
– Can’t remember & never seen
before options
12 physics and 12 chemistry questions
Michael Jennings
Slide 7
Aim of the test
• Is NOT to tell students how much they don’t
know / forgotten
• Is to gauge current mathematical / science
understanding
• Is to tell students what maths/science is used in
their courses so they can choose the appropriate
courses
• Is to identify at-risk students
Michael Jennings
Slide 8
Results
• Consistent over the years
• Students who have done both Intermediate
and Advanced Mathematics (Maths B & C) at
school do better
• Senior topics not done as well as junior ones
• Differentiation and integration
• Correlation between test results, ATAR and
success in first-year engineering
Michael Jennings
Slide 9
Students’ thoughts
Benefit from doing quiz?
%
response
It gave me insight into the knowledge I need for 1st year
It flagged some things I have forgotten and need to
revise
60
I feel this is the beginning of my learning journey at uni
29
It made me feel that my needs were being assessed
No, it achieved none of the above
29
2
82
Michael Jennings
Slide 10
Are you glad you did it?
%
Yes, I have a better idea of what I know and don't
know
73
Yes, I have a better understanding of 1st yr knowledge
expectations
62
Yes, I feel more confident
26
I’m not glad I did it at all
4
What actions will you take as a result?
%
Revise my high school notes
59
Find a relevant text book
40
Ask someone who knows for help
25
Do nothing as I'll pick it up during semester
17
Do nothing because I did well
8
Michael Jennings
Slide 11
What did you dislike about doing the quiz?
%
I haven't disliked doing it at all
38
I forgot a lot of what I knew during the holidays
43
It has scared me
11
It was stressful
6
It didn't help
2
It was time consuming and boring
8
I was still in a holiday mood
35
Kavanagh, L., O’Moore, L, & Samuelowicz, K. (2009). Characterising the first year cohort
knowledge. Proceedings of the AAEE 2009 conference.
Burton, L., Dowling, D., Kavanagh, L., O’Moore, L, & Wilkes, J. (2012). Examining First
Year Students’ Preparedness for Studying Engineering. Proceedings of the AAEE 2012
conference.
Michael Jennings
Slide 12
Question No
2013
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Q13
Q14
Q15
Q16
Q17
Q18
Q19
Q20
Q21
Q22
Q23
Q24
Q25
Concept
Add two numerical fractions
Add two algebraic fractions
Solve an equation
Expand a factorised quadratic
Factorise a quadratic
Solve a quadratic
Solving simultaneous equations
x=
y=
Simplify an expression with powers
Understanding sine, cos, and tan
Straight line equations
What is the slope of the line?
What is the y intercept?
Simple derivative
Optimising a function
Visualising a derivative
Adding different base logarithms
Derivatives - Product rule
Derivatives - Chain rule
Simple integral
Definite integral
Calculate the area under graph
Composition of functions
Correct
Answers
%
97.54
77.05
92.87
89.47
91.95
83.76
70.04
71.98
76.46
84.89
88.99
78.52
92.72
77.67
65.22
64.1
49.38
73.45
89.66
64.81
76.6
64.62
Michael Jennings
Slide 13
Feedback matrix
• http://getset.ceit.uq.edu.au/
Michael Jennings
Slide 14
OLT Extension Grant
2014-2015
• Melbourne, Sydney, Latrobe science,
mathematics and business programmes
• UQ Science
• Macquarie
• Southern Cross, Charles Sturt, Griffith,
UWS
Michael Jennings
Slide 15
What maths is in your course(s)?
What do you want to know?
– Procedural vs conceptual understanding.
With what do you think students struggle?
When are you going to do the test?
What are you going to do with the info from the
test?
QS in Science – www.qsinscience.com.au
Michael Jennings
Slide 16
IT details
• Drupal 7
• Blackboard + other programs
Michael Jennings
Slide 17
Acknowledgements
The 2009 online diagnostic test was funded by a The University of
Queensland Teaching and Learning grant. The project team was
A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and
Information Technology), Dr Liza O’Moore (School of Civil
Engineering), Professor Peter Halley (School of Chemical
Engineering), Professor Paul Lant (School of Chemical Engineering),
Professor Peter Adams (Faculty of Science), Michael Jennings
(School of Mathematics and Physics), Professor Lawrie Gahan
(School of Chemistry & Molecular Biosciences) and Dr Anton Raynor
(School of Mathematics and Physics), .
The OLT funded project involved USQ, UQ, UNE, UTS and
UNewcastle. http://www.olt.gov.au/project-get-set-success-usingonline-self-assessments-motivate-first-year-engineering-studentsengag
The feedback matrix was designed by A/Prof Lydia Kavanagh (Faculty
of Engineering, Architecture and Information Technology), Dr Liza
O’Moore (School of Civil Engineering).
Michael Jennings
Slide 18
References
Burton, Lorelle, Dowling, David, Kavanagh, Lydia, O'Moore, Liza &
Wilkes, Janelle (2012). Examining first year students' preparedness
for studying engineering. In: Llewellyn Mann and Scott Daniel, AAEE
2012 Conference Proceedings. 23rd Annual Conference of the
Australasian Association of Engineering Education (AAEE 2012),
Melbourne, Australia, (1020-1028). 3 - 5 December 2012.
Jennings, M. (2011). The transition from high school to university: The
University of Queensland perspective. In: John Hannah, Mike
Thomas and Louise Sheryn, Te Ara Mokoroa: The Long Abiding
Path of Knowledge - Proceedings of Volcanic Delta 2011. Volcanic
Delta 2011: The Eighth Southern Hemisphere Conference on
Teaching and Learning Undergraduate Mathematics and Statistics,
Rotorua, New Zealand, (139-149). 27 November - 2 December
2011.
Kavanagh, L., O’Moore, L., & Samuelowicz, K. (2009). Characterising
the first year cohort knowledge. 2009 Conference Proceedings.
Annual Conference of the Australasian Association of Engineering
Education (AAEE 2009).
Michael Jennings