Transcript Update – Effects of Digital Quizzes on Student Performance

Digital Assessments in Flipped
Classrooms Workshop
George Szanto
Docent Marketing Management
[email protected] @hippity7
Fontys Toetsconferencie 2014
Agenda
• 14:55 – 15:05 Introduction of workshop participants
• 15:05 – 15:20 Introduction to my flipped classroom; case
& dilemmas
• 15:20 – 15:30 work on dilemmas in pairs
• 15:30 – 15:50 discuss answers to dilemmas as group
• 15:50 – 16:10 Review of George’s efforts & dilemmas:
quantitative & qualitative testing
• 16:10 – 16:15 Wrap-up , Q & A.
Voettekst
2
BRIEF Participant Introductions
•
•
•
•
•
Name
Institute or Fontys Service
Classes/Subjects that you teach
Yes/No Digital testing experience
Yes/No Flipped class teaching experience
Voettekst
3
Why Flip My Class
Motivations of George
International Exchange
Students
Student Complaints
Poor Quality Group Work
Fighting, Disputes in Groups
Limited Resources
History of IBMS Sem. 6 eMarketing
Strategy class 2011 - 2014
Classical Teaching Flipped Classroom Teaching
Fall Spring Fall Spring Fall Spring Fall
2011 2011 2012 2013 2013 2014 2014
Live Lectures
4
4
4
2
2
6
5
Indiv
Indiv
Recorded Lectures
Live Workshops
Group Presentation in Class
1
1
Individual Presentation in Class or Video
Recording
Extra Group Assignment (20%)
Main Group Assignment Report (80%)
Sit down individual final exam (50 - 80%)
5/6 Individual Open Book/Notes In Class
Quizzes
Online discussion board / class participation
2
20% %30%
20%
80%
26
5
Indiv
26
6
27
6
In pairs In pairs
60%
50%
20% 20% 20%
80%
60% 50% 80%
40 % 40 %
20%
10% Peer
Grading
IBMSEM.ORG Flipped Classroom
11
Assessments
Digital On Demand SaaS
Quizzes
€ 45 per year
~ € 2000 per year
€ 350 per year / 200
students / 1 Lecturer
10 – 14 Minute Executive Level
Emarketing Stratety Pitch
14
Introducing the digital testing
dilemmas for this workshop
1.
2.
3.
4.
5.
6.
7.
8.
9.
BYOD or Fontys Computers?
Wifi/Computer Network down – now what?
Asynchronous testing & cheating
Participation in formative quizzes < 20%
Archiving test results for accreditation
Technical issues for a few students/quiz
Online quizzes – full use of Internet/open resources or not?
How to handle students with learning disabilities ?
Summative quizzes do not help raise final score of project –
now what?
10. Can student take summative quiz outside of workshop?
15
Digital Testing Dilemmas
• For the next 5 minutes, please work in
pairs discussing solution(s) to your
assigned dilemma
16
Your Solutions to Dilemmas 1-5
17
Your Solutions to Dilemmas 6-10
18
Key Questions of my Research
• Do digital quizzes improve the
performance and quality of the final
eMarketing Strategy Plan submitted by
students in a flipped classroom?
• What kind of positive and negative
feedback is given about the digital quizzes
by students who have taken the class?
Voettekst
19
Dependent & Independent
Variables
Quiz Scores
Quantity
Quality
Videos
Assigned
Case
# of hrs.
studied
outside
class
Final
Project
Grade
Past student
knowledge
In-class
participation
Student
Motivation
Voettekst
20
Results Final Exam Spring 2013
(no formative quizzes) N=48
Histogram of Grades Using 2 Grading Methods Spring 2013
12
5.5 Linear Cutoff
10
Average Grading (Ave - 3/4
One Std. Dev.)
Number of Students
8
6
Spring 2013
Linear = 23 failures
4
Average = 11 Failures
2
0
1
1.5
2
2.5
3
3.5
4
4.5
5
5.5
6
6.5
7
GRADE (Average Grade = 46; Std = 12 )
7.5
8
8.5
9
9.5
10
Results Fall 2013 (where 70% of final grade
is from the exam; +formative quizzes)
7
Final Exam Scores - Fall 2013 (N=30)
6
Number of Students
5
4
partial credit
3
No partial credit
2
No partial
credit 24 fail
1
With partial
credit 4 fail
0
0
5
10
15
20
25
30
35
40
45 50 55 60 65
Exam Score in Points
70
75
80
85
90
95 100 105 110
22
Fall 2013 Poor correlation between formative
quiz results and final exam score (but N is small!)
Fall 2013 How about students who take few
formative quizzes?
Student Expectations – baseline survey before
class starts (N= 24)
Fall 2013 Other Independent Variables –
Videos Watched (no correlation, N is small)
Independent Variable – class participation
(No correlation, N is small)
Fall 2013 Independent Variable – past GPA full time
students (poor correlation, N is small)
Spring & Fall 2014 Summative Quizzes
Video Pitch Final Grade vs Quiz Grades (N=67)
10
9
y = -0.0075x + 7.0985
R² = 0.0052
Final Video Pitch Grade
8
7
6
5
4
3
2
1
0
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
Total Quiz Grades (5 or 6 quizzes; maximum total score 100 pts)
100.00
Conclusions
• Digital summative quizzes don’t seem to
help produce quality video pitches
• Student feedback:
– Positive, force students to keep up in class
– Test deeper understanding of theoretical
concepts
• Considering reducing time limits; and
introducing harder questions.
Voettekst
30
Additional Resources
• SURF website – see:
• https://www.surf.nl/kennis-eninnovatie/kennisbank/onderwerp/leren-entoetsen/onderwerp/digitaal-toetsen
Voettekst
31
Thank You – Q & A
33