Transcript Slide 1

Judging
Lethbridge
Tuesday 14 May 2013
2
3
4
Thank
You All
Roy Golsteyn
CWSF Chief Judge
Marc Roussel
CWSF Deputy Chief Judge
5
Location
Exhibits – 1st Choice Savings Centre
6
National Judging Committee
Judith Soon
Chair
Jeff Hoyle
Vice-Chair
Caroline
Whippey
Patrick
Whippey
7
National Judging Committee
• Responsible for judging at CWSF
• Responsible for supporting judging process at Regional
Science Fairs
• Ensures integrity and consistency in judging
• Educates about research ethics & academic integrity
• Assesses compliance with YSC research policies
8
Canada Wide Judging Advisory Panel
Roy Golsteyn
Marc Roussel
CWSF 2013
Edwin Tam
Don Thomas
CWSF 2014
Ben Newling
CWSF 2015
Plus the members of the National Judging Committee
9
Canada Wide Judging Advisory Panel
Dianne Fraser
Q.O.P.
James Grant
I.T.
Mark Dzurko
CWSF 2010
Plus the members of the National Judging Committee
10
Judging at CWSF
• CWSF is for and about the finalists
• The judging experience is the raison d’être
• The goal of the CJAP is to run a superb judging
operation, and thus guarantee a successful
CWSF.
11
Numbering the Projects
02 03 16
Challenge
Category
01
02
03
Junior 7 - 8
Intermediate 9 -10
Senior 11 - 12
Counter
01
02
03
Discovery
Energy
Environment
04
05
06
Health
Information
Innovation
04
05
06
07
Resources
07
Energy - Senior - Project 16
01
02
03
12
Ordering the Projects
Projects ordered by Challenge Awards
01 Discovery
02 Energy
03 Environment
04 Health
05 Information
06 Innovation
07 Resources
13
Judging Task
•
•
•
•
To be fair
To be sensitive
To be comprehensive
To be a positive role model
14
Preparation
•
•
•
•
•
•
Check your Registration information is complete
Visit http://judging.youthscience.ca/
Review all the pages on this site
Review the Project Judging Form
Read the Project Reports, available 1 week prior
Prepare questions
15
Judges Orientation
Monday 13 May 2013
From
To
Event
3:30 pm
3:50 pm
Registration for all Team Captains
4:00 pm
5:00
Orientation for all Team Captains, morning & afternoon
4:00 pm
6:00
Registration
5:00
6:30
Supper. Sit at Morning Judging Team Tables
Review morning judging process
6:30
7:30
Judging Workshop in PE 250
7:30
8:00
Sit at Afternoon Judging Team Tables.
Review afternoon judging process
8:00
10:00
View projects without the finalists
Review log books and display
Prepare questions for tomorrow
View extra projects in addition to your own.
16
Judging Timetable - 1
Tuesday 14 May 2013
Start
End
Event
7:00 am
8:30 am
Continental breakfast and orientation in teams
8:20
8:50
Orientation in Teams. Attendance is mandatory,
even if your first judging slot is empty
9:00
12:30
Excellence Award and International judging
12:15
12:30
Lunch for judges without a 12:00 appointment.
12:30
1:45
Lunch and discussion in judging teams
1:45
2:00
Deadline for entry of results into data base
2:00
2:15
Interdisciplinary Award Judges meet in teams
2:15
5:30
Interdisciplinary Award Judging
2:00
2:15
Special Awards judges meet in teams
2:15
3:30
Special Awards judging
17
Judging Timetable - 2
Tuesday 14 May 2013
Start
End
Event
2:15
2:30
Cusp Judges meet in teams
2:30
5:30
Cusp Judging
3:30
5:30
Celebration Judging
5:30
6:00
Upon leaving, hand in all paper work in the boxes
provided.
5:30
5:40
Finalists leave the judging hall
18
Excellence Awards
Participants compete against all others in their
grade category
• 10 Gold - $700
• 20 Silver - $300
• 40 Bronze - $100
Awarded in each of
• Junior – grades 7-8
• Intermediate – grades 9-10
• Senior - grades 11 - 12
19
Judging Criteria
Evaluation of Excellence Awards
• Scientific thought
(50%)
• Originality & Creativity (33%)
• Communication
(17%)
• Visual display
• Oral presentation
• Project report
• Logbook
20
Judging Excellence Awards - 1
• All interviews are scheduled, 9:00am – 12:30pm
• Teams of 4 judges assess 7 projects each
• Judging periods of 30 minutes: 20 minute
interview with finalists; 10 minute write-up
• Each finalist is judged four times
• Every team has a captain
• If there is a fifth judge, pair up with another judge
but evaluate each finalist separately
21
Judging Excellence Awards - 2
•
•
•
•
•
12:30 pm – 1:45 pm over Lunch
Teams of 4+ judges discuss and rank projects
CONSENSUS - complete forms
Each team member has an equal voice
Each project receives an appropriate score, composed
of Level (1 – 4) and Rating (0 – 9)
• Enter results into Database using the Playbooks
• Pass in all paperwork to Admin
22
Project Judging Form - 1
Part A Scientific Thought 50%
Experiment
Innovation
Study
Level 1 - Low
Replicate a known experiment to confirm
previous findings .
Build a model or device to duplicate existing
technology or to demonstrate a well-known
physical theory or social/behavioural
intervention.
Existing published material is presented,
unaccompanied by any analysis.
Improve or demonstrate new applications for
existing technological systems, social or
behavioural interventions, existing physical
theories or equipment, and justify them.
Existing published material is presented, accompanied
by some modest analysis and/or a rudimentary study is
undertaken that yields limited data that cannot support
an analysis leading to meaningful results.
Design and build innovative technology; or
provide adaptations to existing technology or to
social or behavioural interventions; extend or
create new physical theory. Human benefit,
advancement of knowledge, and/or economic
applications should be evident.
The study is based on systematic observations and a
literature search. Appropriate analysis of some
significant variable(s) is included, using arithmetic,
statistical, or graphical methods. Qualitative and/or
mixed methods study should include a detailed
description of the procedures and/or techniques applied
to gather and/or analyse the data (e.g. interviewing,
observational fieldwork, constant comparative method,
content analysis).
Integrate several technologies, inventions,
social/behavioural interventions or design and
construct an innovative application that will have
human and/or commercial benefit.
The study correlates information from a variety of peerreviewed publications and from systematic
observations, and reveals significant new information,
or original solutions to problems. Same criteria for
analysis of significant variables and/or description23
of
procedures/techniques as for Level 3.
Level 2 - Fair
Extend a known experiment with modest
improvements to the procedures, data gathering
and possible applications.
Level 3 - Good
Devise and carry out an original experiment.
Identify the significant variables and attempt to
control them. Analyze the results using
appropriate arithmetic, graphical or statistical
methods.
Level 4 - Excellent
Devise and carry out original experimental
research in which most significant variables are
identified and controlled. The data analysis is
thorough and complete.
Project Judging Form - 2
Part B: Originality and Creativity 33%
Level 1
Level 2
Level 3
Level 4
The project design is
simple with little
evidence of student
imagination. It can be
found in books or
magazines
The project design is
simple with evidence
of student imagination.
It uses common
resources or
equipment. The topic
is a current or common
one.
This imaginative
project makes creative
use of the available
resources. It is well
thought out, and some
aspects are above
average.
This highly original
project demonstrates a
novel approach. It
shows resourcefulness
and creativity in the
design, use of
equipment,
construction and/or the
analysis.
24
Project Judging Form - 3
Part C: Communication 17%
Communication is based on four elements:
visual display, oral presentation, project report with background research, and logbook
Level 1
Level 2
Level 3
Level 4
Most or all of the four
elements are simple,
unsubstantial or incomplete.
There is little evidence of
attention to effective
communication. In a pair
project, one member may
have dominated the
discussion.
Some of the four
elements are simple,
unsubstantiated or
incomplete, but there is
evidence of student
attention to
communication. In a pair
project, one member
may have made a
stronger contribution to
the project.
All four elements are
complete and demonstrate
attention to detail and
substance. The
communication components
are each well thought out and
executed. In a pair project
both members made an
equitable contribution to the
presentation.
All four elements are complete
and exceed reasonable
expectations of a student at this
grade. The visual display is
logical and self-explanatory, and
the exhibit is attractive and well
presented. The project report
and logbook are informative,
clearly written and the
bibliography extends beyond
web-based articles. The oral
presentation is clear, logical and
enthusiastic. In a group project,
both members contributed
equitably and effectively to the
presentation
25
Project Judging Form - 4
Use the rubric to assign a level to Parts A, B and C for the project. In addition to
the Level, please assign a single letter rating: H (high), M (medium) or L (low) that
reflects the quality of the project and its strength relative to the other projects you
have assigned the same level. Note: Finalists will not see this sheet.
Part A: Scientific Thought
3
Level 1 - 4
Judging Notes
H
Rating (HML)
Part B: Originality & Creativity
2
Level 1 - 4
M
Rating (HML)
Part C: Communication
4
M
Level 1 - 4
Rating (HML)
Graphing is weak. Spelling errors
on board. Weak lab notebook
Notes on your verbal feedback
I enjoyed your explanation of
kinetic energy. You should work to
strengthen your understanding of
your graph, and learn about error
bars. Explained Electric Current
26
Judging Team Worksheet
Consensus Scores – Scientific Thought
After filling in the judges’ names and project numbers, enter each judge’s level and rating (H,M or L) for each project.
Following discussion of each project’s scoring by all team members, enter a consensus level (1 - 4) and rating (1 – 9) in the right
hand column .
Note: Consensus values are determined through team discussion, not by mathematical calculation (e.g. mean, median, mode)
Use the Blackberry Playbook to enter the consensus values for each project.
Judge
Consensus
Level
Baker
Combes
Dawkins
Rating
Project
Abbott
Elm
010204
3
M
2
H
2
L
3
M
3
L
3
2
010205
3
M
2
L
2
M
2
L
2
L
2
3
010206
Enter into
Playbook
010209
010211
010214
010220
Repeat for: (b) Originality and Creativity (c) Communication
27
Entering Team Results
Show the Playbook screen here
28
Feedback During Judging
*New this Year*
Give feedback during judging at the end of each interview.
Feedback is very important to the finalists!
Remember: Encourage, encourage, encourage!
Be constructive in your comments
• Balance a thing to improve with two positives about the
project.
29
Feedback During Judging - 2
Make a note of the feedback you provided in the Judging
Notes section of the Judging Form. e.g.
• suggested how to interpret the data better;
• suggested a book or article to be read;
• explained a concept poorly understood e.g. kinetic
energy
As long as any feedback is noted on the judging form, it can
be included in the discussion prior to ranking. It should not
have a substantial impact on the final results.
30
Please Sign your Name
Be sure to sign your name on the
finalist’s timetable before you leave
each project.
31
Lunch
We need to maximize the time spent in discussion.
We will call your table number for lunch.
We will ensure you spend only ten minutes in the line up.
Discussions must be complete by 1:45 pm.
32
Afternoon Judging
Five Judging Activities
• Cusp Judging: Review projects close to boundaries
•
•
•
•
•
•
•
•
Top Gold
Gold – Silver
Silver – Bronze
Bronze – no award
Interdisciplinary Awards
Special Awards
Challenge Awards
Celebration Judging
33
Excellence Award Cusp Judging
Time: 2:30 – 5:30 pm
Team Captains and Category Leaders meet
at your tables for instructions.
Interview projects on the Cusps:
• Top 6 Gold
• Gold – Silver boundary
• Silver – Bronze boundary
• Bronze – no award boundary
34
Excellence Award Cusp Judging 1
Individual Judge
Working with the Team Captain, enter the project numbers to be judged,
as assigned by the Category Leader
Project Number
010205
Rank
010220
Rank
030209
Rank
8
Comment
Project Number
5
Comment
Project Number
1
Comment
5 more projects
35
Excellence Awards Cusp Judging 2
Team Consensus
Enter the Ranking (e.g. 1-8) of each project by each judge on your team. Through reasoned
discussion, determine a consensus rank for each project highest (1) to lowest (8)
Project Number
Judge’s Name
010304
010316
020305
020314 040306 050308
070304
Henry Higgins
8
7
3
1
2
4
5
Ian Ibsen
8
6
4
3
2
1
7
Janet James
6
8
5
2
1
3
7
Kelly Kaczka
7
8
5
2
3
4
6
Lorna Lewis
7
6
8
1
2
4
3
Mandy Maclin
6
8
4
2
1
7
5
Consensus Rank
7
8
4
1
2
3
5
1
more
col
36
Excellence Awards – Final Cusp Ranking
Rank
Project #
Title (abbreviated)
1
010103
Sleep on This
2
020106
Wind Turbines Noise Stress
3
030119
Heavy Metal Mitigation
4
010115
Does An Electric Field Affect Plant Growth?
5
070108
Seed preconditioning to increase crop yield
6
060111
Can Tires Replace Furnace Oil?
7
040102
Can Your Diet Prevent Alzheimers?
8
050109
Distraction
37
Interdisciplinary Awards - 1
Each Finalist can self-nominate for up to three
Interdisciplinary Awards
Examples
• The Manning Innovation Achievement Awards
• Renewable Energy Award
• Canadian Stockholm Junior Water Prize
38
Interdisciplinary Awards - 2
Some projects will not be well matched to the
criteria.
Judge them with enthusiasm.
Our emphasis is on celebrating the finalist’s
achievement, not just on selecting the winner.
39
Interdisciplinary Awards - 3
Round One
Time: 2:00 – 3:30
First round interview is scheduled.
First Interview starts at 2:00 pm
Ten minutes per interview
6 interviews per judge maximum
Each project is judged twice
Eliminate the bottom 80% in round one
40
Interdisciplinary Awards - 4
Repeat the Round 1 process on the remaining 20%
A third round may be required for a few awards
Final result is by consensus
Complete paperwork and hand it in
41
Interdisciplinary Award - 5
060102
Project Results
Frost Buster
Madalon Burnett
Renewable Energy Award - Junior
An outstanding project related to both energy and air quality with a
demonstrated interest in environmental stewardship.
Yes = Top 20%; No = Bottom 80% or the project does not meet the award criteria.
Top 20%
Round 1
Yes
Comments Project on melting ice. Does not know what
Latent Heat means
No

Top 20% go on to Round 2
42
Interdisciplinary Award Final Result
Renewable Energy Award
Junior
An outstanding project related to both energy and air quality with a
demonstrated interest in environmental stewardship.
Please give the winner and one alternate
Project #
Name(s)
Project Title
1
010112
Albert Atkinson
A Better Air Filter
Alt
060105
Barbara Bull
Using Microbes to Remove Metals
Team Leader
Signature
Yardlee Yates
43
Special Awards
Judging based on the Excellence Awards
Self-nomination not required
Examples
• Canadian Association of Physicists Prize
• Award for Excellence in Astronomy
44
Special Award Judging - 1
• 2:00 pm Meet at your tables
• Review the list of highest ranked projects who
are eligible
• Interview the highest ranking candidates
45
Special Award Judging - 2
Individual Judge
Project Number
010205
Meets Criteria
Yes
No
Rank
3
010220
Meets Criteria
Yes
No
Rank
2
030209
Meets Criteria
Yes
No
Rank
Comment
Project Number
Comment
Project Number
1
Comment
4 more projects
46
Special Award Final Result
CAP Physics Prize
Senior
Canadian Association of Physicists
An outstanding project in the Physical and Mathematical Sciences related to Physics
Project #
Name(s)
Project Title
1
010306
Gryb
Carbon Nanotubes
Alt
010315
Hammond
Luminescence in Rare Earths
47
Challenge Awards
• Challenge Awards recognize the top project in
each of the seven Canada Wide Youth Science
Challenges in each Grade Category.
– Junior - $500 and certificate
– Intermediate - $750 and certificate
– Senior - $1000 and certificate
48
Challenge Award Judging - 1
• 2:00 pm Meet at your tables
• Review the list of highest ranked projects who
are eligible
• Interview the highest ranking candidates
49
Challenge Award Judging 2
Individual Judge
02 Energy
Project Number
020205
Meets Criteria
Yes
No
Rank
3
020320
Meets Criteria
Yes
No
Rank
2
020109
Meets Criteria
Yes
No
Rank
1
Comment
Project Number
Comment
Project Number
Comment
4 more projects
50
Challenge Award Final Result
02 Energy
Project #
Name(s)
Project Title
1
020109
Ireland
Liquid Solar Cells
Alt
020205
Jones
Wind Turbines – Friend or Foe?
51
Celebration Judging 1
Typically about 80 finalists will not get judged at all
during the afternoon for:
• Excellence Award
• Interdisciplinary Award
• Special Award
Most will spend 2 hours without an interview
We will give them two Celebration Judgings
53
Celebration Judging - 2
Many judges will finish judging by 4:00 pm
• Go to the Celebration Judging Table
• Select two projects from the list
• Visit those projects for 10 - 15 minutes
• Celebrate the work done
• Give as much feedback as you can – be
constructive
• Encourage! Encourage! Encourage!
• Goal: Every finalist has two afternoon interviews
54
Ambassadors
• Dressed in UV shirts
• All are previous winners at the CWSF
• Support finalists and resolve any issues
– My Judge has not shown up
– My computer just died
– I am not feeling well
55
Mentorship - 1
Level
Description
0
I did not receive any mentoring.
1
I exchanged a few emails or phone calls, and/or met with my mentor once or twice
to discuss my ideas.
2
I had occasional contact with my mentor by email or phone, and/or met occasionally
with my mentor who provided some advice or materials.
3
I had regular contact with my mentor by email or phone, and/or met regularly with
my mentor who provided advice, materials, assistance with design/testing, or data
analysis.
4
I had regular face-to-face contact with my mentor and regular access to advice,
materials, space, equipment, design/testing, or other personnel in a specialized
facility.
5
I worked closely with my mentor over an extended period of time to develop the
project idea, plan and conduct the research/development, and analyze the results or
56
test the innovation.
Mentorship - 2
•
•
•
•
All professional scientists receive extensive mentoring.
Read the section Projects – Mentorship here:
http://cwsf.youthscience.ca/fairs/current?tid=163
Does the finalist have a good grasp of the project, and did
he/she do the work?
Do not discount a project just because it was mentored.
57
Non-Disclosure Agreement
• Judging information is confidential and is
not discussed outside the judging hall.
• Intellectual property belongs to finalists
• All digital notes and 5 page reports are to
be deleted after judging is over
• Do NOT discuss judging matters on social
media eg Twitter, Facebook.
58
Conflict of Interest
IF YOU
•
•
•
•
are related to the finalist
have judged the project before
have mentored the project
have other potential conflicts of interest
THEN
You must consult the Chief Judge
59
Keep All Paper
PLEASE
DO NOT
TAKE ANY PAPER AWAY
All paper is sorted and filed for a year
60
Judging 101
A Dramatic Presentation
Patrick & Caroline Whippey
61
The Judge as seen by…
Fellow Judges
Finalists
62
62
Oops !
The following are based on real events, and
they have all happened.
Viewer Discretion is advised.
63
63
Over-enthusiastic Judge
• do not give finalists false hope
• “I enjoyed meeting you.”
• “I particularly liked….”
64
64
Sarcastic Judge
• this is not a Msc/PhD examination
• do not belittle - be joyful, not judgmental
• Every project is to be enjoyed and valued
65
65
Insensitive Judge
• Never discuss the projects in the exhibit hall
where finalists are present
66
66
“Helpful” Feedback
• Give constructive feedback: balance positive and
negative
• Encourage, encourage, encourage
67
67
In Summary
We want every finalist to finish judging and say,
“Wow, that was a fantastic experience”
Please help us make that happen!
68
68
Questions?
69
Thank
You
Again !
Roy Golsteyn
CWSF Chief Judge
Marc Roussel
CWSF Deputy Chief Judge
70