Transcript Slide 1

Creating a Standard Arts Assessment:
Written Evaluation vs. Project Based
An Informal Case Study
Ruth Currey
ARE 6905: Research Trends in Art Education
Purpose
Determine and evaluate the possibility of creating a National
Assessment for computer-based visual arts classrooms
Goals and Processes
• Conduct an informal study that evaluates students on technological processes in both a
written format as well as a skills test format
• Evaluate and consider the data collected from each form of assessment
• Make preliminary conclusions concerning modes of assessment
• Create a potential options for Technology-Based Art Classrooms based on Industry
Standard Software
Personal Interest & Current Status
• With Animation I classrooms packed to the brim, written exams may be the only
feasible option due to time constraints
• Although National Exams such as the NAEP are available for a portion of the arts, these
exams do not cover any technology based skills or processes
• Relevant competency exams are needed for technology based art classrooms
• No data exists currently that accurately demonstrated technology based learning
Questions to Consider
• Will creating a written standard assessment truly demonstrate the artistic knowledge of the students?
• Can multiple choice tests identify a student’s artistic potential?
• Students are currently allowed to enroll in an AP Drawing class based on PSAT, FCAT, and
Stanine scores in Reading and Math
• Can a project based assessment be created?
• What would it look like?
• Would separate standards have to be created by artistic content area?
• Would standards be set based on grade level?
• Does the assessment tool measure what you need it to measure?
• Realistically, can a set of National Standards be designed based on the art
making process?
Possible Limitations to Consider
Assumption – Students will have the same sequence of classes to ensure receipt of all necessary
information
• Depending on the school and guidance department, students are given the opportunity to skip
classes in a sequence due to scheduling
Assumption – Students will have access to the necessary supplies needed to meet the National
Standards for project based assessment art supplies and media availability change from school to school
• Funding is not available to all schools in order to provide the exact same material at every
single school
Preliminary Case Study Examples
Student A
Computer Animation I – Written Skill Exams
Process Sequential Order
Quiz Score: 12/12 (100%)
Fill in the Blank Process
Quiz Score: 11/12 (92%)
Preliminary Case Study Examples
Student B
Computer Animation I – Written Skill Exams
Process Sequential Order
Quiz Score: 3/12 (25%)
Fill in the Blank Process
Quiz Score: 2/12 (17%)
Project Based Assessment
Animation I Student A
Ball Bounce using Motion Guides
Score: 49/50 (98%)
Oral Skills Demonstration Quiz
Score: 23/25 (92%)
Animation I Student B
Ball Bounce using Motion Guides
Score: 47/50 (94%)
Oral Skills Demonstration Quiz
Score: 25/25 (100%)
Review of Literature
The Teacher as Stakeholder in Student Art
Assessment and Art Program Evaluation
By: Charles Dorn
Art Education, 55(4), pp. 40-45
“What cannot be tested cannot be taught, the arts in the near future may
face being left out of the curriculum.” (Dorn, 2002, p. 40)
Article Discusses
• National Education Goals and National Art Standards
• NAEP Assessment
• Authentic Assessment
• What teachers teach and assessment based on information taught
• What needs to be taught
• Models for assessing arts performances
Recommendations
• Teachers develop their own authentic assessment instruments
• Teachers develop school and district assessment plans using a peer review process
• Teachers develop ways to document student progress and establish sensible and appropriate
record keeping systems that will meet the agreed upon goals of the district and state
“In the end, we must realize that US school children are not equal in their aesthetic abilities and US
schools are also not equal in the aesthetic opportunities they provide. But if we can at least entertain
the possibility that either one or both of these conditions are reversible, our best hope lies in deciding
what it is that kids need to know and be able to do and make that the primary focus in reforming
schools and schooling.” (Dorn, 2002, pp. 44-45)
Performance Assessment in the Arts
By: Robin E. Clark
Kappa Delta Pi Record, Fall 2002
“Despite the difficulty of assessing a non quantifiable field like art, teachers in the arts
and other areas should creatively embrace different methods of performance assessment
to evaluate student work more accurately.” (Clark, 2002, p. 32)
What is Performance Assessment?
• In a standards-based curriculum, student performance generally is expected to be the most important
indicator that learning is taking place
• Teachers are required to provide evidence of learning based on multiple assessments
• According to Clark, art should be leading the way in performance based assessment
Process
• The entire range of activities required to produce a work of visual or performing art from
concept to creation
Product
• The aesthetic quality of the student’s finished result of work of art
Art teachers typically recognize the need for both forms of assessment
• A finished product or performance does not necessarily reveal skills, knowledge, creative
thought, and personal meaning of the process that created it
“A balanced approach to grading considers both student engagement in the process of
creation and the quality of the resulting product or performance” (Clark 2002, p. 30).
Performance Assessment in the Arts
By: Robin E. Clark
Kappa Delta Pi Record, Fall 2002
Assessment Methods
• Portfolios: self-assessment and collections of student work over a period of time
• Critique: causes students to reflect on his or her work and hypothesize a means for improvement
• Demonstrations: makes student knowledge level visible based on teaching their peers
• Working Exhibitions: opportunities to actively discuss their all works on display
• Annotated Sketchbooks: documents essential points along their creative process
• Other forms of assessments can be modified to accommodate performance based
assessment
• Written research papers, presentations, class discussions, tests quizzes and exams
Time-Efficient Assessment
• Teachers are open to suggestions and tips regarding time management particularly with assessment
• Well-constructed rubrics
• Provide a way for teachers to evaluate performance easily without compromising divergent
qualities of the individual creative processes as well as the final product performances
• Needs to be a balance between subjective and objective elements
• Essential to have a clear statement regarding what students should know by the end of the
project
• Students should receive copies of the rubric prior to the start of the lesson
“Students’ performance can offer solid proof that they know and are able to do what
you are trying to teach them and what the standards require” (Clark, 2002, p. 32).
Measuring Student Learning in Art Education
By: Donald D. Gruber
Art Education, September 2008
“No single aspect of assessment can provide a representative and accurate measure of
student learning in art” (Gruber, 2008, p. 42).
Difference between Evaluation and Assessment
Evaluation
• Encompasses the global aspects of the curriculum and measures program efficacy
Assessment
• More tightly focused measurements at the level of the individual student and their interactions
within the art program
Developing Assessment Criteria
Process Criteria
• Considering the development of learning and growth within a program
• Teachers report attendance, effort, class participation, homework, etc.
• Reliable measures of student growth and establish a baseline to gauge growth
Product Criteria
• Significance of where students are is more important than how far they have come
• Overall assessments, final exams scores, and final products
• Provide a valid estimate of student learning as an indication of current ability and knowledge
Measuring Student Learning in Art Education
By: Donald D. Gruber
Art Education, September 2008
Methods of Assessment
Written Tests
• Most widely used source of assessment information
• Researchers warn about relying on test validity due to their inability to measure aesthetic
response
Observations
• Consistent surveillance during all phases of the lesson including
• Performance
• Attitude (level of engagement / sharing discoveries / attention to task)
• Work habits and behavior
• Other observational strategies
• Checklists: lists of attributes of projects that can be compared
• Rubrics: detailed guides for scoring student products, performance, and portfolios
Portfolios
• Formative Portfolios: contain all aspects of a particular lesson from initial conceptualizations
through research and revisions to final products
• Summative Portfolios: contain collections of finished products that give indications of abilities at a
given point in the instruction (usually at the end)
Assessment Strategies
• Must consider objectives
• Unproductive to assess without regard to stated objectives
• Measure how well the objectives of each lesson are being met
• Based assessment on a set of established standards
Research Results & Data
Data Charts
• Students we separated based
on educational status
• Gifted=G
• Mainstream=M
• ESE=E
• Numbers reflect points earned
for each assessment method
• A total of 95 students are included
in this study with the breakdown
being:
• Gifted = 10 students (10%)
• Mainstream = 70 students (74%)
• ESE = 15 students (16%)
*Final analysis will include charts and graphs for each subdivision of students
Overview of Results
Overview for Gifted Students
Gifted Breakdown
Sequential Order Average Score: 12/12 (100%)
Fill in the Blank Average Score: 11/12 (91%)
Overall Written Assessment Average: 96%
Ball Project in Flash Average Score: 48/50 (96%)
Oral Skills Test Average Score: 24/25 (96%)
Overall Project Assessment Average: 96%
0% Differential
• No student scored below an 84% on any
written assessment
• All scores for project based assessment
ranged between 92% to 100%
• Gifted students demonstrated a level
consistency across assessment modes
Overview for ESE Students
ESE Breakdown
Sequential Order Average Score: 5/12 (41%)
Fill in the Blank Average Score: 4/12 (33%)
Overall Written Assessment Average: 38%
Ball Project in Flash Average Score: 40/50 (80%)
Oral Skills Test Average Score: 21/25 (83%)
Overall Project Assessment Average: 82%
44% Differential
• Scores ranged from 8% to 66% on written
assessments
• Student scores increase significantly for
project based assessment ranging from 72%
to 100%
• Students successfully demonstrated learning
gains primarily in animation processes through
project based assessment
Mainstream Breakdown
Sequential Order Average Score: 8/12 (66%)
Fill in the Blank Average Score: 8.6/12 (72%)
Overall Written Assessment Average: 69%
Ball Project in Flash Average Score: 41.5/50 (83%)
Oral Skills Test Average Score: 22.25/25 (89%)
Overall Project Assessment Average: 85%
16% Differential
• Scores for written assessment modes
exhibited fairly large learning gaps between
the two poles ranging from 16% to 100%
• Project based scores had a significantly
lower learning gap ranging from 75% to 100%
• Scores demonstrated a statistically significant
increase from written assessment to project
based assessment modes
• Data exhibits a consistent incline as the mode
of assessment shifts from written to project
Mainstream Student Data
Preliminary conclusions
• Preliminary research indicated that the most effective means of assessment was to
maintain a balanced approach
• Based on the data collected, I would be inclined to agree
• Gifted students seemed to excel across the board (96% for both written & project based)
• Mainstream students maintained a steady incline as the assessments moved towards
project based (69% written / 85% project based)
• ESE students responded significantly higher when assessed through project based (38%
written / 82% project based)
• Data also indicated multiple occurrences where ESE students out performed mainstream
students on project based tasks
• Written assessment seems to yield lower scores for mainstream and ESE students
• Preliminary data indicates failing scores for written assessments, however the point
values are lower in comparison to the project based values
• For this reason, the data was informally compared to the final grade received
• When comparing results to the final grades, most students from each subgroup
were able to successfully complete the course with a “C” or higher
• However…
• Does adding the written assessments automatically decrease their final
grade potential?
• Should that matter?
• Consideration also needs to be give to other potential factors such as lack of reading or
writing skills
Possible Solutions
Written Assessment
• Lends itself well to overcrowded introductory level courses
• Approximately two-thirds of the subgroups struggled when faced with written assessments
Project Based Assessment
• Does not lend itself to individualized work due to overcrowding and lack of available technology
• Success rates are higher for all subgroups
Based on the data collected, the best option would be to create a balanced curriculum that requires
multiple modes of assessment in order to provide the most opportunities for success for the students
A Balanced Combination
• Students will be assessed in a manner that reflects the principles of differentiated instruction
• Students will be exposed to written as well as project based assignments in order to fully
demonstrate where their strengths and weaknesses lie
• Students will have the opportunity to work with partners to compensate for overcrowding,
However, individual oral exams as well as project accountability must be built into the rubric
and the curriculum
Possible Limitations
• Provides the best opportunity for students to succeed and demonstrate their learning
gains and individual skills if the value attached to the assessment is balanced over the entire
curriculum
• Does not lend itself to overcrowded classroom due to the time needed to log in each individual
student for individual assessment.
• Exam periods prove difficult
Personal Learning Gains & Reflection
• Statistics and I have decided to maintain our love hate relationship
• Teaching and assessing students effectively are trial and error processes
• When working in a relatively new aspect of a content area there are no absolutes
• More research needs to be conducted in the field of technology based art
education before any definitive recommendations can be made
• Just from my own personal research, I have learned such a great deal about how
my students function in my classroom and how they respond to the assessments
and activities I present
• If more teachers took the time to consider how their assessments are actually
evaluating their students, more effective assessments may be possible
• Even with all of the data that was collected, I am still undecided on how to proceed
with the assessments for the Animation I course
• The written assessments demonstrate a task oriented understanding based on
the representation and interpretation of language
• The project assessments demonstrate a literal grasp of the information and exhibit
direct learning gains based on a tangible product
References
Clark, R. (2002). Performance Assessment in the Arts. Kappa Delta Pi Record, 39(1),
pp. 29-32.
Dorn, C. (2003). Models for Assessing Art Performance (MAAP): A K-12 Project. Studies
in Art Education, 44(4), pp. 350-370.
Dorn, C. (2002). The Teacher as Stakeholder in Student Art Assessment and Art Program
Evaluation. Art Education, 55(4), pp. 40-45.
Gruber, D. (2008). Measuring Student Learning in Art Education. Art Education, 61(5),
pp. 40-45.