Transcript EAP and Student Placement

EAP and Student
Placement
Karen Borglum, Ed.D.
Assistant Vice President
Curriculum and Articulation
Walmart “GO” Grant
• Spring of 2010: EAP program receives a two-year grant from
Walmart
• Outcomes of grant
• Develop an EAP/Gen Ed curriculum designed to increase student
academic success
• Establish a common course repository
• Implement an aligned curriculum during two consecutive
semesters
• Engage faculty in data and curriculum evaluation
• Engage targeted students in community-building activities
• Disseminate the curriculum model
Next Steps
• Dollars were then used to address placement issues
• A UCF Statistician was hired to review data and make
suggestions
Present Placement Process
• Four tests given: essay writing (graded by a faculty member)
worth 50%; reading test, grammar test, and usage/mechanics
test graded by Accuplacer system for the remaining 50%
Goals of Statistical Analysis
• To determine if faculty assessments of student essays were
reliable across graders.
• To determine which of the computerized essay evaluation
software packages, most closely agree with assessment of
three panel graders
Methodology
• Sixty students were given entrance essays and were graded by
11 faculty members collegewide.
• The three faculty scores were averaged to produce a
composite score for comparison in Accuplacer (present
system) and the Compass (new system)
Findings
• Inter-rater reliability when using our faculty to grade
essays is .66 (n=60 essays)
• Spearman’s Rho coefficients comparing faculty and
computer scoring:
• Accuplacer-.747
• Compass-.851
• There is substantial variability between faculty graders
(especially across campuses), but the average scores
produced by groups of graders match up with
computerized grading
Meetings
• Faculty and Deans met over the course of about a year to
discuss results, review data, run tests, and determine next
steps
Meeting History
• May 3, 2012: Discussion of history of EAP,
review of statistical analysis, preliminary
discussion of systems.
• Need to discuss Compass testing with Frank Potter from Compass
• Need only 3 tests for placement instead of 4: Essay, Reading, and
Listening
• May 24, 2012: Sent e-mail to Campus Presidents
explaining EAP Placement Issue
• June 4, 2012: Met with Frank Potter to look at
possible cut scores
• Indian River, Jacksonville, Miami Dade and Palm Beach
use Compass
Meeting History Continued
• September 10, 2012: Discussion of how to establish cut
scores in Compass
• 300 units were provided to run a Beta test in reading similar to
writing
• November 12, 2012: Reviewed data from Beta tests
• Determined that we needed linear regression done from
Compass
• February 6, 2013: James May reviewed his work with
Compass Statistician to determine cut-scores
Cut-Score Decision
• Two issues emerged from Compass data:
• 1. Restriction of range issue: not a true population since our
students can only place with a score of 66.
• 2. Equitable forms issue: the line cut scores are an average of 4
tests, so the placement is obscured
Final Decision
• Take an average of reading and essay scores to place students
into reading, grammar and writing
• Collect data on the listening tests to make a determination on
a separate placement for the speaking course
Cut Scores
Course Number
Reading Score
Writing Score
EAP 0281
(combined skills)
15-29
1-2
EAP 0320 (reading)
30-39
EAP 0340 (writing)
3
EAP 0420 (reading)
40-49
EAP 1520 (reading)
50-69
EAP 1540 (writing)
EAP 1620 (reading)
5-6
70-94
EAP 1640 (writing)
Test out of EAP
4
7-9
95-100
10-11
Cost
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Previous year:
# of students tested – 1,100
# of units used – 3,300
Price per unit - $ 1.75 (Accuplacer ESL, AKA LOEP)
Total cost - $ 5,775
Projection (with Listening test):
# of students to be tested for 1 year – 1,200
# of units needed – 8,320*
Price per unit - $ 1.50 (ACT Compass ESL)
Total cost – 12,480
Projection (without Listening test):
# of students to be tested for 1 year – 1,200
# of units needed – 7,120*
Price per unit - $ 1.50 (ACT Compass ESL)
Total cost – 10,680
• *Based on projection that 1/3 of our students are retaking the test – there is a slightly
lower cost for a retake for the listening and reading sections. E-Write uses 3.5 units
independently of whether it is a first time or retake.
Cost Analysis
• The cost of Compass is more than Accuplacer; however, it is a
better placement tool. There will be a more accurate and
consistent placement for the essay portion of the test.
• The Director of Standardized Testing is seeking approval to
implement a retake fee of $10 or $15 to offset cost.
• The retake fee is NOT being explored because of the possible
new placement test; it was already in the works with the
current system. We already have a retake fee for students
retaking the PERT.
Additional Information
• Valencia’s Continuing Ed Division is also looking to use
Compass for their placement work (we can get a pricing
discount)
• Faculty and Deans have voted to approve cut scores
• Deans have discussed results with their Campus Presidents
• The Director of Standardized Testing is supportive of this
switch
• Agreement to review placement information January 2014 to
determine if placement needs to be tweaked
• There are mechanisms in place if students/faculty feel that a
student is improperly placed upon initial testing
Voting
• Supportive of new placement structure
• Non-supportive of new placement structure