High-Stakes Assessment:

Using Evidence for Admission Policy Decisions

Cheryl Shaffer PhD, RN Associate Dean School of Nursing

Background

O First-time licensure pass rates have been used as a quality indicator for nursing program success.

O The Accreditation Commission for Education in Nursing (ACEN) require evidence of achievement of the licensure examination pass rates being at or above the national mean.

Background

O The National Council of State Boards of Nursing publishes a new NCLEX (Licensure Examination) test plan and passing standard every three years.

O The latest version of the NCLEX-RN test plan (April 2013) will remain in effect until March 31, 2016.

The Problem

O Nationwide, the competency level for safe and effective entry-level nursing practice has significantly and permanently changed.

O As such, the April 2013 passing standard rise was

the largest single increase ever in the history of the RN licensure exams.

First time NCLEX Pass Rates

2011 2012 2013 * 2014 ** National Average 87.89% 90.34% 83.04% 84.69 % School of Nursing 80.56% 82.96% 73.02% 67.67 % New York Average 84.05% 87.15% 78.14% 82.34% * 2010 and 2013 test plan ** Only January graduates

Previous Admission Criteria

O 3.0 cumulative grade point average for: ENG101, PSY101, BIO130 and BIO132 (and SOC 101 for Advanced Placement Program) O A minimum grade of C must be earned for BIO130 and BIO132 O All prerequisite courses may be retaken only once to achieve a higher grade. The second grades will be used for admission consideration.

Previous Admission Criteria

O Admission to the nursing programs has always been competitive O Based on the NCLEX results, it was hypothesized the current criteria for admission was no longer an accurate predictor of program success

Repeated Course Withdrawals

O Approximately 25% of admitted students withdraw or repeat their science courses O Incentives to withdraw, opposed to course completion from such prerequisite courses may be attributed to their use as preadmission criteria. O Students understand that final averages are crucial in competitive selections.

Repeated Course Withdrawals

O Through regression analysis, repeated attempts in preadmission science courses were found to be a significant predictor of NCLEX-RN outcome in SCCC graduates from 2010 and 2011 (N=335) O There was a significant, negative correlation between the number of science course attempts and passing the NCLEX-RN on the first attempt (

r

= -0.284,

p

=0.000). O Repeating science course grades were found to be a significant predictor of NCLEX success, accounting for 25 percent of the total variance

(F 1, 164 =

5.732, R 2 = 0.254, p = 0.000).

Shaffer, C., McCabe, S. (2013). Evaluating the predictive validity of preadmission academic criteria: high stakes assessment.

Teaching and Learning in Nursing,

8 (4), 157-161.

Repeated Course Withdrawals

O 82 students did not graduate May 2014 due to withdrawal from a nursing course or achieving a grade less than a C O Of those 82 students, 42 % withdrew 1 or more times from a Science prerequisite

Review of the Literature

O Nursing program admission criteria in the literature and other SUNY community colleges and universities on Long Island was reviewed.

O The most common admission criterion was administration of the entrance exam, the Assessment Technologies Institute (ATI) Test of Essential Academic Skills (TEAS V).

O This result was combined with pre-requisite course grades and the use of a scoring rubric.

The TEAS V

O Computerized test, 209 minutes, 150 multiple choice items O Measures students basic or entry-level program readiness in the major content areas of Reading, Mathematics, Science and English/Language usage. O The TEAS V score report will identify students overall academic preparedness necessary to support learning of nursing course content.

The TEAS V

O Academic Preparedness Level category of: O O O Exemplary- high level Advanced-high level Proficient-moderate level O O Basic- low level Developmental-very low level

Assessment Project

O June 2014, all Fall 2014 (N=225) newly admitted students took the TEAS V post admission O Students individual performance profile can was used to guide additional preparation for the nursing program over the summer.

Academic Preparedness

Results

At-Risk Area Qualified for Admission Unqualified for Admission Reading Mathematics Science 72% 32% 11% 20% 30% English Language 12%

Results- Math

Percent of Students Deficient

Numbers and Operations Identify decimal placement Compare rational numbers Divide fractions Solve word problems Algebraic Applications Add, subtract, multiply polynomials solve equations or inequalities Measurement Convert quantities from 1 scale to another 64% 61% 57% 55% 62% 62%

Nursing Students Computation Expectations

O O O O O O O Read a dosage calculation word problem Set up dosage calculation in fractional form Ratio and proportion method Cross multiplying fractions Substitute converted units into proportion Metric system and weight measurement conversions Round to nearest tenths and hundredths

Results- Reading

Percent of Students Deficient Paragraph Comprehension Identify characteristics of different passages Identify topic, main idea, supporting details, themes Analyze and evaluate use of text structure 61 % 68 % 55 %

Results- English/Language

Percent of Students Deficient

Grammar and Word Meanings in Context Apply subject-verb agreement rules Use of grammar for stylistic effect and enhance clarity Use context clues to determine meaning of words Use dialogue correctly Spelling and Punctuation 65 % 60% 61% 58% Capitalization 80%

Nursing Students Writing Expectations

O O O Using APA style for reference page for various sources O (textbook, periodicals, internet sources), Proper use of punctuation or grammar, organization of ideas Apply evidence in the literature to patient assignments

Results- Science

Percent of Students Deficient Life Science

Identify parts of a cell Explain changes in DNA Explain heredity information Compare and contrast chromosomes and genes

Physical Science

Explain physical and chemical properties Balance and identify chemical reactions Distinguish among energies

Scientific Reasoning

Recognize and analyze alternative explanations and models 67 % 68 % 62 % 61 % 63 % 62 % 56 % 70 %

Nursing Students Science Expectations

O Build on prerequisite courses to apply knowledge of the human body anatomy and physiology, and principles of microbiology to learn concepts of pathophysiology O Development of critical thinking, problem solving, analysis and synthesis to provide patient and family nursing care

O

Common Themes

Students demonstrated ability of subject domain knowledge and comprehension O Students were deficient in ability to: O O O O O O O O O Apply Convert Evaluate Explain Solve Distinguish Analyze Compare and contrast Develop alternative explanations

Use of TEAS V Data for Decision Making

O Development of a proposal to change admission policy O Prioritize curricular goals, identify and correct gaps O Improve teaching of higher level cognitive processes O Allocate resources O Interdisciplinary collaboration

New Admission Policy ( in addition to prerequisite course work)

O All* RN program applicants are required to take the pre-admission Test for Essential Academic Skills (TEAS V) and achieve a minimum of Proficient Level O Within an academic year (September 1 through August 31), applicants may only take the TEAS V twice, and must register and pay all fees each time.

Answers to FAQ

O TEAS V is now offered at the Sayville Center O Cost $56 O Directions on how to register is on the School of Nursing Website O Prospective students are advised use the TEAS study guide manual and take practice assessments online prior to taking the exam

Expected Outcomes

O Admitted students will possess the academic preparedness for program success O Students with lower level of academic preparedness can focus on areas to be strengthened before applying to program O Take additional course O O Academic labs Review course