Transcript PSAT SOAS Report Workshop Power Point

PSAT/NMSQT®
Connections:
Using PSAT/NMSQT Tools to
Inform and Improve Teaching
and Learning
Belinda Chung, College Board Consultant
Today’s agenda
January 15, 2011
•
•
PSAT/NMSQT Overview
PSAT/NMSQT Score Report Plus
•
AP Potential
•
Sample Questions Work
•
SOAS Report Overview
•
SOAS Group Work
•
Group Insights/Recommendations & Action Plan
•
Evaluation
Mission
The College Board's mission is to
connect students to college success
and opportunity. We are a not-for-profit
membership organization committed to
excellence and equity in education.
PSAT/NMSQT
Overview
PSAT/NMSQT Overview
Content and Question Types
•
Performance compared to the state and nation
•
Question-by-question analysis
•
Early wake-up call
•
•
3.57 million students (2010)
•
55.5% of test takers are pre-juniors (2010)
Utilize tools to help you when Understanding by Design
(backwards planning)
What is the PSAT/NMSQT?
•
A comprehensive academic reasoning test, which focuses on
skills that all students need to succeed in high school and
college.
•
These reasoning skills focus on the three main academic
areas, but their attainment affects all areas of learning, as
they are transferable process skills that are needed to
manage and apply knowledge, regardless of content
orientation.
•
These skills are relevant to all students, and are needed by
all students, thus the strong focus on inclusion when testing
with the PSAT/NMSQT.
PSAT/NMSQT Overview
•
The PSAT/NMSQT is a school-based test
•
Comprised of five sections:
• 2 Critical Reading
• 2 Mathematics
• 1 Writing Skills
•
Critical Reading, Mathematics and Writing Skills scores
are on a 20-80 scale
PSAT/NMSQT Overview
Content and Question Types
Critical Reading
Two 25-minute sections
48 questions total :
- 13 sentence completion
- 35 passage based reading
50 mins
Mathematics
Two 25-minute sections
38 questions total:
- 28 multiple choice
- 10 student produced response
50 mins
Writing Skills
One 30-minute section
39 questions total:
- 20 improving sentences
- 14 identifying sentence errors
- 5 improving paragraphs
30 mins
PSAT/NMSQT Overview
Content and Question Types
•
•
•
Performance compared to the state and nation
Question-by-question analysis
Early wake-up call
•
3.57 million students (2010)
•
55.5% of test takers are pre-juniors (2010)
•
Utilize tools to help you when Understanding by Design
(backwards planning)
Average Test Scores
For Juniors
Average Test Scores
For Sophomores
Critical Reading
Overview
PSAT/NMSQT Overview
Critical Reading
Questions focus on:

Student’s ability to read critically

Student’s ability to think logically

Student’s ability to analyze

Student’s ability to evaluate
PSAT/NMSQT Overview
Critical Reading
Critical Reading
Two 25-minute sections 48 questions total
- 13 sentence completion
- 35 passage based reading
5 skills
•
Determining the Meaning of Words
•
Author’s Craft
•
Reasoning and Inference
•
Organization and Ideas
•
Understanding Literary Elements
50 mins
PSAT/NMSQT Overview
Sentence Completion
5. Because their behavior was ------- , Frank and Susan
served as ------- models for the children under their
excellent care.
(A) incorruptible . . pernicious
(B) lamentable . . flawed
(C) commendable . . exemplary
(D) erratic . . unimpeachable
(E) reputable . . imperfect
PSAT/NMSQT Overview
Sentence Completion
5. Because their behavior was ------- , Frank and Susan
served as ------- models for the children under their
excellent care.
(A) incorruptible . . pernicious
(B) lamentable . . flawed
(C) commendable . . exemplary
(D) erratic . . unimpeachable
(E) reputable . . imperfect
Difficulty Level = Easy
Skill = Determining the Meaning of Words
PSAT/NMSQT Overview
Critical Reading Sample Skill
Determining the Meaning of Words
Use vocabulary skills, context, roots, prefixes, and
suffixes to determine the meaning of words.
PSAT/NMSQT Overview
Passage Based Reading
Passage:
Line1
Advertisers are interested in finding out what exactly
makes pop-up ads on the Internet irritating to viewers.
Why are these ads seen as intrusive? One 2002 marketing
study determined that when Internet users are focused,
Line 5
they perceive interruptions as more severe than when they
are not focused. What seems to get the attention of viewers
(rather than merely annoy them) is to expose them to popup ads only at breaks in content—for example, when they
are switching between pages. Another marketing strategy
Line 10
is to increase the relevance of the ad. If a consumer is
browsing an automobile Web site, a pop-up ad for a car
will seem less intrusive than will a travel ad.
PSAT/NMSQT Overview
Passage Based Reading
9.
The primary function of the sentence in lines 3-6
(“One . . . not focused”) is to
(A) paraphrase the question posed in line 3
(B) present a relevant research finding
(C) hypothesize about a solution to a problem
(D) shift the focus to a different medium
(E) address a concern of Internet users
PSAT/NMSQT Overview
Passage Based Reading
9.
The primary function of the sentence in lines 3-6
(“One . . . not focused”) is to
(A) paraphrase the question posed in line 3
(B) present a relevant research finding
(C) hypothesize about a solution to a problem
(D) shift the focus to a different medium
(E) address a concern of Internet users
Difficulty Level = Medium
Skill = Organization and Ideas
PSAT/NMSQT Overview
Critical Reading Sample Skill
Organization & Ideas
Understand the organization of a reading passage,
and identify the main and supporting ideas.
Mathematics
Overview
PSAT/NMSQT Overview
Mathematics
The primary aim of the math section is to assess how
well students understand and reason within a math
framework:
 Can
they apply what they already know to new
situations, i.e. transfer their learning and knowledge?
 Can
they reason and problem-solve within a high
school math context?
PSAT/NMSQT Overview
Mathematics
Mathematics
Two 25-minute sections
38 questions total
50 mins
- 28 multiple choice
- 10 student produced response
4 Content Skills
5 Process Skills
•
Number and Operations
•
Problem Solving
•
Algebra and Functions
•
Representation
•
Geometry & Measurement
•
Reasoning
•
Data, Statistics, & Probability
•
Connections
•
Communication
PSAT/NMSQT Overview
Mathematics
Question 30 (Student–Produced Response)
Each year from January 1 through January 8, the
number of daily customers c at a music store is
estimated by c =1200 + 2x, where x is the day of the
month. How many more customers are estimated to
be at the music store on January 5 than on January
1?
PSAT/NMSQT Overview
Mathematics
Question 30 (Student–Produced Response)
Each year from January 1 through January 8, the
number of daily customers c at a music store is
estimated by c =1200 + 2x, where x is the day of the
month. How many more customers are estimated to
be at the music store on January 5 than on January
1?
The correct answer is 30. For January 1, the day of the month, x, is equal to 1, and
so the estimated number of customers on January 1 is c = 1200 +21 = 1202. For
January 5, the day of the month, x, is equal to 5, and so the estimated number of
customers on January 5 is c =1200 +25 = 1232. Therefore, there are estimated to
be 1232–1202 = 30 more customers at the music store on January 5 than on
January 1.
Level of Difficulty = (M)
Skill(s) = Algebra and Functions
Problem Solving
PSAT/NMSQT Overview
Student-Produced Response
Algebra and Functions
Solve problems using algebraic expressions and symbols to represent
relationships, patterns and functions of different types.
Problem Solving
Solve abstract and practical problems, applying and adapting a variety of
strategies. Monitor progress and evaluate answers in terms of
questions asked.
Writing Skills
Overview
PSAT/NMSQT Overview
Writing Skills
Writing Skills questions test
students’ ability to edit written work
and find the most effective means to
communicate a thought.
PSAT/NMSQT Overview
Writing Skills
Writing Skills
One 30-minute section
39 questions total:
- 20 improving sentences
- 14 identifying sentence errors
- 5 improving paragraphs
30 mins
5 Skills
•
Manage Word Choice and Grammatical Relationships between Words
•
Manage Grammatical Structures Used to Modify or Compare
•
Manage Phrases and Clauses in a Sentence
•
Recognize Correctly Formed Sentences
•
Manage Order and Relationships of Sentences and Paragraphs
PSAT/NMSQT Overview
Identifying Sentence Errors
Understanding the difference between criticism
A
and insult enhances one’s ability to engage in
B
C
D
constructive argument. No error
E
PSAT/NMSQT Overview
Identifying Sentence Errors
Understanding the difference between criticism
A
and insult enhances one’s ability to engage in
B
C
D
constructive argument. No error
E
Level of Difficulty = (E)
Skill = Correctly formed sentences
PSAT/NMSQT Overview
Identifying Sentence Errors
Correctly Formed Sentences
Recognize correct sentence structure.
Student Feedback:
Paper Score Report Plus
&
My College QuickStart
Score Report Plus
•
Scores and percentiles
Personalized feedback on
skills
•
•
Student answers
•
Next Steps
My College QuickStart™
Signing in
Students use the access code on their
PSAT/NMSQT® paper score report to sign in at
www.collegeboard.com/quickstart.
My Online Score Report
Look Beyond Scores
• Performance on each
skill measured
• State and national
percentiles
• Projected SAT®
score ranges
My Online Score Report
Your Answers
• Questions and
answer explanations
• Students can
filter questions
My SAT Study Plan
Prepare for the SAT®
• Personalized skills
to improve
• SAT practice questions
• An official SAT practice test
My SAT Study Plan
Review and Practice
• Review PSAT/NMSQT®
questions missed
• Answer SAT ® practice
questions of the same type
My SAT Study Plan
My Skills Feedback
• Based on PSAT/NMSQT results
• Hundreds of practice questions,
organized by skill
My SAT Study Plan
Warm Up for the
SAT® Essay
• Strategies for success
• Official scoring guide
• Practice essay
questions
• Sample essays for
the each score 1-6
Express SAT® Sign-Up
My Personality
My Personality
My Personality
My Major & Career Matches
Connect to the
Possibilities
•
Major a
student chose
•
Related majors
•
Related careers
My Major & Career Matches
Research Majors
•
Description of
majors
•
Courses
students should
take in high
school to
prepare
•
Courses they
will take in
college
•
Jobs students
can expect
achieving
different degree
levels
My College Matches
Start the College
Search
•
Starter list of
colleges
•
Criteria to
customize
search
•
Ability to save
searches
My College Matches
Get the Facts
•
Admissions
•
Cost and financial aid
•
Majors offered
•
Sports, housing
and more
My College QuickStart Tips
Arrange for a computer lab
session to introduce students
to My College QuickStart
Use free lesson
plans to guide students
through structured activities
Check out the demo for educators
http://professionals.collegeboard.com/k-12/prepare/quickstart
AP Potential
What is AP Potential?
•
A web-based tool
•
Free for schools that administer the
PSAT/NMSQT ®
•
Provided by the maker of the AP ® Exams
•
Designed to ensure that no student is overlooked
•
Based on proven research
AP Potential
•Identifies
“diamond-in-the-rough” students
•Promotes
equity
Helps identify students who initially might have been
overlooked for AP courses
•Is
a useful tool for principals, teachers, and counselors to
Expand AP programs
Increase enrollment in current AP courses
The PSAT/NMSQT and AP Connection
Research shows a strong relationship between PSAT/NMSQT and AP
•
Studies conducted in 1998 and 2006
•
Analyzed performance of more than a million students
•
Showed strong correlations between PSAT/NMSQT® scores and AP®
Exam results
•
Download the research reports at www.collegeboard.com/appotential
Traditional Methods of Identifying Students
for AP Courses
•Teacher
recommendations
•Self-nomination
•History
of courses completed
•Student
grades
•Student
discussion
Access to AP is Important
•AP
access is linked to success in college. Rigorous
high school curriculum is key
•Participation
in AP is linked to college completion. It
is a stronger indicator than socioeconomic status
and GPA.
Signing in to AP Potential™ :
www.collegeboard.com/appotential
Signing in to AP Potential
www.collegeboard.com/appotential
AP Potential
Expectancy Tables
Expectancy Tables
See data from the
correlation study
showing the
percentage of
students at any
given PSAT/NMSQT
score range.
AP Potential
Select PSAT/NMSQT® Administration Year
Step 1 of 4
•
Select the
PSAT/NMSQT ®
administration
year for which
you’d like to
generate a roster.
AP Potential
Select Subjects
Step 2 of 4
•
Select the AP
subject areas your
school offers.
AP Potential
Select Pool
Step 3 of 4
•
Choose a grade level
and define your pool.
AP Potential
Student Roster
Step 4 of 4
Read the
instructions for using
the roster and see
your list of students
at the bottom of the
screen.
•
AP Potential
Subject Detail
•
Student names
•
Correlations and
Student scores
•
Ethnicity
•
Gender
AP Potential
Student Detail
•
All courses
selected noted
•
Status of
meeting
the criteria
•
Probability for
scoring a 3 noted
AP Potential Tips
Export data into an Excel
spreadsheet to make it easy
to read and use.
Create letters to parents.
AP Potential™ provides
sample letters in English
and Spanish.
AP Potential Tips
•After
considering potential AP students using your school’s
standard process, check AP Potential for additional
students that might have been overlooked.
•Use
AP Potential to make a case for professional
development - more teachers will need to be trained to
handle an increased number of AP courses.
AP Potential for 9th and 10th graders in preparation
for 11th or 12th grade AP courses and ensure that they are
taking the correct preparatory courses
•Review
Activity A:
Getting to know the Test
SOAS Introduction
Audience
•
Administrators
•
Department heads
•
Curriculum specialists
•
Teachers
•
Other staff interested in students’
college preparatory skills
Note: the more inclusive the testing within each grade, the
more meaningful and applicable the results.
Instructions: Part I, Solo Activity (20 minutes)
Activity A
•
Review the PSAT/NMSQT skills list in your binder in order to become
familiar with the skills for your academic area.
•
You will have 20 minutes to take sample PSAT/NMSQT
•
Mark your answers in the exam booklet
•
Immediately after answering a question, refer to the List of Academic
Skills list and, in the test book next to the question number, identify the
skill group(s) that you feel is needed to answer the question.
Instructions: Group Activity (20 minutes)
Activity A
•
Gather in groups by subject matter (no more than 7 per
group). Each group needs a Leader and a Recorder.
•
You will have 20 minutes to review at least 5 questions
from the test-taking activity, to compare notes on the skills
needed to the answer questions:
•
•
English teachers, do at least 3 Critical Reading
questions and 2 Writing Skills questions
•
Math teachers, do at least 3 multiple choice questions
and 2 SPR questions
Take 15 -20 minutes to answer the following questions:
Discussion Topics:
Activity A
1.
How would the content covered in these questions be
familiar to your 10th grade students? 11th grade
students?
2.
How are these test questions similar to question types
you would present in class?
3.
Was there agreement in the group on the skills coding?
If no, what kinds of issues caused disagreement?
4.
How do these skills compare to the skills you would
expect your students to have? (10th? 11th?)
5.
How did the “group’s” coding match the test developer’s
coding?
The Summary of
Answers and Skills
(SOAS)
Logging In
Repeat Users
www.collegeboard.com/results
Logging In
New Users
If it’s your first time,
you’ll need your
school’s SOAS
access code
Supporting Materials
SOAS Resources
Your Reports
Report Orientation
Table of Contents
Page(s)
•
Title Page
•
Performance Overview
1
•
Skills Analysis
2, 6-7, 11
•
Question Analysis
3-4, 8-9, 12-13
•
Comparable Group Analysis
5, 10, 14
Performance Overview
Page 1
•
Number of students
in report
•
Mean scores and
•
score distribution
•
College Readiness
Benchmarks
Skills Analysis
Pages: 2, 6-7, 11
•
Skills
•
Triangles show a
comparison to the
state and nation
•
Each test question is
linked to a skill
Question Analysis
Pages: 3-4, 8-9, 12-13
•
Student
responses and
answer patterns
•
Comparisons to
the state and
nation
Question Analysis
Tips:
Look for questions your
Students answered
incorrectly that the
students in the state
and nation answered
correctly
Look for commonly
Wrong answers
Comparable Group Analysis
Pages: 5, 10, 14
SOAS Comparable Group
What is a comparable group?
•
A useful statistical model
•
A statistically created group (virtual group)
•
Mirrors your group’s performance profile
•
This creates an expected performance indicator
for your group on each question.
•
Provides more “actionable” feedback than state
or national averages on questions/skills.
Comparable Group Analysis
Pages: 5, 10, 14
Darker blue =
significantly below the
comparable group.
Lighter blue =
significantly above the
comparable group
Resources
Resources
Test Questions and Answer Explanations
How does it all add up?
1. Identify
2.
Find
3.
View
a skill
the
measur
questio
for
eddetails
by
n
review
this
questio
n
Other Relevant Resources:
Before, During, and After SOAS review
Understanding PSAT/NMSQT Scores
•
PSAT score related FAQ’s
•
Information on percentiles
and mean scores
•
Score conversions
PSAT/NMSQT Summary Reports
PSAT/NMSQT Summary Reports
Summary Reports are provided to all schools that tested at least 50 juniors
or sophomores.
State level summary reports are also available (College Bound
Juniors/Sophomores reports) www.collegeboard.com/reporting
Reports contain:
•
number of test-takers
•
ethnic background
•
participation in Student Search Service
•
self-reported GPA
•
performance by score range
•
college major
Activities
Activity B
Review Skills Analysis
Instructions
•
Work within your own school group/academic area, using your school’s
SOAS report.
•
Identify the skill that your students had the lowest performance on.
•
Using the Test Question and Answer Explanations, look at each question
aligned to that skill.
•
Things to look for:
Skills furthest below the state/national average
Skills that are furthest to the left (i.e. those that are the weakest
for the group).
Skills that came to teachers’ attention after review of Question
Analysis.
Activity B
Review Skills Analysis
Questions for Consideration
Within your group, answer the following questions:
1.
Is there a pattern in the skills you identified as
problematic?
2.
Are there actions or strategies that might help address
the issues?
Activity C
Review Question Analysis
Instructions
•
Work within your own school group/academic area, using your
school’s SOAS report.
•
Refer to the Test Questions and Answer Explanations to make note
of problem questions and how they might be addressed.
•
Things to look for:
Easier questions (difficulty level 1-3) where less than a 2/3 of the
students answered correctly.
A larger proportion of your students answered incorrectly or
omitted an answer in comparison to the state/nation
Commonly wrong answers
Review any questions that generated teacher interest when you
took the test in Activity A.
Activity C
Review Question Analysis
Questions for Consideration
Within your group, answer the following questions:
1.
Are there any patterns in the questions you identified as
problematic?
2.
Are there actions or strategies that might help address the issues?
3.
Summarize your findings for sharing.
Activity D
Comparable Group Analysis
Instructions
•
Work within your own school group/academic area, using your school’s
SOAS report.
•
Using the Test Question and Answer Explanations, look at each question.
•
Things to look for:
Questions that fall into the dark blue area, where your students
performed below expectation.
Questions that fall into the light blue area, where your students
performed higher than expectation.
Activity D
Comparable Group Analysis
Questions for Consideration
Within your group, answer the following questions:
1.
Is there a pattern in the skills aligned with the test questions your
students performed below expectation on?
2.
Is there a pattern in the skills aligned with the test questions your
students performed expectation on?
3.
Are there actions or strategies that might help address the issues?
For Group Discussion
To Help Students and Parents
1. How does the school educate parents about the wealth of
information on Score Report Plus?
2. What plan does the school have in place to make sure
students receive and understand their scores?
For Group Discussion
Using Your SOAS Data
•Identify
the skills associated with the questions students
answered incorrectly.
•Make
inferences about what led to errors.
•Identify
where in the curriculum the skills are (or are not)
taught.
•Work
with departments and classrooms to review common
concerns.
•What
recommendations can you make to your school for
increased student success?
Closing Thoughts
Using Assessments for School Improvement
•
Identify skills to help guide instruction and curriculum planning
•
Connect more students with challenging high school courses and
college attendance
•
Increase Advanced Placement Program courses in your school
system; more professional development for teachers
•
Close the achievement gap in your community
The Excellence and Equity Challenge
“Without preparation,
opportunity is an
empty promise.”
Alan Page, former NFL star, current Minnesota State
Supreme Court Justice
Austin American Statesman, 10.25.02
Looking Ahead:
2011 PSAT/NMSQT
Wednesday, October 12th
Saturday, October 15th