Smarter Balanced Assessments 2014-15 Webinar #1 October 20, 2014 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Division of Assessment and Student Information.
Download ReportTranscript Smarter Balanced Assessments 2014-15 Webinar #1 October 20, 2014 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Division of Assessment and Student Information.
Smarter Balanced Assessments 2014-15 Webinar #1 October 20, 2014 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Division of Assessment and Student Information OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Today’s Topics Big Picture Overview Digital Library Interim Assessments Summative Assessments Other State Assessments (Exit Exams) October 20, 2014 | Slide 2 Big Picture Overview 3 October 20, 2014 Big Picture Overview State Assessments in 3 content areas OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION ELA and Math assessed with Smarter Balanced Science still assessed with MSP and Biology EOC Smarter Balanced Components: Digital Library, Interim Assessments, Summative Assessments Transition from grant to vendor Accountability HS changes from Grade 10 to Grade 11 October 20, 2014 | Slide 4 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Summative Assessments 2014–15 and beyond English/LA Mathematics Grade 3 Smarter Smarter Grade 4 Smarter Smarter Grade 5 Smarter Smarter Grade 6 Smarter Smarter Grade 7 Smarter Smarter Grade 8 Smarter Smarter MSP Grade10 Smarter ELA exit exam Year 1or Year 2 EOC exit exam EOC Biology exit exam (until NGSS) Smarter Smarter (or below for EOCs) (until Class of 2019) Grade 11 Grade 11 or 12 (class of 2015 & 2016) Science (no change) MSP HSPE Reading and/or Writing Retake Smarter=Smarter Balanced Assessment; MSP=Measurements of Student Progress; EOC=End of Course exams; HSPE=High School proficiency Exam October 20, 2014 | Slide 5 Smarter Balanced Assessments Common Core State Standards specify K-12 expectations for college and career readiness Summative: College and career readiness assessments for accountability Teachers and schools have information and tools they need to improve teaching rand learning Formative Educator Resources: Digital Library with instructional and professional learning resources that include embedded formative assessment processes All students leave high school college and career ready Interim: Flexible and open assessments, used for actionable feedback Page 6 A System of Assessments Smarter Balanced Sustainability OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION The Smarter Balanced grant is coming to an end 4 years of development – remarkable accomplishments Now housed at UCLA Member states pay UCLA to use assessments Joe Willhoft is retiring as of Dec 31; Tony Alpert has been named executive director Each state implements on their own, with common protocols Most states, including Washington, need a vendor to assist with implementation October 20, 2014 | Slide 8 New Smarter Balanced Vendor to Support Our State’s Implementation OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Multi-Agency Assessment Cooperative (MAAC) Contract awarded to our new service provider Washington, Hawaii, Idaho, South Dakota, US Virgin Islands, West Virginia partnered on an RFP Smarter Balanced assessments (common to all) and state specific (Science MSP and EOC, new state exit exams) American Institutes for Research (AIR) Kick off meeting held last week (10-16-2014) Current vendors assisting through fall/winter October 20, 2014 | Slide 9 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Training Plan in Context of MAAC Smarter Balanced AIR – MAAC Customization OSPI Customization October 20, 2014 | Slide 10 Tests for Accountability OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION (95% participation expected) October 20, 2014 | Slide 11 Digital Library Educator resources for formative assessment practices to improve instruction 12 October 20, 2014 Digital Library Access OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION District Assessment Coordinators (DACs), or their designees, should enroll instructional and support staff who work directly with students in their districts. Functionality The Digital Library offers professional learning resources and instructional materials—like lesson plans, units, and learning modules—for educators to use daily with students and staff. Resources span K-12 and are aligned to Washington State Learning Standards for English language arts and mathematics. Resources are also aligned to formative assessment practices. October 20, 2014 | Slide 13 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Digital Library How to Add Users (Demonstration of what ought to happen) Known issues and solutions/recommendations October 10, 2014 Slide 14 Digital Library Known Bugs: OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Email addresses cannot have apostrophes Email addresses are case-sensitive Coordinators sometimes see users from other states But cannot edit or change them Export only includes users listed on the current screen It’s possible create two accounts that only differ in case of email addresses. We recommend creating all accounts using lower case; if it’s too late for that, the correct case can be found in the original upload file, by searching in ART* or in the welcome email* This is causing problems with OSPI’s effort’s to support password resets Estimate is 2-3 weeks for these issues October 10, 2014 Slide 15 Digital Library The Big Problems (and what we’re doing to fix them) OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION If user has a state and district role, District Coordinators cannot see the records in ART Short Term Solution: contact OSPI to remove STATE records Long Term Solution: working with Smarter to get the complete list of WA’s users and removing all STATE level records (may disrupt some users) User never logged in during preview and doesn’t have temp email Short Term Solution: School or District Coordinator can reset oneby-one (unreasonable in most districts) Long Term Solution #1: OSPI resets every password in the state Long Term Solution #2: allow all users to use ‘forgot password’ even if they don’t have a security question for a limited time period October 10, 2014 Slide 16 Digital Library Big Problems (continued) OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Emails not coming through (NOTE: emails only created during initial account creation) Whitelist [email protected] BEFORE uploading lists to ART Check school/district level spam/junk/quarantine folders to see if the emails hit the server or not to isolate if the issue is local or remote Issue seems to be consistently district wide (suggests local issue, but working to confirm) OSPI working on providing dedicated staff for firewall issues, but can provide additional sender info to help configure firewalls (“Tier 2 Support”). Password issues should be resolved by early next week; email triage on a case-by-case basis October 10, 2014 Slide 17 Digital Library Basics • • • • The Digital Library is accessed through a single sign-on with user permission levels so educators have access to all of the instructional and professional learning resources for each grade (Grades K-12). There are resources that educators can share or use with students and families, but students and families will not be able to access materials directly. All resources will have the formative assessment process embedded within them. But the Digital Library WILL NOT be a “bank” of formative assessment items alone. All submitted materials will be vetted through a Quality Criteria Review Process by reviewers across the nation. • Each resource reviewed and rated by at least 3 trained educators • If resources do not meet the quality criteria, they are not included in the library Page 18 Resources in the Digital Library Assessment Literacy Modules Exemplar Instructional Modules Educational Resources • Commissioned professional development modules • Resources for students and families • Frame formative assessment within a balanced assessment system • Articulate the formative assessment process • Highlight formative assessment practices and tools • Commissioned professional development modules • Instructional materials for educators • Instructional materials for students • Demonstrate/support effective implementation of the formative process • Focus on key content and practice from the Common Core State Standards for Mathematics and English Language Arts • High-quality vetted instructional resources and tools for educators • High-quality vetted resources and tools for students and families • Reflect and support the formative process • Reflect and support the Common Core State Standards for Mathematics and English Language Arts • Create Professional Learning Communities * Resources include the following file types: Video, HTML5, Audio, PPT, Excel, Word, and PDF. Page 19 Digital Library Functionality • Enables State Networks of Educators and State Leadership Teams to submit, review, and publish resources • Allows educators to view, download, and rate resources • Uses state-of-the-art tagging and search to quickly find resources by CCSS and other topics • Enables educators from across the Consortium to collaborate and share their knowledge • Facilitates access to resources that are stored in participating libraries Slide 20 Filtering for the Resource and Modules Types Page 21 Difference Between Educational Resources and Modules Educational Resources Modules A Green Check in the top corner marks the resource as “posted with distinction.” Page 22 Digital Library: Collaboration Page 23 Digital Library: Collaboration Page 25 Digital Library: Collaboration Page 26 Digital Library: Collaboration Page 27 Grounded in this Definition of Formative Assessment Process • Formative Assessment is a deliberate process used by teachers and students during instruction that provides actionable feedback that is used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals. ~ Compiled by the Digital Library National Advisory Panel Page 29 All Resources are Grounded in the Four Attributes of the Formative Assessment Process and Reviewed using Quality Criteria Clarify Intended Learning Elicit Evidence Act on Evidence Interpret Evidence Page 30 Interim Assessments 34 October 20, 2014 Interim Assessment Design Principles OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Administered through the same system as Summative Items drawn from same pool as Summative; full array of item types Can be administered at various points in the year Optional for districts Online administration Adhere to Usability, Accessibility, and Accommodations Guidelines Adaptive as appropriate (2015-16) Not intended for accountability decisions October 20, 2014 | Slide 35 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Interim Assessment Components Interim Assessment Interim Comprehensive Assessment (ICA) Interim Assessment Blocks (IAB) October 20, 2014 | Slide 36 Interim Assessment User Interface, Scoring and Reporting OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION User interface Details for accessing items are not yet determined. Interim test engine is still in development. Scoring Interim assessments will have various item types, most of which will be machine scored Hand scoring will be a local (school/classroom) responsibility Rubrics and training will be provided online as part of the system October 20, 2014 | Slide 37 Interim Assessment User Interface, Scoring and Reporting, continued OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Reporting Interim Comprehensive Assessment (ICA) Overall composite scale score Achievement levels Claim level information Interim Assessment Blocks (IABs) Report same information as Summative Assessment Report information consistent with claim level information Item level reporting is a future feature (not on current work plan) October 20, 2014 | Slide 38 Interim Assessments Timeline OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Interim Assessments will be available Jan 6th Jan 6th – Comprehensive Interim Jan 27th – Interim Assessment Blocks Initial release will include a fixed forms; adaptive forms will be available as the item pool allows October 20, 2014 | Slide 42 Current Timelines OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION All components of Smarter Balanced will be operational in 2014-15 Digital Library now available! Interim assessments available Jan 6th (Comprehensive) and Jan 27th (Blocks) Summative assessments administered in spring 2015 11th grade – last 7 weeks of year 3-8th grade – last 12 weeks of year (3rd gr ELA more limited) Grade 3 ELA results must be returned in time for principals to meet with parents before the end of the year so window is short and early Student score reports available June 2015 (or three weeks after school’s testing is completed if online) October 20, 2014 | Slide 43 Summative Assessments 44 October 20, 2014 Summative Assessments OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Administration windows Paper/pencil option AIR Portal Cut scores October 20, 2014 | Slide 45 Summative Assessments Administration Windows OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Smarter Balanced 2015 Testing Windows Grade 3 ELA March 9 10 to April 23 – online March 9 10 to April 15 – paper/pencil Grade 3 Math Last 12 weeks of school, but no earlier than March 9 10 and no later than June 15 – online March 9 10 to May 20 – paper/pencil Grades 4–8 ELA and Math Last 12 weeks of school, but no earlier than March 9 10 and no later than June 15 – online March 9 10 to May 20 – paper/pencil Grade 11 ELA and Math Last 7 weeks of school, but no earlier than April 6 and no later than June 15 – online Last 3 weeks of school, but no later than May 30 – paper/pencil October 20, 2014 | Slide 46 Summative Assessments Administration Windows Measurements of Student Progress (MSP) 2015 Testing Windows OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Grades 5 and 8 Science April 20 to June 5 – online April 20 to May 19 – paper/pencil High School Exit Exams (HSPE) (Off Grade) (EOC) (ELA) 2015 Testing Windows Grades 11 and 12 HSPE Retake o Reading o Writing Grade 10 ELA exit exam March 17 March 18–19 Math EOC exit exams Last 12 weeks of school, but no earlier than March 10 and no later than June 15 – online May 12–13 OR May 19–20 paper/pencil May 11 to June 12 Biology EOC exit exam May 11 to June 12 Off Grade Level testing (formerly known as DAPE) Last 12 weeks of school, but no earlier than March 9 10 and no later than June 15 – online October 20, 2014 | Slide 47 Summative Assessments – Paper/Pencil OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Smarter Balanced 3–8 and 11 paper/pencil tests will cost $6.00 per test per content area ($12.00 per student if using paper for both ELA and Math). Additional costs do not apply to MSP science Grades 5 and 8, HSPEs, Off Grade Level tests, EOCs, or Grade 10 ELA. Selection of paper/pencil is at a school by grade by content area (e.g., a school use online except for 5th grade math). Print-on-demand tests required for visually impaired students (documented on the Individualized Education Program), are available through the Smarter Balanced testing engine at no cost to districts. October 20, 2014 | Slide 48 Summative Assessments – Paper/Pencil OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Paper/pencil tests are fixed forms – not computer adaptive. This means students will likely take longer to complete paper/pencil tests than online tests. The online test engine for Smarter Balanced incorporates a multitude of embedded usability, accessibility, and accommodations – districts will need to provide those accommodations or have individual students who need accommodations test online. Scoring paper/pencil tests will take longer than scoring online tests. Shipping paper/pencil tests back to the scoring vendor will necessitate a shorter testing window to minimize the delay in scoring and reporting. October 20, 2014 | Slide 49 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Washington’s Smarter Balanced Portal (Mockup) October 20, 2014 | Slide 50 Alphabet Soup Assessment and Registration Tool (ART) OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION • • Digital Library User Sign-on Test Information Distribution Engine (TIDE) • • • • • Registering students Enrolling new students Managing and editing users/students Processing and viewing test invalidations Test Delivery System (TDS) • • • Test Administrator Interface Student Interface October 20, 2014 | Slide 51 Training Plan OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Webinars for District Assessment Coordinators (OSPI/AIR) Smarter Balanced training modules (customized for MAAC/OSPI) What is a CAT Technology Requirements for Online Testing Performance Tasks Overview Student Interface for Online Testing Accessibility and Accommodations Universal Tools Tech Readiness Test Administrator (TA) Interface Test Administration Training Test Administrator course (for test proctors) How to: Start and administer tests Approve students for testing Monitor a test session October 20, 2014 | Slide 52 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Smarter Balanced + Higher Ed Washington has had unprecedented movement to use mandatory high school tests in college placement Recently announced - Agreement to provide access to credit bearing courses to 11th graders who score a 3 or 4 on the Smarter Balanced ELA and math tests Washington’s Community Colleges Washington’s Baccalaureates October 20, 2014 | Slide 56 Setting a New Baseline OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION New content standards: Clear expectations for educators, students and parents about the knowledge and skills students need to be on track for college and/or career New assessments: Challenging and engaging for students, provide teachers and school leaders with better information to help students New performance baseline: Proficiency scores developed by educators that reflect high expectations for students (achievement level setting) October 20, 2014 | Slide 57 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION What is “Achievement Level Setting”? Formerly known as “standard-setting” Process of establishing one or more scores for proficiency on a test Allows state and local education agencies to create categories of performance for students October 20, 2014 | Slide 58 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Achievement Level Setting Timeline April 30 • State chiefs approve plan/process for Achievement Level Setting October 6-17 • Online Panel opened to thousands of educators and other constituents October 13-19 • In-Person panels convened to recommend grade-level performance standards October 20 • Vertical Articulation by subgroup of In-Person panels looks at across-grade coherence October 30 • Technical Advisory Committee reviews proceedings November 6 • State chiefs vote to endorse achievement level recommendations October 20, 2014 | Slide 59 What about Graduation? OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Legislature has given the State Board the option to set a different cut score for graduation purposes The ELA exit exam cut score will be determined from 10th graders’ performance on the Smarter Balanced HS ELA test (for classes of 2015-2018, at least). Math EOC exit exam cut scores will be determined from this year’s performance on new EOC tests (for classes of 20152018). Exit exam cut score on Smarter Balanced math may also be set next summer – SBE discussion next month. October 20, 2014 | Slide 60 Other State Assessments (Exit Exams) 61 October 20, 2014 Other State Assessments – HS Exit Exams OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION ELA Exit Exam Retakes and Alternatives for High School Graduation Supt. Dorn’s Legislative Proposals October 20, 2014 | Slide 62 ELA Exit Exam OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Legislation calls for OSPI to develop a comprehensive ELA exit exam for 10th graders to use to fulfill assessment graduation requirement, using Smarter Balanced items Limited items available to states would necessitate paper/pencil format Paper/pencil precludes CAT Test map would be same as 11th grade test Items would be the same as 11th grade test Why not just use “11th grade” test? October 20, 2014 | Slide 63 ELA Exit Exam – News! OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Wa’s ELA Exit Exam will use the same test engine and item bank as HS Smarter Balanced ELA test (used in 11th grade for school and district accountability) In other words, the Smarter Balanced ELA test will be taken by 11th graders for accountability but should be available to10th graders for graduation purposes (to allow time for remediation and retakes) An Exit Exam cut score will be set by the State Board for graduation purposes based on 10th graders’ performance The College & Career Ready cut score set by the consortium will be used for accountability (11th graders) October 20, 2014 | Slide 64 ELA Exit Exam – more detail OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Wa’s ELA Exit Exam will now be available online – administration within last 12 weeks of school (but no later than June 15). Previously expected to be paper/pencil only. Fixed form paper/pencil can be used (at no cost) – administration May 12–14 OR May 19–21. Only online test will be Computer Adaptive. If a student meets or exceeds the College and Career Ready cut score in 10th grade, we will bank their score so they won’t need to be tested in 11th grade. October 20, 2014 | Slide 65 High School Assessment Graduation Requirements Vary by ‘Class of’ Cohort OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Class of… English Language Arts Writing HSPE Mathematics Science 2014 Reading HSPE 2015 Reading HSPE 2016 HSPEs or Smarter Balanced (exit exam cut score) 2017 & 2018 Smarter Balanced (exit exam cut score) Either Algebra 1/Integrated Math 1 or Geometry/Integrated Math 2 EOC/EOC Exit Exam or Smarter Balanced (exit exam cut score) 2019 and beyond Smarter Balanced (college & career ready cut score) Smarter Balanced (college & career ready cut score) Writing HSPE Either Algebra 1/Integrated Math 1 or Geometry/Integrated Math 2 EOC Either Algebra 1/Integrated Math 1 or Geometry/Integrated Math 2 EOC/EOC Exit Exam Biology EOC (until Next Gen Science Standards) October 20, 2014 | Slide 66 Assessment Menu – English Language Arts OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION General Assessments Smarter Balanced ELA Alternatives Exit Exam Cut Score College/Career Ready Cut Score HSPE in Reading and Writing (retakes for 11th and 12th graders) Collection of Evidence College Entrance Scores SAT, ACT, IB, AP Out of State Tests Grades Comparison Recent Transfer Waiver Special Education Alternatives October 20, 2014 | Slide 67 Assessment Menu - Mathematics OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION General Assessments Smarter Balanced Math Yr1 & Yr2 EOC Exit Exams Alternatives Collection of Evidence College Entrance Scores SAT, ACT, IB, AP Out of State Tests Grades Comparison Recent Transfer Waiver Special Education Alternatives October 20, 2014 | Slide 68 Assessment Menu - Science OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION General Assessments Biology EOC Exam (until new NGSS comprehensive exam is developed) Alternatives Collection of Evidence College Entrance Scores IB, AP Out of State Tests Grades Comparison Recent Transfer Waiver Special Education Alternatives October 20, 2014 | Slide 69 Superintendent Dorn’s legislative proposals related to assessments OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Plan A: Eliminate assessment graduation requirements Use Common Core and Next Generation Science Standards to prepare students for college and career. Use Smarter Balanced assessments in grades 3-8 and 11 to signal if students, teachers and systems are on track for graduating students college and career ready. Use the 11th grade tests as a “Green,Yellow, Red” indicator of services/instruction needed in senior year. Switch resources used for testing retakes and alternatives for passing exit exams on direct instruction and preparation of students. October 20, 2014 | Slide 70 Superintendent Dorn’s proposals OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Plan B: Update and streamline options to fulfill assessment graduation requirements Collection of Evidence in Math – reduce to just Year 1 math (eliminate Geometry COE) Out of state test – must be test used for accountability or exit exam in the other state Grades Comparison in Alg 1, Int 1, Geometry or Int 2 – eliminate minimum GPA for access to this alternative Additional HS Credit (1.0) in content area – add this new alternative College Credit in content area – add this new alternative Transfer Waiver – change date of transfer to January 1 of senior year (from March 1) October 20, 2014 | Slide 71 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION Thank you! October 20, 2014 | Slide 76