Transcript Document
Chapter 1 Psychological Testing and Assessment Twelve Assumptions in Psychological Testing and Assessment • Assumption 1: Psychological traits and states exist – Observed (manifest) vs. latent – Can you study something invisible? – Logical positivism, Behaviorism • Assumption 2: Psychological traits and states can be quantified and measured – Can you really assign a numeric value to everything? – Rasch: objective measurement • Assumption 3: Various approaches to measuring aspects of the same thing can be useful (Methodology Triangulation) Twelve Assumptions in Psychological Testing and Assessment • Assumption 4: Assessment can provide answers to some of life’s most momentous questions – Tracking (in the UK system) – screening and placement • Assumption 5: Assessment can pinpoint phenomena that require further attention or study – National (?) assessment: No Child Left behind (The objective of assessment is…) – International assessment: PISA, TIMSS What is PISA? Not these • PISA: Program for International Student Assessment is a series of assessments on reading literacy, mathematics literacy, and science literacy, sponsored by Organization for Economic Cooperation and Development (OECD), and administered internationally to 15-year-olds • TIMSS: The Trends in International Mathematics and Science Study (TIMSS) is administered every four years internationally to both fourth and eighth grade students. Source: OECD/PISA and Wikipedia Source: TIMSS and Wikipedia Obama's "Sputnik Moment" • In 1957 the USSR launched the first artificial Earth satellite, Sputnik. In order to fill the technological gap, the US government awarded financial support to promote math and science education. • In response to the performance gap between the US students and Asian/European students, President Obama introduced the phrase "Sputnik moment" to argue for more investment in science, technology, engineering, and technology (STEM) education. Twelve Assumptions in Psychological Testing and Assessment (Cont’d) • Assumption 6: Various sources of data enrich and are part of the assessment process (Data triangulation) • Assumption 7: Various sources of error are part of the assessment process Error is everywhere! • Sources of error – Measurement error – Sampling error • Types of error – Bias: Systematic e.g. self-report height and weight – Variance: Random Twelve Assumptions in Psychological Testing and Assessment (Cont’d) • Assumption 8: Tests and other measurement techniques have strengths and weaknesses – Classical test theory – Item response theory – Rasch model Twelve Assumptions • Assumption 9: Test-related behavior predicts non– test-related behavior • Assumption 10: Present-day behavior sampling predicts future behavior – Academically adrift: Limited learning on college campuses -- After 4 years, 36% college graduates showed no gains in higher order thinking skills. US corporations complain that college graduates don’t have the critical thinking skill required by the jobs. – Corporations launch their own assessment programs e.g. MCSE, CCNA. Examples of industry certification/assessment program Twelve Assumptions in Psychological Testing and Assessment (Cont’d) • Assumption 11: Testing and assessment can be conducted in a fair and unbiased manner • Assumption 12: Testing and assessment benefit society Who, What and Why? • Who Are the Parties? – – – – The test developer The test user The test taker Society at large The test developer • Subject matter expert/content expert: item authoring • Psychometrician: item analysis, factor analysis • Database programmer/IT personnel: item banking • Program manager: put all together Evaluating the Quality of Tests • What is a “Good” Test? – Reliability – Validity – Other settings • Reference Sources for Test Information – – – – – Test catalogues Test manuals Journal articles Online and CD-Rom databases Other sources • http://www.creativewisdom.com/pub/efficacy.pdf Exercise (2 points) • Form a group consisting of 3-4 people. One or two must have a Web-enabled laptop. • Access the APU library • Look for Mental Measurements Yearbook (the electronic version) • Search for instruments that measure psychological constructs (e.g. anxiety, depression, happiness…etc) • Read the comments at the end • Discuss which one is the best.