Transcript Slide 1
Background Questionnaires: Why Ask About Social Identity? Ruth A. Childs & Orlena Broomes Ontario Institute for Studies in Education University of Toronto Paper presented at the annual meeting of the Canadian Society for the Study of Education, Montreal, May 2010. Why have Background Questionnaires? Large-scale assessments provide important information about what students know and what they know how to do But, not which students are performing well and which are struggling – or, more importantly, why Background Questionnaires often include questions about… Students Social identities Experiences Attitudes & Beliefs Goals Teachers Social Identities & Experiences Instruction & Assessment Practices Beliefs Resources Schools Community What do we mean by “Social Identity”? Carla O’Connor, a professor at the University of Michigan describes social identity as “how people are differentially positioned in the social world” She emphasizes that “any one individual reflects multiple social identities” “these identities are simultaneously structured and cultured and operate differentially across place and time” Why ask about Social Identity? The Ontario Human Rights Code gives four reasons for collecting social identity data: To monitor and evaluate potential discrimination To identify and remove systemic barriers To ameliorate or prevent disadvantage To promote equality One of the Ontario Ministry of Education priorities is “Reduced gaps in student achievement” A gap is a difference between groups To reduce gaps in achievement, we have to first identify the gaps To identify the gaps, we need to have information about group membership An Example Ontario Secondary School Literacy Test (OSSLT) High School graduation requirement In 2007/08, 156,151 Grade 10 students who were eligible to sit the English-language version of the test Results 122,324 students passed 23,279 failed 4,357 were absent 6,191 received permission to defer taking the test OSSLT 2007-2008 140,000 120,000 100,000 80,000 All Students 60,000 122,309 40,000 20,000 23,275 0 Successful Not Successful 4,355 6,191 Absent Deferred OSSLT 2007-2008 140,000 120,000 100,000 62,806 80,000 Female Male 60,000 40,000 59,503 20,000 8,809 14,466 2,131 2,224 2,480 3,711 Not Successful Absent Deferred 0 Successful OSSLT 2007-2008 140,000 120,000 2,326 100,000 80,000 ELL 60,000 Not ELL 119,983 40,000 1,591 20,000 21,684 0 Successful Not Successful 184 4,171 2,020 4,171 Absent Deferred OSSLT 2007-2008 140,000 120,000 759 21,747 100,000 80,000 Locally Developed English Course Applied English 60,000 Academic English 97,548 40,000 20,000 3,094 13,592 4,151 0 Successful Not Successful 409 1,942 1,038 Absent 1,518 1,553 1,038 Deferred Writing the Questions Choose the dimensions that matter in the particular context (this may vary regionally and by the academic subject being tested) Explain why the information is important and how it will be used – why would someone respond? Specify if the information will be anonymous or confidential Describe social identity dimensions clearly (e.g., sex vs. gender; race vs. ethnicity) Choose categories thoughtfully What distinctions are important? What level of specificity is needed? Gerber (1999) found that respondents “paid close attention not only to the category that applied to their own group, but to the categories available for others as well,” wanting to “make sure that the categories were evenhanded, and did not give preferential treatment to specific groups” (p. 230) What is the precise meaning of terms? What terms will respondents recognize? Include an “I prefer not to respond” option Testing the Questions Approaches Expert review Cognitive interviewing (think-alouds) Questions How do respondents decide whether to answer? How do respondents decide how to answer? Do students at different ages understand the questions differently? Roger Levine in testing NAEP questions found differences by age.