Transcript TESTING AND EVALUATION IN INDONESIAN EDUCATION

TESTING AND EVALUATION IN
INDONESIAN EDUCATION
By
Didi Sukyadi
Universitas Pendidikan Indonesia
20 July 2011
Position of English in Indonesian
Education System
1) English is the first foreign language in Indonesia
2) English is not used as a medium of instruction at
any level of education.
3) English is a compulsory subject at the lower and
upper secondary schools
4) English is an optional subject at the university
level
5) English may be taught as early as Primary Four at
the school’s discretion ( local content or muatan
lokal)
Current Trends and Policies
in Indonesian TEFL
1) The establishment of international standard schools at every local
government (districts, municipalities, provinces) starting from 2006.
Because of severe criticisms, we postpone having a new one.
2) In International standard schools, English is encouraged to use as a
medium of instruction.
3) The implementation of School-based Curriculum which is developed
by each school (KTSP)
• Central government only develops its basic competence and
standard competence.
• Indicators and syllabus are developed by each school.
4) The adoption of genre-based approach to teaching English starting
from 2004 to accompany our competence-based curriculum.
5) The establishment Board of National Education Standards in 2005,
which stipulates 8 standards: graduates, curriculum, process,
teachers and education administrators, facilities, management,
finance, and testing
Types of Testing
1)
2)
•
•
•
a.
b.
c.
d.
High stake testing: set and administered in a large scale by central
goverment called National Examination (Ujian Nasional, UN for
short).
Subjects tested:
Elementary schools: Bahasa Indonesia, mathematics, and natural
science
Junior Secondary Schools: Bahasa Indonesia, English, mathematics,
and natural science
Senior Secondary Schools:
Natural science stream: Indonesian, math, English, physics,
chemistry, biology
Social science stream: Indonesian, math, English, economy,
sociology, geography
Language stream: Indonesian, English, Indonesian literature,
history/anthropology, selected foreign languages.
Vocational schools: Indonesian, math, English, vocational theories
Functions of High Stake testing
1) Nationally assessing students’ learning achievement at
the end of each education institution level (junior and
senior secondary)
2) Provide information on learning achievement of every
individual student throughout the country,
3) Inform about the success and failures of teaching and
learning endeavours at the schools, districs,
municipality, and provincial levels
4) Results so far have been used for passing criteria and
school completion certification.
5) The results of examination at every school level will be
used for the admission to the higher level institution.
Criteria of passing
1) Passing an education institution level (Junior or Senior Sec)
- (0,6 x UN score) + (0,4 x school scores).
- Passing grade is decided by the schools and subjects to
annual change
2) School scores:
- School exam scores (subjects may overlap or the same as
ENE subjects)
- The average of scores of semester 1-5 for each ENE
subjects.
3) Testing for Movement to higher level (e.g. Year 7 to 8):
- Formative tests (60%)
- Mid semester test (20%)
- Ujian Kenaikan Kelas or Final Semester Test (20%)
Types of School Tests
1) Written Tests:
- Developed by Year 9 teachers in cooperation with MGMP (Subject
Specific Interest Groups).
- Mostly in the form of multiple choice items,
- Testing Reading and Writing Skills (Listening and speaking are not
included)
- Used for Schools’ exams and its Try Out or UN Try Out
2) Portfolio Assessment: writing products or projects
3) Performance tests: Schools may choose whether testing speaking or
Listening (whichever easier):
Speaking: interview, role play, play or drama
Listening: dictations, songs, or audio taped-materials
4) Teachers develop rubrics for portfolio and performance assessment.
5) The results of porfolio assessment and performance assessment are
integrated to school’s scores.
Specs of National Exams
(Senior Secondary)
• Multiple choices consisting of 50 items, testing
writing and reading.
• Reading: certain information, detail information,
explicit and implicit information, reference, word
meaning, purpose of communication, main idea,
synonyms or antonyms
• Text type: announcement,
advertisement/brochure, news item, recount,
explanation, exposition, discussion and review
• Writing: Cloze test, arranging words into
sentences, and sentences into paragraphs
Junior Secondary Schools
• Reading:
- Areas tested
• General information, Main idea, Explicit detail information,
Implied information, Reference, Meaning of words, phrase
and sentences
- Text types: Short Functional Texts: Caution, Greeting
card, Short message, Invitation, Announcement, Label,
Advertisement, and Letter
- Writing:
• appropriate words to complete short report texts.
• appropriate word arrangement to to make sentences
• appropriate sentence arrangement to make paragraphs
Steps In Test Item Writing
(National Exam)
• Developed by Education Evaluation Center (Pusat
Pengujian, EEC)
• ECC staff and experienced teachers set test item
specifications.
• ECC staf carefully read and revise the specs
• ECC send the specs to regional offices of education
• Well-experienced teachers use the specs for test item
writing (about a month)
• The items are sent to EEC to be reviewed by teachers or
university lecturers
• ECC arranges and reviews items and prepare them for trial
• The results of the trials are analysed using computer
program such as ITEMAN and BIGSTEPS
• ECC Identifies the results of the trials and preparing the
final version
Tenets of Test Item Writing
•
•
•
•
•
•
•
The item should match the indicator
The option should be homogeneous and logical
There should be on correct answer only
The stem must be concise and firm
The stem has no clues for correct answer
The stem does not contain double negative
Figures, graphs, tables, and diagrams must be
functional and clear.
Tenets of Test Item Writing (Cont)
• Options must be relatively equal in length
• Options do not contain “all answers above are wrong)
• Option in the form numbers should be place
chronologically
• Test item should not be dependent on preceding or
succeeding test items
• An option should not reapet the same words or
phrases taken precisely from the stem
• The stem and option should contain necessary stament
only
Testing at Tertiery Level (State Univ)
• National Selection for University Admission (SNMPTN):
large-scale, centralized and administered by a
university consortium
• Subjects tested:
a) Basic Subjects (Indonesian, Basic Math, English)
b) Natural science stream: math, chemstry, physics,
biology
c) Social science stream: sociology, history, geography,
economy
• Items used deal with higher order thinking in Bloom
Taxonomy covering Reading skills only
• For 2011: the English test consits of 15 items: text 1
and 2 five questions each, twin texts: five questions
Washback Effect of National Exams
• A study by Mardiani (2010) as reported in her
theses and will be published in “K@ta” Journal
2011 which I co author, tries to explore the
washback effects of English National Examination
(ENE) in Indonesian secondary education context,
involving three Secondary Schools categorized
based on their NE achievement.
• The results of the study indicate that English
teachers and students from the schools involved
have different perceptions on ENE.
ENE Washback
• English National Examination has an
influential impact on teachers’ teaching in the
aspect of: activity/time arrangement, teaching
materials, teaching contents, teaching
methods, teaching strategies, ways of
assessing, and on the feelings and attitudes of
the students. ENE also affects the students’
learning in the classroom in which teachers
mainly teach to test, practice the test and
develop test-taking strategies.
ENE Washback
• Negative washback only take place at schools
categorized as low achievers
• Both teachers and students in higher achiever
schools experience a positive washback of
National Examination
• Negative washback only takes place for a short
period time when students and teachers are
preparing for the exams.
• When exams finished, everything backs to
normal.
Testing at Tertiery Level (State Univ)
2. Testing administered by every university (Ujian
Mandiri)
• Subject tested: decided by each university:
commonly involved Indonesian, English, and Test
of Academic Potential (TPA)
• It is required to admit 40% of the total candidates
3. Invitation Selection: (Jalur Undangan) based on
the academic track record of the candidates,
usually scores of semster 1 to 5
SNMPTN and Invitation selection are to recruit
60% of the total candidates.
Other forms of Test
1) Directorate General of Secondary Education has developed
Test of English Proficiency (TOEP)
• The emphasis is on reading and Listening: (writing and
speaking are not yet used.
• TOEP is used to substitute TOEFL, which is now expensive
and using TOEFL products will violate copyright laws
• TOEFP is also designed mostly for Junior and Senior
Secondary Schools
2) Some universities require their postgraduate students to
have English proficiency certificate before completing their
program
• Commonly, the testing institutions use ITP TOEFL or
develop their own English proficiency test.
THANK YOU
HATUR NUHUN