Transcript Bureau for International Language Coordination

Bureau for
International
Language
Coordination
Julie J. Dubeau
BILC Secretary
Istanbul, Turkey
May 24, 2010
Outline of Presentation

General information

LNA report

BAT Report
What does BILC do?

Annual Conference in Spring

June 2009 hosted by Italy - “Bridging the Gap:
Language Requirements vs. Language Reality”

May 2010 hosted by Turkey “Mapping The Road: Success in Language Training”

Professional Seminar in Fall

October 2009 hosted by Denmark –
"The 21st Century Classroom: Keeping up with the Times!“

October 2010 hosted by Bulgaria – “Aligning Training and
Testing in Support of Interoperability”
Responsible for STANAG 6001
Ratified by the nations and promulgated
by NSA in February 2009
 English and French versions can be
downloaded from the BILC website
www.bilc.forces.gc.ca
 Steering Committee will be discussing a
proposal for admin change during
conference

What else does BILC do?




Language Training Assessments
Assistance to National Testing Programmes
Language Testing Seminars:
 LTS & ALTS
Special Projects:
 Benchmark Advisory Test (BAT)
 Language Needs Analysis (LNA)
General Information

ACT is tasked by IMS to investigate the
feasibility of introducing a computer based
learning tool for use by nations to
familiarize with NATO terminology…

BILC is investigating requirement through
JSSG
Complementary ADL Solutions
IMS Report Dated November 26 2009, Bratislava Follow-up Tasking on
Counter Insurgency (COIN)

“ACT should develop ADL solutions that will
complement nations’ efforts in language training,
understanding of different local government
structures and understanding familial, clan and
tribal cultures”.

ACT has requested BILC’s assistance

Please contact the BILC Secretariat if you are
interested in providing an ADL product
LANGUAGE NEEDS ANALYSIS OF
NATO CRISIS ESTABLISHMENT (CE) POSTS
November 2008 Chairmen’s Meeting of
the NTG in Brussels
 The aim of this study was to show whether
language requirements appeared to be set
at appropriate levels to enable military
personnel to perform their duties
adequately in the NATO OPS context, in
this case ISAF.

LANGUAGE NEEDS ANALYSIS OF
NATO CRISIS ESTABLISHMENT (CE) POSTS

To broadly identify whether the mandatory
SLP requested for the post was:



At Level ~ A: meaning that from the tasks described
in the JD, the profile appeared to be adequate;
High ~ H: meaning that the SLP requested for the
post appeared to be higher than the functions
defined in the job description; or
Low ~ L: meaning that the SLP requested for the
post appeared to be lower than the functions
defined in the job description.
Summary of Results
Ratings
Total = 609 CE Post JDs
A
- At Level
355 (58.2%)
H
- High
203 (33.3%)
L
- Low
41 (6.7%)
H/L - Mixed
10 (1.6%)
Results: Levels 1 & 2









Job Descriptions requiring SLPs of Level 1:
30 JDs required SLPs of 1111 & all were High.
Job Descriptions requiring mixed SLPs of level 1
& 2:
4 required SLPs of 2211 & all were at level.
2 required SLPs of 2221 & both were at level.
Job Descriptions requiring SLPs of level 2:
61 JDs required SLPs of 2222.
15/61 (26.2%) were at level.
43/61 (70.4%) were High.
Results: Levels 2 & 3
Job Descriptions requiring mixed SLPs of Levels 2 & 3:
 156 (25.6%) JDs required mixed SLPs of Levels 2 and 3.
 Of these 156, 66 (42.3%) were considered as having
SLPs that were at level
Further Breakdown:
 1 required SLP 3221 (High/Low depending on skill)
 9 required SLP 3222 (7/9 High)
 1 required SLP 3223 (High)
 4 required SLP 3232 (3High/1Low)
 118 required SLP 3322 (47 at level (39.8%), 56 High
(47.4%) & 3 Low (2.5%))
 23 required SLP 3332 (17 at level, 6 Low note that these
6 also req Dari)
Results: Level 3
Job Descriptions requiring SLPs of Level 3:




301 (49.4%) required SLP 3333
240/301 (79.9%) were considered at level
44/301 (14.6%) were considered High
13 were considered as Low SLPs for functions, (all
req expert language knowledge with 3s in Dari and
Pashto)
Results: Levels 3 & 4
Job Descriptions requiring mixed SLPs of level 3 & 4:


43 profiles of 4343, from which 22/43 were at level
(51.1%), and 21(48.8%) were not scored at level (some
H some L)
1 profile of 4344 (Low), 1 profile of 4443 (High)
Results: Level 4

Job Descriptions requiring SLPs of level 4:

8 profiles of 4444 (6/8 were considered at level, with
comments referring to the functions as requiring expert
language ability, diplomatic or sophisticated language
use, 2 were H)

Out of the 53 JDs requiring some skills at level 4, 10
(18.8%) also req languages other than Eng such as
French, Dari, Pashto, etc.
Out of these 10, 6 (60%) were at level.

Recommendations




NATO CE Post JD SLPs should be reviewed
SLPs should be based on an analysis of the
tasks performed by incumbents in relation to
STANAG 6001 Ed 3.
It is strongly recommended that the next
analysis of requirements be done with the
assistance of BILC language experts.
CE posts master sheet should include the SLPs
requested for the posts.
What’s next for interoperability?

IMS-Tasking on Language Proficiency and
Education on NATO Standards & Terminology.
12 February 2010
 Analysis of CE/PE posts
 Setting realistic requirements
 Review of linguistic requirement for Officers for operationally
deployed duties‘ has been initiated
 Benchmarking
Benchmark Advisory Test
(BAT)- PURPOSE

To provide an external measure against which
nations can compare their national STANAG test
results

To promote relative parity of scale interpretation
and application across national testing programs

To standardize what is tested and how it is
tested
Benchmark Advisory Test
(BAT)
Allocation to 11 Nations (200 ‘free’ tests)
 Tests administered by LTI, the ACTFL
Testing Office via the Internet and
Telephone
 January 2010 – End of Benchmarking
Process
 Positive Feedback from Nations

Benchmark Advisory Test (BAT)
Results
Listening
Speaking
Reading
Writing
Black
(11)
64%
(18)
72%
(10)
60%
(11)
55%
White
(18)
61%
(18)
56%
(18)
94%
(18)
39%
Red
(18)
89%
(18)
83%
(18)
83%
(18)
50%
Blue
(20)
85%
(19)
47%
(20)
55%
(20)
60%
Teal
(8)
0%
--
--
(8)
75%
(8)
38%
Maroon
(16)
69%
(15)
47%
(14)
64%
(16)
44%
Pink
(18)
67%
(18)
50%
(18)
78%
(18)
67%
Purple
(12)
8%
--
--
(13)
54%
(13)
62%
Orange
(13)
77%
(13)
46%
(13)
54%
(13)
62%
Yellow
(17)
24%
(18)
0%
(18)
33%
(18)
0%
Green
Nation did not release national test scores, but indicated that, on average, they were higher
than those attained on the BAT.
FACTORS CONTRIBUTING TO NONALIGNMENT







Test purpose
Testing method
Use of plus ratings
Alignment of author purpose, text type, and
reader/listener task
Inadequate tester/rater norming (productive
skills)
Inconsistencies in the interpretation of STANAG
6001
Cut-off score setting, etc.
BAT – Way Ahead

Project successful
Demand for administrations will dictate future
development needs and modes

BILC SC to formulate recommendations

Nations who require BILC support (post-BAT)
can request it

QUESTIONS?
Enjoy the Conference!