Transcript Slide 1

LEMMA: Learning Environment in
Multilevel Modelling and
Applications
Fiona Steele
London School of Economics & Political Science
Director NCRM LEMMA node, University of Bristol
http://www.bristol.ac.uk/cmm/learning/online-course/index.html
LEMMA Training
• Capacity building in the analysis of data with
complex structure
– Ultimate goal is to move learners to “take-off”, i.e.
conducting and publishing multilevel analyses
• Different modes of delivery
– Face-to-face workshops
– Web-based materials in a virtual learning environment
LEMMA e-learning: First steps
• Originally planned to host searchable repository of
materials
• BUT …
– Written by different authors using different styles and notation
– Not clear how to tag and organise materials
• Concluded that we needed new materials organised as
a course
– Coherent structure – provide ladder for novices
– Ensure consistent terminology and notation
→ LEMMA online course launched April 2008
Lessons from Face-to-Face Training
• Learners need motivation and time
– Best motivated are those with multilevel data and research
questions
• Learners often do not possess prerequisites for
multilevel modelling
• In practical sessions, learners tend to focus on
mechanics of using software rather than interpretation
LEMMA online course: principles
• Accessible to anyone with basic statistics training
• Modules to have two integrated components: concepts
and practical
• Facility for learner’s self-evaluation (quizzes)
• Collect data to evaluate materials
• Design materials so they can be modified by other
trainers
A typical module
• Concepts
– Approx 40-50 pages, split into lessons
– Illustrative examples based on a single dataset
– Draw links between fitted model equations, graphs of predictions and
verbal interpretation
– No reference to software
• Practical
– MLwiN, Stata and R (SPSS coming soon)
– Detailed analysis of one dataset (different to that used in Concepts)
• Quizzes
7
Current modules (1)
1. Using quantitative data in research
2. Introduction to quantitative data analysis
3. Multiple regression
4. Multilevel structures and classifications
5. Introduction to multilevel modelling
6. Regression models for binary responses
7. Multilevel models for binary responses
8. Multilevel modelling in practice
Note: Modules 1-3 and 6 not on multilevel modelling
Current modules (2)
9. Single-level and multilevel models for ordinal responses
10. Single-level and multilevel models for nominal responses
11. Three-level multilevel models
12. Cross-classified multilevel models
13. Multiple membership multilevel models
14. Missing data
15. (coming soon) Multilevel models for repeated measures
Choice of e-learning application
• In 2005 choice was between Moodle and
Blackboard
• Moodle chosen because:
– Free and open-source
– Excellent on-line community support
– More easily customised than Blackboard
Growth in number of registered users
Launched April 2008
Now ~14k users
Intended use of materials*
n
%
11967
87.7
Recommended reading for a course
726
5.3
Preparation for LEMMA workshop
363
2.7
595
4.3
Learning
Self learning
Teaching
Total
*All figures correct as of 16/6/14
13651
Users by sector
n
%
Student
8085
58.6
Academic
3803
27.6
Non-academic
1907
13.8
Total
13795
Users by country
72% users from outside the UK
Country
n
Country
n
UK
3861
Spain
316
US
2701
Italy
310
Germany
604
India
287
Netherlands
514
Belgium
284
Australia
504
Switzerland
242
Canada
474
France
216
Users by primary discipline
n
%
Medical sciences
1962
14.2
Statistics, methods and computing
1886
13.7
Psychology
1778
12.9
Economics
1243
9.0
Sociology
1184
8.6
Education
932
6.8
Human geography and demography
784
5.7
Management and business studies
683
5.0
Political science and management studies
663
4.8
Biological sciences
504
3.7
2176
15.8
Other
User familiarity with statistical methods
User familiarity with mathematical statistics
Evaluation of LEMMA online course
• Survey of UK users conducted by NCRM in 2013
• 264 responses received
• Full report at
http://eprints.ncrm.ac.uk/3261/1/MOLEY_NCRM_
Impact_2011-2013.pdf
Summary statistics on usage
• Median time spent on course is 10 hours (SD =
14.8)
• ~ 65% had at least partially completed modules
on Introduction to Quantitative Research and
Multiple Regression (Modules 1-3)
– So important to provide introductory materials as
preparation for multilevel modelling modules
Source: ESRC NCRM survey
Reasons for registration
Combining with face-to-face training
“The course enabled me to get better value from face-to-face training, by
providing in-depth preparatory reading. Together with the face-to-face
training, I improved my knowledge of MLM to a sufficient degree to engage
in detailed discussions about analytical methods with academic
collaborators.”
“…I’d like to take further classes in this, but I need the online course first
so I’m prepared for the further courses.”
Respondents to NCRM survey.
Online course modules are recommended reading for
LEMMA face-to-face courses.
Impact of LEMMA online course
Source: ESRC NCRM survey
Resource considerations
• Writing online materials is resource intensive
– Materials unsupported so need to be clear and thorough
– Ensure consistency of style and notation across modules and
cross-reference previous modules
– Writing quizzes and feedback especially time-consuming
• Wide range of technical expertise required
–
–
–
–
–
Installation and maintenance of Moodle
Programming skills for customising Moodle
Web design
Database management
Ongoing technical support and trouble-shooting
Future developments
• New materials in pipeline
– SPSS practicals for selected modules
– Module on longitudinal data analysis
• Convert materials to interactive e-books
– New ESRC project led by William Browne
Thanks
• Funding for 3 phases of the NCRM LEMMA node, 20052015
• Technical team
–
–
–
–
Sacha Brostoff
Hilary Browne
Christopher Charlton
Hugh Garner
• Authors