Transcript How to train a Dragon: a comparative investigation into

How to train a Dragon: a comparative
investigation into attitudes and efficiency
of voice dictation software for formative
and summative feedback
Vicky Reed
Melvyn Chimes
Introduction
What we didn’t do
What we did
2
What is Dragon? How does it
work?
• Voice recognition/dictation software (VRS) v.12
(we worked with v.11)
• It turns talk into text (grading, emails, reports). It
has a 99% recognition accuracy.
• Edits and revises documents.
• Frees you from the constraints of keyboard and mouse.
3
Why?
Increasing
student
numbers, but,
staffing levels
staying the
same.
More
assessments to
mark
Other physical
and mental
stress
Time
consuming
Sedentary
Katzmarzyk
(2010) life
expectancy
study
4
Introduction - Aims
1
• To enable lecturers to provide effective
feedback in a timely manner.
2
• To enhance the quality of (formative) and
summative feedback in order to improve
student success and achievement.
3
• To utilise new learning technologies to
develop more effective summative
feedback mechanisms at UCP.
5
Introduction - Outcomes
Predicted Outcomes
• More formative feedback.
• Improved quality of feedback.
Predicted Outcomes
• Positive engagement with new technology.
• Trained staff that can disseminate and train other
members.
Predicted Outcomes
• Better support for non – traditional learners.
• Better feedback mechanisms for staff with
developmental reading disorder.
6
Background literature
Technological factors
• Honeycutt (2003, 2008) –
A good quality sound card
and a fast processor are
needed.
• Spector (2001) this type
of educational technology
requires significant
investment in resources
and training.
Physical and Mental factors
• Yengin (2007) 70.4% of
44 lecturers used
computers in excess of 6
hrs a day.
• Blatter and Bongers
(2002) working on a
computer in excess of 6
hrs a day is associated
with an increase in work
related upper limb
disorders.
7
Background literature
User related factors
•
Mauri et al (2006) – user friendly, but time consuming.
•
Coniam (2004) – VRS can produce results as reliable as those performed
by hand.
•
Hux et al (2000) - Dragon NaturallySpeaking was significantly better and
more consistent at voice recognition than other VRS systems.
•
Roberts (2003) - Dragon software has improved implications for persons
with learning difficulties.
Batt and Wilson (2008) VRS represents a valuable tool for producing end
comments.
•
Anderson et al (2009) Technology has been shown to have mixed results as
a tool for staff and students with mild disabilities.
8
Methods & methodology
Short (open-ended questions)
questionnaire (Gillham 2008)
Unstructured interviews
(Kvale 2008)
• Administered electronically
for ease.
• Rich, narrative driven
qualitative data.
• Questions were more
detailed.
• Mainly employed a
sentence completion model,
with some completely open
ended questions.
• Valid data - Emphasis on
depth, exploration and
generation
• Questions are more ‘flexible’
and ‘more negotiated’ – there
are no limits or restrictions
• This may generate new
meanings
9
Methods & methodology
Grounded theory
Glaser & Strauss
(1978)
Rigour:
Fit and relevance
Workability
modifiability
Uncover basic
social processes
simultaneous
collection and
analysis of data
Memo writing
Open, axial, and
selective coding
Literature (review)
can be used as
data
10
Results
Questionnaire
Technology: Hardware,
Software
Interviews
Technology: Hardware,
Software
Time and location:
getting used to, ‘talking
feedback’ and the time
it takes to form this and
access to ‘quiet’ areas
Time and location:
problems speaking slowly
in a quiet room, time lost in
corrections, ‘not worth it ‘
Confidence: No to
training others, not
aware of all features
and how to access them
Confidence: Improves
with practice, still feels
the need to be ‘tied to
the chair’
11
Summary
• In both the interviews and questionnaires the
answered focused on three main themes/areas
• The memos showed that the technological issues
and those related to usability (time and location)
and user confidence emerged as the main
themes.
• As a result, it proved quite difficult to align the
emerging themes with any of the predicted
outcomes.
12
Evaluation
This project is best described as a ‘pilot study’.
The main aims and predicted outcomes were not achieved,
however the data generated was useful.
The questions may have needed to be developed ‘openended’ free flowing narrative is unsuitable.
Semi-structured focus groups (as originally planned) would
probably be more benefical.
13
Conclusion
• Despite all the complications associated
with staff absence and radically revised
methods etc, the research did provide
some illuminating and useful information.
• VRS is a useful tool, but how it is used or
implemented may need further
investigation.
14
Recommendations
Future Research
• A larger study is needed,
with more participants over
longer period of time.
• More focused training and
specific time given for
training.
• Updated hardware to cope
with the demands of the
software.
The use of Dragon for
feedback
• Once adequate training has
been given all participants
recommend that VRS be
utilised as part of the ‘lecturers
toolbox’.
• Investigation into its use in
assessment development for
those with learning disabilities
has been suggested.
• Shared comment banks would
be a useful addition.
15
Bibliography
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Charmaz K (2006) Constructing Grounded Theory: A Practical Guide through Qualitative Analysis London Sage
Publications
Crooks DL (2001) The importance of symbolic interaction in grounded theory research on women’s health Health
Care for Women International 22,11-27
Dey I (1999) Grounding Grounded Theory Guidelines for Qualitative Inquiry, San Diego: Academic Press.
Glaser BG Strauss AL (1967) The discovery of grounded theory: Strategies for qualitative research New York:
Aldine de Gruyter
Glaser BG (1978) Theoretical sensitivity, California, The Sociology Press
Glaser BG (1992) Basics of grounded theory analysis Emergence vs. forcing California: Sociology Press
Melia, K M (1996) Rediscovering Glaser, Qualitative Health Research 6(3) 368-373.
Strauss A Corbin J (1990) Basics of qualitative research Grounded theory procedures and techniques Newbury
Park: Sage Publications
Strauss A Corbin J (1998) Basics of qualitative research techniques and procedures for developing grounded
theory Second edition Thousand Oaks: Sage Publications
Gillham, B. (2008). Developing a questionnaire (2nd ed.). London, UK: Continuum International Publishing Group
Ltd.
Kvale & Brinkman. 2008. InterViews, 2nd Edition. Thousand Oaks: SAGE.
Blatter, B.M. & Bongers, P.M.(2002)Duration of computer use and mouse use in relation to musculoskeletal
disorders of neck or upper limb, International Journal of Industrial Ergonomics, 30, (4–5), p295-306.
De La Paz, S. (1999). Composing via dictation and speech recognition systems: Compensatory technology for
students with learning disabilities. Learning Disability Quarterly, 22, p173-182. Cited in: Honeycutt, L. (2003).
Researching the use of voice recognition writing software. Computers and Composition. 20, p77-95.
Gross, N. & Judge, P. (1998) Let’s talk!: Speech technology is the next big thing in computing, but will it put a PC
in every home? Business Week, 3566, p60-72
Hacifazlioglu, O., Sacli O.A., & Yengin, I. (2007) Lecturer’s attitudes towards the use of technology: Alternative
strategies for faculty administrators. In: ERIC (Educational Resources Information Centre) 7th International
Educational Technology Conference, North Cyprus, 2007.
Honeycutt, L. (2003). Researching the use of voice recognition writing software. Computers and Composition. 20,
p77-95.
16