Transcript Web-based searching

Web-based searching
Understanding student experiences to enhance the
development of this critical graduate capability
Sylvia Lauretta Edwards
Dip.Lib.(RMIT), GCEd(HE), MIT(Res.), AALIA, MACS
Abstract:
Web-based information searching is a critical graduate capability. As teachers we can impressed
by the students’ ability to "talk the talk" and make it sound like they understand the process of
searching. Unfortunately, later, some of these same students do not "walk the walk". Ongoing
research to understand the variation in students’ searching experiences has been the first step.
On the basis of student perceptions we are challenged to find activities that may bring about the
ways of experiencing searching that we want them to engage in. This workshop will outline the
research findings in brief and then allow participants to explore the issues the research has
revealed in designing assessment instruments to bring about desirable learning outcomes in their
own teaching.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
1
Workshop Outline
• You will be given information on a phenomenographic study
of students’ experiences of web-based information
searching, aiming to explore student conceptions of the
searching experience.
– This will include:
•
•
•
•
An outline of the research inspiration
The brief methodology
The main findings from the research, and
Provide examples of using the research findings in assessment & curriculum
design
• Workshop will then allow you time to explore the issues and
apply this to your own work, or assessment and teaching.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
2
Inspiration
Background to the study:
Where did idea come from?
• Teaching in an IT Unit - Information Resources (ITB322/ITN322)
– Student’s learn about a variety of information resources and
their uses, independent of the format
• Information retrieval techniques for all of them, how to find and keep current on
information resources in future career, try to improve information searching skills
so that students learn to retrieve a lower recall of resources more relevant to their
needs.
– Their information explosion awareness increases
• Over 600 library/dialog databases
• Noticed that students often display little understanding of the tools
they are using, and seem to show little evidence of reflection upon
how to improve their results
• The study began as PhD research with 3rd yr. and PG IT students
and has now been expanded into 1st yr students in various faculties
using the Teaching Fellowship research.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
3
Research Aims
1.
2.
To determine variation in the ways IT students approach information
searching when using web-based databases.
To determine variation in IT students’ ways of learning how to search for
information when using web-based databases.
3. To recommend teaching and learning strategies for curriculum
design that are based on managing student’s experiences.
4.
To determine if there are different levels of sophistication in information
searching, or other differences in student information searching
behaviour approaches.
5. If levels do exist, to identify any triggers which allow students’
to move from one level of searching sophistication to the next
level.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
4
Participants and Method
• Phenomenographic Study - with transcript analysis, aiming to
uncover the variation (32 students and 44 Interviews)
• Different cultures, ages and genders represented
– 1st Round of Interviews
• Twenty ITB322/ITN322 students
– 1st Interviews conducted Wk 3-6 of semesters, Dual audio taped and video recorded.
(Transcripts of both)
– Weekly diary entries also analysed weekly looking for any changes in the way the
respondents appear to be reflecting upon the process.
• Twelve 1st Yr. students from IT, Science and Creative Industries
– 2nd Round of Interviews
• 12 of the previous students re-Interviewed at the end of the Teaching period Audio
Taped (Transcripts)
– Key Questions
• Has information searching experience changed, How, why, reasons for the
change? Information searching is …
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
5
Refer to EARLI 2002
• For the students undertaking the unit
– All students believed they were searching differently as
a result of unit
– students identified reasons for change as
• assignments designed to encourage reflection
• the ability of the teaching staff
•
NB: No student identified the OLT as one of the reasons for change
FOR MORE INFO...
Edwards, S. L., & Bruce, C. (2002). The assignment that triggered …change: Assessment and the relational learning
model for generic capabilities. Paper presented at the EARLI/Northumbria Assessment Conference Learning
communities and assessment cultures: connecting research with practice, 28-30 August 2002, Longhirst Hall,
University of Northumbria, Newcastle upon Tyne, UK. Available full-text on EducatiOn-line and AEHE (special
edition) (in press)
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
6
1st and 2nd
interviews used
to identify …
The Outcome
Space
Teaching Fellowship
findings Confirm
framework with the
1st year students
FOR MORE INFO...
Edwards, S.L. & Bruce, C.S. (2002) Needles, haystacks, filters and me: the IT confidence dilemma.
Refereed Conference Paper presented at Lifelong Learning Conference 2nd: Yeppoon, Central Queensland,
Australia, 16-19 June 2002. [Lifelong Learning Conference: refereed papers from the 2nd International
Lifelong Learning Conference, Yeppoon, Qld.] pp. 165-171. ISBN: 187 6780 19 3
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
7
Category 1: Information searching is seen as
“looking for a needle in a haystack”
“Once I understand the topic I am looking for then I can find it.
• Student’s focus is on the topic.
• Aware of info. environment but not the
details of the tool or the environment
structure
• Often confusion b/t different tools and tool
searching options.
• Comfortable with IT & moderate confidence
– Not confident to push further into the environment
• Planning is poor, or possibly non-existent
• Little, if any, reflection occurring
– Eg: searcher switches tools & terms at same time.
• Likely the search process will be abandoned
• Assumes information required is not
available at this source, or the tool in use is
inferior or deficient in the subject
Int 1:5(p.6) truncation is more like, let’s
say you have the ‘or’ ‘not’ ‘and’ and you
try to use these words. Lets say you
want both (topic 1) and (topic 2), you
put the ‘and’ in it so you come up with
both of them. That means it has to have
both of these terms in it. And ‘or’ will be
either this or either this, so any of them
will do.
Category 1: Information Searching is finding a needle in a haystack
Category 2: Information searching is seen as
“finding a way through a maze”
I must understand the topic, & plan to retrieve results in a useable format
• Students focus on the topic
• Strong emphasis on terms/synonyms, DB
selection, & retrieving results
• Planning process commenced
• Aware of Info. Environment structure & tool
search options.
• Begins to use advanced search features
Int 1:3 (p.4) …I try and find a search
• Aware of Info. Quality (not a major focus)
engine that has an advanced search
option and do it that way and specify
• Talk about tools interchangeably
whereabouts I want the web site to be.
• IT confidence good but still miss own mistakes
• Comfortable with online searching
– Show signs of reflection, see results and changing terms
– Likely to persist, consider alternatives, & persevere
• Still blame the tool rather than question their own
abilities.
Category 2: Information Searching is finding a way through the maze
Category 3: Information searching is seen as
“using the tools as a filter”
I may/may not understand the topic, but I can use tools to help me find it
• Students use tools to understand topic & to find
the required information.
• Focus is the tools, the topic is second
• Strong awareness of Info. Environment structure
& search tool options (adapt searching based on the tool)
• Info. Quality not important in search
• Tools refine topic & filter results size
• IT confidence high
• BUT Students aware of mistakes
– take steps to correct mistakes
• Planning evident
– term analysis & pronounced attempt at synonyms
• Reflection evident
– During search, terms & strategies change
• Students search is successful
Int 1:6 (p.1) … I had to search Internet
definitely, but I also had to search some
library to get a general information about
(topic), and I also had to find the recent
updates because the issue had to be
current. … Then I would probably go to
the library and find books about (topic)
and (sub-topic), because it is needed for
general definitions.
Category 3: Information Searching is using the tools as a filter
Category 4: Information searching is seen as
“panning for gold”
I can use the appropriate tools to find primary resources of information
• Use tools to understand topic, as a filter, & to
limit to high quality (retrieve primary sources)
• Focus on tool structure, then topic, & then on
info. quality
– Tools used to both refine topic & search to filter
out poor quality items
– ie: domain name searching or scanning
•
•
•
•
IT confidence very high
Aware of mistakes, check & correct them
More inclined to ask for help from peers.
Strong planning and reflection are evident
– reflection on information character/quality
– More inclined to stop a search, reflect upon
improvement, then reattempt later a previously
failed search.
• Successful in searching.
Int 1:4 (p.5)… but if you do a
search on say (topic) or
something, and you’ll end up
with thousands of pages. But
if you put lecture notes or
tutorial on the end of it, it
narrows it down quite a bit.
Category 4: Information Searching is panning for gold
Structure of Awareness …
Using the categories of description showing the variations
•
Introduced changes to assessment materials to lead
students into the structure of the experience considered
desirable
–
–
•
The next step is on online tool to encourage reflection
across the structure of awareness.
–
•
Success of this is not yet known
Structure of awareness section of each category has revealed elements that
need to be attended to, to allow the student to be aware of what is happening in
many areas of the experience.
There will be trial with 3rd yr. IT “crash test dummies” in Semester 1, 2004.
Decided to existing model already used in teaching a reflective
internet searching approach
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
16
An Action Research Model for Reflective Internet Searching
Edwards, S. L., & Bruce, C. S. (2002). Reflective Internet searching: an action research
model. The Learning Organization: an international journal, 9(3/4), 180-188.
Plan
the Net Search
Reflect
Critical Evaluation
Ways of thinking about the
Internet
Ways of learning about the
Internet
Act
Conduct Search
Constantly Changing
Internet Environment
Record Results
Print, Save, Bookmark
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
Edwards& Bruce ©2002
17
Assignments used in the unit
• 3 Items of Assessment originally developed
– Reflective Journal (20%)
– Information Consultants Search (30%)
– Information Resource Guide (50%)
• pedagogical underpinnings should be apparent as we go
through
Discuss these through looking at the design and
development of each as we go
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
18
Reflective Journal
• Submitted weekly, marked & within 1 week.
• Briefly this includes:
– Write a summary of weeks materials, readings completed, general
comments on unit and current studies
– Gives them the chance to reflect on what was covered and develop own
ideas
– Must reflect upon their current searching, what was new or different this
week in their searching approach, or what did they learn this week about
searching
– includes a section for questions to the lecturer or suggestions for
improvement
– NB: They get weekly feedback!
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
19
Reflective Journal
• Students attributing change in searching to this assignment stated
that the effort of having to reflect upon the content each week had
caused them to change their experience.
… if I wasn’t doing the journals this semester I don’t
think I would have rethought a lot of the things that
I’d learned straight away, and I think that a lot of the
good things that I’d learned may not have, you
know, may have just disappeared.
Male student, UG 21-25 yrs (Int 2001:2.2 p.8)
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
20
Information Search
• Given a Topic, which they must search using The
Library databases, Internet Search Engines and
commercial database vendors
• Interview the lecturer as if they are the “client”
– think about end users needs, what are they really looking for when
they search, matching information needs to end results
– Must produce a search report explaining their search strategies, tool
selection, and reflect on approaches
– learning about changing their strategies across different search tools
and databases, and evaluating resources
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
21
Search Report
• Students attributing change in searching to this assignment stated
that being forced to reflect and report on assumptions, keyword
selection, or other aspects of the search process, made them
realise that they should change the way they approached
searching.
What if I just blame it on assignment two … That – big time change
for me. Well you see it had to be comprised of all these different
components. Like ok, your client …synopsis, your assumptions,
your keywords and synonyms, your … there were so many
different areas. …. So you had to make sure you that you know
all these in order to put down in your report if you really wanted
to do well in this report. … So I came up with all these
assumptions and was like, oh ok I’ve never thought about these
before. Female student, UG 21-25 yrs (Int 2001:2.3 p.11)
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
22
Resource Guide
• Encouraged to select a topic that will be in the area
they want to work in when they are finished at QUT
– Team project (working in real-life)
– Find all resources that would be necessary to help
them keep up to date in their field over their working
life
• What books would they want on the office bookshelf? What
journals? What electronic conferences or Usenet Newsgroups
should they look at, what Internet sites should they bookmark
(add to “favorites”); what newspapers and databases should be
scanned regularly, etc.
• Must select the “Desert Island Top 5” best resources found and
give justification for their choice
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
23
Reviewing Assignments against the Categories
• Category 3 is built into Assignments
– It appears Assign’t 1 & 2 intuitively mapped with Category 3
• Both assignments require students to reflect
• Assignment 2 in particular asks students to refine their topics, devise
alternative search terms, and adapt their searching based on the tool in
use.
• But Category 4 ?
– Assign’t 2 & 3 - limited extent, no process reflection
• Assignment 2 requires a limit to high quality resources only.
• Assignment 3 asks for “Desert Island Top 5” and reasons
– But assessment marks did not reflect the significance of this, so little
attention/weight given to the exercise by the students.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
24
Where to from here …
• On the basis of student perceptions, assessment is now
being carefully re-constructed to bring about desired
learning outcomes
– to encourage reflection to level 4
– need to design the assessment to make the assignments work
harder, to lead students into the structure of the experience that is
considered desirable. In that way helping them to move into a
higher level of information searching experience.
• How ?
– A series of questions designed to encourage reflection about their choices and
report on the reasons for the choice, could be added, with appropriate
weighting in assessment criteria.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
25
Assessment in my other Units in IT
• In the information searching expected in a first year
unit, students are asked to undertake one or two of
the QUT Pilot project modules,
– Expected to complete the exercises and write a 300
word discussion of what they have learnt from the
exercise, reflecting in particular on their searching
experiences and how they search.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
26
Learning Theories
• Learning theories state that successful learning
includes (Marton, Dall'Alba & Beaty, 1993).
–
–
–
–
–
the students ability to increase their knowledge,
to memorise and reproduce that knowledge,
to apply it and understand what was done,
to see something in a new way,
and finally to change as a person.
• Furthermore information literacy, which includes the
ability to search for information, is a major part of the
lifelong learning process that all graduates need in
order to remain employable.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
27
Lessons Learnt
While embedding generic capabilities into assessment is vital to trigger changes in
conceptions or experiences, the ability and quality of the teaching staff should not
be ignored. Their ability, to arouse curiosity and make the learning environment
both stimulating and enjoyable, helps promote interest in the topic, which can also
lead to changes in learning outcomes.
• Based on the experiences in this FIT unit, a word of caution
needs to be sounded.
–
–
–
–
Students will not automatically engage in interaction via the OLT.
Students will engage when the assessment marks require it.
The OLT itself does not create a community of learners.
The unit coordinator needs to design the elements of the unit to enable the OLT
to be used to its maximum potential.
The whole learning experience must be crafted.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
28
Let’s list a few of your ideas
1. Can you suggest a few classic assessment items,
or even a few different forms of assessment?
2. Can you suggest how information searching skills
can be used within a unit or item of assessment?
–
–
–
to increase their searching skills?
to apply it and understand what was done?
to come to see searching in a new way?
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
29
Your Turn – the “work”shop begins
Alone or in Pairs - 20mins
• How can you apply this in a unit you teach?
– Select one of your units and/or choose one item of assessment to
either redesign or to develop from scratch
– Develop a list of basic criteria for this assessment item – what do
you want them to learn?
• What else can you do as a teacher to craft the learning
experience in your unit?
– ie: remember that the learning design can be used to develop a
learning community
• If you consider your unit design as a holistic process,
what role does information searching play?
– Include both present and any future ideas.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
30
Your Turn in Small Teams
Small
Groups
20mins
Workshop presented October 1st, 2003
1.
Select one unit/assessment that someone in the
team is currently developing or teaching.
2.
Reflect upon the ideas so far. Decide how
searching skills used in the unit could help the
students develop their skills further.
3.
Decide now an alternative set of assessment items
and/or teaching strategies to encourage the
development of searching skills.
4.
Using a spokesperson, share with the group your
final ideas.
© Edwards, S.L. (2003) Web-based Searching Workshop
31
Questions anyone?
Get more Information on assessment from
TALSS http://www.talss.qut.edu.au/
or try
1.
2.
Workshop presented October 1st, 2003
TEDI at UQ
http://www.tedi.uq.edu.au/Teaching/
Alternative modes of teaching and learning at UWA
http://www.acs.uwa.edu.au/csd/altmodes/
© Edwards, S.L. (2003) Web-based Searching Workshop
32
Further References
1.
2.
3.
4.
5.
6.
Candy, R.C., Crebert, R.G. (1991). Ivory tower to concrete jungle: The transfer of
learning skills from the academy to the workplace. Journal of Higher Education,
62(5), 570-592.
Candy, P., Crebert, G. & O'Leary, J. (1994). Developing lifelong learners through
undergraduate education. Commissioned Report No. 28. NBEET. Canberra:
AGPS.
Carlson, T., & Spegel, N. (1996). Adventure into education: Innovation in teaching
in tertiary settings. ERA/AARE Joint Conference Papers, Singapore.
Clanchy, J. & Ballard, B. (1995). Generic skills in the context of higher education.
Higher Education Research and Development, 14(2) pp. 155-166.
QUT MOPP Policy C/1.3 Generic attributes of QUT graduates
http://www.qut.edu.au/admin/mopp/C/C_01_03.html last accessed 1st October
2003
Raths, D. (1999). Next-century skills. (Industry trend or event) InfoWorld, 21(16),
pp.97-99.
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
33
Concluding Thoughts
What is happening to our individual/group view of
information searching this morning?
• Remember … Use materials already available
–
–
–
–
Library Pilot Modules
Edwards & Bruce “Reflective Searching Model”
Website/library database searching tips and “hands-on”
Most important …
Build desired experiences into your Assessment
requirements
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
34
Thank you for participating
Sylvia Edwards
QUT Teaching Fellow
and Lecturer,
School of Information Systems, FIT
[email protected]
http://www.fit.qut.edu.au/~edwardss/
Workshop presented October 1st, 2003
© Edwards, S.L. (2003) Web-based Searching Workshop
35