Workshop: Utilizing concept maps and other approaches for acquiring, eliciting, representing, and comparing structural knowledge By Roy Clariana http://www.personal.psu.edu/rbc4 University of Oulu, Finland EDTECH Team seminar 14 and 15 of March, 2005 (9am till 1pm)

Tuesday March 15th

Mindmaps for qualitative data collection, a working seminar

Activity 1  Activity 2  The goal today is to determine if mindmapping of interview data is useful (ECOL)   Identify themes by extracting and sorting concept terms Capture meaning by extracting and representing propositions Products for today  Activity 1 – One (or possibly more) holistic map(s) that represents the themes across the 6 interviews  Activity 2 – One representational map of each interview

The connection

 

The purpose of most qualitative research is to explore the relationships between the data and ideas in the project

(p.27,

NVivo

Getting Started Guide, 2002)

Concept maps… are particularly effective for representing the organization that students see among concepts

(p.64, Turns, Atman, & Adams, 2000)

Using a group drawn mindmap

during an interview

Interview 1

text text text text

In future, experiment with this approach, contrast one group using yellow stickies on big pieces of paper to respond to the interview questions compared to regular (no map) interview. See if there is a difference in the data…

Qs

The capability and experience of the person coding the text is critical…

examples

interviewer

See: William M. K. Trochim

Using a researcher drawn mindmap

after an interview

Interview 1

Transcript text text text text text text text text text text

Interview 1

memo text text text text

i.e., attribute theory note issue The capability and experience of the person coding the text is critical…

examples

coders

The connection

Activity 2 Activity 1    

Daley, 2004

Concept maps can provide one strategy to deal with the methodologic challenges of qualitative research. A mindmap can be used to: frame a research project (develop the model) reduce qualitative data (20 page interview to one page map, visual identification of theme, one page maps facilitate comparison between groups and sites) analyze themes and interconnections (e.g., our cluster analysis activity), and present findings (similar to using participant comments, maps provide peremptory example).

Mindmaps and qualitative research are philosophically consistent

Daley, 2004

You must determine the unit of analysis for each pass through the interview    Concept (one or two words, usually nouns) Phrase (several words, nouns and verbs) Proposition (noun-verb-noun combination)

The connection

Words--



Phrase--



Words--



Phrase--



Daley, 2004

Concept maps can provide one strategy to deal with the methodologic challenges of qualitative research. A mindmap can be used to: frame a research project (develop the model) reduce qualitative data (4 page interview to one page map, visual identification of theme, one page maps facilitate comparison between groups and sites) analyze themes and interconnections (e.g., our cluster analysis activity), and present findings (similar to using participant comments, maps provide peremptory example).

Mindmaps and qualitative research are philosophically consistent

Daley, 2004

Activity 2 – reduce qualitative data

   An example of a 20 page interview to one page map, visual identification of theme, one page maps facilitate comparison between groups and sites phrases

ECOL data

Activity 1 – Lets look at the semi-structured interview audio tape from 2 groups over 3 sessions The first activity is to derive themes (from terms) The second activity is to represent the interview (with phrases)

Activity 1 - Identifying broad themes



text abstraction

An alternative to brainstorming for generating a set of statements for concept mapping. A text abstraction method involves identifying text statements that are imbedded in some larger text and extracting them for use as separate statements in concept mapping. For instance, one could review the transcripts of a focus group discussion and identify key conclusion statements that were made. These could each be extracted and entered as a concept map statement set so that the results of the focus group discussion could be mapped and used in subsequent work (e.g., decision making or pattern matching). http://www.conceptsystems.com/kb/00000376.cfm

Using all 6 interviews, if possible

Activity 1 – Cluster analysis, as 2 groups

Brainstorming (corpus list) Sorting (move like terms closer) Use yellow posit notes on large paper Merging & Pruning (combine like terms, delete or move unlike terms, synthesize terms) Finally, links may be added Then document (save/print) Naming Clusters (name the categories/ themes )

and if necessary

Sorting Clusters (move like clusters closer) Naming broad themes (name the cluster of clusters)

thoughts

 Keep note of the evolution of the process for later discussion, especially what works and what did not   Each team presents their map Discussion of Activity 1 process…

Activity 2a: Let’s try converting ECOL interview data directly into a map

Method 1  Read a sentence, add a phrase or proposition to the map, repeat  when done, play with structure and links  Compare our maps  Debrief Activity 2a

How to copy from interview, double click paste on CMAP

Interview CMAP Activity 2a

Activity 2b: Converting ECOL interview data into a map using propositions

Method 2  Write propositions into a text file   Open text file with cmap tools Play with structure and links   Compare our maps Debrief Activity 2b

How to convert propositions



a map

     Write propositions into a text file Open text file with cmap tools Play with structure and links Compare our maps Debrief Activity 2b

How to convert propositions



a map



Makes a “likes” text file

 

Import into CMAP Tools Select Format



Auto layout

 View hierarchical  View force directed Activity 2b

Summary

  Additional approaches – I will use your corpus terms and see what

ALA Reader

can determine from the interviews Other comments?

readings

Monday Turns, J., Atman, C.J., & Adams, R. (2000). Concept maps for engineering education: A cognitively motivated tool supporting varied assessment functions.

IEEE Transactions on Education, 43

(2), 164-173 Cicognani, A. (2000). Concept mapping as a collaborative tool for enhanced online learning.

Educational Technology and Society, 3

(3), 150-158.

van Boxtel, C., van der Linden, J., Roelofs, E., & Erkens, G. (2002). Collaborative concept mapping: provoking and supporting meaningful discourse.

Theory Into Practice, 44

(1), 40-46.

Chiu, C.H., Huang, C.C., & Chang, W.T. (2000). The evaluation an influence of interaction in network supported collaborative concept mapping.

Computers and Education, 34 International Journal of Science Education, 23

, 17-25.

Kinchin, I.M. (2001). If concept mapping is so helpful to learning biology, why aren’t we all doing it? (12), 1257-1269.

Tuesday Daley, B.J. (2004). Using concept maps in qualitative research. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds.,

Concept maps: theory, methodology, technology

http://cmc.ihmc.us/papers/cmc2004-060.pdf , vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17. Daley, B.J. (2004). Using concept maps with adult students in higher education. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., F.M.Gonzales, Eds.,

Concept maps: theory, methodology, technology

http://cmc.ihmc.us/papers/cmc2004-059.pdf

Concept maps: theory, methodology, technology

http://cmc.ihmc.us/papers/cmc2004-215.pdf , vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17. Basque, J., Pudelko, B., & Leonard, M. (2004), Collaborative knowledge modeling between experts and novices: a strategy to support transfer of expertise in an organization. In A.J.Canas, J.D.Novak, and , vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17.

Tuesday post workshop – Lessons learned

Only able to complete Activity 1 for 1 interview (5 page transcript), allow more time or fewer activities next time Lessons learned for Activity 1  There are “unsaid” ideas that will not be captured; solution, add “memos”       Underlining key terms is quick, about the same as just reading the transcript Individual’s underline, compare for inter-rater reliability We noted that the corpus terms depended on the perspective of the reader, and concluded that the corpus term list may be under represented and decided to do it again taking separate roles when underlining Have individuals read with different hats in order to capture different Themes began to emerge just simply during underlining Write “a has” in the margin when you think of it            To prepare the interview, reduce names (i.e., James becomes J: ), number each exchange Note episodes while reading by drawing a line across the page On the yellow postit note, include line number and person (i.e., 12, J: ) as well as the phrase It takes a lot of time to go from the underlined text to the postit notes; solution, prepare postit notes ahead of time but d on’t tell the group until they are ready to write the postit notes, then add their phrases as additional postit notes Need a really big white board to stick the postits on and to be able to move them around and write on it 5 pages of interview produced about 50 stickies We first used a matrix to sort the stickies, students down the left column and time going from left to right. We noticed that certain students tended to start discussions and take certain roles, like leading or group regulation (this specific matrix can be done ahead of time, but is fairly quick to complete); need a way to capture this matrix because the next step moves all the stickies The next sort was open-ended, moving like terms together etc. we needed to do this as a big group, though several people were at the board at the same time moving things, we all discussed the names of clusters, moved more stickies, and then reached consensus on the THEMES, the themes that emerged were goals, group dynamics, external factors, regulation, positive feelings and negative feelings Looking at these clusters, we asked “Who was the first to mention a theme” and “Who occurred the most within a theme cluster” We drew a new matrix with these themes down the first column and time going across the top, we observed a pattern of group dynamic-external factor-regulation that repeated several times. A group dynamic issue seems to demand a regulation response to resolve it.

We concluded that this kind of “pre analysis work” as a team before going to NVIVO analysis may be critical in order to elicit emerging themes See Monday powerpoint at http://www.personal.psu.edu/rbc4/edtech_monday.ppt