Visual Thinking and Visual Thinking Tools: Space, Time and Simple Cognitive Models to Support Design Colin Ware Data Visualization Research Lab, CCOM, University of New Hampshire

Architecture for visual thinking

Change Blindness Simons and Levin

Central Problem: How do we perceive the world in all its rich detail?

The Nature of Visual Space

Capacity of visual working memory (Vogal, Woodman, Luck, 2001)  Task – change detection 1 second  Can see 3.3 objects  Each object can be complex

Sequential comparison task

We can remember about 3-8 locations Object File Object File Object File Egocentric Coordinate Map

Gist Semantic content

Solution    “The world is its own memory” O’Regan Task-related active vision “What you see is what you need” Treish et al. (2003)  Seeing is a process that helps us solve problems

Task-related eye movements Hayhoe and Ballard, 2005

Example 1: How to get focus and context?

 Zooming (Bedersen)  Linked windows (Fowler & Ware)  Fisheye (Furnas, Carpendale)

Zooming Vs Multiple Windows (Matt Plumlee)  Problem: When do we need extra windows?

 Comparing parts of a visual scene.

 2 solutions: Zooming, multiple windows

Task: searching for target patterns that match

Conditons: Zooming vs Windows + eye movements 1,3,5,7 items per cluster

Cognitive Model (grossly simplified)  Time = setup cost + number of “visits” x time per visit  Number of visits is a function of number of objects to be compared and visual working memory capacity.

Visits = n/M

Prediction Results As targets (and visual working memory load) increases, multiple Windows become more attractive.

Design heuristic  When we need to compare more two or three simple pattern components add windows.

Example 2  Tools for finding new underwater behaviors from humpback whale tag data

(Why turning time into space is a good idea)

The gear Antenna Big Eyes

Dave Wiley DTAG Mark Johnson

Task: find new behaviors  = stereotyped patterns Cognitive Algorithm  repeat  Review behavior sequence looking for patterns. Remember patterns.

 Look for more instances.

 until no new patterns

The old way

Solution 1. GeoZui 4D

Cognitive process for finding new behaviors  stereotyped patterns Cognitive Algorithm  repeat  Review behavior sequence looking for patterns by playback . Remember patterns using space-time notes .

 Look for more instances. May involve reviewing all other whale tracks.

  Until no new patterns Cost k*playback time.

Solution 2: trackplot

Foraging patterns

Traversing

04 2006 Mostly 07 06 06

Process for finding new behaviors  stereotyped patterns Cognitive algorithm  Get to a good viewpoint    repeat   Review behavior sequence looking for patterns eye movements . Remember patterns using visual working memory .

Look for more instances. May involve reviewing all other whale tracks. Can be posted on the wall until no new patterns Cost Nav + Eye Movement time *pattern matching.

Gain in efficiency – from playback tool to pattern finding tool  Many hours (with playback)  A few minutes (with patterns)  Approximately a factor of 100

Design heuristic  Whenever possible: Turn time into a spatial pattern – one that converts critical events into shapes or patterns  Try to make natural mappings – proper use of texture color, etc.

Example 3: Network diagram

Degree of relevance highlighting  User clicks on something  Computer shows related items  User conducts a visual search for task relevant information

ME Graph Constellation

Order of magnitude gain in size  Layout problem simplified  Applicable to many problems  Depends on information scent

Review Display Features A B Patterns D Verbal Working Memory OBJECT FILES “Nexus” Visual Working Memory GIST Visual Query Egocentric object and Pattern map a

Where are we going?

 Simple cognitive process models involving  Perceptual and cognitive operations  Interaction methods

For Design