Transcript Basic QSL Specwriting

Where is market research
technology leading us?
It’s time to put the necessity
back into invention
Tim Macer
Managing Director
meaning ltd
United Kingdom
AGENDA
1.
2.
3.
4.
5.
State of the market in 2002
Where technology could/should be taking us
Impact of the Internet, after ‘boom and bust’
Standardization
The challenges of technology for all
1. The state of the market in 2002
How to back a winner in a 230 horse race
What is the state of the art?
 Relational databases
 Open interfaces
 Web enablement
 Process automation
 Ease of use
What’s hot
 ASP-delivered CATI
 Sample and panel management systems
 Portals and portal development tools
 Open interfaces making customization easier
 New visual interviewing tools
 Multi-mode interviewing
 Integration with enterprise databases and
data warehouses
The state of the market in 2002
 MR software now a $241 million industry,
worldwide1
 Software companies employ >1600 people1
Thousands more employed in agencies and
research buyers
 Over 100 specialist software manufacturers2
 Over 230 commercial MR software products2
82 web interviewing products
50 CATI products
1Source:
Research Software Guide, Jan 2002
2Source: Research Software Central
The problems of over-supply
 Years of under-investment showing
 Endurance of the ‘old way of doing things’
Programming scripts
New suppliers offering functionality the old
ones don’t
New suppliers not offering everything the old
ones do
 Developers can’t do everything
The way forward
 Consolidation through acquisition and merger
Difficult to see how
 Specialization: the niche within a niche
Already starting to happen
 Co-operation through open standards
Allow customers to pick the ‘best of breed’
solution
Integration across vendors
2. Where technology could (or
should) be taking us
It’s time to solve the difficult problems
The challenges
 To work faster
Less time on
repetitive tasks
 To work better
improved accuracy;
fewer errors
 To give more to the
client
Richer insights,
more connected
 To give more to the
respondent
More rewarding
experience
 To do all this for less
money
and still make a
profit
The right technology can help us reach these goals!
Multimodal data collection
 Serial
Phone/mail/phone
Telephone recruit/web interview
 Parallel
Respondent chooses: phone, web, mail in
Respondent can switch
 The challenges
Eliminating any reprogramming or reformatting
Overcoming modal differences
Multimode: who’s doing what
 Eliminating modal bias—web, CAPI and kiosk
Opinion One CAVI
 Web-based CATI/CAPI/CASI
GMI, NEBU, Pulse Train
 Modal templates
Askia
 Interviewer assisted web interviewing
Surveyguardian
 Multi-mode interview players
SPSS MR Dimensions (in development)
Candidates for automation
 Regular reporting
 Managing change on continuous projects
 Developing question libraries
 Testing and error detection in scripts
 Coding, editing and cleaning data
 Questionnaire scanning
 Eliminating paper records and storage
Automation: who’s doing what
 Report automation
Confirmit
 Good change management NEBU
 Question libraries
various but weak
 Computer-assisted coding
Ascribe
 Process automation
SPSS MR Script
Going Paperless: who’s doing what
 Paperless report distribution and archiving
E-tabs
 Report library and research admin for clients
Research Reporter
 Paperless interviewing
Techneos Entryware, J-TECH Electronic
Comment Card, M-CAPI Express
 Questionnaire Scanning
Bellview Scan, Eyes & Hands/mrPaper
Mixed visual/syntax questionnaire
authoring
 Who does the survey authoring?
PD, technician, programmer?
 We need editors that work like HTML page
design tools such as Frontpage or Dreamweaver
Syntax preferred by expert users
Visual better for non-technical users
Syntax more efficient for repetitive functions
Visual more efficient for look and feel
Better authoring: who’s doing what
 The ‘Dreamweaver’ of survey authoring
Pulse Train’s Visual QSL
 Cross-platform authoring tool
NIPO and CfMC visual survey editor
 Easy importation of Word documents
Askia
 Reading others’ questionnaire scripts
GMI
Visual QSL
Pulse Train
Combining data from multiple sources
 Mixing research data with hard data: some
techniques
Match by actual customer
Cluster analysis and segmentation
Time overlay
 Creating and using norms for comparison
Specialist market analysts are taking the lead
Integrating with CRM processes
 Adding the ‘why’ into knowing ‘what’
 Example: Egg, Internet bank in the UK
Customer panels
Event driven research
Ad hoc
Extranet for project definition and approval
Real-time reporting
Ethics: setting clear boundaries between
marketing, MR and CSS activities
Who’s doing what: multi-source & CRM
 Visual tools to merge survey and warehouse
data for analysis
SPSS Clementine
 Ability to read directly from a data warehouse
SPSS MR Data Model, Pulse Train’s Pulsar
 Sampling from CRM systems + adding data back
into the enterprise database
Centurion’s MaRSC
Clementine
SPSS
In the future… expert systems?
 research design
 survey authoring
 translation
 sampling
 data cleaning
 coding
 automated analytics and data mining
3. The real impact of the Internet,
after the ‘boom and bust’
Sure, if you can prove it’s faster, cheaper
and better
After the froth
 Continued growth for data collection in US
Slower elsewhere
 The web as the means to deliver computing
 More focus on delivery: growth of the ‘portal’
 Greater collaboration
Client
Client
Outworkers
Outworkers
Agency
Supplier
Supplier
Agency
Agency
Portals
 Client portals provide real-time access to their
research data, current and past projects
Many large research agencies are providing web
access for their clients
Realtime, online or published reports?
Project ordering and briefing
Survey testing and approval
 Respondent portals allow panelists to manage
their own profiles, surveys and create ‘online
communities’
Who’s doing what: Portals
 Portal building tools for client and respondent
interfaces
GMI
 Web-based analytics
Confirmit, GMI, Pulse Train, SPSS MR
 Respondent portals
NEBU
Net-MR Portal Management
Global Market Insite
Working visually
 Most web surveys still text based
 Next generation will be more visual and more
multi-media, sound, animation etc
 Not just a gimmick
 Aim to improve the interview experience
 Faster to complete
 Closer to reality
Who’s doing what: Going visual
 Visual interviewing for CASI, web and CAPI
OpinionOne CAVI
 Web-based quali/quant image-based evaluation
and tradeoff
GMI in association with 2ndSight
 Web-based and ASP-delivered conjoint
IdeaMap.Net
CAVI
OpinionOne
4. Standardization
Enabling us to go for ‘best of breed’
Proprietary imports and exports are
not good enough
 Too many competing packages; you may be in a
universe of one using your combination
 Proprietary data formats can change without
notice
 Too much effort wasted converting from one
format to another
 Data are useless without the definitions and
texts - the metadata
XML is not a standard language
 According to W3C,
“The Extensible Markup Language (XML) is the universal
format for structured documents and data on the Web”
 Close relative of HTML: both originated from
SGML
 Lets you create your own definition languages
 Increasingly used within MR software for data
definition and execution logic
 Everyone’s XML is different
Easier to translate but not mutually intelligible
Initiatives from around the world
Triple-s
www.triple-s.org
 First published 1994
 Originated in the UK but now implemented by
30 vendors worldwide
 Exchange data and metadata via exports and
imports in a generalized format
Version 1.1 introduced XML support
New version 1.2 adds filters, weighting and
multi-language support
<VARIABLE ID="4">
<NAME>Q4</NAME>
<LABEL>Which sights have you visited?</LABEL>
<TYPE>MULTIPLE</TYPE>
<POSITION>41 TO 42</POSITION>
<SPREAD>2</SPREAD>
<VALUES>
<VALUE CODE="1">Statue of Liberty</VALUE>
<VALUE CODE="2">Empire State Building</VALUE
<VALUE CODE="3">Times Square</VALUE>
<VALUE CODE="4">Central Park</VALUE>
<VALUE CODE="5">Rockefeller Center</VALUE>
<VALUE CODE="9">Other</VALUE>
</VALUES>
</VARIABLE>
Example triple-s code
Initiatives from around the world
Open Survey
www.opensurvey.org
 Not-for-profit organization that promotes open
standards for software
 Encourages production of open source software
for market research
 AskML is its proposed XML standard for
exchanging survey instruments
 TabSML is its working standard for exchange of
cross-tabular reports in a generalized format
 Also endorses triple-S
Initiatives from around the world
Object Management Group
www.omg.org
 Looks after the Common Warehouse Metamodel
 Pan-industry initiative to provide an agreed
framework for defining the storage and
handling of large amounts of data in data
warehouses
 Has an Analytic Data Management SIG with links
to triple-S and Open Survey
Initiatives from around the world
Data Documentation Initiative
www.icpsr.umich.edu/DDI
 Based at the University of Michigan
 Developing XML-based tools and open standards
 Aims to create machine readable archives of
textual descriptions and metadata for past
surveys, independently of the original
proprietary data formats used
 Will overcome future problem of surveys
defined in defunct languages
Initiatives from around the world
SPSS MR Dimensions Data Model
www.spssmr.com
 A new open (though proprietary) metadata
model for survey data
 Can be licensed independently of all SPSS MR
products (don’t have to use SPSS software)
 Comes with a developers’ library of tools for
building applications that will read or write
data via the Data Model
5. The challenges of ‘Technology
for all’
Are you licensed to drive that system?
Technology for all
 Researchers and clients want more control and
more freedom to access surveys, data and
management tools for themselves
 Is this the freedom to make mistakes
Bigger, faster, more expensively…?
 Are we putting dangerous weapons in the hands
of the innocent ?
 Is it the best use of the researcher’s time?
The wider issues
 End of the road for the specwriter and survey
programmer?
 New technology often means new ways of doing
things: new processes and new procedures
 Roles and responsibilities will shift
Who gets the blame if the logic or the
weighting is wrong?
 The need for the internal technology consultant
and advocate
“Every time I have seen it go wrong it is because
senior management did not understand the
technology. Expecting the systems or DP
department to come up with the answers is very
likely to give you a systems led solution rather
than a research led solution.”
John O’Brien, Chairman, BMRB
International, on implementing new
technology, interviewed in Research
World, March 2001 (Esomar)
Take a multi-disciplinary approach
 New technology won’t work if it is simply
molded around existing working models
 Set up an implementation strategy team,
involving all the operational units
 Make sure everyone learns the new technology
 Support for the new technology must come
from the top
get someone as senior as possible with their
hands on the technology
The software gets simpler
The system gets more complex
 The standard ‘turnkey package’ is on its way
out
 Multi-vendor solutions and going for ‘best of
breed’ can mean integration issues
 Open interfaces are making it easier to
customize software
Customization is increasingly desirable
Changing skills
 Less programming in proprietary languages like
Quancept and Survent
 Expertise needed in Java, HTML, XML, SQL,
Visual basic and so on
 Research people need to know more about
technology
 Technology people need to know more about
research
Wising up - short term
 Make training specific to your application of the
software
about how it will be used, not about what it can do
use real examples
 Aim for understanding so that people can rapidly learn
to solve their own problems
 Offer as many ways of learning as possible: courses,
self-study, lunchtime seminars, web-based or distance
learning
Wising up - long term
 Professional training in research must include
something on technology
 Where are the professional training and qualifications
for the research technologist?
 Proposed MRS Advanced Certificate in Research Practice
and Technology
Modular courses to provide ongoing or continuous
professional development for the research IT
professional
Need to involve vendors as well as research agencies and
buyers
Thank you.
Over to you…
For references, information and reviews of the
software mentioned in this presentation,
please visit www.meaning.uk.com