The Random Sampling/Tracking Tool: A Response to OverSurveying Steve Graves Intel Corporation Overview Problem Statement Recommended Solution Features User Interface Demo Benefits Concerns
Download
Report
Transcript The Random Sampling/Tracking Tool: A Response to OverSurveying Steve Graves Intel Corporation Overview Problem Statement Recommended Solution Features User Interface Demo Benefits Concerns
The Random Sampling/Tracking
Tool: A Response to OverSurveying
Steve Graves
Intel Corporation
Overview
Problem Statement
Recommended Solution
Features
User Interface
Demo
Benefits
Concerns
Next Steps
Possible tool enhancements
Beyond Intel
Summary
Problem Statement
Perception that too many surveys are deployed to
end users from IT – Source: Annual Partnership
Excellence Survey with top Intel Execs
Likely causes identified (based on experience as a
survey consultant)
– Groups tended to survey entire population of interest
instead of a sample (e. g. survey of habits of frequent
travelers -- 26,000 people)
– When criteria for a population is only partially available a
screening survey is necessary – results in more people
surveyed
– Simple random sampling will tend to hit a small
percentage of people multiple times
General
Population
Study A
Study B
Same
Participant
Problem Statement
Effective and efficient process to stop oversurveying does not exist
– Previous process relied on incomplete information and
“Guardians” – go between groups
– No mechanism to track who has received surveys
– Guardians may be overly cautious
– Meetings used to determine what to do – time consuming
and inefficient
– Process is not followed universally
Recommended Solution
Sampling tool containing a repository of
Intel employee data
– Utilizes data from employee database (nonconfidential)
– Generates random samples of target audience
– Tracks who has been sampled over a given time
period – currently 5 months
– Sampled individuals will not be selected again
for during this period
Recommended Solution
Database perspective
Employee Database
Retrieval Database
Sampling Database
Sampling Database
Randomly Selected Employees
Tracking Table
Storage Database
Recommended Solution
Database perspective – previously sampled individuals
removed before Sample is generated
Tracking Table
Generated table based on Criteria
Table that meets criteria and contains no excluded individuals
Recommended Solution
Incorporate into an overall process where
researchers are required to utilize the tool and
consult with sampling advisors
– Advisors can advise researchers on appropriate sampling
techniques – so researchers to survey an entire
population (e.g. 3,000 instead of 26,000)
Currently implemented using RAD database
programming language
–
–
–
–
Tool has been in use for almost 2 years
Tool is performing reliably and customers are satisfied
Usage is only through word of mouth
Approximately 60 samples generated last year
Sampling Tool Features
Current features
– Tracking and recording mechanism
– Recording of samples generated by others
– Random Sampling of entire Intel population
– Random sample from group that uses criteria not part of
employee database (for example: Employees who use
PDAs)
– Random sampling of Sub-Groups (For example:
Employees in Folsom California)
– Stratified sampling
User Interface
Demo (Double Click to Start)
Benefits
Saves researchers time
–Reduces the need for screening surveys
–No need to re-invent the wheel
–Saves time compared to previous
process
Benefits
Improvement in sampling process
–Reduce or eliminate “over-sampling”
– Delay surveys when target population is not
available
– Researchers receive consultation on using
appropriate sample sizes
–Distribute sampling more evenly across
groups
Concerns
May cause sampling bias in some
situations
Next Steps
Need to develop more sophisticated business rules
around sampling process
Currently working to make this an IT wide process
– Early results are promising
Looking at making it an Intel wide process
Port to SQL server and have a web based solution
– Current access is through Remote Desktop Connection
Develop strategies to mitigate sampling bias
Possible Future Tool
Enhancements
Complex samples through UI – currently
requires custom query
Linking to Other database that contains
product usage information
Beyond Intel
Similar tools could be used in similar
situations in other companies or
institutions
Summary
Problem: Over-surveying by our IT
organization
Response: Random Sampling/Tracking
Tool in conjunction with business rules
to manage survey sampling process
Benefits: Saves researchers time,
reduces over-surveying, improves
sampling process
Working to make this an IT wide official
process and eventually Intel wide
Summary
Next steps:
–Link to other databases
–Improve business rules
–More robust software platform
–Strategies to mitigate sampling bias
Similar tools/business rules could be
used in other institutions