Transcript Data Capture Process Stages UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice.

Data Capture
Process Stages
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Overview
 Objective
 Major Process Stages




Document Scanning operations
Recognizing operations
Verifying operations
Coding Assistance
 Factors/Considerations
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Objective
 To provide an overview of the major
process stages associated with optical
data capture and quality assurance
considerations
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Major Process Stages
Scanner Speeds are dependent on process chosen
Document Scanning
Recognizing
Recognizing is dependent on the sophistication of
the recognition engine
Automatic Electronic Verification
Verifying
Non-Successful Electronic Verification
Coding Assistance
prepare data in a form
suitable for entry into
computer
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Document Scanning Stage
 Key feature: scanning speed
 Scanning speed will be determined by:
 Quality of the scanner machines
 Size of non-drop out color
 Paper quality, cleanness & weight
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Recognizing Stage
 The recognizing process is to interpret images
 Accuracy of interpretation will be determined
by:
 Recognition engine/memory dictionary;
 Configuration threshold
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Verifying Stage
 Processing can be in geographic order or
in random order:
 Automatic electronic verification
 Non successful electronic verification: Need to
compare the value of the interpreted image with
the real image of the form.
Image manipulation
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Verifying Stage (cont.)
 Image Manipulation:
Electronic questionnaires can be sent to
specialist operators then back to the original
operator if necessary (in some cases, the same
questionnaire can be worked on simultaneously
by two or more persons)
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Coding Assistance Stage
 Process in which census questionnaire entries are
assigned numerical and/ or alphanumeric values
 Objective is to prepare data in a form suitable for
entry into computer
 Done by setting up possible responses to each
question in the census questionnaire
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Factors to be considered
 Questionnaire Design & Preparation
 Data Collection & Processing Considerations
 Field Operation
 Staff Training
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Thank You
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
 Additional material
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Questionnaire Design & Preparation
Form Design Advise
 Consider the number items to be included in a form
 Pre-print codes near the place where the box for ticks are
located
 Considering the speed of the data capture process - it is
advisable to use marks or “ticks” as much as possible
 Define drop out color properly; use registration marks
(allows for quicker recognition)
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Questionnaire Design & Preparation
 Form Design Advise
 Maintain consistent pattern in which the information
to be collected will be located
 Do not disturb the visibility of the ticks and marks
with titles, labels or instructions
 Avoid putting "answers" of one field to another page
of the questions;
 Avoid using open ended questions
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Questionnaire Design & Preparation
 How to Obtain Good Results of Scanning
 Select adequate paper quality
 Select a reliable printing press
 Use appropriate ink, considering drop out color
(for the questionnaires paper heavier than 80
grams per square meter can help avoid paper
crashes in scanner)
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Data Collection & Processing Considerations
 Field Operation
 Field Operators should have basic knowledge of the
data capture process chosen
 Staff Training
 A set-up of required training for staff will ensure
quality and effectiveness of the data captured
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Field Operation Considerations
 Reasons of Error-Reading of OCR:
 Bad condition of the form because of dirt,
folded, crumple, etc
 Unnecessary lines of characters such as
points, decorative strokes, hooks, etc
 Checking the questionnaires for completeness
and consistencies
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Training for Processing Staff
 Installation and set-up break-down of
equipment (e.g. hardware and software)
 Basic software knowledge
 Scanner operating procedures
 Troubleshooting (e.g. solutions to common
problems/issues)
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008
Control steps
 Control steps should be taken if the information
image is partial or no information to assure the
quality of generated files
 Value Checking Steps
 Control for Blank
 Missing Questionnaire
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region:
Contemporary technologies for data capture, methodology and practice of data editing
Doha, State of Qatar, 18-22 May 2008