the Coltrane project
Download
Report
Transcript the Coltrane project
Redesigning French Internet
Business data collection at Insee:
the Coltrane project
4th International Workshop on Internet survey
methods (Daejeon)
Béguin Jean-Marc
Business statistics director (Insee)
13/09/2013
Outline of the presentation
Thanks and context
1. The existing system in France
1. Surveys
2. Tools
2. The Coltrane project (four work packages)
1.
2.
3.
4.
2
Authentication
Contacts management
Portal of customized services
Data collection platform
4th International Workshop on Internet survey methods
09/13/2012
1.1 Business surveys and National Statistical
Authorities (NSA)
NSA : Insee and 5 or 6 ministerial statistical Offices
(MSO) (In 2010 Sessi, the MSO for industry, & Insee have merged)
80 Business surveys (of many kinds)
- 30 Insee and 50 MSO (+ others)
- About 500,000 enterprises (in over 3,5 millions) receive at
least one questionnaire
- The sum of all the samples sizes is 700,000
- Taking into account the periodicity of the surveys (12
questionnaires for a monthly survey), 1 800,000 forms are
sent each year by the NSA.
"sent" can be
- By mail for paper forms
- Through a website for dematerialized forms
3
4th International Workshop on Internet survey methods
09/13/2012
1.1 The present situation regarding Internet Usage
So far, put surveys on the web has not been a priority
2012 situation:
Insee
Number of surveys
Web surveys
Ratio of Internet usage
Sum of the samples sizes
Web surveys
Ratio of Internet usage
30
MSOs
47
22
16
73.3%
34%
380,000
305,000
178,000
46.8%
80,000
26,2%
Number of questionnaires
895,000 875,000
(weighted by periodicity)
Web surveys
654,000
253,000
Ratio of Internet usage
4
4th International Workshop on Internet survey methods
73,1%
28.9%
09/13/2012
1.1 Factors explaning the ratio of Internet answers
Heterogeneous from one office to another
Heterogeneous from one survey to another
Possible factors
- The age of the process
- The type of variables collected
- The number of variables and the length of the
questionnaire
- The frequency of the survey
- The size of the responding firms
- The renewal of the samples
- The ergonomics of the human interface
- the efforts made by Insee staff
- ...
5
4th International Workshop on Internet survey methods
09/13/2012
1.2 The tools to implement Internet Surveys
First Internet survey : 2000 (Sessi)
- Presently a second-generation application (ASP)
- Now Insee took over the portal (since 2010)
Beginning at Insee : 2004 (application called CRPI)
- This application is now 8 years old (JAVA)
- Designed for repetitive short term surveys
For specific purposes, new ways of collecting
dematerialized data have been developed recently:
- Upload / download application (eg EXCEL sheets)(2009)
- Blaise IS (Dutch software) (2009)
- Voozanoo (a free PHP software from Epiconcept) (2008)
Conclusion: many tools !
Complex for us and for enterprises
(e.g. one portal for each tool)
6
4th International Workshop on Internet survey methods
09/13/2012
1.2 An analysis grid to compare the tools (2010)
Originally based on the GSBPM
Within the sub-processes we have defined 23 functions
such as :
- Build data collection instrument (create the form)
-
Create a form on one web page…………………………
Create a form on several web pages or screens ……..
Create simple filters (from one question to another)….
Create complex filters (from page to page)…………….
- Build or enhance process components
enterprise)
- Generate both login & password……………………….
- Take into account the campaign dates ………………..
- Customize the questionnaire (e.g. with previous answers of the same
This grid helped us to analyse the pros & the cons of the
tools and of the whole system
7
4th International Workshop on Internet survey methods
09/13/2012
1.2 There is a need for a new project
CRPI is the best; but
-
old (9 years)
based on a proprietary software (to implement java)
its maintenance is costly;
Not designed for big annual surveys
No common directory nor common governance nor
common portal
Technically complex
moreover: in april 2011, the government requested that
all statistical surveys should be put on the web; existing
tools couldn’t allow it (specially for the SBS survey)
8
4th International Workshop on Internet survey methods
09/13/2012
2 The Coltrane project
Platform offering a range of technical & business
services
Four independent work packages:
-
Authentication
Contact management
Portal of customized services (for contacts)
Data collection platform (divided into 4 blocks)
Each WP is composed of several functions accessible
through services
A specific batch is planned to transform the SBSsurvey into a web-based survey
- Innovative because forms should be generated from
their DDI description
9
4th International Workshop on Internet survey methods
09/13/2012
2.1 The Coltrane project: authentication
Independent from the collection itself
We do not think about using PKI (Public Key Infrastructure)
but a LDAP directory to authenticate all the contacts
Any « contact » in an enterprise should have a unique
login & password
We shall offer the possibility of using this service to
all the MSOs
- Enterprises do not make any difference whether the
survey is carried out by Insee or a MSO
- They receive questionnaires from both
But each ministry has its own Internet policy and have
already built something for their own web-sites
we are not sure to be successfull !
10
4th International Workshop on Internet survey methods
09/13/2012
2.2 The Coltrane project: contact management
Independent from the collection itself, but linked to the
authentication service (to exchange credentials)
Can be used to manage all kinds of contacts (for example,
people buying data)
Insee staff will be able to:
- Create or delete a contact and its respective
characteristics
- Renew credentials
- Display the list of contacts of a given enterprise
- Display the list of the surveys of a given contact
- Assist the web respondents and inform them about the
coming surveys
- Communicate with all the contacts of a given survey
- …..
11
4th International Workshop on Internet survey methods
09/13/2012
2.3 The Coltrane project: portal of customized
services
A unique Internet portal will present all the surveys
(different from today)
Including the MSO surveys (as far as they have
accepted to use our portal)
The portal will be customized for each contact so that,
once authenticated, he can:
- only see the surveys he is interested in
- modify his own personal parameters (phone number, email, address, etc.)
- transfer his “answering power” to a colleague
- display his previous answers
- look at the remaining surveys he still has to answer as
well as their deadlines
12
4th International Workshop on Internet survey methods
09/13/2012
2.4 The Coltrane project: Data collection platform
Main part of the project
Block 1: offering different collection modes
-
Classical web form
Paper form (exchanged by mail)
Electronic data Interchange (EDI)
Upload / download service
If several collection modes exist, the questionnaires
should be generated from the same metadata flow
Block 2: running the collection (eg building management tools)
13
Opening/closing a campaign
Edit statistics for the collection
Organize reminders
Start litigations with non respondents
4th International Workshop on Internet survey methods
09/13/2012
2.4 The Coltrane project: Data collection platform
Block 3: generating questionnaires
- Aims at maintaining the coherence between a questionnaire
& its description (metadata)
- We will use a DDI model to model the questionnaire
- Then the form should be generated from its DDI description
- We are testing some softwares helping either the generation
or the DDI description
- This could allow us to put the SBS survey on the web
Block 4: managing the collected data
- In our view, centralizing all the collected data (whatever the
data collection mode) is of utmost importance
- The following functions will be developed : retrieve and
control the data, adapt the formats of data flows, keep an
image of the enterprise answer
14
4th International Workshop on Internet survey methods
09/13/2012
2.5 The Coltrane project: Some specific issues
Give the possibility for one enterprise to keep a copy of
its answer and for several respondents in the same firm
to fill up a single form
Find the best trade-off between
- Sending back quickly new credentials to a contact (when he
has lost them)
- Keeping a good level of security
Use the dialogue possibilities of Internet
- to help firms to answer to questions with hundreds of
possible codes
- to implement dynamic controls (eg with previous answers)
or explanations (in case of “peculiar” answers)
Make a proof an enterprise has received a web form and
has not replied (for mandatory surveys, to impose fines)
- Legal and technical issue
15
4th International Workshop on Internet survey methods
09/13/2012
The Coltrane project is still in a conceptual
phase and should be completed by 2014 or 2015.
Thanks for your attention !
Insee
18 bd Adolphe-Pinard
75675 Paris Cedex 14
www.insee.fr
Contact
M. Jean-Marc Béguin
Tél. : +33 1 41 17 50 41
Courriel : [email protected]
Informations statistiques :
www.insee.fr / Contacter l’Insee
09 72 72 4000
(coût d’un appel local)
du lundi au vendredi de 9h00 à 17h00