Transcript Project Overview

THE 4TH INTERNATIONAL WORKSHOP ON
INTERNET SURVEY METHODS
Statistical Center of Statistic Korea, Daejeon, South Korea
13 September 2012
Public Opinion Online:
The Hong Kong Experience
Robert CHUNG and Winnie LEE
Public Opinion Programme (POP)
The University of Hong Kong
About HKU POP
•
Established in June 1991
•
To collect and study public opinion on topics which
could be of interest to academics, journalists, policymakers, and the general public
•
The first of its kind established by an academic
institution in Hong Kong
•
Now directly under the Faculty of Social Sciences of
the University of Hong Kong
2
POP Service Policy
All POP Polls shall be published for public consumption, preferably as soon as they
are completed. Flexibility, however, is allowed on the timing of the release in order
to let clients make full use of the research findings.
The POP Team shall be fully responsible for the research design, including the
sampling method, questionnaire design, fieldwork supervision, data analysis, and
report writing.
The POP Team reserves the right to add opinion or demographic questions to a
survey which may not be of interest to the clients, provided that such questions are
methodological sound and do not impair the original design.
The POP Team shall retain the copyright of all the data gathered from the POP Polls,
this right can be shared with the client.
Full acknowledgement shall be given to POP and the client in any subsequent
publication using whole or part of the data collected in any POP Poll.
POP shall deposit one set of reports at the Library of the University of Hong Kong at
such time when the research findings are ready for public consumption.
3
Conventional Public Opinion Surveys
•
By June 2012, POP has conducted over 1,200
independent surveys, mostly supported by outside
funding
•
Common types of surveys: Random telephone
surveys, online surveys, face-to-face interviews,
mail surveys, household surveys, focus groups
•
Areas of interest: Electoral study, government
policy, media performance, and general political,
social, livelihood and health issues
4
POP Tracking Polls
Popularity of the Chief Executive
Popularity of the Government
Popularity of Principal Officials
Popularity of Councilors
Appraisal of Current Conditions
Social Indicators
Freedom Indicators
Rule of Law Indicators
Media Performance
Annual Series
National Issues
Election Surveys
5
Popularity of HKSAR Government
6
Popularity of Government Officials
7
POP Online Surveys (Selected Studies)
Sampling
method
Target
Convenient
Public
Census
Members
- Student Quizzes on eHealth Record System (2010/11)
Census
HKU students
- Hong Kong Police Force Staff Opinion Surveys (2004, 2007,
2010)
Random
Staff
- Hong Kong Lesbian, Gay, Bisexual and Transgender (LGBT)
Climate Study (2011/12)
- Members survey of Hong Kong Institution of Engineering
(2010, 2012)
- July 1 Rally Online Surveys (2003 – 2010, annual)
- Participant Survey for Hong Kong Council of Social Service
“Caring Company Scheme” (2009)
- Online Survey on Hong Kong Harbour Fest (2003)
- Public’s Awareness of Product Logos (2002)
Convenient
Census
Convenient
Random
Rally
participants
Company list
supplied
Fest
participants
Public
The Studies
9
Hong Kong Lesbian, Gay, Bisexual and Transgender
(LGBT) Climate Survey (2011/12) - 1
Background:
Commissioned by Community Business, a non-profit making organization
dedicate to advocate Corporate Social Responsibility
Objective:
To gauge the local working population’s awareness and attitude towards LGBT
individuals in Hong Kong, and to uncover issues faced by the local LGBT
employees in the workplace
Study Design:
1) a representative survey of the Hong Kong working population by random
telephone interviews, and
2) an anonymous online focus survey of the Hong Kong LGBT working
population which targeted at LGBT individuals who were working full-time,
part-time or seeking jobs then
Survey Period:
12 November 2011 to 14 January 2012
Participant
Recruitment:
Via online promotion as well as leaflet distribution at LGBT gathering places,
with the help of LGBT concerned groups
Instrument:
22 questions, all closed-ended questions
Sample Size:
628 submissions in total, among them, 626 cases were regarded as valid after
data cleaning, of which 548 cases were from LGB individuals whereas 78 cases
were from T individuals
Response rate:
N/A
10
Hong Kong Lesbian, Gay, Bisexual and Transgender
(LGBT) Climate Survey (2011/12) - 2
Merits and Limitations
Merits
- Makes the unreachable reachable, at a low cost
- Complete anonymous and self-administered mode suitable for collecting
views on potentially sensitive / embarrassing issues
- Time-efficient
Limitations
- Difficult to avoid multiple submissions (beside I.P. address & identical
answers checking)
- Difficult to ensure authenticity of answers & eligibility of respondents
- Difficult to work out response rate, hence representativeness unclear
11
Members Survey of Hong Kong Institution of
Engineering (2010, 2012) - 1
Background:
Commissioned by Hong Kong Institution of Engineering in 2010 and 2012
Objectives:
To collect members’ views on the Institution’s service provision and on its
official Journal namely, “Hong Kong Engineer”
Study Design:
Both surveys targeted ALL its current members with email accounts
Survey Periods: 12 February to 5 March 2010
31 January to 15 March 2012
Participant
Recruitment:
Email invitations each with the link to the survey platform and a unique
password, were sent to ALL current members with email account, followed by
a couple of reminder mails
Total number of invitations sent: 19,374 in 2010, 17,440 in 2012
Instruments:
52 questions in 2010, 27 questions in 2012, mainly closed-ended, plus a openended questions in both surveys
Sample Sizes:
1,140 valid submissions in 2010 (sampling error: +/-2.8% at 95% conf. level)
1,166 valid submissions in 2012 (sampling error: +/-2.8% at 95% conf. level)
Response Rates: 5.9% in 2010
6.7% in 2012
12
Members Survey of Hong Kong Institution of
Engineering (2010, 2012) - 2
Merits and Limitations
Merits
- Cost effective
- Paperless
- Time-efficient
- Unique password log-in system prevents multiple submissions
Limitations
- Low response rate
- Impersonal, limit the ability to probe in-depth opinions
13
Student Quizzes on eHealth Record System (2010/11) - 1
Background:
Commissioned by the Hospital Authority and the Food and Health Bureau of the
HKSAR Government
Objectives:
To review the effectiveness of difference promotion approaches for public consultation
exercises
Study Design:
Two separate online quizzes, namely Quiz A (without incentives) and Quiz B (with
incentives), were launched in sequence to test the effectiveness of using online quiz as a
means of public consultation targeting at young people, as well as the effectiveness of
incentives, if any
Survey Periods: Quiz A: 26 November to 2 December 2010
Quiz B: 3 December to 9 December 2010
Participant
Recruitment:
Invitations were sent out to basically all undergraduate and postgraduate students in the
University of Hong Kong via its bulk mail system, divided into two groups according to
their year of study and faculty, who were then invited to take part in either Quiz A or
Quiz B. Only the invitation message for Quiz B stated clearly that the first 50
participants who answered all questions correctly would be entitled to receive an
incentive, i.e. HK$25 Starbucks coupon. Total number of invitations sent: 11,194 for
Quiz A, 7,893 for Quiz B.
Instruments:
5 multiple choice questions on their knowledge of eHealth Record
Sample Sizes:
39 valid submissions for Quiz A
94 valid submissions for Quiz B
Initial Response 0.3% for Quiz A
1.2% for Quiz B
Rates:
14
Student Quizzes on eHealth Record System (2010/11) - 2
Merits and Limitations
Merits
- Paperless
- Time-efficient
- Log-in system using student ID number prevents multiple submissions
- Convenience
- Incentives encourage participation
Limitations
- Low response rate
- Impersonal, limit the ability to probe in-depth opinions
15
Hong Kong Police Force Staff Opinion Surveys
(2004, 2007, 2010) - 1
Background:
Commissioned by Hong Kong Police Force to conduct regular staff opinion survey once
every three years since 2004
Objectives:
1) assess the level of staff job satisfaction and commitment to the Force; 2) gauge the
extent of acceptance, understanding, and action with respect to values, policing change
readiness, performance of the Force, and other dimensions that were emerging; and 3)
identify any other emerging concerns the staff might have
Study Design:
Target population was all in-service members of the Force, including Regular, Civilian
and Auxiliary staff of all ranks, but excluding the Commissioner rank, stratified random
samples were selected according to the staff members’ Unique Identification Numbers
Survey Periods: 28 December 2010 to 28 January 2011; 27 December 2007 to 18 January 2008; 16
December 2004 to 4 January 2005
Participant
Recruitment:
Invitation letters were disseminated to the targets by designated Training and Staff
Relation Officers (TSROs) in batches according to the Locations the targets belong to,
and the TSROs were responsible for ensuring a satisfactory response from the selected
members in their Locations
Total numbers of invitations sent: 5,467 in 2010; 5,498 in 2007; 5,518 in 2004
Instruments:
A respective of 118, 87 and 102 mostly closed-ended questions in 2010, 2007 and 2004
Sample Sizes:
2010 survey: 3,685, with 3,656 online, 29 telephone (error: +/-1.6% at 95% conf. level)
2007 survey: 3,695, with 3,681 online, 14 telephone (error: +/-1.6% at 95% conf. level)
2004 survey: 3,950, with 3,906 online, 44 telephone (error: +/-1.4% at 95% conf. level)
Response Rates: 2010 survey: 67.4%; 2007 survey: 67.8%; 2004 survey: 72.4%
16
Hong Kong Police Force Staff Opinion Surveys
(2004, 2007, 2010) - 2
Merits and Limitations
Merits
- Cost effective
- Time-efficient
- Convenience
- Unique log-in number prevents multiple submissions
- High response rate (with frequent chasers from TSROs)
Limitations
- Impersonal, limit the ability to probe in-depth opinions
- Privacy concern (fear of identity being mapped)
17
July 1 Rally Online Surveys
(2003 – 2010, Annual) - 1
Background:
July 1 rally is an annual protest rally in Hong Kong originally led by the
Civil Human Rights Front since the Hong Kong SAR establishment day in
1997
Part of POP’s self-initiated study of the July 1 Rally besides head-counting
Objective:
To collect opinions on current social issues from the marchers
Study Design:
Rally marchers were invited to participate in the survey by convenience
sampling
Survey Periods: 1 to 11 July 2003 and 1 to 4 July in 2004 – 2007, 2010
Participant
Recruitment:
POP interviewers to distribute leaflets with passwords at the assembly point
to invite the rally marchers to log on to a designated website to fill in an
online questionnaire
Total numbers of invitations distributed:
19,350 (2003), 12,270 (2004), 12,853 (2005), 4,616 (2006), 5,500 (2007),
6,000 (2010)
Instruments:
17 to 52 mostly closed-ended questions in different surveys
Sample Sizes:
2,413 (2003), 736 (2004), 411 (2005), 257 (2006), 362 (2007), 73 (2010)
Response Rates: 12.5% (2003), 6.0% (2004), 3.2% (2005), 5.6% (2006), 6.5% (2007), 1.2%
(2010)
18
July 1 Rally Online Surveys
(2003 – 2010, Annual) - 2
Merits and Limitations
Merits
- Cost effective
- Time-efficient
- Convenience
- Flexible on locations
- Unique log-in number prevents multiple submissions
Limitations
- Low response rate
- Impersonal, limit the ability to probe in-depth opinions
Participant Survey for Hong Kong Council of Social Services
“Caring Company Scheme” (2009) - 1
Background:
Commissioned by the Hong Kong Council of Social Service (HKCSS)
Objective:
To gauge the participants’ opinions on various aspects of the “Caring
Company Scheme”
Study Design:
Online survey targeted all participating companies of the “Caring
Company Scheme” in recent years
Survey Period:
6 August to 12 August 2009
Participant
Recruitment:
E-mail invitations sent to the 1,765 participating companies of the
Scheme, followed by multiple chasers
Instrument:
A questionnaire comprising a total of 9 key questions, mainly in closedended format
Sample Size:
402 submissions were received, while 397 was confirmed to be valid
cases (sampling error: +/-4.4% at 95% conf. level)
Response Rate:
22.5%
20
Participant Survey for Hong Kong Council of Social Services
“Caring Company Scheme” (2009) - 2
Merits and Limitations
Merits
- Cost effective
- Time efficient
- Convenience
- Flexible on locations
Limitations
- Low response rate
- Impersonal, limit the ability to probe in-depth opinions
- Privacy concern (fear of identity being mapped)
Online Survey on Hong Kong Harbour Fest (2003) - 1
Background:
Commissioned by the Audit Commission of the HKSAR Government
Objective:
To gauge opinions of the concert-goers on Hong Kong Harbour Fest
Study Design:
Online survey targeted audience of the three particular Hong Kong
Harbour Fest concerts held on 6, 7 and 9 November 2003, of age 6 or
above
Survey Period:
6 to 11 November 2003
Participant
Recruitment:
Leaflets, each printed with unique password, were distributed at three
locations of the concert venue, to the audience
A total of 12,065 leaflets were distributed
Instrument:
A questionnaire comprising a total of 18 questions, mainly in closedended format
Sample Size:
902 submissions were received, while 718 was confirmed to be valid
cases (sampling error: +/-3.8% at 95% conf. level)
Response Rate:
6.0%
Online Survey on Hong Kong Harbour Fest (2003) - 2
Merits and Limitations
Merits
- Cost effective
- Time efficient
- Convenience
- Flexible on locations
Limitations
- Low response rate
- Impersonal, limit the ability to probe in-depth opinions
Survey on Public’s Awareness of Product Logos (2002) - 1
Background:
Commissioned by a local beverage company
Objective:
To understand public’s awareness of its brand and their abilities to
distinguish different brand logos
Study Design:
Random online survey targeted local population aged 15 or above who
have a habit of using internet
Survey Period:
17 July to 7 August 2002
Participant
Recruitment:
1) Respondents of a random telephone survey on the same topic were
invited to take part in the online survey after the interview
2) Cold call made to randomly drawn telephone numbers to invite
eligible respondents to participate
Instrument:
A questionnaire comprising a total of 12 closed-ended questions with
visual images of product logos
Sample Size:
590 valid cases (sampling error: +/-4.2% at 95% conf. level)
Response Rate:
25.7%
Survey on Public’s Awareness of Product Logos (2002) - 2
Merits and Limitations
Merits
- Allows the use of visual aids in questionnaires
- More flexible as compared to mail survey or face to face surveys
Limitations
- Low response rate
- Skewed attributes of internet population
- Impersonal, limit the ability to probe in-depth opinions
“The HKU POP Experience”
•
Online survey can be very successful for staff opinion and membership
survey, if the organization involved can instill trust in the target
respondents in terms of privacy and confidentiality, while allowing the
researcher to keep track of all responses and non-responses.
•
If the target population is the general public, online survey is most
appropriate for niche studies where respondents can hardly be reached by
other methods - examples include studies of sexual orientation.
•
Online survey gives more flexibility to respondents in terms of interview
environment and time commitment. This may be important for concert
goers, public demonstrators, practitioners at work, and so on.
•
Online surveys are limited by target respondents’ computer literacy,
online habit, and interest in the survey, especially when the study is not
aided by personal contact and other conventional approaches.
New Development – PopVote
PopVote (http://popvote.hk)
3.23 Civil Referendum
28
PopVote (http://popvote.hk)
3.23 Civil Referendum
Background
- POP held a “civil referendum” on March 23 & 24, 2012 to echo with the
fourth Chief Executive election held on March 25, with the mere goal to
let the general public vote via the civil referendum in order to express
their support towards different candidates.
Objectives
1) to integrate with the results of public opinion surveys to form a
comprehensive reference for the public and the election committee,
2) to construct a civil society by promoting civil participation, and
3) to demonstrate the electronic voting system.
29
PopVote (http://popvote.hk)
PopVote (http://popvote.hk)
Polling station
Independent
voting area
Independent
voting area
Voting
Machine
Voting
Machine
Voting
Machine
Voting
Machine
Voting
Machine
Reg.
Machine
Reg.
Reg.
Machine
Reg.
Machine
Reg.
Entrance
Machine
Queue
Queue
Machine
Queue
Independent
voting area
Queue
Independent
voting area
Queue
Independent
voting area
PopVote (http://popvote.hk)
 Total number of voters: 222,990
 Onsite voters: 85,154
 Offsite website voters: 66,005
 Offsite mobile app voters: 71,831
PopVote (http://popvote.hk)
New Development – PopCon
34
PopCon (http://popcon.hk)
An online platform to interact with the younger
generation, at http://popcon.hk/
35
PopCon e-Platform
CE Election Guessing Game
36
PopCon (http://popcon.hk)
Other Guessing Games
CE Primary Election
Guessing Game
CE Nomination
Guessing Game
Taiwan Presidential Election
Guessing Game
37
POP Site Resources
http://hkupop.hku.hk
38
Five Lines of POP Development
1) Standard conventional surveys: Constant enhancement of
content and methodology, with a high degree of IQ
2) Deliberative polling: Introduction of quality deliberation in
opinion formation and expression, via DF and DP
3) Mass engagement projects: Development
referendums and electronic voting
of
civil
4) Niche engagement projects: Especially with young people on
electronic platforms
5) International projects: Like WAPOR 2012 Conference, World
Public Opinion, Greater China Region projects
Basic groundwork development: WebCATI,
WebCAPI using mobile devices, etc.
WebOMR,
39
Three key websites of POP
PopSite for conventional polling studies –
http://hkupop.hku.hk/
PopCon for online engagement –
http://popcon.hk/
PopVote for civil referendums –
http://popvote.hk/
40
Q&A
Thank you!