Transcript Civic Engagement or Token Participation? Perceived Impact of the Citizen Review Panel Initiative in Kentucky Valerie Bryan, CSW, PhD Blake L.

Civic Engagement or Token Participation?
Perceived Impact of the Citizen Review Panel
Initiative in Kentucky
Valerie Bryan, CSW, PhD
Blake L. Jones, LCSW, PhD
University of Kentucky College of Social Work
Training Resource Center
Objectives for this Presentation
 Describe Background
and Context of
Evaluation Study of
Kentucky’s CRPs
 Describe Methodology
and Results of Study
 Give you some “News
You Can Use” in your
state
Citizen Review Panels in Kentucky
 Three free standing citizen review panels established in
1999. Two more Panels added since then. At the time of
this study, all five regional panels were active, ranging
in size from 9 to 17 members.
 In 2001, The Cabinet contracted with the University of
Kentucky College of Social Work Training Resource
Center to provide technical assistance and
administrative support to CRPs in their efforts.
 Able to do a more comparable analysis across regional
Panels
Prior Studies on Citizen Participation
 To date, only two recent studies have been conducted
specifically on CRPs, both with methodological
limitations .
 Other studies have been conducted pertaining to citizen
advisory boards involved in a wide range of initiatives
(mental health, law enforcement, environmental cleanup
efforts).
 Findings suggest that citizen advisory boards are most
effective when they have clearly communicated goals
and objectives and are given access to needed
information to make decisions (Zander, 1993; Callahan,
1999).
Other Important Factors
* top management commitment to the
public participation process
* two-way communication and
education;
* interactive and iterative public
participation; adequate resources;
* development of provisional trust
between agency and public;
* giving priority to trust building
actions;
* openness of the agency (Peelle,
Schweitzer, Munro, Carnes, & Wolfe,
1996)
Evaluation Process and Design
 Evaluation Request was made to the Cabinet for Health and
Family Services
Research Questions
 1.) How effectively do citizens review panel members perceive
their efforts to be in positively affecting child protective
services delivery?
 2.) How well do panel members work together collaboratively
toward their federally mandated goals?
 3.) How effectively are the citizen review panels and the child
welfare agency communicating with each other?
 4.) How responsive is the Cabinet to the panels’
recommendations given in their annual reports?
Research Questions
 5.) How prepared are panel
members to carry out their
mandated duties?
 6.) What do state child welfare
agency officials perceive the
role of the panels to be in
developing and enacting child
protection policies?
 7.) What can the panels do to
further assist the child welfare
agency in developing and
implementing effective child
protection policies?
Research Design
A multi-method evaluative
design was developed,
centered around the
investigation of these
questions, including
interviews with Cabinet
personnel, focus groups
with citizen review panels,
a survey of citizen review
panel members, and
content analysis of CRP
reports and Cabinet
responses.
Focus Group Procedures
 Five focus groups with citizen review panels. A total of
34 panel members participated
 Panel members were asked nine questions related to the
evaluation aims.
 Notes were taken by a practicum student in attendance
while the researcher also wrote the responses on an easel
board visible to all participants.
 These notes were qualitatively analyzed for the
identification of primary themes.
Survey Development and Procedures
 Survey instruments developed from instrument used in previous
study
 Three subscales measuring 1) information flow between agencies,
2) group cohesion, and 3) self-governance were embedded in the
instrument.
 The group cohesion subscale was adapted from the Group
Cohesion Scale-Revised, developed by Treadwell, Lavertue, Kumar,
& Veeraraghavan (2001), and the self governance subscale was
adapted from the Governance Effectiveness Quick Check (Gill,
Flynn, & Reissing, 2005).
 The wording of questions was adapted to more clearly apply to
citizen review panel members
 The survey was distributed to all 81 panel members via CRP listserv
email
Survey Procedures, cont…
 Two questions were added to assess
panel members’ perceptions of
adequate community
representation on their panels, a
requirement of the federal mandate.
 Two open-ended questions asked
respondents to give feedback on the
three largest obstacles to citizen
involvement in child welfare and
three recommendations for
improved partnership with the
Cabinet.
 Distributed to 81 members via
email/ A total of 34 panel members
responded, with 30 completing all
items (42% response rate).
Cabinet Personnel Interview
Procedures
 Interviews were conducted with
Cabinet liaisons, Senior Regional
Administrators (SRAs) and state
headquarter personnel (Central
Office).
 Fifteen interviews with Cabinet
representatives were completed, in
which they were asked questions
about their perceptions of the
impact of CRPs, the purpose of
the mandate which established
CRPs, barriers to citizen
involvement in public child
welfare, and recommendations for
future involvement.
Content Analysis Procedures
 Annual reports prepared by the panels and written
responses were available from 2003 through 2006. This
process was conducted from October through November
2006.
 Content analysis of panel annual reports to the Cabinet
and the Cabinet responses to these recommendations was
conducted in order to describe and assess the quality of
communication exchange between the Cabinet and
CRPs.
Content Analysis
 We wanted to know:
1) what activities were conducted,
2) what recommendations were
developed from those activities,
3) how recommendations were
addressed,
4) to what extent responses from the
Cabinet reflected an adequate
recognition of problem areas
identified by the CRP reports, and
5) what, if any actions or strategies
were planned to implement
appropriate and necessary changes.
Synthesized Evaluation Findings
How effectively do citizen review panel members
perceive their efforts to be in positively affecting child
protective services delivery?

Panel members’ perceptions regarding their effectiveness in making a positive impact
upon service delivery were mixed.

Panel members view themselves as primarily a source of support to the Cabinet and to
the Cabinet’s goals, by being expert community partners, advocates, and neutral
observers of the child welfare system.

CRP members perceived themselves as more effective when working at the local level
on community issues rather than statewide policy matters.

Panel members identified communication problems and a lack of authentic partnership
with the Cabinet, as well as resource and membership recruitment issues, as obstacles
to reaching their full potential.

Responses to suggestions for more effective citizen involvement indicated that panel
members would prefer a more clearly defined purpose and scope of work, and reemphasized the need for improved communication with the Cabinet.
Synthesis of Findings, cont…
How well do panel members work together
collaboratively toward their federally mandated goals?
 CRP members reported working together very successfully toward common
goals in their online survey responses.
 Panel members who participated in the survey experienced few or no
difficulties in working together toward common goals, and that most panel
members perceived their CRPs to be cohesive workgroups.
 Survey items inquiring about panels’ capabilities to self-govern were also
responded to positively, suggesting that panel members perceive themselves
as good managers of CRP initiatives and activities.
Synthesis of Findings, cont…
How effectively are the citizen review panels and the
Cabinet communicating with each other?
 Findings from all sources suggest that effective communication between the
Cabinet and CRPs presents a serious obstacle needing attention.
 Panel members identified communication problems as a primary obstacle to
fulfilling their duties effectively; all panels wanted to improve the quality
and frequency of communication.
 Some offered that this could be accomplished through the Cabinet liaisons
who attend panel meetings.
 SRAs and liaisons echoed these concerns about communication by
recommending that more experienced liaisons be appointed to all panels,
who could serve as the key intermediary between the Cabinet and the panels
more effectively.
Synthesis of Findings, cont…
 Most survey respondents disagreed with the statement, “Citizen input is
sought by the child welfare system early in the policy-making process,
before decisions are made,” indicating that the panel members who
participated in the study did not perceive themselves to be true partners
with the Cabinet.
 As a group, they do not believe that they have actual influence upon
Cabinet policy makers, which likely affects how effective they perceive
their efforts to be.
 A primary focus group theme which emerged was the need for Cabinet
accountability, most notably with regard to the annual reports.
 A few Central Office interviewees expressed concern with the quality of
recommendations CRPs offer in their annual reports, however, suggesting
that the panels at times make recommendations without really
understanding what the process of change may entail.
Synthesis of Findings, cont…
 Despite these communication
difficulties and
misunderstandings, Central
Office interviewees also
recognized that the Cabinet
needs to respect the panel
members’ efforts.
 One respondent stated that the
panel members should be “at
the table” in the decision
making process and they should
offer thoughtful input, with the
understanding that the Cabinet
must make the final decisions
about policy and practice
changes and improvements.
Synthesis of Findings, cont…
How responsive is the Cabinet to the panels’
recommendations given in their annual reports?

Findings from focus groups, SRA and liaison interviews, survey results and content
analysis indicate that the Cabinet could improve the quality of its written response to
CRP reports.

Many panel members criticized the Cabinet’s responses as merely reframing or
restating the panel recommendations without directly acknowledging if or how they
may address the identified issue.

SRAs and liaisons interviewed also expressed uncertainty about the level of
cooperation between Central Office and the panels.

Though content analyses of the panels’ and the Cabinet’s written reports revealed
examples of effective communication, several panel recommendations were made
without adequate justification and context.

Some of the Cabinet responses failed to mention any potential actions they might take
to address the recommendation.

This problematic communication may be negatively influencing the panels’ and the
Cabinet’s perceptions of each other and the role the panels can fulfill in improving
child welfare practice and policy.
Synthesis of Findings, cont…

How prepared are panel members to carry out their
mandated duties?
Findings from focus groups, survey results, and Cabinet interviews
pertaining to the level of CRP member preparedness to carry out their duties
were mixed.
 Many panel members identified the need for more members who will
regularly attend meetings, and the need for more representative
membership. A need to retain experienced members was also identified.
 Several said they would like to see former clients become panel members
but they also voiced concerns over several barriers to their recruitment and
retention. SRAs and Central Office interviewees concurred.
 SRAs recommended recruitment of new members who were not child
welfare professionals, including former clients, to attain a more
representative perspective.
Synthesis of Findings, cont…
 Central Office interviewees were concerned with panel members’
lack of knowledge regarding the bureaucratic nature of the child
welfare system and the unrealistic nature of some CRP
recommendations.
 The federal mandate which established the panels was criticized
by all sources as a barrier to the panels effectively carrying out their
duties, citing its broad, vague scope, while only requiring panels to
meet quarterly.
 Cabinet interviewees thought it was impossible for panel members
to effectively meet the intent and purpose of the mandate as it is
currently written.
 Because of this, it is not clear what roles a citizen review panel
member must, can, or cannot fulfill to successfully adhere to the
mandate, or what activities are sufficient to meet the purpose of the
mandate.
Synthesis of Findings, cont…
What do Cabinet officials
perceive the role of
the panels to be in developing
and enacting child
protection policies?

Cabinet interviews indicated that they
view the citizens review panel
members’ primary roles as those of
“objective third party,” agency
advocate, staff support, and community
partner.

Most SRAs and liaisons viewed the
evaluative role the CRPs carry out as
valuable, and helped to offset the
agency’s “tunnel vision” about policies
and practices.

Interviewees frequently emphasized the
support and advocacy CRPs provide to
the agency.
Synthesis of Findings, cont…
 Central Office valued their ability to serve as a neutral group who could
view the child welfare system “with fresh eyes,” a strength that helps
services and practices to evolve.
 Some did recall, however, past difficulties and misunderstandings
encountered when CRP members misinterpreted the boundaries of their role
or brought personal agendas into their CRP efforts.
 Like others, Central Office interviewees identified the most beneficial work
of the CRPs occurring at the local level.
What can the panels do to further assist the Cabinet in
developing and implementing effective child protection
policies?
 Cabinet interviews indicated that they do not view the role of CRPs as that
of policy developers or policymakers, but rather as expert community
partners who can provide an objective viewpoint.
Synthesis of Findings, cont…
 It should be noted, however, that the mandate’s language indicates that
panels should examine and evaluate existing state and agency policy; in so
doing, it would be expected that recommendations they offer would impact
the shaping of policy.
 Again, the broad language of the mandate may be the source of much
miscommunication, role confusion and misinterpretation of CRP
responsibility.
 Cabinet interviewees indicated that CRPs could help front line workers the
most by advocating for human and technological resources.
 One Central Office interviewee suggested that because child welfare issues
exist beyond the Cabinet’s scope, systemic issues need to be investigated in
a broader manner, including other involved elements of child welfare, such
as the family courts, public health, and community-based agencies who
work with children and families.
Synthesis of Findings, cont…
 Central Office personnel also
wanted to see solutions offered
more frequently to the problems
CRPs identify in their reports.
 They also recommended that
CRP members take on the role
of intermediary between
different parties and providers,
explore public relations
endeavors, and educate
legislators about DCBS work,
in the hopes of securing more
workers and funding for
initiatives.
Recommendations
Analysis and synthesis of findings from all sources resulted in the
following recommendations:
 Work to create and pass state-level legislation which would clarify the
purpose, role, and duties of citizen review panels, in order to identify the
scope of their efforts and in what ways the citizens review panels, the
Cabinet, and other partners responsible for child welfare within the state are
accountable to each other as a result of the federal mandate.
 Improve the level of specificity within the Memorandum of
Understanding between the Cabinet by including a feasible protocol for
ongoing exchanges of information which will facilitate the ability of all
parties to follow through with action plans and to keep all informed about
the result of recommendations throughout the change process.
Recommendations, cont…
 Improve communication between all levels of the Cabinet and the
panels through first, adequate preparation and orientation of
appointed liaisons who will serve in a consistent manner as an
intermediary between the agency and the panels. Secondly, propose
this issue as an open topic of discussion among involved parties to
identify mutually acceptable ways in which ongoing, constructive
communication can be achieved.
 The Cabinet, the CRP Program Coordinator, and the panels should
consider developing a unified, strategic plan to recruit panel
members from a broader base of applicants to assure adequate
representation of each region’s citizenry on the panels.
Recommendations, cont…
 The citizen review panels and the Cabinet should work to identify
areas together in which panel members may lack knowledge or
training regarding child welfare issues, policies or practice, and the
Cabinet should offer ways in which panel members can be educated
in these areas at no cost to the members, in order to improve the
quality of recommendations offered by all panels every year. The
CRP Program Coordinator in partnership with liaisons, SRAs and
chairpersons can help to identify areas of need.
 The Cabinet should seek to improve the quality of responses
offered to each recommendation presented in the panels’ annual
reports, addressing all items offered with at the least, a reasoned
explanation as to why recommendations are not feasible or timely
when appropriate. Action plans, prospective timelines, and updates
as they become known should be offered to the panels in response to
recommendations which the Cabinet expects to act upon. The
recommendation for a follow-up protocol as agreed upon within the
MOU as listed above should support this process.
Lessons Learned
 It was clear that both the Cabinet and the CRPs shared
responsibility for the difficulties encountered in working
together.
 The results support prior study findings on the
necessary conditions for effective citizen participation
 The obvious communication problems identified led
over time to deeply rooted resentments experienced by
the CRP members. This led to a sense of powerlessness
to change the situation and to make meaningful
contributions to child welfare policy and practice.
Lessons Learned
 As one panel member stated,
“The reports and responses
should be the start of a
conversation, not the end.”
Making communication a
process rather than an event,
and adhering to a systematic
exchange of information is the
key change needed.
 In order to address the offered
recommendations,
communication problems must
first be repaired.
References
Callahan, K. M. (1999). Citizen participation: The utilization and
effectiveness of citizen advisory committees in local government.
Unpublished doctoral dissertation, Rutgers University.
Gill, M., Flynn, R., & Reissing, E. (2005). The Governance SelfAssessment Checklist: An instrument for assessing board
effectiveness. Nonprofit Management & Leadership, 15 (3), 271294.
House Report 104-081, 104th Congress, Second Session. (1996).
Jones, B. (2004b). Citizen participation in public child welfare: A multistate study of citizens review panel members’ perceptions of
effectiveness. Dissertation Abstracts International, 65 (07), 2765A.
(UMI No. 3140108)
References, cont…
Jones, B.L., Litzelfelner, P. & Ford, J.P. (2003). The value and role of citizen review
panels in child welfare: Perceptions of citizens review panel members and child
protection workers. Child Abuse & Neglect: The International Journal, 27 (6),
699-704.
Peelle, E., Schweitzer, M., Munro, J., Carnes, S., & Wolfe, A. (1996). Factors
Favorable to Public Participation Success. No. AC05-96OR22464, report
submitted to the US Department of Energy.
Treadwell, T., Lavertue, N., Kumar, V., & Veeraraghavan, V. (2001). The Group
Cohesion Scale-Revised: Reliability and validity. International Journal of
Action Methods: Psychodrama, Skill Training, and Role Playing, 54 (1), 3-12
U.S. Department of Health and Human Services (1998). Establishment of the
Citizen Review Panel Requirement Under the Child Abuse Prevention and
Treatment Act (ACYF-PI-CB-98-01). Washington, DC: Author on Team
Functioning: An Empirical Investigation. Human Factors, 38:87-100.
Zander, A. (1993). Making boards effective: The dynamics of nonprofit governing
boards. San Francisco, CA: Jossey-Bass.
This presentation based on….
Bryan, V., Jones, B.L., Allen, E. & CollinsCamargo, C. (in press, Child and Youth
Services Review). Civic Engagement or
Token Participation? Perceived Impact of the
Citizen Review Panel Initiative in Kentucky