Transcript UFE STEP Applied to Indigenous Case

Utilization Focused Evaluation Case Studies:
UFE in Multijurisdictional/Tribal Contexts
Session 757 – Room 211 C/D – 8:00 AM to 9:30 AM
Presenter: Nicole Bowman (Munsee/Mohican)
President, Bowman Performance Consulting
UFE’s Practical Application in Indian Country…..
UFE: It’s not just about the flowchart…..
Or the checklists…..
…..or the Pretty Faces of
those of us who use it…..
BTW--ELDER? That is a good thing in my community…
UFE is so much more!
•Process
•Relationships
•Analogies
•Stories
•Responsive
•Responsible
•Meaningful
•Useful
•Complex
•Nuanced
•Integrated
•And so on…..
•Nii allagomahteet (all my relations;
all things are related)
•Wunj iin Daaptoonaakanum Niiloona
(stories of our local history, usefulness,
and relevance to community)
•Wiciwinknewuk (four sacred directions,
four sacred teachings, medicine wheel)
•UFE has both traditional and
contemporary connections
Responsibility & Relations
• UN Declaration of Indigenous Rights encourages States to
comply with and effectively implement all their obligations as
they apply to indigenous peoples under international
instruments, in particular those related to human rights, in
consultation and cooperation with the peoples concerned
• Emphasizing that the United Nations has an important
and continuing role to play in promoting and protecting
the rights of indigenous peoples
• How does the UN Declaration inform my work and the
choices I use for evaluation design and related activities?
• Remember the 4 R’s in Indian Country: Respect,
Relationships, Reciprocity, Responsibility – UFE allows
evaluators to incorporate all of these constructs
UN Declaration of
Indigenous Rights Facts:
•Adopted by UN in 9/2007
•144 nations voted in favor
•11 abstentions
•4 nations voted against
•Guess who they were?
•All 4 nations eventually
adopted the Declaration
•Guess who was last?
•Guess what year it was
finally adopted by this
nation?
•Why does this matter???
Grounding Indigenous Evaluation
The UN Declaration on the Rights of Indigenous Peoples:
• Pulls aside scales of blindness that a culture of inequity has bred
• Provides that its recitation of fundamental rights “constitute the
minimum standards for the survival, dignity and well-being of the
Indigenous Peoples of the world.”
• The Declaration’s preamble echoes the Universal Declaration of
Human Rights in proclaiming it “as a standard of achievement
to be pursued in a spirit of partnership and respect.”
• Obliges all governments to examine how they will come to
evolve systems that recognize and embrace the “minimum
standards” set forth in the Declaration
• 46 articles with subsections are in UN Declaration
• Articles 31 & 32 are the foundation for Indigenous knowledge &
intellectual property rights protections
•Robinson, 2010. “Minimum Standards: The UN Declaration on the Rights of Indigenous People”
•UN Nations Permanent Forum on Indigenous Issues Website:
http://social.un.org/index/IndigenousPeoples/DeclarationontheRightsofIndigenousPeoples.aspx
UN Indigenous Declaration:
Selections from Article 31
•….Indigenous peoples have the right to maintain,
control, protect and develop their cultural heritage,
traditional knowledge and traditional cultural expressions,
as well as the manifestations of their sciences, technologies
and cultures, including human and genetic resources,
seeds, medicines, knowledge of the properties of fauna and
flora, oral traditions, literatures, designs, sports and
traditional games and visual and performing arts.
•…..They also have the right to maintain, control, protect
and develop their intellectual property over such cultural
heritage, traditional knowledge, and traditional
cultural expressions.
UN Indigenous Declaration:
Selections from Article 32
•…States shall consult and cooperate in good faith with the
indigenous peoples concerned through their own
representative institutions in order to obtain their free and
informed consent prior to the approval of any project
affecting their lands or territories and other resources,
particularly in connection with the development, utilization or
exploitation of mineral, water or other resources.
•…States shall provide effective mechanisms for just and
fair redress for any such activities, and appropriate measures
shall be taken to mitigate adverse environmental, economic,
social, cultural or spiritual impact.
•UFE has been an effective mechanism in Indian Country….
But remember meanings, perceptions and feelings that
came before & extend beyond the UN policy document
The Case
• 5 year federally funded project at over $5,000,000
• Partnership between a state university & Tribal college
• Education & STEM workforce development focus
• Major grant activities/purposes:
• Professional development for K-12 STEM teachers through
summer academies, academic year trainings, and research
internships with university faculty
• K-12 curriculum development in eco-diversity, bio-energy, and
sustainability aligned to cultural, contextual, and workforce needs
• Curricular and professional development incorporation of
private sector, traditional community, and public/tribal schools
• Student activities for STEM career interest, engaging/relevant
STEM curriculum and school experience, and after school clubs
or internships in STEM
• Synergistic and collaborative K-16+ activities designed to
strengthen systems supporting STEM education
UFE Steps 1-4 in Indigenous Context
UFE STEP
1. Assess/build
organizational
readiness
Applied to Indigenous Case
IHE IRB; Tribal IRB; funding for evaluation available; history of
IHE organizational experience with Tribes/Tribal college and
vice-versa
2. Assess/enhance Strong practical knowledge, cultural/interpersonal
evaluator readiness/ competence, reflective practice, and project management;
competence
less skilled on “rigor” (AKA generalizability/use of nationally
normed instruments) as IHE wanted it but understood that this
wouldn’t be appropriate or realistic given the n size and
context; challenges and adaptations are continual
3. Identify,
organize, & engage
primary users
Initial evaluation development team to co-construct evaluation
comprised of evaluators, IHE, and Tribal; design and
instruments are member checked; data is shared as a process
for building capacity, documenting process, and impacts
4. Situational
analysis with
primary users
Context and relations; staffing changes; sample size; relation
building; participation in program; etc. all impacted evaluation;
implementation was impacted too which delayed evaluation
UFE Steps 5-8 in Indigenous Context
UFE STEP
Applied to Indigenous Case
5. Identify/prioritize
primary intended
users by determining
primary purposes
Evaluation purposes were established based on major
grant activities/treatment; other purposes developed based
on local needs and use; presentations & work groups on
evaluation capacity/purpose completed; developmental,
knowledge generation, and accountability were primary
uses for data
6. Consider/build in
process uses if/as
appropriate
Evaluation updates and work groups built into major activity
agenda’s, leadership team, and community team sessions;
instruments and methods continue to be discussed and
modifications are made as needed
7. Focus primary
evaluation questions
5 grant goals with objectives are noted however “best” data
was with the cultural / academic “process” for building
curriculum and with teacher PD impacts in classrooms
8. Check that
fundamental
evaluation inquiry
areas are addressed
Outcomes evaluation helps to answer to what extent goals
were met; process evaluation helps to answer the
challenges and supports for implementation; and new
program/treatment = developmental use for evaluation
UFE Steps 9-12 in Indigenous Context
UFE STEP
9. Determine what
intervention model/theory
of change is being
evaluated
Applied to Indigenous Case
PD process for curriculum development and training
institutes utilizing a socio-cultural process with elders,
academics, teachers, community members, and
students = relevance in STEM curriculum and stronger
impacts in classroom and STEM career pathways
10. Negotiate
Methods and instrument choices challenging in year 1;
appropriate methods to
appropriateness for culture/context/N size challenged;
generate credible findings modifications made in year 2 and 3; ongoing process
11. Make sure intended
users understand
controversies and
implications
Mix method data will be produced; frequent visits back to
the evaluation plan and questions with PI is warranted;
keeping costs down and methods/instruments changes
less frequent is helpful; R vs. E is always an issue
12. Stimulate use of
findings
Abstract discussions need concrete frameworks;
expectations needed to be better aligned; time up front
for data analysis needed to be increased; use/value of
data by Indigenous users stronger than IHE users
UFE Steps 13-16 in Indigenous Context
UFE STEP
Applied to Indigenous Case
13. Gather data with
ongoing attention and
use
Management of data collection and reporting needed to be
clarified; quarterly PI meetings set up; creating templates
and expectations for reporting at 2-3 events annually was
developed; continual reminders and requests for
data/sticking to the plan are needed
14. Organize/ present
data for
interpretation/use by
primary users
Re-connecting in reports/presentations (use of visual
organizers, data maps, and summary eval tables used);
audience is key to reports so academic vs. summary vs.
community reports or data chats are utilized; requests for
feedback on emerging data, initial findings, and reports is
always welcomed in a non-defensive and positive manner
15. Prepare reports to
facilitate use,
disseminate findings,
and expand influence
Presentations/executive summaries for leadership team
and curriculum team; talking circles for community and
elders; formal evaluation report to funder and PI annually;
alternative explanations needed for N & culture/context
16. Follow up with
primary users to
enhance use
Findings and recommendations discussed; annual self-eval
and monitoring of recommendation use, implementation,
and completion; eval/project/budget modifications made
UFE Step 17: Meta-evaluation
UFE Step 17: Meta-evaluation
of Useof Use
•Accountability
•Learning
•Improvements
•Accountability in many forms:
• Participants
• Community
• PI Team
• Partners
• Funding Agency
• Evaluation Agency (BPC) for
project, time, budget management
and cultural, contextual, professional
standards
• Next 7 generations…….
UFE Step 17: Meta-evaluation of Use, con’t.
• Traditional vs. contemporary
uses and impacts for Indigenous
participants—delicate balance
•Reflective practice for what the
evaluation did well and what didn’t
go as planned
• Analyze what evaluation has
control over and negotiate the rest
with others
• Set evaluation priorities and
recommendations for coming year
• Monthly monitoring (self) of
improvements and quarterly with
PI team
Indigenous evaluation requires walking
and working between/across the worlds
(multiple & dynamic contexts)
Community and/or
Intended Participants
*Evaluation Provider
And/or Project Staff
Funding Source
UFE is a framework, process, and tool that
helps us make this a more honorable walk…
Use Strengths / Asset Based
Approaches
Much diversity exists within
Indigenous communities…..
Contextual
Family/
Individual
Indigenous
Diversity
Socio-economic
Cultural/
Linguistic
Concluding Thoughts….
• Create a shared space that is negotiated and respectful
• Linda’s story of “the scientists felt vulnerable because
their knowledge wasn’t being acknowledged…”
• UFE says cultural competence is interwoven with our
work; everyone brings culture to the project so address it
• Make evaluation work responsive to local needs and
useful for the next 7 generations
• UFE’s design and use of meta-evaluation in Indigenous
contexts helps create evaluation work that is helpful in the
present as well as builds capacity for the future
• Nii allogomahteet and laapi uchkneewal
• Relationships are built and sustained with trust,
recognition/value of each other, and authenticity
• Leave good “moccasin tracks” during and after the project
ADD TEXT HERE
I hope my Indigenous perspectives on
UFE support your evaluation journey…
Anushiik & Laapi uch Keneewal!
Future Contact: Nicole Bowman (Mohican/Munsee)
E-Mail: [email protected]
Phone: 715-526-9240
URL: www.bpcwi.com
www.bpcwi.com