Transcript Slide 1
ARE WE DOING ANY GOOD? ARE WE DOING ANY GOOD? Measuring, Monitoring and Evaluating Medway and Blyth Comparing the two….IMD Benchill NW 1 Hertsmere East 8414 Blyth NE 342-455 Medway SE 1181-1472 Cowpen Croft Kitty Brewster Twydall Holcombe Town % 11-18 smoking = 37% % 11-18 smoking = 43% In the past…. Health interventions usually small scale + local Evaluation often driven by need to obtain + retain funding Few lessons for wider use in policy + practice More recently…. Growing interest in community-based interventions New opportunities for exploring new evaluation approaches + outcome measurement Culture of accountability So what is Evaluation? Evaluation is the systematic collection of data that will allow a judgement to be made about the value of a programme or intervention. The evidence from an evaluation should allow practitioners to make suitable changes to the programme to make it more effective for participants. HDA 2001 So what is Evaluation? Evaluation is the systematic collection of data that will allow a judgement to be made about the value of a programme or intervention. The evidence from an evaluation should allow practitioners to make suitable changes to the programme to make it more effective for participants. HDA 2001 Assessing whether projects are effective and acceptable Nottingham HAZ 2000 Example – the FAB Fruit and Vital Veg Project Questionnaire pre and post the intervention Measured consumption, knowledge and attitudes to fruit and vegetables Results: Average fruit+veg per day Like veg Like fruit Aware of 5-a-day Pre Post 1.62 73% 94% 35% 2.30 84% 96% 80% And also…. Needs supportive teacher School needs to be ready and committed to the project Need a mix of activities including: Art and drama to help promote image of fruit and veg Tasting sessions to increase confidence in trying fruit + veg Work with parents on packed lunches Work with school cooks and caterers etc etc. Evaluations give…. Indicators and measures …of success and change …of what worked and what didn’t …and a rich picture of ‘the intervention’ Evaluation should…. Measure effectiveness, efficiency and cost benefits Measure progress towards aims and objectives – short and long term Inform and improve decision making Give credibility Inform funding decisions Give deeper understanding Develop the skills of those involved Develop a systematic evidence base .…be enjoyable and fun! Where do we start? Setting the Baseline… Identify objectives for the research – our evaluation questions Establish Baseline Position Review Identify indicators and measures, and how these are to be collected Establish current position (baseline) - using data already available or data you collect Continuous review will allow us to ensure that these measures continue to be appropriate as the programme develops. Identify Appropriate Measures and Indicators Identify Objectives of Research Components of an Evaluation BASELINE Review PROCESS EVALUATION Effectiveness of planning and delivery, acceptability, accessibility etc IMPACT EVALUATION Immediate benefits OUTCOME EVALUATION Longer-term benefits, sustainability A Formative Approach Improvements Baseline Learning Evaluation The worries…. It’s there to help us To make the project better Make evaluation the central element We all own the evaluation We’re not being assessed Sources of help Various evaluation resource packs Other projects experience Health Development Agency NICE http://www.nice.org.uk/ …and many references to other work Example – School-based project Project – Active programme promoting lifestyle education in schools (APPLES) Intervention – project team assisted schools develop action plans, trained teachers and provided resources Objective – to implement and evaluate a school based intervention designed to reduce risk factors for obesity Design – Randomised controlled trial (RCT) Sample – 10 primary schools in Leeds – half received the programme, half were a control and didn’t Output measures: Teachers evaluation of training and other input Success of school action plans Content of school meals Children’s knowledge of healthy living and self-reported behaviour The Results…. The Results…. And also…. Children who had received programme showed greater understanding of health benefits of diet and physical activity Children reported changes in behaviour Positive changes in playground activities and tuck shops Things to remember…. Plan clearly at the start Evaluation doesn’t need to be difficult or intimidating, but does need thought and planning. Build it in as a central item as early as poss! There is no ‘one best way’ to do an evaluation Include stakeholders – they will have contradictory views sometimes – but it’s worth it! Choose most suitable data collection methods for answering your evaluation questions Use Qual and Quant Document all lessons learnt Share your findings Thank you [email protected]