Transcript revised OSDV tool

Revised OSDV Tools and
Processes
LFA PSM expert workshop
28-30January 2014
Session objectives
1. Explain why revision was necessary
2. Revised OSDV Templates – What has
changed?
3. Clarify changes on the PSM aspects of the
OSDV
4. Explain the changes to the OSDV process
and requirements
5. Share some field experiences on OSDV
implementation
Session structure
1. Presentation - the revised OSDV templates
2. LFA presentation - piloting the revised tool
3. Q&A
Why the revision?
•Simplify the tool
•Respond to feedback from LFA
•Reduce some redundancy - Streamline with
other tools
•Ensure internal consistency - analysis,
reporting and interpretation
•Include provision for Quality of Services
verification
What has changed?
• Modified cover page to provide a snap shot of Data
Quality Rating and top Data Quality and PSM issues
for Grant
• Revised M&E and PSM system assessment
questionnaires
• Excel tool scaled down into a worksheet for facilitating
the assignment of Data Quality Ratings for indicators
• Revised the logic for assignment of Data Quality Rating
• Expanded PSM assessment - included quality of
services questions
• Revised the related tools in OSDV Package
OSDV tool revisions (1) – M&E Assessment
Previous Version
1
2
Area for Improvement
Key findings
• Key areas for follow
embedded in
up needed to be
text and
more succinct and
summarized only easily visible
at the end of the
report
Separate form
used for
identifying
indicators and
site selection,
which was then
copied into final
report template
• Required re-entry
of data that had
been captured in
proposal form
• Required
completion and
storage of two
documents
Revised Version
Modified cover page to
provide a snap shot of
DQ ratings per indicator
and highlight top DQ and
PSM issues
Integrated indicator and
site selection into the
main report template
OSDV tool revisions (2) – M&E Assessment
Previous Version
Area for Improvement
Revised Version
M&E systems
were assessed at
facility,
intermediate and
central levels, with
raw data provided
for each site
• Central level
questionnaire
duplicates
Implementation
Assessment tool
• Detailed information
rarely used for
decision-making
Simplified M&E system
assessment
questionnaires.
- A single consolidated
M&E System
Assessment required in
OSDV Report – based
on LFA analysis and
synthesis
Excel addendum
included detailed
information for
each indicator and
each site
assessment
Comprehensive data
was time-consuming
to analyze and not
systematically used
for decision-making
Excel tool scaled down
into a worksheet to
calculate error margins
and assign Data
Quality Rating.
OSDV tool revisions (3) – PSM section
Previous Version
7
Area for Improvement
Revised Version
OSDV tool included a
limited number of PSMrelated questions
mainly focused on
inventory data
verification
Site visits offer
Revised and expanded PSM
opportunities to collect
section, with quality of
data on facility-based PSM services questions
issues including
consumption and patient
data
No information
available on the
alignment of data
between facility register
and pharmaceuticals &
health products data
Site visits provide key
opportunity to assess the
level of coherence
between patient and
health products data
PSM section includes
question on the link between
facility registers and
pharmacy data. e.g.: health
products and patient
information verification
PSM systems
assessments and
findings were not
consolidated
Detailed information was
not really used for
decision-making
Require consolidated PSM
assessment in OSDV report
- based on LFA review
analysis and synthesis of
findings.
OSDV tool revision (4) - Data Quality Rating
Data Quality Rating based on Overall Average Weighted Error
Margin and the % average report availability for the indicator
• Overall Good Data Quality: If the Overall Absolute
Average Error Margin is less than 10% AND the average
% reports availability for the indicator is equal to or more
than 90%
• Minor Data Quality Issues: If the Overall Absolute
Average Error Margin is between 10% and 30% AND %
average reports availability is equal to or more than 70%
• Major Data Quality Issues: If the Overall Absolute
Average Error Margin is over 30% OR % reports
availability is less than 70%
OSDV tool revision (4) - Data Quality Rating
Extreme cases with Overall Average Error Margin >30% or
<-30%) or overall average availability of reports <70% are
flagged as follows:
• Major– Over reporting: If the Overall Average Error
Margin is more than 30%
• Major – Under reporting: If the Overall Average Error
Margin is less than 30% and or % average report
availability is less than 70%
Note: If the Error Margin is equal to or over 30% at any
level, the rating is automatically ‘Major Data Quality
Issues’. A rating of ‘Major Data Quality Issues’ also
results if the ‘Overall Absolute Error Margin’ is over 30%
or the ‘% average report availability’ is less than 70%.
The Revised OSDV templates
Report template – Visual display
Workbook - Visual display
Revised OSDV Processes & Requirements
• LFA OSDV Guidelines have been revised
• OSDV – requirement risk-based
• In case of OSDV/RSQA - ideally conducted at the
same time
• Excel template to be submitted together with the
main OSDV Report
• Expanded role for LFA PSM experts >> However
questions have been designed so that LFA
Programmatic/M&E experts are able to collect raw
data from facilities.
Expanded Role of LFA PSM Expert
• At a minimum, the LFA PSM Expert should
participate in:
- Preparing the OSDV Planning Tool
- Briefing Programmatic/M&E Expert on PSM issues
- Analysis of data for PSM-relevant sections of the
report
- Should be involved in reviewing the findings and
verifying the summary of issues and making
relevant recommendations
• For some countries (e.g. countries with multiple
PSM issues and/or HI Countries CT may require
LFA PSM experts to accompany OSDV field team
Revised OSDV Package
Revised OSDV package (available online on the
Global Fund website)
• LFA Guidelines for OSDV/RSQA
• LFA OSDV/RSQA Planning Template
• OSDV Tool Templates (Report template + Excel
Workbook)
• OSDV User Guide Form
http://www.theglobalfund.org/en/me/documents/dataq
uality/
LFA Presentation:
Piloting the revised tool
Q&A