Transcript Document
Quality feedback: Validation of TDI Standard Table submitted in 2005 (2004 data) Sandrine Sleiman, Xavier Poos September 2006 TDI and Quality Assurance • Quality assurance at EMCDDA • TDI data under … I) Quality Assurance in data reporting • Tool for EMCDDA • Tool for NFPs and their partners • To improve quality of data collection instruments • To improve quality of data sets • To reduce workload in validation phase • To improve credibility of EMCDDA and NFPs As an example : • EMCDDA deals with a vast amount of information: 2002: 493 data tables 2004: 723 data tables 2005: 759 data tables and 3111 pages of NRs Quality Management at the EMCDDA EMCDDA – Data input, internal data management and feedback Data submission National data providers Quality Control Quality Assurance Validation Quality feedback Validation Protocol Guidelines Protocols Guidelines Data collection Data collection instruments instruments National data providers Quality Control and standard tables • Compliance with guidelines : • Deadline • Methodology instruction • References (bibliography, abbreviation,…) • Reliability of data : • Calculation • Consistency with previous data II) Overview of implementation level of TDI indicator (**) n=26 14 14 12 10 7 8 6 2 3 4 2 0 Good implementation Most recent data Data quality from or before 2003 limited/not in line with EMCDDA definitions No implementation (**) Implementation Profile for selected indicators (profiles sent to NFP for check/approval early May 2006) a) Respect of deadline • 1st table: 6th sep 2005 last table: 25th jan 2006 • 17 countries have matched the deadline for uploading • 2 countries have uploaded the table with delay Upload of 2004 TDI table (N=19) 24-03-2006 02-02-2006 14-12-2005 25-10-2005 05-09-2005 17-07-2005 28-05-2005 b) TDI respect of deadline in comparison … Respect of deadline (TDI, ST07, ST08)* 02-02-2006 13-01-2006 24-12-2005 04-12-2005 14-11-2005 25-10-2005 deadline 05-10-2005 15-09-2005 26-08-2005 06-08-2005 17-07-2005 TDI ST07 ST08 c) Control of double counting (n=19) • Double counting is reported to be checked in 13 countries • Double counting is not checked in 6 countries d) Rating of overall clarity/understanding of TDI info in national reports Overall level of details (n=27) Clarity and understanding (n=27) 1 0 2 0 8 18 25 Insufficient Key information covered Too much details Insufficient Sufficient Good Very good II) Main problems during validation period Types of queries • Approximately 40 initial mails have been sent for data clarification 17% 46% • 81 queries (query = each single question sent to NFPs regarding data uploaded) 37% Quantitative Qualitative Inconsistencies III) Common problems observed • Quantitative information • 46% of all queries are related to missing values, errors of calculation, not understandable data, wrong data reported and/or reported in the wrong place. • Qualitative information • 37 % of all queries are related to clarifications of provided data, abbreviation used unclear, term/definition unclear, references provided but unclear, methodological info provided but unclear. Other problems: • Not available values ≠ missing • Inconsistencies (discrepancies between years, between STs and NRs) • Upload interface Most of those errors can be avoided IV) Future developments “Quality should include reducing surprises by highlighting realities in time to prepare for possibilities…” • Foster development of quality control and assurance at NFP level • More systematised quality control (FONTE) • Further consolidation of EMCDDA quality reports THANK YOU FOR YOUR ATTENTION [email protected]