Statistical presentation in international scientific publications

2. A symptomatic review

Malcolm Campbell

Lecturer in Statistics, School of Nursing, Midwifery & Social Work, The University of Manchester Statistical Editor, Health & Social Care in the Community

2. A symptomatic review

Contents

• 2.1 Statistical reporting • 2.2 General (but not definitive) advice • 2.3 The structure of a paper • 2.4 Statistical errors in submitted papers • 2.5 What should we do?

26 March 2008

Statistical presentation - 2. A symptomatic review

2

2.1 Statistical reporting

Presenting statistics in scientific papers

• A scientific research paper should contain sufficient information to allow readers to 1. assess the authors’ aims, material, methods, findings and interpretation 2. decide whether their conclusions are justified 3. apply the authors’ conclusions to their own practice 4. repeat the study in their own setting, if required • Quantitative research papers show (or hide?) information statistically

– numbers, estimates, statistics, test results, p-values…

– but the reader still needs to be able to do 1 to 4 26 March 2008

Statistical presentation - 2. A symptomatic review

3

Statistical reporting in research papers

Lies, damned lies and submissions to journals

• Papers submitted to healthcare journals have a generally poor standard of statistical reporting – many papers do not have the right structure – occasionally the study design is at fault – most authors choose the correct methods to analyse their data but fall down on the description of the methods and the presentation of the results – others present sequences of p-values without explanation, inviting the reader to “guess the test” – many brush non-significant p-values out of the way – tables of findings are often inconsistent, too complicated or depend on the text for their explanation 26 March 2008

Statistical presentation - 2. A symptomatic review

4

2.2 General (but not definitive) advice

Conventions, recommendations and suggestions

• These presentations give general advice on how to present statistical findings in papers – there are common conventions, style and sense – some recommendations on good practice • There are few absolute rules – different journals (and editors) have different styles • always read the journal’s “Instructions to Authors” – it may be necessary to bend the rules occasionally

– accept that research in healthcare may not be perfect!

• Important bottom line: readability 26 March 2008

Statistical presentation - 2. A symptomatic review

5

The Good, the Bad and the Ugly

Examples of good, bad and ugly practice

• We will look at examples of all three in published

papers

– “The Good” are ideals – “The Bad” may be slightly wrong – but “The Ugly” should have been corrected before printing 26 March 2008

Statistical presentation - 2. A symptomatic review

6

2.3 The structure of a paper

Quantitative papers are usually laid out as follows

• Title: What was the study about, in a line?

• Authors: Who “performed” the study?

• Abstract: What was the study about, in a paragraph?

• Introduction: Why was the study carried out?

• Methods: How was the study carried out?

• Results: What did the study find?

• Discussion: What do the findings mean?

• Acknowledgements: Who should be thanked?

• References: What essential references were used?

26 March 2008

Statistical presentation - 2. A symptomatic review

7

IMRaD for healthcare papers

The standard structure for research papers (ICJME, 2005)

• Many (but not all) healthcare journals adopt this structure for main text of research papers – Introduction, Methods, Results and Discussion – these sections may be broken down into subsections • It reflects the process of scientific investigation – why the study was done, how it was done, what was found, what it means • though not always the order in which things were done!

– use IMRaD even when planning for a journal that uses other sections to get overall structure right

• should be easy to make minor changes to the structure

before submission 26 March 2008

Statistical presentation - 2. A symptomatic review

8

Structures in selected journals

Sections used by healthcare journals (* = structured Abstract) BMJ HSCC Midwifery J Adv. Nursing

Abstract* Abstract Abstract* Abstract* Introduction Introduction Introduction Introduction/ Background Lit review Methods Methods Methods The study

J Clinical Nursing

Abstract* Introduction/ Background Methods

Nurse Educ Today

Abstract Introduction/ Lit review Background Methods Results Discussion Results Discussion Findings Discussion Results Discussion Conclusions Results Discussion Conclusions Results Discussion Conclusions Acknowledg Refs (Vanc) Acknowledg Refs (Harv) Acknowledg Refs (Harv) Acknowledg Refs (Harv) Refs (Harv)

Refs (Harv)

26 March 2008

Statistical presentation - 2. A symptomatic review

9

Common faults with authors’ structure

How not to lay out a quantitative paper (eg Hall et al, 1998)

• The Introduction should not include any findings or conclusions • The Methods should not include any findings – information not known before the data were collected should ideally be in the Results • The Results should not repeat detailed findings already summarised in tables • The Discussion should not introduce findings not already in the Results • The References should not be in the style of another journal (ie not the BMJ!) 26 March 2008

Statistical presentation - 2. A symptomatic review

10

Methods or Results?

What goes where (eg ICJME, 2007)

• International Committee of Medical Journal Editors (2007) at http://www.icmje.org/ : – “The Methods section should include only information that was available at the time the plan or protocol for the study was written; all information obtained during the conduct of the study belongs in the Results section. ” • I would expect the Results to start with – number of participants, participation rate, details of characteristics of the participants… • Sometimes better for readability to include some “results” in the Methods • Not all papers or journals follow this! 26 March 2008

Statistical presentation - 2. A symptomatic review

11

The Bad

Lack of structure confusing

• Papanikolaou et al (2003) – Pressure ulcer risk assessment: application of logistic analysis, J Advanced Nursing 44(2), 128-136 • title misleading – they meant logistic regression • does not follow IMRaD structure – characteristics of participants in Methods section – analysis of variables used for logistic regression with percentage breakdown per group also in Methods section – Results starts with the main analysis, logistic regression • [also no sample size calculation (only 25 patients with pressure ulcers v 473 without) • inconsistent percentages and p-values of “0.00”] 26 March 2008

Statistical presentation - 2. A symptomatic review

12

2.4 Statistical errors in submitted papers

100 consecutive non-RCT peer reviews +

Yes

(Altman, 1998)

Unclear No Objective clear?

Appropriate study design?

Source of subjects? * Sample size calculation? * Satisfactory response rate? *

Methods described adequately?

Statistical analyses appropriate?

Statistical presentation satisfactory?

Confidence intervals given?

(CIs given inappropriately) Conclusions justified?

Paper statistically acceptable?

If not, could it become acceptable?

47 41 14 51 (+8) 40 83 72 83 0 49 4 89 6 25 6 0 23 0 37 0 0 49 0 6 11 3 10 63 2 53 22 86 41 11

96

1

* not always applicable 26 March 2008

+

for BMJ 1991-1993

Statistical presentation - 2. A symptomatic review

13

2.5 What should we do?

Never fear - help is at hand! – Three sources of advice

1. ICMJE uniform requirements for manuscripts – International Committee of Medical Journal Editors (2007) at http://www.icmje.org/ • eg use double spacing, pages should be numbered, start each section/table/figure on a new page 2. Journal’s instructions to authors – please read them!

3. Published reporting guidelines for different study designs – eg CONSORT, STARD, TREND, STROBE… 26 March 2008

Statistical presentation - 2. A symptomatic review

14

Guidelines for statistical reporting

Standardised reporting guidelines

• There are published guidelines for reporting certain types of study designs – cover the type of information to be reported and in what order/sections of the paper – but not the finer details of numerical reporting • refer to journals’ guidelines to authors • and more later!

• Go to the EQUATOR Network for more details – a new initiative, Enhancing the QUAlity and Transparency Of health Research Network – http://www.equator-network.org/ 26 March 2008

Statistical presentation - 2. A symptomatic review

15

The CONSORT statement (2001)

Checklist/flow diagram for reporting randomised trials

• CONsolidated Standards Of Reporting Trials – for reporting (parallel) randomised controlled trials – the statement – eg Moher et al (2001) – explanation and elaboration – eg Altman et al (2001) – http://www.consort-statement.org/ • download both documents and templates of checklist and flow diagram 26 March 2008

Statistical presentation - 2. A symptomatic review

16

The STARD statement (2003)

Guidelines for studies of diagnostic accuracy

• STAndards for Reporting of Diagnostic Accuracy – for studies comparing outcomes from new diagnostic test against reference standard – the statement – Bossuyt et al (2003a) • CONSORT adapted for studies of diagnostic accuracy – explanation and elaboration – Bossuyt et al (2003b) – http://www.stard-statement.org/ • download both documents and templates of checklist and flow diagram 26 March 2008

Statistical presentation - 2. A symptomatic review

17

The TREND statement (2004)

Checklist for reporting non-randomised trials

• Transparent Reporting of Evaluations with Non randomized Designs – for reporting comparative trials without randomisation – the statement – Des Jarlais et al (2004) – • CONSORT + theoretical basis of intervention, tests of baseline equivalence and corresponding adjustments of main analyses http://www.trend-statement.org/ • download the statement 26 March 2008

Statistical presentation - 2. A symptomatic review

18

The STROBE statement (2007)

Checklist/flow diagram for reporting observational studies

• STrengthening the Reporting of OBservational studies in Epidemiology – reporting of cohort studies, case-control studies and cross-sectional studies (surveys) – the statement – eg von Elm et al (2007) • CONSORT + sources of potential bias + outcomes related to type of design + potential confounders and corresponding adjustments of main analyses – explanation and elaboration – eg Vandenbroucke et al (2007) – http://www.strobe-statement.org/

• download both documents and templates of checklist

and flow diagram 26 March 2008

Statistical presentation - 2. A symptomatic review

19

Other guidelines

For other kinds of studies

• Extensions to CONSORT for other kinds of randomised trials – cluster RCTs – Campbell et al (2004) [no relation!] – non-inferiority/equivalence trials – Piaggio et al (2006) – herbal medicinal interventions – Gagnier et al (2006) – (coming) non-pharmaceutical interventions – http://www.consort-statement.org/?o=1044 • download all documents 26 March 2008

Statistical presentation - 2. A symptomatic review

20