#### Transcript Statistical Concepts and Methodologies for Data Analyses

Statistical Concepts and Methodologies for Data Analyses Benilton Carvalho Computational Biology and Statistics Group Department of Oncology University of Cambridge FROM RANDOM VARIABLES TO HYPOTHESIS TESTING Random Variables • Function that associates probability to: – Countable items (discrete random variable); • Tumor vs. Normal; Yes vs. No; Head vs. Tail; – Uncountable items (continuous random variable): • Log-expression; weight; height; • Characterized by a distribution function: – Bernoulli; Binomial; Geometric; NegativeBinomial; Poisson; – Normal; Student’s t; Gamma; Examples – Discrete Distributions Examples – Continuous Distributions Common Uses of Different Distributions • Bernoulli: probability of 1 success; • Binomial: probability of K successes; • Geometric: probability of K failures before 1st success; • Negative-Binomial: probability of K failures before R successes; • Poisson: probability of K rare events; The Questions • Investigation of populations or groups within a population leads to questions: – How does BRCAI behave across groups? – Can genotype predict drug response? – Does transcript abundance change as a function of time? The Experiment • A procedure used to answer the questions; • Comprised of multiple items: – Population; – Sample; – Hypotheses; – Test statistic; – Rejection criteria; Population • Superset of subjects of interest; • Ideally, every subject in the population is surveyed; • Issues with the “census approach”; Sample • • • • Select some subjects from the population; We refer to this subset as sample; Subject in a sample can be called replicate; Replicate: technical vs. biological; Hypotheses • Sets that define the “underlying truth”; • Null Hypothesis (H0): default situation. – Cannot be proven; – Reject (in favor of H1) vs. fail to reject; • Alternative Hypothesis (H1): alternative (duh!) – Complements H0 on the parametric space; – Assists on the definition of the rejection criteria. Examples of Hypotheses – P1 • Comparing expression: Tumor vs. Normal: – Expression on tumor is at most as high as on normal; – Expression on tumor is higher than on normal; Examples of Hypotheses – P2 • Comparing expression: Tumor vs. Normal: – Expression on tumor is at least as low as on normal; – Expression on tumor is lower than on normal; Examples of Hypotheses – P3 • Comparing expression: Tumor vs. Normal: – Expressions on tumor and normal are the same; – Expressions on tumor and normal are different; Test Statistic • • • • • Summary of the data; Built “under H0”; Independent of unknown parameters; Known distributions; Compatibility between data and H0; Test Statistic • What the statistician see… Rejection Criteria • Function of three factors: – Test statistic; – Hypotheses; – Type I Error (False Positive), α; • Determines thresholds used to reject H0: – One threshold: one-sided tests; – Two thresholds: two-sided tests; • Defines what is “extreme” for the experiment; Rejection Criteria From Rejection Criteria to P-value! p-value Rejection Criteria From Rejection Criteria to P-value! p-value Rejection Criteria From Rejection Criteria to P-value! p-value Sampling and testing Discrete observations Random sample of 10 balls from the box #red = 3 When do I think that I am not sampling from this box anymore? How many reds could I expect to get just by chance alone! 10% red balls and 90% blue balls 24 Sample Discrete observations Random sample of 10 balls from the box #red = 3 Test statistic Rejection criteria (based on your observed sample, do you have evidence to reject the hypothesis that you sampled from the null population) 10% red balls and 90% blue balls Null hypothesis (about the population that is being sampled) 25 Sample Continuous observations 4, 2.3, 5.2, 4.7, 2.1, 3.5, …….. mean = 3, sd = 0.6 Test statistic Rejection criteria (based on your observed sample, do you have evidence to reject the hypothesis that you sampled from the null population) Null hypothesis (about the population that is being sampled) 26 Summary of the Experiment 1) hypotheses 4) decision 2) sample 3) test statistic Useful Facts • The Law of the Large Numbers guarantees that the larger the sample size is, the closer the sample average is to the actual mean; • Normality assumption isn’t that important with large sample size; • The Central Limit Theorem states that the average is asymptotically normal; Useful Facts • The Z-score depends on the precise knowledge of the variance term: • Estimating the variance changes the distribution of the test statistic: Useful Facts • The Student’s t distribution is similar to the Normal distribution, but has heavier tails; • Larger sample size, more d.f.; • More d.f., closer to Normal; Multiple Testing • We are doing high-throughput experiments; • Comparing thousands of units simultaneously; • At this scale, we can observe several instances of rare events just by chance: – Event A: 1 in 1000 chance of happening; – Event B: 999 in 1000 chance of happening; – And the experiment is tried 20,000 times; – We expect 20 occurrences of Event A to be observed, although Event B is much more likely; Multiple Testing • • • • • • Similar scenario, for example, with DE; Most genes are not differentially expressed; High-throughput experiments; Differential expression is tested for 20K genes; Need to protect against false positives; Suggestion: use non-specific filtering; DATA MODELING What is a model? Statistical Models • • • • There is no “correct model”; Models are approximations of the truth; There is a “useful model”; Understand the mechanisms of the system for better choices of model alternatives; Revisiting Microarrays • • • • • • Scanned images; Fluorescence intensities; Proportional to target abundances; Restricted dynamic range; Asymmetrical distribution; Log-Intensities behave better; Revisiting Microarrays Intensities Log-Intensities Back to Data Modeling Linear Regression / ANOVA • Nature of the data: continuous; • Linear regression often used; • For subject i, known factors/covariates are candidates to predict log-intensities of a gene: • Residuals expected to be Normal; Interpreting Coefficients • Statisticians indicate that a parameter is estimated by using a “hat” on top of it: • Assuming that X = 0 for normal tissue: • Assuming that X = 1 for tumor tissue: Interpreting Coefficients Average log-intensity for normal tissue Change in average log-intensity associated to the tumor tissue Average log-intensity for tumor tissue GLM • • • • Generalized Linear Models; Generic framework; Accommodates different types of data; Special cases: Linear regressions and ANOVAs; Example – GLM Binomial Family • Responses: yes/no; dead/alive; sick/healthy; • Predictors: Gene expression / genotype / age; • Example: – Response: Cytogenetic abnormalities (Yes/No); – Predictors: Log-expression of probeset 1059_at; Log-Expression vs. Abnormalities Modeling a Binary Response • Response in the previous example: – Observed cytogenetic abnormalities; – Did not observe cytogenetic abnormalities; • Linear regression does not work: 1059_at 1 0.8 0.6 0.4 0.2 0 3 3.2 3.4 3.6 3.8 4 Modeling a Binary Response • Instead of modeling the actual response, we model the probability of that response; • Linear regression still fails; Prob(Observe Cytogenetic Abnormalities) 200% Invalid Results 150% 100% Valid Results 50% 0% -50% -100% 2 2.5 3 3.5 4 4.5 5 Logistic Regression - Rationale • Probability is restricted to the [0, 1] interval; • Linear regression isn’t; • Need to transform probability; Logistic Regression - Rationale • Instead of probability, model the odds: • Odds range from 0 to Infinity; • A linear regression approach would still fail; Logistic Regression - Rationale • Instead of odds, model the log-odds: • Log-odds range from -Infinity to Infinity; • An approach like linear regression, using the log-odds scale, would work fine; Back to GLM • In the previous example: Link function: logit Linear Predictor Interpreting Coefficients on a Logistic Model • b0: average log-odds for normal tissue; • b1: average change in log-odds on tumor; • Suppose b0 = 10.87 and b1 = -3.46: – How do we interpret? Model Selection • Likelihood measures the probability of observing the data under a certain model; • Given two models, M1 and M2 (M2⊃M1): – Get L1: likelihood of the data under M1; – Get L2: likelihood of the data under M2; • LRT = -2 log(L1/L2) is known; – Small LRT: choose M1; – Large LRT: choose M2; MODELING STRATEGIES FOR SEQUENCING DATA Sequencing – Rationale Technical Replicate • Sample j, transcript i is generated at rate λij; • A fragment attaches to the flow cell with a (low) probability pij; • Number of observed tags, yij, is Poisson distributed with rate proportional to λijpij; Adapted from notes by Tom Hardcastle Poisson Probability function: Analysis method: GLM Expected count of region i in sample j Noise Part Design matrix Deterministic Part Library size effect (Differential) effect for region i Need to account for extra variability technical rep – consistent with Poison biol. rep – not consistent with Poison Based on the data of Nagalakshmi et al. Science 2008; slide adapted from Huber; Sequencing – Rationale Biological Replicates • For subject j, on transcript i: • Different subjects have different rates, which we can model through: • This hierarchy changes the distribution of Y: Negative Binomial Probability function: Adding an additional source of variation smooth dispersion-mean relation α CONSIDERATIONS ON EXPERIMENT DESIGN Consideration • Sample size is crucial. The larger, the better; • With differential expression, one can observe this more easily; • Is RNA-Seq really worth it when we consider: – Cost, – Strategies for analysis, and – Technical requirements? Differential Expression Across Groups Flow Cell Confounded With Group 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 Flow Cell 1 Flow Cell 2 Flow Cell 3 Flow Cell 4 Group A Group B Group C Group D 7 8 Differential Expression Across Groups Randomize Samples wrt Flow Cell 1 2 3 4 5 6 Flow Cell 1 7 8 1 2 3 4 5 6 Flow Cell 2 7 8 1 2 3 4 5 6 Flow Cell 3 7 8 1 2 3 4 5 6 Flow Cell 4 7 8 Differential Expression Across Groups Barcoding vs. Lane Effect 1 2 3 4 5 6 Flow Cell 1 7 8 1 2 3 4 5 6 Flow Cell 2 7 8 1 2 3 4 5 6 Flow Cell 3 7 8 1 2 3 4 5 6 Flow Cell 4 7 8 CONSIDERATIONS ON DATA PROCESSING Normalization • Samples are sequenced in different depths: Gene Gene 1 … Gene N Total Reads Sample 1 Sample 2 500,000 … 500,000 … 0 500,000 15,000,00 0 30,000,00 0 • Genes with higher expression on Sample 2; • Adjusting by total reads can be misleading; Normalization • Length can affect relative inference of expression across genes; • Gene A K-times longer than B is expected to have K-times more reads than B: Gene A Gene B