Evaluating Machine Learning Approaches for Aiding Probe

Transcript Evaluating Machine Learning Approaches for Aiding Probe

Mining High-Throughput Biological Data

David Page Dept. Biostatistics and Medical Informatics Dept. Computer Sciences

University of Wisconsin-Madison

Some Data Types We’ll Discuss

Gene expression microarray Single-nucleotide polymorphisms Mass spectrometry proteomics and metabolomics Protein-protein interactions (from co immunoprecipitation) High-throughput screening of potential drug molecules

image from the DOE Human Genome Program http://www.ornl.gov/hgmis

How Microarrays Work

Probes (DNA) Labeled Sample (RNA) Hybridization Gene Chip Surface

Two Views of Microarray Data

Data points are genes   Represented by expression levels across different samples (ie, features=samples ) Goal: categorize new genes Data points are samples (eg,   patients ) Represented by expression levels of different genes (ie, features=genes ) Goal: categorize new samples

Two Ways to View The Data

Person Gene A28202_ac AB00014_at AB00015_at . . . Person 1 1142.0 321.0 2567.2 . . . Person 2 586.3 586.1 759.0 . . . Person 3 105.2 559.3 3210.7 . . . Person 4 42.8 692.1 812.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Data Points are Genes

Data Points are Samples

Supervision: Add Class Values

Person Gene A28202_ac AB00014_at AB00015_at . . . Class Person 1 1142.0 321.0 2567.2 . . . normal Person 2 586.3 586.1 759.0 . . . cancer Person 3 105.2 559.3 3210.7 . . . normal Person 4 42.8 692.1 812.0 . . . cancer . . . . . . . . . . . . . . . . . . . . . . . . . . .

Supervised Learning Task

Given

: a set of microarray experiments, each done with mRNA from a different patient (same cell type from every patient) Patient’s expression values for each gene constitute the features, and patient’s disease constitutes the class

: Learn a model that accurately predicts class based on features

Location in Task Space

Data Points are: Genes Samples Clustering Supervised Data Mining Predict the class value for a patient based on the expression levels for his/her genes

Leukemia

(Golub et al ., 1999) Classes Acute Lymphoblastic Leukemia (ALL) and Acute Myeloid Leukemia (AML) Approach Weighted voting (essentially naïve Bayes) Cross-Validated Accuracy Of 34 samples, declined to predict 5, correct on other 29

Cancer vs. Normal

Relatively easy to predict accurately, because so much goes “haywire” in cancer cells Primary barrier is noise in the data… impure RNA, cross-hybridization, etc Studies include breast, colon, prostate, lymphoma, and multiple myeloma

X-Val Accuracies for Multiple Myeloma

(74 MM vs. 31 Normal) Trees Boosted Trees SVMs Vote Bayes Nets 98.1 99.0 100.0 100.0 97.0

More MM (300), Benign Condition MGUS (Hardin et al., 2004)

ROC Curves: Cancer vs. Normal

ROC: Cancer vs. Benign (MGUS)

Work by Statisticians Outside of Standard Classification/Clustering

Methods to better convert Affymetrix’s low-level intensity measurements into expression levels: e.g., work by Speed, Wong, Irrizary Methods to find differentially expressed genes between two samples, e.g. work by Newton and Kendziorski But the following is most related…

Ranking Genes by Significance

Some biologists don’t want one predictive model, but a rank-ordered list of genes to explore further (with estimated significance) For each gene we have a set of expression levels under our conditions, say cancer vs. normal We can do a t-test to see if the mean expression levels are different under the two conditions: p-value Multiple comparisons problem: if we repeat this test for 30,000 genes, some will pop up as significant just by chance alone Could do a Bonferoni correction (multiply p-values by 30,000), but this is drastic and might eliminate all

False Discovery Rate (FDR) [Storey and Tibshirani, 2001]

Addresses multiple comparisons but is less extreme than Bonferoni Replaces p-value by with this p-value or lower that really don’t have different means in the two classes (false discoveries) package q-value : fraction of genes Publicly available in R as part of Bioconductor Recommendation: Use this in addition to your supervised data mining… your collaborators will want to see it

FDR Highlights Difficulties Getting Insight into Cancer vs. Normal

Using Benign Condition Instead of Normal Helps Somewhat

Question to Anticipate

You’ve run a supervised data mining algorithm on your collaborator’s data, and you present an estimate of accuracy or an ROC curve (from X-val) How did you adjust this for the multiple comparisons problem?

Answer: you don’t need to because you commit to a single predictive model before ever looking at the test data for a fold—this is

only one comparison

Prognosis and Treatment

Features same as for diagnosis Rather than disease state, class value becomes life expectancy with a given treatment (or positive response vs. no response to given treatment)

Breast Cancer Prognosis

(Van’t Veer et al ., 2002) Classes good prognosis (no metastasis within five years of initial diagnosis) vs. poor prognosis Algorithm Ensemble of voters Results 83% cross-validated accuracy on 78 cases

A Lesson

Previous work selected features to use in ensemble by looking at the entire data set Should have repeated feature selection on each cross-val fold Authors also chose ensemble size by seeing which size gave highest cross-val result Authors corrected this in web supplement;

accuracy went from 83% to 73%

Remember to “tune parameters” separately for each cross-val fold!

Prognosis with Specific Therapy (Rosenwald et al ., 2002) Data set contains gene-expression patterns for 160 patients with diffuse large B-cell lymphoma , receiving anthracycline chemotherapy Class label is five-year survival One test-train split 80/80 True positive rate: 60% False negative rate: 39%

Some Future Directions

Using gene-chip data to select therapy Predict which therapy gives best prognosis for patient Combining Gene Expression Data with Clinical Data such as Lab Results, Medical and Family History Multiple relational tables, may benefit from relational learning

Unsupervised Learning Task

Given

: a set of microarray experiments under different conditions

: cluster the genes, where a gene described by its expression levels in different experiments

Location in Task Space

Data Points are: Genes Samples Clustering Supervised Data Mining Group genes into clusters, where all members of a cluster tend to go up or down together

( Green

Example

= up -regulated, Red = down -regulated)

Experiments (Samples)

Visualizing Gene Clusters (eg, Sharan and Shamir, 2000)

Gene Cluster 1, size=20 Gene Cluster 2, size=43 Time (10-minute intervals)

Unsupervised Learning Task 2

Given

: a set of microarray experiments (samples) corresponding to different conditions or patients

: cluster the experiments

Location in Task Space

Data Points are: Genes Samples Clustering Supervised Data Mining Group samples by gene expression profile

Examples

Cluster samples from mice subjected to a variety of toxic compounds (Thomas et al ., 2001) Cluster samples from cancer patients, potentially to discover different subtypes of a cancer Cluster samples taken at different time points

Some Biological Pathways

Regulatory pathways  Nodes are labeled by genes   Arcs denote influence on transcription G1 codes for P1, P1 inhibits G2’s transcription Metabolic pathways   Nodes are metabolites, large biomolecules (eg, sugars, lipids, proteins and modified proteins) Arcs from biochemical reaction inputs to outputs  Arcs labeled by enzymes (themselves proteins)

Acetyl CoA

Metabolic Pathway Example

H 2 0 HSCoA

Citrate

citrate synthase

cis -Aconitate

aconitase H 2 0

Oxaloacetate

NADH MDH NAD +

Malate

H 2 0 FADH 2 fumarase

Fumarate

(Krebs Cycle,

Isocitrate

TCA Cycle, Citric Acid Cycle) NAD + IDH NADH + CO 2

a -Ketoglutarate a

-KDGH NAD + + HSCoA succinate thikinase

Succinate Succinyl-CoA

NADH + CO 2 FAD GTP + HSCoA GDP + P i

Regulatory Pathway (KEGG)

Using Microarray Data Only

Regulatory pathways  Nodes are labeled by genes   Arcs denote influence on transcription G1 codes for P1, P1 inhibits G2’s transcription Metabolic pathways   Nodes are metabolites, large biomolecules (eg, sugars, lipids, proteins, and modified proteins) Arcs from biochemical reaction inputs to outputs  Arcs labeled by enzymes (themselves proteins)

Supervised Learning Task 2

Given

: a set of microarray experiments for same organism under different conditions

: Learn graphical model that accurately predicts expression of some genes in terms of others

Some Approaches to Learning Regulatory Networks

Bayes Net Learning (started with Friedman & Halpern, 1999, we’ll see more) Boolean Networks (Akutsu, Kuhara, Maruyama & Miyano, 1998; Ideker, Thorsson & Karp, 2002) Related Graphical Approaches (Tanay & Shamir, 2001; Chrisman, Langley, Baay & Pohorille, 2003)

Bayesian Network (BN)

Data geneA Expt1 Expt2 Expt3 Expt4 geneB Note: direction of arrow indicates dependence not causality P(geneA) 0.5 0.5

geneA P(geneB) 1.0 0.0

0.5 0.5

geneB

Problem: Not Causality

A B A is a good predictor of B. But is A regulating B??

Ground truth might be: B A A C B A C B B C A Or a more complicated variant

Approaches to Get Causality

Use “ knock-outs ” (Pe’er, Regev, Elidan and Friedman, 2001).

organisms.

Use time-series But not available in most Networks data and Dynamic Bayesian (Ong, Glasner and Page, 2002).

But even less data typically.

Use other data sources, eg sequences upstream of genes, where transcription regulators may bind. (Segal, Barash, Simon, Friedman and Koller, 2002; Noto and Craven, 2005)

A Dynamic Bayes Net

gene 1 gene 2 gene N gene 3 gene 1 gene 2 gene N gene 3

Problem: Not Enough Data Points to Construct Large Network

Fortunate to get 100s of chips But have 1000s of genes    E. coli: ~4000 Yeast: ~6000 Human: ~30,000 Want to learn causal graphical model over 1000s of variables with 100s of examples (settings of the variables)

Advance: Module Networks

[Segal, Pe’er, Regev, Koller & Friedman, 2005] Cluster genes by similarity over expression experiments All genes in a cluster are “tied together”: same parents and CPDs Learn structure subject to this tying together of genes Iteratively re-form clusters and re-learn network, in an EM-like fashion

Problem: Data are Continuous but Models are Discrete

Gene chips provide a real-valued mRNA measurement Boolean networks and most practical Bayes net learning algorithms assume discrete variables May lose valuable information by discretizing

Advance: Use of Dynamic Bayes Nets with Continuous Variables

[Segal, Pe’er, Regev, Koller & Friedman, 2005] Expression measurements used instead of discretized (up, down, same) Assume linear influence of parents on children (Michaelis-Menten assumption) Work so far constructed the network from literature and learned parameters

Problem: Much Missing Information

mRNA from gene 1 doesn’t of mRNA from gene 2 directly alter level Rather, the protein product from gene 1 may alter level of mRNA from gene 2 (e.g., transcription factor) Activation of transcription factor might not occur by making more of it, but just by phosphorylating modification) it (post-translational

Example: Transcription Regulation DNA P O geneR T mRNA R P O R geneA geneB geneC T mRNA

Approach: Measure More Stuff

Mass spectrometry (later) can measure protein rather than mRNA   Doesn’t measure all proteins Not very quantitative (presence/absence) 2D gels can measure post-translational modifications, but still low-throughput because of current analysis Co-immunoprecipitation (later), Yeast 2 Hybrids can measure protein interactions, but noisy

Another Way Around Limitations Identify smaller part of the task that is a step toward a full regulatory pathway   Part of a pathway Classes or groups of genes Examples: Chromatin remodelers Predicting the operons in E. coli

Chromatin Remodelers and Nucleosome [Segal et al.

2006]

Previous DNA picture oversimplified DNA double-helix is wrapped in further complex structure DNA is accessible only if part of this structure is unwound Can we predict what chromatin remodelers act on what parts of DNA, also what activates a remodeler?

The

E. Coli

Genome

Finding Operons in

E. coli

(Craven, Page, Shavlik, Bockhorst and Glasner, 2000) g3 g2 g4 g5 Given: Do: known operons and other E. coli data predict all operons in E. coli Additional Sources of Information  gene-expression data  functional annotation

Comparing Naive Bayes and Decision Trees (C5.0)

Using Only Individual Features

Single-Nucleotide Polymorphisms SNPs: Individual positions in DNA where variation is common Roughly 2 million known SNPs in humans New Affymetrix whole-genome scan measures 500,000 of these Easier/faster/cheaper to measure SNPs than to completely sequence everyone Motivation …

If We Sequenced Everyone…

Susceptible to Disease D or Responds to Treatment T Not Susceptible or Not Responding

Example of SNP Data

Person SNP 1 2 3 . . . CLASS Person 1 C T A G T T . . . old Person 2 C C A G C T . . . young Person 3 T T A A C C . . . old Person 4 C T G G T T . . . young . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Phasing (Haplotyping)

Advantages of SNP Data

Person’s SNP pattern does not change with time or disease, so it can give more insight into susceptibility Easier to collect samples (can simply use blood rather than affected tissue)

Challenges of SNP Data

Unphased Algorithms exist for phasing (haplotyping), but they make errors and typically need related individuals, dense coverage Missing values are more common than in microarray data (though improving substantially, down to around 1-2% now) Many more measurements. For example, Affymetrix human SNP chip at a half million SNPs.

Supervised Learning Task

Given

: a set of SNP profiles, each from a different patient.

Phased: nucleotides at each SNP position on each copy of each chromosome constitute the features, and patient’s disease constitutes the class Unphased: unordered pair of nucleotides at each SNP position constitute the features, and patient’s disease constitutes the class

: Learn a model that accurately predicts class based on features

Waddell et al., 2005

Multiple Myeloma, Young (susceptible) vs. Old (less susceptible), 3000 SNPs, best at 64% acc (training) SVM with feature selection (repeated on every fold of cross-validation): 72% accuracy, also naïve Bayes. Significantly better than chance. Old Young Old 31 9 Actual Young 14 26

Listgarten et al., 2005

SVMs from SNP data predict lung cancer susceptibility at 69% accuracy Naïve Bayes gives similar performance Best single SNP at less than 60% accuracy (training)

Lessons

Supervised data mining algorithms can predict disease susceptibility at rates better than chance and better than individual SNPs Accuracies much lower than we see with microarray data, because we’re predicting who will get disease, not who already has it

Future Directions

Pharmacogenetics: predicting drug response from SNP profile   Drug Efficacy Adverse Reaction Combining SNP data with other data types, such as clinical (history, lab tests) and microarray

Proteomics

Microarrays are useful primarily because mRNA concentrations serve as surrogate for protein concentrations Like to measure protein concentrations directly, but at present cannot do so in same high-throughput manner Proteins do not have obvious direct complements Could build molecules that bind, but binding greatly affected by protein structure

Time-of-Flight (TOF) Mass Spectrometry

(thanks Sean McIlwain) Detector Measures the time for an ionized particle, starting from the sample plate, to hit the detector Laser Sample +V

Time-of-Flight (TOF) Mass Spectrometry 2

Matrix-Assisted Laser Desorption-Ionization (MALDI) Crystalloid structures made using proton rich matrix molecule Hitting crystalloid with laser causes molecules to ionize and “fly” towards detector Sample +V Detector Laser

Time-of-Flight Demonstration 0 Sample Plate

Time-of-Flight Demonstration 1 Matrix Molecules

Time-of-Flight Demonstration 2 Protein Molecules

Time-of-Flight Demonstration 3 Laser Detector +10KV Positive Charge

Time-of-Flight Demonstration 4 Laser pulsed directly onto sample Proton kicked off matrix molecule onto another molecule + +10KV

Time-of-Flight Demonstration 5 Lots of protons kicked off matrix ions, giving rise to more positively charged molecules + + + + + +10KV

Time-of-Flight Demonstration 6 The high positive potential under sample plate, causes positively charged molecules to accelerate towards detector + + + + + +10KV

Time-of-Flight Demonstration 7 + + + + + Smaller mass molecules hit detector first, while heavier ones detected later + +10Kv

Time-of-Flight Demonstration 8 The incident time measured from when laser is pulsed until molecule hits detector + + + + + + +10KV

Time-of-Flight Demonstration 9 + + + + + + Experiment repeated a number of times, counting frequencies of “flight-times” +10KV

Example Spectra from a Competition by Lin et al. at Duke These are different fractions from the same sample.

M/Z

Trypsin-Treated Spectra

M/Z

Many Challenges Raised by Mass Spectrometry Data Noise: extra peaks from handling of sample, from machine and environment (electrical noise), etc.

M/Z values may not align exactly across spectra (resolution ~0.1%) Intensities not calibrated across spectra: quantification is difficult Cannot get all proteins… typically only several hundred. To improve odds of getting the ones we want, may fractionate our sample by 2D gel electrophoresis or liquid chromatography.

Challenges (Continued)

Better results if partially digest proteins (break into smaller peptides) first Can be difficult to determine what proteins we have from spectrum Isotopic peaks: C 13 for a single peptide and N 15 atoms in varying numbers cause multiple peaks

Handling Noise: Peak Picking

Want to pick peaks that are statistically significant from the noise signal Want to use these as features in our learning algorithms.

Many Supervised Learning Tasks

Learn to predict proteins from spectra, when the organism’s proteome is known Learn to identify isotopic distributions Learn to predict disease from either proteins, peaks or isotopic distributions as features Construct pathway models

Using Mass Spectrometry for Early Detection of Ovarian Cancer [Petricoin et al., 2002]

Ovarian cancer difficult to detect early, often leading to poor prognosis Trained and tested on mass spectra from blood serum 100 training cases, 50 with cancer Held-out test set of 116 cases, 50 with cancer 100% sensitivity, 95% specificity (63/66) on held-out test set

Not So Fast

Data mining methodology seems sound But Keith Baggerly argues that cancer samples were handled differently than normal samples, and perhaps data were preprocessed differently too If we run cancer samples Monday and normals Wednesday, could get differences from machine breakdown or nearby electrical equipment that’s running on Monday but not Wed Lesson: tell collaborators they must randomize samples for the entire processing phase and of course all our preprocessing must be same Debate is still raging… results not replicated in trials

Other Proteomics: 3D Structures

Other Proteomics: Interactions

Figure from Ideker et al.,

Science

292(5518):929-934, 2001 each node represents a gene product (protein) blue edges show direct protein-protein interactions yellow edges show interactions in which one protein binds to DNA and affects the expression of another

Protein-Protein Interactions

Yeast 2-Hybrid Immunoprecipitation   Antibodies (“immuno”) are made by combinatorial combinations of certain proteins Millions of antibodies can be made, to recognize a wide variety of different antigens (“invaders”), often by recognizing specific proteins

antibody protein

Protein-Protein Interactions

Immunoprecipitation

antibody

Co-Immunoprecipitation

antibody

Many Supervised Learning Tasks

Learn to predict protein-protein interactions: protein 3D structures may be critical Use protein-protein interactions in construction of pathway models Learn to predict protein function from interaction data

ChIP-Chip Data

Immunoprecipitation can also be done to identify proteins interacting with DNA rather than other proteins Chromatin immunoprecipitation (ChIP): grab sample of DNA bound to a particular protein (transcription factor) ChIP-Chip: run this sample of DNA on a microarray to see which DNA was bound Example of analysis of such new data: Keles et al., 2006

Metabolomics

Measures concentration of each low molecular weight molecule in sample These typically are “ metabolites ,” or small molecules produced or consumed by reactions in biochemical pathways These reactions typically catalyzed by proteins (specifically, enzymes) This data typically also mass spectrometry, though could also be NMR

Lipomics

Analogous to metabolomics, but measuring concentrations of lipids rather than metabolites Potentially help induce biochemical pathway information or to help disease diagnosis or treatment choice

To Design a Drug:

Identify Target Protein Knowledge of proteome/genome Relevant biochemical pathways Determine Target Site Structure Synthesize a Molecule that Will Bind Crystallography, NMR Difficult if Membrane-Bound Imperfect modeling of structure Structures may change at binding And even then…

Molecule Binds Target But May:

Bind too tightly or not tightly enough.

Be toxic.

Have other effects (side-effects) in the body.

Break down as soon as it gets into the body, or may not leave the body soon enough.

It may not get to where it should in the body (e.g., crossing blood-brain barrier).

Not diffuse from gut to bloodstream.

And Every Body is Different:

Even if a molecule works in the test tube and works in animal studies, it may not work in people (will fail in clinical trials).

A molecule may work for some people but not others.

A molecule may cause harmful side effects in some people but not others.

Typical Practice when Target Structure is Unknown

High-Throughput Screening (HTS): Test many molecules (1,000,000) to find some that bind to target ( ligands ).

Infer (induce) shape of target site from 3D structural similarities.

Shared 3D substructure is called a pharmacophore .

Perfect example of a machine learning task with spatial target.

An Example of Structure Learning

Common Data Mining Approaches

Represent a molecule by thousands to millions of features and use standard techniques (e.g., KDD Cup 2001) Represent each low-energy conformer by feature vector and use multiple-instance learning (e.g., Jain et al., 1998) Relational learning  Inductive logic programming (e.g., Finn et al., 1998)  Graph mining

Supervised Learning Task

Given

: a set of molecules, each labeled by activity -- binding affinity for target protein -- and a set of low-energy conformers for each molecule

: Learn a model that accurately predicts activity (may be Boolean or real-valued)

ILP as Illustration: The Logical Representation of a Pharmacophore

Active(X)

if:

has-conformation(X,Conf), has-hydrophobic(X,A), has-hydrophobic(X,B), distance(X,Conf,A,B,3.0,1.0), has-acceptor(X,C), distance(X,Conf,A,C,4.0,1.0), distance(X,Conf,B,C,5.0,1.0).

This logical clause states that a molecule X is active if it has some conformation Conf, hydrophobic groups A and B, and a hydrogen acceptor C such that the following holds: in conformation Conf of molecule X, the distance between A and B is 3 Angstroms (plus or minus 1), the distance between A and C is 4, and the distance between B and C is 5.

Background Knowledge I

Information about atoms and bonds in the molecules atm(m1,a1,o,3,5.915800,-2.441200,1.799700).

atm(m1,a2,c,3,0.574700,-2.773300,0.337600).

atm(m1,a3,s,3,0.408000,-3.511700,-1.314000).

bond(m1,a1,a2,1).

bond(m1,a2,a3,1).

Background knowledge II

Definition of distance equivalence dist(Drug,Atom1,Atom2,Dist,Error): number(Error), coord(Drug,Atom1,X1,Y1,Z1), coord(Drug,Atom2,X2,Y2,Z2), euc_dist(p(X1,Y1,Z1),p(X2,Y2,Z2),Dist1), Diff is Dist1-Dist, absolute_value(Diff,E1), E1 =< Error.

euc_dist(p(X1,Y1,Z1),p(X2,Y2,Z2),D): Dsq is (X1-X2)^2+(Y1-Y2)^2+(Z1-Z2)^2, D is sqrt(Dsq).

Central Idea: Generalize by searching a lattice

Lattice of Hypotheses

active(X) active(X)

active(X)

has-hydrophobic(X,A) has-donor(X,A) active(X)

has-acceptor(X,A) active(X)

distance(X,A,B,5.0) active(X)

active(X)

has-hydrophobic(X,A), has-donor(X,A), has-acceptor(X,A), has-donor(X,B), has-donor(X,B), has-donor(X,B), distance(X,A,B,4.0) distance(X,A,B,6.0)

etc.

Conformational model

Conformational flexibility modelled as multiple conformations:   Sybyl randomsearch Catalyst

Pharmacophore description

Atom and site centred      Hydrogen bond donor Hydrogen bond acceptor Hydrophobe Site points (limited at present) User definable Distance based

Example 1: Dopamine agonists

Agonists taken from Martin data set on QSAR society web pages Examples (5-50 conformations/molecule) OH OH H 3 C OH OH OH N OH H 2 N H 2 N H 3 C N CH 3 OH OH H 2 N OH OH H N OH OH

Pharmacophore identified Molecule A has the desired activity if: in conformation B molecule A contains a hydrogen acceptor at C, and in conformation B molecule A contains a basic nitrogen group at D, and the distance between C and D is 7.05966 +/- 0.75 Angstroms, and in conformation B molecule A contains a hydrogen acceptor at E, and the distance between C and E is 2.80871 +/- 0.75 Angstroms, and the distance between D and E is 6.36846 +/- 0.75 Angstroms, and in conformation B molecule A contains a hydrophobic group at F, and the distance between C and F is 2.68136 +/- 0.75 Angstroms, and the distance between D and F is 4.80399 +/- 0.75 Angstroms, and the distance between E and F is 2.74602 +/- 0.75 Angstroms.

Example II: ACE inhibitors

28 angiotensin converting enzyme inhibitors taken from literature  D. Mayer et al ., J. Comput.-Aided Mol. Design, 1, 3-16, (1987) H S O N N N O COOH H O O P OH N H CH 3 O N COOH COOH CH 3 N H N O COOH

ACE pharmacophore

Molecule A is an ACE inhibitor if: molecule A contains a zinc-site B, molecule A contains a hydrogen acceptor C, the distance between B and C is 7.899 +/- 0.750 A, molecule A contains a hydrogen acceptor D, the distance between B and D is 8.475 +/- 0.750 A, the distance between C and D is 2.133 +/- 0.750 A, molecule A contains a hydrogen acceptor E, the distance between B and E is 4.891 +/- 0.750 A, the distance between C and E is 3.114 +/- 0.750 A, the distance between D and E is 3.753 +/- 0.750 A.

C B A

Pharmacophore discovered

Distance Progol Mayer Zinc site 4.9

5.0

H-bond acceptor 3.8

3.8

8.5

8.6

B A C

Additional Finding

Original pharmacophore rediscovered plus one other   different zinc ligand position similar to alternative proposed by Ciba Geigy

4.0

3.9

7.3

Example III: Thermolysin inhibitors

10 inhibitors for which crystallographic data is available in PDB Conformationally challenging molecules Experimentally observed superposition

Key binding site interactions

Asn112-NH O OH S2’ O=C Asn112 Arg203-NH Zn NH O O O P R NH S1’ O=C Ala113

Interactions made by inhibitors

Interaction Asn112-NH S2’ Asn112 C=O Arg 203 NH S1’ Ala113-C=O Zn 1HYT   1THL            1TLP        1TMN    2TMN      4TLN         4TMN        5TLN        5TMN    6TMN      

Pharmacophore Identification

Structures considered 1HYT 1THL 1TLP 1TMN 2TMN 4TLN 4TMN 5TLN 5TMN 6TMN Conformational analysis using “Best” conformer generation in Catalyst 98-251 conformations/molecule

Thermolysin Results

10 5-point pharmacophore identified, falling into 2 groups (7/10 molecules)   3 “acceptors”, 1 hydrophobe, 1 donor 4 “acceptors, 1 donor Common core of Zn ligands, Arg203 and Asn112 interactions identified Correct assignments of functional groups Correct geometry to 1 Angstrom tolerance

Thermolysin results

Increasing tolerance to 1.5Angstroms finds common 6-point pharmacophore including one extra interaction

Example IV: Antibacterial peptides [Spatola et al., 2000]

Dataset of 11 pentapeptides showing activity against Pseudomonas aeruginosa   6 actives <6 4m g/ml IC 50 5 inactives

Pharmacophore Identified

A Molecule M is active against Pseudomonas Aeruginosa if it has a conformation B such that: M has a hydrophobic group C, M has a hydrogen acceptor D, the distance between C and D in conformation B is 11.7 Angstroms M has a positively-charged atom E, the distance between C and E in conformation B is 4 Angstroms the distance between D and E in conformation B is 9.4 Angstroms M has a positively-charged atom F, the distance between C and F in conformation B is 11.1 Angstroms the distance between D and F in conformation B is 12.6 Angstroms the distance between E and F in conformation B is 8.7 Angstroms Tolerance 1.5 Angstroms

Clinical Databases of the Future (Dramatically Simplified)

PatientID Gender Birthdate P1 M 3/22/63 PatientID Date Physician Symptoms Diagnosis P1 1/1/01 Smith palpitations hypoglycemic P1 2/1/03 Jones fever, aches influenza PatientID Date Lab Test Result P1 1/1/01 blood glucose 42 P1 1/9/01 blood glucose 45 PatientID SNP1 SNP2 … SNP500K P1 AA AB BB P2 AB BB AA PatientID Date Prescribed Date Filled Physician Medication Dose Duration P1 5/17/98 5/18/98 Jones prilosec 10mg 3 months

Final Wrap-up

Molecular biology collecting lots and lots of data in post-genome era Opportunity to “connect” molecular-level information to diseases and treatment Need analysis tools to interpret Data mining opportunities abound Hopefully this tutorial provided solid start toward applying data mining to high throughput biological data

Thanks To

Jude Shavlik John Shaughnessy Bart Barlogie Mark Craven Sean McIlwain Jan Struyf Arno Spatola Paul Finn Beth Burnside Michael Molla Michael Waddell Irene Ong Jesse Davis Soumya Ray Jo Hardin John Crowley Fenghuang Zhan Eric Lantz

If Time Permits… some of my group’s directions in the area

Clinical Data (with Jesse Davis, Beth Burnside, M.D.) Addressing another problem with current approaches to biological network learning (with Soumya Ray, Eric Lantz)

Using Machine Learning with Clinical Histories: Example

We’ll use example of Mammography to show some issues that arise These issues arise here even with just one relational table These issues are even more pronounced with data in multiple tables

Supervised Learning Task

Given

: a database of mammogram abnormalities for different patients (same cell type from every patient) Radiologist-entered values describing the abnormality constitute the features, and abnormality’s biopsy result as benign or malignant constitutes the class

: Learn a model that accurately predicts class based on features

Mammography Database

[Davis et al, 2005; Burnside et al, 2005] Patient Abnormality Date Mass Shape … Mass Size Loc Be/Mal P1 1 5/02 Spic 0.03 RU4 B P1 2 5/04 Var 0.04 RU4 M P1 3 5/04 Spic 0.04 LL3 B … … … … … … …

Original Expert Structure

Level 1: Parameters

Shape Be/Mal Size

Given

: Features (node labels, or fields in database), Data, Bayes net structure

Learn

: Probabilities. Note: probabilities needed are

(Be/Mal),

(Shape|Be/Mal),

(Size|Be/Mal)

Level 2: Structure

Shape Be/Mal Size

Given

: Features, Data

Learn

Bayes net structure

and probabilities. Note: with this structure, now will need

(Size|Shape,Be/Mal) instead of

(Size|Be/Mal).

Mammography Database

Patient Abnormality Date Mass Shape … Mass Size Loc Be/Mal P1 1 5/02 Spic 0.03 RU4 B P1 2 5/04 Var 0.04 RU4 M P1 3 5/04 Spic 0.04 LL3 B … … … … … … …

Mammography Database

Patient Abnormality Date Mass Shape … Mass Size Loc Be/Mal P1 1 5/02 Spic 0.03 RU4 B P1 2 5/04 Var 0.04 RU4

P1 3 5/04 Spic 0.04 LL3 B … … … … … … …

Mammography Database

Patient Abnormality Date Mass Shape … Mass Size Loc Be/Mal

1 5/02 Spic 0.03 RU4 B

2 5/04 Var 0.04 RU4

M P1

3 5/04 Spic 0.04 LL3 B … … … … … … …

Level 3: Aggregates

Shape Be/Mal Size

Avg size this date Given

: Features, Data, Background knowledge –

aggregation functions

such as average, mode, max, etc.

Learn

: Useful

aggregate features

, Bayes net structure that uses these features, and probabilities. New features may use other rows/tables.

Mammography Database

Patient Abnormality Date Mass Shape … Mass Size Loc Be/Mal P1 1 5/02 Spic 0.03 RU4 B P1 2 5/04 Var 0.04 RU4 M P1 3 5/04 Spic 0.04 LL3 B … … … … … … …

Mammography Database

Patient Abnormality Date Mass Shape … Mass Size Loc Be/Mal P1 1 5/02 Spic 0.03 RU4 B P1 2 5/04 Var 0.04 RU4 M P1 3 5/04 Spic 0.04 LL3 B … … … … … … …

Mammography Database

Patient Abnormality Date Mass Shape … Mass Size Loc Be/Mal P1 1 5/02 Spic 0.03

RU4 B P1 2 5/04 Var 0.04 RU4 M P1 3 5/04 Spic 0.04 LL3 B … … … … … … …

Level 4: View Learning

Shape change in abnormality at this location

Be/Mal

Increase in average size of abnormalities

Avg size this date

Given

: Features, Data, Background knowledge – aggregation functions and

intensionally-defined relations

such as “increase” or “same location”

Learn

: Useful

new features defined by views

(equivalent to

rules

SQL queries

), Bayes net structure, and probabilities.

Shape Size

Example of Learned Rule

is_malignant(A) IF: 'BIRADS_category'(A,b5), 'MassPAO'(A,present), 'MassesDensity'(A,high), 'HO_BreastCA'(A,hxDCorLC), in_same_mammogram(A,B), 'Calc_Pleomorphic'(B,notPresent), 'Calc_Punctate'(B,notPresent).

ROC: Level 2 (TAN) vs. Level 1

Precision-Recall Curves

1 0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 0

All Levels of Learning

Level 4 (View) Level 3 (Aggregate) Level 2 (Structure) Level 1 (Parameter) 0.1

0.2

0.3

0.4

0.5

Recall

0.6

0.7

0.8

0.9

SAYU-View

Improved View Learning approach SAYU: Score As You Use For each candidate rule, add it to the Bayesian network and see if it improves the network’s score Only add a rule (new field for the view) if it improves the Bayes net

0.4

0.3

0.2

0.1

0 0.9

Relational Learning Algorithms

SAYU-View 0.8

Initial Level 4 (View) 0.7

0.6

0.5

Level 3 (Aggregates) 0 0.1

0.2

0.3

0.4

0.5

Recall

0.6

0.7

0.8

0.9

0.12

0.10

0.08

0.06

0.04

0.02

0.00

Average Area Under PR Curve For Recall >= 0.5

SAYU-View Initial Level 4 Level 3

Clinical Databases of the Future (Dramatically Simplified)

Another Problem with Current Learning of Regulatory Models

Current techniques all use greedy heuristic Bayes net learning algorithms use “sparse candidate” approach: to be considered as a parent of gene 1, another gene 2 must be correlated with gene 1 CPDs often represented as trees: use greedy tree learning algorithms All can fall prey to functions such as exclusive-or – do these arise?

Skewing Example

[Page & Ray, 2003; Ray & Page, 2004; Rosell et al., 2005; Ray & Page, 2005] Female SxL Gene Gene 3 … Gene 100 Survival 0 0 1 1 0 1 0 1 0…0000000 0…0000001 … 1…1111111 0…0000000 0…0000001 … 1…1111111 0…0000000 0…0000001 … 1…1111111 0…0000000 0…0000001 … 1…1111111 0 1 1 0

Drosophila

survival based on gender and

Sxl

gene activity

Hard Functions

Our Definition: those functions for which no attribute has “gain” according to standard purity measures (GINI, Entropy)  NOTE: “Hard” does not refer to size of representation  Example: n -variable odd parity  Many others

Learning Hard Functions

Standard method of learning hard functions (e.g. with decision trees): depth-

Lookahead  O(

k+1

-1 ) for

examples in

variables We devise a technique that allows learning algorithms to hard functions efficiently learn

Key Idea

Hard functions are not hard for all data distributions We can skew the input distribution to simulate a different one  By randomly choosing “preferred values” for attributes Accumulate evidence over several skews to select a split attribute

Example: Uniform Distribution 1 1

x 1

0 0 0 1

x 2

0 1

x 3

0 1 0 1 0 1 0 1 1 0

0 1

GINI

(

)  0 .

GINI

(

;

x i

 0 )  0 .

GINI

(

;

x i

 

GAIN

(

x i

)  0 1 )  0 .

1 1

x 1

0 0 0 1

x 2

0 1 Example: Skewed Distribution (“Sequential Skewing”, Ray & Page, ICML 2004)

x 3

0 1 0 1 0 1 0 1 1 0

0 1 Weight 1 4 1 4 3 4 3 4 1 4 1 4 3 4 3 4

GINI

(

)  0 .

GINI

(

;

1  1 )  0 .

GINI GINI

( (

f f

; ;

x x

2 3  1 )  0 .

25  1 )  0 .

25 

GAIN

(

1 )  0

GAIN

(

2 ) 

GAIN

(

3 )  0 0