Transcript The Neural Network Objects RhoNNO: A ROOT package for general use

The Neural Network
Objects
RhoNNO: A ROOT package
for general use
Marcel Kunze
Institut für Experimentalphysik 1
Ruhr-Universität Bochum
3rd ROOT Workshop, M. Kunze
1
Why ?
Neural Network Objects are around us and tested in
various applications since several years (“NNO”)
ROOT seems very appealing wrt. object persistence and
interactivity
NNO has been overhauled to re-use ROOT functionality
(“RhoNNO”)
Integration of other ROOT applications (Neural Network
Kernel of J.P.Ernenwein)
RhoNNO complies to ROOT‘s coding standards
3rd ROOT Workshop, M. Kunze
2
Neural Network Models
Supervised
Multi-Layer Perceptron (TMLP, TXMLP)
Training Models Fisher Discriminant (TFD)
Supervised Growing Cell Structure (TSGCS)
Supervised Growing Neural Gas (TSGNG)
Neural Network Kernel (TNNK)
Unsupervised
Learning Vector Quantization (TLVQ)
Training Models Growing Cell Structure (TGCS)
Growing Neural Gas (TGNG)
Implementation of the most popular supervised and unsupervised
network models
3rd ROOT Workshop, M. Kunze
3
Growing Neural Gas
(GNG)
Example: Adaptation to a multi-dimensional PDF (1D-2D-3D)
Fractal growth process: Density and topology conservation
3rd ROOT Workshop, M. Kunze
4
Architecture
VNeuralNet
VSupervisedNet
VUnsupervisedNet
TNNK
TSGCS
TLVQ
TSGNG
TGCS
TGNG
TXMLP
TFD
TMLP
VNeuralNetPlotter
TSimpleNeuralNetPlotter
RhoNNO class hierarchy
3rd ROOT Workshop, M. Kunze
5
Interactive Control
Positive Samples (“pro”)
Negative Samples (“con”)
The plotter highlights training progress
- Output functions
- Error functions
3rd ROOT Workshop, M. Kunze
6
Management of Data Sets
TDataServe
A mini database to support management of input/output vector
relations
- Add vector pairs to data sets
- Partition data sets for training/testing
- Shuffle data sets
- Serve vector pairs
- Save and load data sets using ROOT persistence
3rd ROOT Workshop, M. Kunze
7
VNeuralNet Interface
// Abstract interface for all networks
virtual void AllocNet() = 0;
virtual void InitNet() = 0;
virtual void WriteText() = 0;
virtual void WriteBinary() = 0;
virtual void ReadText() = 0;
virtual void ReadBinary() = 0;
virtual Double_t* Recall(NNO_INTYPE* in,NNO_OUTTYPE* out=0) = 0;
virtual Double_t Train(NNO_INTYPE* in,NNO_OUTTYPE* out=0) = 0;
// Training and testing
Double_t
TrainEpoch(TDataServe *server, Int_t nEpoch=1);
Double_t
TestEpoch(TDataServe *server);
void
BalanceSamples(Bool_t yesNo = kTRUE);
virtual void
SetMomentumTerm(Double_t f);
virtual void
SetFlatSpotElimination(Double_t f);
Abstract base class of all network models
3rd ROOT Workshop, M. Kunze
8
NetworkTrainer
A RhoNNO sample application to train and test neural
networks
- Assemble training and test data sets out of ROOT trees,
based on TFormula
- Define network architecture and transfer functions
- Define and execute a training schedule
- Persist networks
- Generate C++ code to perform network recall
3rd ROOT Workshop, M. Kunze
9
Example: PID Tagging
Qc
e
m
p
K p
P [GeV/c]
P [GeV/c]
Identify charged tracks using dE/dx, QC etc.
3rd ROOT Workshop, M. Kunze
10
The ROOT Training File
Measurement and
shape variables
MC-Truth:
1 = electron
2 = muon
3 = pion
4 = kaon
5 = proton
Likelihood values
Arbitrary standard tree to provide training and test sample
3rd ROOT Workshop, M. Kunze
11
Steering File
# Example: Training of PIDSelectors with NNO
#define the network
xmlp 7 15 10 1
transfer TR_FERMI
momentum 0.2
balance true
plots true
test 10000
start 1
stop 200
topology and training schedule
# MLP with 2 hidden layers
# Transfer function
# Momentum term
# Assure same statistics for pro and con samples
# Show updating error plots on training progress
# Number of test vector pairs to reserve
# First training epoch
# Last training epoch
#define the data source,
datapath ../Data
networkpath ../Networks
file PidTuple1.root
file PidTuple2.root
take two input files
# Directory to look up data files
# Directory to persist network files
# First file to get input from
# Second … (ad infinitum)
x
#set up the input layer (use branch names)
tree PidTuple
# This is the tree to look up data
cut mom>0.5&&dch>0&&dch<10000 # Preselection of samples
input mom:acos(theta):svt:emc:drc:dch:ifr #Input layer
autoscale true
# Apply a scale to assure inputs are O(1)
#set up the output layer (use branch names)
#Particles pid = {electron=1,muon,pion,kaon,proton}
output abs(pid)==3
# Perform training for pions
3rd ROOT Workshop, M. Kunze
12
Generation of Recall Code
// TXMLP network trained with NNO NetworkTrainer at Fri Apr 27
// Input parameters mom:acos(theta):svt:emc:drc:dch:ifr
// Output parameters abs(pid)==3
// Training files:
//../Data/PidTuple1.root
//../Data/PidTuple2.root
#include "RhoNNO/TXMLP.h"
Double_t* Recall(Double_t *invec)
{
static TXMLP net("TXMLP.net");
Float_t x[7];
x[0]
= 0.76594 *
invec[0];
x[1]
= 2.21056 *
invec[1];
x[2]
= 0.20365 *
invec[2];
x[3]
= 2.2859 *
invec[3];
x[4]
= 1.75435 *
invec[4];
x[5]
= 0.00165 *
invec[5];
x[6]
= 0.85728 *
invec[6];
return net.Recall(x);
}
//
//
//
//
//
//
//
3rd ROOT Workshop, M. Kunze
mom
acos(theta)
svt
emc
drc
dch
ifr
13
Execute the Example
NetworkTrainer <steering file> <first epoch> <last epoch>
3rd ROOT Workshop, M. Kunze
14
Summary
The re-use of ROOT functionality improves NNO a lot
TFormula works great to pre-process samples
Run training either from CINT C++ or from ASCII steering file
A RhoNNO GUI and/or Wizard is still missing
RhoNNO comes as part of the Rho package, but can be used
independently:
Installation of the shared lib plus the headers is sufficient.
The NetworkTrainer application runs standalone
Documentation:
http://www.ep1.ruhr-uni-bochum.de/~marcel/RhoNNO.html
3rd ROOT Workshop, M. Kunze
15