Transcript Title

Workarounds in Public Transit
Modelling with EMME and Perl
Matt Carlson & John Armstrong
12th October 2007
21st International EMME Conference, Toronto
Contents
• Introduction to need for workarounds
• Case Studies
• EMME Wish Lists
Introduction
• EMME – Established in the UK in large public
transport models:
• Railplan (Transport for London)
• Docklands Public Transport Model (Docklands Light Railway,
London)
• PLANET (Department for Transport)
• However, there are cases where additional tools are
needed in addition to EMME:
• Processing electronic timetable data for rail services input to
EMME
• ‘Cleaning up’ EMME outputs of transit lines
• Reversing of EMME transit lines
• Hence scripting languages used
Scripting Languages
• AWK used with EMME/2 for decades
• Python used increasingly with Emme 3
• Perl used recently in Arup
• Doesn’t matter too much: just need to get the job
done
Case Study 1 – Exporting EMME Transit Lines
• No matter how ‘readable’ the data is when imported
to EMME, it is exported like this…
• Awkward spacing
• No ‘clues’ about nodes
(multiple 6-digit nodes at
stations don’t easily convert to
unique and readable 4
character labels)
Step 1
• Remove carriage returns where us3 values are
‘orphaned’ on to new lines
Step 2
• Read all elements from the tidier file…
• …and re-export desired data with Names
Wish List 1
• To be able to choose which data is exported by
EMME, including Database attributes
Case Study 2 – Reversing Transit Lines
INRO has included a facility in the network editor for
reversing lines interactively, but what if:
1. An entire network needs to be transposed (such
as to create a PM network from an AM)?
2. The reverse journeys required different nodes?
Use a script to reverse the lines
Example Network – DLR
110592
110591
E
133
D K 3 2E
1
DK
32 W
DK1 3W
3
DK1
110572
110573
110590
3E
13
D K 32 E
1
DK
2W
13
D K 33 W
1
DK
D
D K13
K1 0
11 N
N
D
D K11
K1 1
30 S
S
111190
111191
D
DK K130
11 N
1N
DK
1
D K 1 1S
13
0S
110791
110790
W
132
D K 33 W
1
DK
3E
13
D K 3 2E
1
DK
111091
111090
110972
110973
DK133W
DK132W
N
111
D K 130 N
DK
S
130
D K 1S
11
DK
DK132E
DK133E
91191
91190
80390
80391
120491
120490
3E
11
D K 0E
80491
80490
DK113E
DK110E
D
DK K1
13 1 1 S
0S
DK113E
DK
11
80690
80691 DK11 0E
2
DK DK1 E 80591
11
80590
2W 10W
70191
D K1
70190
13W
DK110W
DK113W
1170390
DK 70391
70290
70291K110W
D
3W
11
DK
80791
80790
80890
80891
0S 2E
13 1 1
D K D K 11S
1
DK
1N
11
D K 12 W
1
D K 30 N
1N
11 W
K1
D K 112 0N D
DK K13
D
0E E
DK133E
DK110E
DK133E
DK110E
DK133E
DK110E
D K133
E
D K110
E
D
DK K13
11 3 W
DK DK
0W
13 11
3E 0E
E
DK133E DK133
DK110E
DK110E 100590
100591
DK133E
D
K1
DK110E
100790
D 10
100890 100990
100791
100891
K1 W100690
100991
100290
100390
100291
100391
33 100691
DK110W
101091
DK110W DK1 101090
DK110W
W
DK110W
DK110W
10W
DK110W
DK133W
DK133W
DK133W
D
33W
K1
DK133W
DK1
33W
DK133W
DK132E
132E
DK
DK131E
DK132E
131E
DK
DK113E
DK131E
101590
DK113E
101591
DK113E
101390
101690
101391
101691
DK113W
101490
101491
DK113W
DK131W
DK113W
131W
DK
DK
DK131W
132W
DK132W
DK132W
3W
11
D K 131 W
D K 132 W
DK
DK111S
DK112E
70490
70491
820590
820591
70890DK DK1
70891
1 1
DK112E
DK111S
DK112W
DK111N
11 2E
DK DK1
S
11 1 1
2W N 70791
70790
820690
820691
DK
1
D K 1 2W
111
N
0W
11
D K 113 W
DK
W
100491
100490
2E
13
D K 31 E
1
D K 13 E
1
DK
80990
80991
DK
1
D K 1 1S
112
E
DK110
D K112 W
DK113 W
W
D
K1
K1 30N
11
N
80290
80291
D
DK110W
DK112W
DK113W
DK112W
DK111N
D
91390
91391
120372
DK1
12037310E
120380
120390
120391
D
D K1
DK1 100190
K1 3
1100191
0
1 3
3W
11 1 W
DK K13 W
D 32
1
DK
51
DK110W
3E
11 E
DK K110
D
2E
13 E
D K 131
D K 13 E
1
DK
R6
W
DK110E
2W
13 W
D K 133
DK
DK113W
11490
11491
1U
3E
13 E
DK K132
D
DK112W
5
R6
W
DK1
13E
DK1
12E
DK1
10E
S
DK130 S
DK111
N
DK111
N
DK130
10590
10591
DK113E
DK112E
DK110E
DK113E
DK112E
820190
820191
2W
11 N
DK K111
1S
D
11 2E
DK K11
D
390890
390891
2E
11
D K 1 1 1S
DK
1N
11 W
D K 112
DK
2E
11
D K 11 S
1
DK
N
111
D K 12W
1
DK
390990
390991
390590
390591
110592
110591
32W
DK1
33W
DK1
33E
DK1 32E
DK1
110572
110573
110590
DK
DK 13
11 0N
1N
111190
111191
DK
DK 11
13 1S
0S
DK
DK 130N
11
1N
110791
110790
W
132 W
DK
133
DK
32W
DK1 33W
DK1
E
133 E
DK
132
DK
111091
111090
DK
11
DK 1S
13
0S
D
01D
653
U
01U
653
E
133 E
DK
132
DK
PD
TT0
PD
TT0
Example Network (Inset)
110972
110973
DK133W
DK132W
11N
DK1 30N
DK1
30S
DK1
11S
DK1
DK132E
DK133E
91191
91190
80390
80391
120491
120490
DK110W
DK112W
DK113W
80290
80291
3E
11
DK 110E
DK
DK11
3E
DK11
2E
DK11
0E
DK11
DK11 0W
DK11 2W
3W
DK
DK 13
11 0N
1N
91390
91391
DK113
DK110
80690
80691 DK112 E
E
80491
80490
DK113E
DK110E
DK
DK 11
13 1S
0S
DK110W
DK DK110 E 80591
112
W 80590
WDK113
70191
70190
W
13E
DK1 10E
70390
DK1 70391
70290
70291
10W
DK1 13W
DK1
DK110W
DK113W
80791
80790
100491
100490
120372
DK1
12037310E
120380
120390
120391
DK
DK
DK1 100190
100191 11 133E
10W
0W
11
DK 113W
DK
DK
DK 13
11 3W
DK DK
0W
13 11
3E 0E
DK133E
DK133E
0E
DK133
DK133E
DK133E DK133E
E
DK133E
DK
DK110E
DK110E
DK110
DK110E
DK110E
DK110E 100590
11
E
DK110E
100591
100790
DK 0W
100890 100990
100791
100891
100991
100290
100390
100291
13 100690
100391
100691
101090
DK110W
101091
3W
DK110W DK110
DK110W
DK110W
DK110W
DK110W
DK133W
DK133W DK133 W
DK133W
DK133W
DK133W
W
DK133W
DK132E
DK132E
DK131E
DK132E
DK131E
DK113E
DK131E
101590
DK113E
101591
DK113E
DK
DK110E
2E
13
DK 131E
DK 3E
11
1D
3W
11 1W
DK 13
DK 2W
13
DK
WR
65
W
132 W
DK 133
DK
DK113W
11490
11491
1U
3E
13
DK 132E
DK
1N
DK11
0N
DK13
0S
DK13 1S
DK11
DK112W
65
DK113E
DK112E
DK110E
DK113E
DK112E
10590
10591
WR
101390
101391
80890
80891
101690
101691
DK111S
DK112E
DK112W
DK111N
4D
820590
820591
DK112W
DK111N
DK112E
DK111S
DK1
DK1 12W
11N
820690
820691
DK1
DK1 11S
12E
3U
DK113W
DK131W
DK132W
70890DK DK11
70891
11 2E
1S
DK DK11
11 1N
2W 70791
70790
TT00
TT00
DK113W
DK131W
DK132W
W
113
DK 131W
DK 132W
DK
70490
70491
• Note separate nodes by direction
101490
101491
820190
W
820191
112 N
DK 111
S
DK
111 2E
DK 11
DK
390890
390891
E
112
DK 111S
DK
1N
11
DK 112W
DK
2E
11
DK 1S
11
DK
11N
DK1 12W
DK1
• To enable line-to-line interchange movements at
complex stations
390990
390991
390590
390591
HX503U
HX503
D
2D
HX50
HX50
2U
D
K1
K1 30N
11
N
80491
80490
D
D
DK K1
13 11 S
0S
80690
80691
DK113
DK110E E
DK
11
2E
80591
2W
0N
13
D K 111 N
DK
DK110W
DK113W
80590
1S S
11 30
DK DK1
70191
70190
DK111N
DK112W
DK130N
DK111S
DK112E
DK130S
DK
11
DK113E
DK110E
DK11
0W
DK11
3W
0409
ka
80791
80790
E
132 E
DK
131 E
DK
113
DK
0S 2E
13 11
DK DK 1S
11
DK
N
111 W
DK
112
DK N
N
130
111 W
DK 112 N DK
DK 130
DK
DK113W
DK131W
DK132W
80990
80991
Reverse Lines
• Use a script to read a ‘tidied’ output:
• Input all values into an array
• Export in reverse order
• Look up opposite node
Wish List 2
• Splitting of stations into so many nodes would be
un-necessary if line-to-line transfer information was
more easily extracted
• Currently there is a need to re-code the network with one node
per line and extract the line-to-line data (as transfer.mac* and
transfer.awk*)
• Perhaps if something similar to auto turn movements for transit
were output at the end of an assignment…
* See transfer.zip by Heinz Spiess at http://www.inro.ca/en/download/macros.php
Case Study 3 – Processing Transit Line Data
• Railplan Model (Rail, Metro,
Tram, Bus)
• Rail is more complicated:
• Stopping patterns vary,
• Trains join and split,
• Timetables change more
frequently
• Other modes simple by
comparison
• Usually headways only difference
• Need an automated approach
Data Sources 1 – Network Rail CIF
• Network Rail use a
‘Common Interface
Format’ (CIF)
• This contains all rail
movements in a given
timetable period
including
• Freight Trains
• Empty stock movements
• 3 Million+ Lines
• Cryptic Format
• Example: Manchester-London
Data Sources 2 – Issues
• Very little vehicle information
• E.g. EMU 125mph – data suited to train pathing not passenger
use
• No capacity information (4/8/12 car???)
• Need to convert to multiple station nodes
• Need to convert to transit line codes
Methods - Software
• Perl is used
• Perl = Practical
Extraction and
Reporting
Language
• Open Source
• Cross Platform
www.perl.org
Methods - Overview
1. Extract subset of ‘relevant’ trains from CIF;
2. For ‘relevant’ trains extract the subset of ‘relevant’ nodes
required;
3. Look up ‘relevant’ nodes dependent on direction and TOC;
4. Subtract journey times between ‘relevant’ nodes;
5. Aggregation of identical lines;
6. Allocate a Railplan service code to each line;
7. Assign Vehicle Types;
8. Export in Railplan format;
9. Import to EMME
10. Harmonise times & Fix Join-Split
1 - Extract subset of ‘relevant’ trains from CIF
• Is a passenger service
• Not freight or empty stock
• Is in a relevant TOC
• Runs in the south east of the UK
• Is within the correct time period
• Passes most important station between 7-10am / 10am-4pm
TOC
TIPLOC
SW
WATRLMN
Hierarchy
1
Name
London Waterloo
SW
SOTON
2
Southampton Cent
SW
RDNGSTN
3
Reading
SW
RDNG4AB
3
Reading 4A/B
SW
PSEA
4
Portsmouth & S
ES
WATRINT
1
Waterloo International
ES
ASHFKI
2
Ashford (Kent) International
SN
VICTRIC
1
London Victoria
SN
CHRX
2
London Charing Cross
SN
LNDNBDE
3
London Bridge
SN
KENOLYM
4
Kensington Olympia
SN
ECROYDN
5
East Croydon
SN
WCROYDN
6
West Croydon
SN
BRGHTN
7
Brighton
SE
CHRX
1
London Charing Cross
SE
CANONST
2
London Cannon Street
SE
LNDNBDE
3
London Bridge
2 - For ‘relevant’ trains extract the subset of ‘relevant’
nodes required
• Skeletal nature of model means stops near edge of
model are ignored
3 - Look up ‘relevant’ nodes dependent on direction
and TOC
• Stations are often split into separate nodes by
direction
MOOR
MOOR
G ener al wor ksheet
CITY
CITY
BLAC
BLAC
BLAC
BLAC
• Note:
CANN
FENC
CANN
FENC
CHAR
CHAR
LOND
LOND
LOND
LOND
LOND
LOND
LOND
LOND
WATE
WATE
WATE
WATE
WATE
WATE
WATE
• 4 nodes at Raynes Park
• 7 nodes at Waterloo
WATE
VICT
VICT
VICT
VICT
VICT
VICT
VICT
VICT
ELEP
ELEP
G ener al wor ksheet
HAYD
VAUX
VAUX
VAUX
VAUX
HAYD
WIMB
WIMB
Ra ilp la n M a in CI F Co n v e r s io n
WIMB
WIMB
Sc e n a r io
8200:
I P 0. 8
2 0 0 7 - 0 5 - 0 2 1 6 : 3 9 ( M a t t . Ca r ls o n )
B'SEB'SE
B'SEB'SE
QUEE
WATERLOO EUROSTAR(BOTH)
310180
WATERLOO S.W. (WINDSOR) (DOWN)
310182
WATERLOO S.W. (WINDSOR) (UP)
310183
WATERLOO S.WEST (LOW NUMBERS) (DOWN)
310166
WIMB
WIMB
RAYN
WATERLOO S.WEST (LOW NUMBERS) (UP)
310167
RAYN
RAYN
RAYN
WATERLOO S.WEST (MID-NUMBERS) (UP)
310185
SOUT
SOUT
NEW
NEW
MORD
MORD
MOTS
MOTS
Ra ilp la n M a in CI F Co n v e r s io n
Sc e n a r io
8200:
I P 0. 8
2 0 0 7 - 0 5 - 0 2 1 6 : 4 2 ( M a t t . Ca r ls o n )
ST
ST
WATERLOO S.W. (MID NUMBERS) (DOWN)
310184
4 - Subtract journey times between ‘relevant’ nodes
• The journey time is stored in us3
• This avoids problems with times for ‘irrelevant’
nodes
• i.e. subtract the times between modelled nodes after discarding
‘irrelevant’ nodes
5 - Aggregation of identical lines
• Lines with identical stopping patterns are
aggregated
• This includes:
• noboa
• noali
6 - Allocate a Railplan service code to each line
•Lines are named according to TOC, O-D Pair, Direction
TOC
Origin
CC
CC
CC
CC
CC
CH
CH
CH
CH
CH
ES
ES
FENCHRS
GRAYS
UPMNSTR
UPMNSTR
HWYCOMB
MARYLBN
Destination
UPMNSTR
GRAYS
PITSEA
FENCHRS
MARYLBN
BHAMSNH
PADTON
PADTON
WATRINT
WATRINT
Updated
Identifier
FN
KO
KO
KO
FN
MR
TO
MR
XX
XX
ER
ER
Direction
D
U
D
D
U
U
D
D
U
D
D
U
7 - Assign Vehicle Types
•Vehicles are assigned according to:
•TOC
•Timetabled Type
•Speed
TOC
CC
CC
CC
CC
CH
CH
CH
CH
CH
CH
TrainType
EMU
EMU
EMU357
EMU357
DMUE
DMUE
DMUN
DMUN
DMUS
DMUT
Speed
75
100
100
75
75
RailplanTrainAM
326
326
326
326
226
226
212
212
212
212
RailplanTrainIP
325
325
325
325
225
225
210
210
210
210
8 - Export in Railplan format
• Note: Non-interpolated times
9 - Import to EMME
• Interpolate us3 times with splitime.mac
• Reset noboa and noali flags from us1 and us2
• This allows splitime to work with ‘timing points’ as well as stops
10 – Modify in EMME
• Harmonise times for common stop-stop sections
• Fix join-split times
10a – Harmonise Times
• All Stop-to-Stop pairs are consistent and rounded to an
integer us3
• Including common Stop-to-Stop pairs on different routes
52
PD
D
PD523
3D
PD523D
PD
52
3D
8D
PD51
8D
PD51
51
PD
8D
D
5183D
PPD
D52
10b – Fix Join-Split Times
• Sections ‘x’ minutes long where trains stop at a
dummy node are:
• x-0.01 minutes on main leg
• 0.01 minutes on dummy leg
0
3.00
3.00
5.0
0
4.0
4.9
3.0
3.0
0
9
3.00
0
0
0
2.00
3.00
5.0
4.00
1
0.0
0.0
1.37
1
1.07
0.99
1.28
7.0
Transit Lines are Output (as Case Study 1)
• Transit Lines
exported:
• Node Names
shown for clarity
• us3 times
interpolated and
‘bucket-rounded’ to
2 d.p.
• Un-necessary info
(us1, us2)
removed from file.
Wish List 3
• To model join-split trains as a
single line, e.g. Y-shape
• To be able to view transit lines
in terms of stop-stop times,
not node-node times:
• Similar to configurable attributes
• Underlying segment data could still
be stored as now
10
57226
27057
35
2
459586
261850
73172
36762
15
14
32
66528
27273
38518
20991
12
6
38491
29973
21
26
7
25691
308493
96010
404746
60815
94441
31
75809
48433
75909
175978
29
73049
66331
19
31385
16060
74613
86542
34
13
869
3494
11
85443
107304
59888
29189
23
38011
30118
33
28
113712
103370
24
75575
31860
92467
48187
41020
25686
25
22
16
1
47307
36712
20
95137
172975
9
71010
29588
17
52147
21625
41821
25085
4
77753
37835
18
27
54720
14892
5
50722
17637
59889
29802
27939
21839
8
30
25642
16222
74977
44945
3
40127
19688
Conclusions
• Various tedious or tricky operations are made
possible by use of a scripting language
• Learning a scripting language pays off very quickly
Matt Carlson
Arup
13 Fitzroy Street
London
W1T 4BQ
UK
+44 20 7755 4114
[email protected]
www.arup.com
linkedin.com/in/mattcarlson