Transcript Layout of the OSMOSIS OSCB

Layout of the OSMOSIS
Optical Switch Controller Board
using Expedition
or
IS hindsight nearly always 20/20 … ?
Outline









OSMOSIS Project
Design Entry
Board Structure, Materials
Signals, Rules, Constraints
1st Approach
2nd Approach
3rd Approach
“Final” Approach
Conclusion
2
pdi, Layout of the OSMOSIS OSCB, May 2006
Design Entry
DesignView
.
.
.
I/O
Designer
8 FPGA
Designs
3
pdi, Layout of the OSMOSIS OSCB, May 2006
HyperLynx
PreLayout
Simulations
OSCB Chassis
Test chassis with a
pre-version of the
OSCB Board
Pre-OSCB
OSCI Interface Cards
4
pdi, Layout of the OSMOSIS OSCB, May 2006
OSCB Board
Size: 431.8 x 573mm (17" x 22.5")
- fits into a 19" chassis
3644 Components
7 FPGAs, 1704 pins
1 FPGA, 1020 pins
40 Slot connectors
20 on front
20 on back
each slot connector has
125 pins single ended,
120 diff pin pairs; total of 285 signal pins
5
pdi, Layout of the OSMOSIS OSCB, May 2006
Connector Fanout Problem
OSCI Conn
Blocked Routing Channels
OSCI Conn
OSCB
Use Blind Vias
Free
Routing Channels
Half Board Thickness
2.0mm minimum (78mils) (16 Layers)
6
pdi, Layout of the OSMOSIS OSCB, May 2006
1.8 mm
View from Top
1704 pin FPGA (Xilinx FF1704 Package)
12 Rows of
signal pins
min 12 internal layers
(outer layers not used)
• ≥ 16 signal layers
7
pdi, Layout of the OSMOSIS OSCB, May 2006
High speed design requirements

12954 Nets
—
—




4072 diff pairs (clocks + data)
5000 single ended
Clock frequency: 125MHz, 156MHz
I/O Technology used:
- LVPECL
Master Clock
- LVDS
Data, Clk, Sync
- LVCMOS
Control, uncritical signals
Impedances:
- LVPECL
100 Ω balanced
- LVDS
100 Ω balanced
- LVCMOS
60..70 Ω single ended
Long Wires:
up to 750mm (30")
8
pdi, Layout of the OSMOSIS OSCB, May 2006
Board Structure 16s16p
Half Board Thickness ~2.3mm (90mils)
Blind Via 1-16
GND
+2.5V
Blind Via 1-16,
Drill: 0.2mm (8mil)
Aspect Ratio 1:11
Thru Via
S1,H
S2,V
B.C. Plane, (high εr)
Thru Via,
Drill: 0.45mm (18mil)
Aspect Ratio 1:10
S3,H
GND
S4,V
+1.5V
GND
B.C. Plane, (high εr)
S5,H
S6,V
GND
S7,H
Danger!
GND
S8,V
+3.3V
GND
9
pdi, Layout of the OSMOSIS OSCB, May 2006
"LoFlow" Prepreg
Excessive
Material
Board Material
Isola IS620





Low Dielectric Loss: <0.01 @ 2..10GHz
(FR4: 0.02)
Low Permitivity: εr = 3.5 @ 1GHz
(FR4: 4.4)
Low vertical CTE: 40ppm/ºC
(FR4: 175ppm/ºC)
Lower risk of torn vias!
Cost: ~3x FR4 (“moderate”)
FR4 compatible, but process parameter tuning
required
10
pdi, Layout of the OSMOSIS OSCB, May 2006
Rules / Constraints




Done in CES:
- 86 Signal Classes
- 1 additional Scheme for BGA Areas
- 7 Clearance Rules for Netclasses
General Rules Trackwidth (60..70 Ω):
Track Width
outer: 200 μm (8 mils),
inner: 150 μm (6 mils)
Track-to-Track outer: 200 μm (8 mils),
inner: 100 μm (4 mils)
Diff Pairs on inner Layers (100 Ω):
100-140-100, 100-115-100 (μm)
11
pdi, Layout of the OSMOSIS OSCB, May 2006
Wiring challenge
Exception:
Local diff pair wiring
Each star consists of
~900 Signals
12
pdi, Layout of the OSMOSIS OSCB, May 2006
Challenges



Board Manufacturing:
- Size alone not a problem, but...
- 100μm Structures alone not a problem, but…
- 32 Layers alone not a problem, but…
- IS620 alone not a problem, but…
 All together, - can that be done at all?
Wiring:
- 12900 Signals is much, but…
 4000 Diff Pairs is incredibly much! but…
 most of these pairs wired in global stars
This is going to be tough!
13
pdi, Layout of the OSMOSIS OSCB, May 2006
1st Approach
Placement
Top: all major comps
Bottom: "Chickenfood"
"Standard" routing
method:
- PWR/GND
- Critical Signals
-
Top
14
pdi, Layout of the OSMOSIS OSCB, May 2006
Bottom
1st Approach - Observations (1)
- 6 week effort
Only 80% completion
(still 1500 opens!)
- Autorouter takes
looong (days!)
Almost impossible to
do test runs
- manual completion not
feasible
need breakthrough!
15
pdi, Layout of the OSMOSIS OSCB, May 2006
1st Approach - Observations (2)
Autorouter cannot convert
blind vias into thru vias:
poor FPGA fanout
12 rows
 Manually add thru vias
under the inner 4 rows of
signal pins
8 outer rows
4 inner rows
Power
Supply
Thru Via
Blind Via (1-16)
(2 between)
16
pdi, Layout of the OSMOSIS OSCB, May 2006
Thru Via (1 btw)
Blind Via (1-16)
(2 between)
1st Approach - Observations (3)
Autorouter cannot connect a diff
pair to different vias
blind vias
thru vias
Diff Pairs
 Manually convert blind vias to
thru vias, where necessary
Autorouter does not know
fences (hard or soft)
17
pdi, Layout of the OSMOSIS OSCB, May 2006
"Workaround":
Use route obstructs to guide
router
1st Approach - Observations (4)
Question of an expert:
Why are all FPGAs on the
same side?
18
pdi, Layout of the OSMOSIS OSCB, May 2006
Discussion with manufacturer:
FPGAs on both sides can be done
 place 4 FPGAs on top side and
4 FPGAs on bottom side
2nd Approach


New FPGA placement – 4 on top, 4 on bottom
Rewire from scratch - except master CLK and
supply
(0v2, 10sep05)
19
pdi, Layout of the OSMOSIS OSCB, May 2006
2nd Approach - Observations (1)
Much better results!
(still ~1000 opens)
 Did not solve the problem,
Asked Mike Bare from Mentor
Graphics:
Are we doing something wrong?
 No principal mistakes,
approach seems to be OK.
settings seem to be OK.
Asked US top PCB „Guru“:
Can this board be done?
 Feasible; forget autorouter!
Would do it manually,
would need only 12 layers,
would take 3 months
– too late!
MG Switzerland runs an
autorouter test without diff
pair definition (all signals
single ended)
 99.8% Completion!
15 opens finished manually
in under 1 hour
20
pdi, Layout of the OSMOSIS OSCB, May 2006
Need breakthrough!
2nd Approach - Observations (3)
Congestion in the top
connector area:
Wide single ended bus causes
partial blockage of diff pairs
21
pdi, Layout of the OSMOSIS OSCB, May 2006
Conclusion:
Turn top daughtercard slots 180º
2nd Approach - Observations (4)
On the edge of despair…
Expedition very, very slow
- e.g. „Save“ takes about 10 minutes
- e.g. move and drop a simple component (e.g. Cap)
can take 10-15 seconds
- 2GB of memory not sufficient  crashes
- Routing passes can take several days
- CES seems to extremely slow down Expedition
22
pdi, Layout of the OSMOSIS OSCB, May 2006
3rd Approach



Turn top daughtercard section 180º
Buy new PC
- Athlon 64 X2 Dual Core 4800+
- 4GB Memory
- Latest MB technology
Rewire from scratch (except power supplies)
23
pdi, Layout of the OSMOSIS OSCB, May 2006
3rd Approach - Observations (1)
Further improvement
???? opens
 Still not the final solution,
Need breakthrough!!!
Difficult / often impossible to
place a diff pair of vias,
even if there is enough space
 No immediate solution,
Manually place 2 diff vias
individually!
Too many open diff pairs after
autoroute:
Router seems to have real
difficulties with diff pair
fanout out of the BGAs
 No immediate solution,
Manual routing! Mostly easy!
Even with new PC:
Slow performance still almost
unbearable
 Use ExtremePCB and
ExtremeAR!
24
pdi, Layout of the OSMOSIS OSCB, May 2006
 Measure:
Use 4 more layers
4th Approach – Add 4 more layers
(possible without changing board thickness)
16s16p
25
pdi, Layout of the OSMOSIS OSCB, May 2006
20s16p
S1,H
S2,V
S1,H
S2,V
S3,H
S3,H
S4,V
S4,V
S5,H
S6,V
S5,H
S6,V
S7,H
S7,H
S8,V
S8,V
S9,H
S10,V
4th Approach


Use 4 more layers
Install XtremePCB,
setup server + 2 client sessions, and
Add one more person



Install XtremeAR
setup server + 3 client processes
Remove CES
CES slows down Expedition even more with the
new pre-release required for running XtremeAR
Rewire from scratch
26
pdi, Layout of the OSMOSIS OSCB, May 2006
4th Approach – Use LDIR
Vertical layers are more utilized,
especially 5 inner PFGAs need
more vertical routing space
27
pdi, Layout of the OSMOSIS OSCB, May 2006
4th Approach – Routing Method (1)
- All PWR / GND
- Flow Control Bus (single ended)
- Master Clock (DP) and Sync Signals (SE),
tuning,
manual clean-up
- Diff Pairs:
Partial route (per FPGA)
1) route opposite side
- force thru vias
2) route FPGA side
- force blind vias
3) route both sides
4) manual Clean-Up
28
pdi, Layout of the OSMOSIS OSCB, May 2006
4th Approach – Routing Method (2)
- Repeat Partial Routing / Clean-up
.
.
.
- Single Ended Signals
- Tuning
- DRC
- Clean-Up
- DRC
- Generate Data
29
pdi, Layout of the OSMOSIS OSCB, May 2006
• Supply all done
30
pdi, Layout of the OSMOSIS OSCB, May 2006
• Flow Control Bus
31
pdi, Layout of the OSMOSIS OSCB, May 2006
• Flow Control Bus
32
pdi, Layout of the OSMOSIS OSCB, May 2006

• Flow Control Bus
• Global Sync Signals
33
pdi, Layout of the OSMOSIS OSCB, May 2006

• Flow Control Bus
• Global Sync Signals
34
pdi, Layout of the OSMOSIS OSCB, May 2006


• Flow Control Bus

• Global Sync Signals

• Master Clock (Diff Pairs)
35
pdi, Layout of the OSMOSIS OSCB, May 2006
• Flow Control Bus

• Global Sync Signals

• Master Clock (Diff Pairs) 
36
pdi, Layout of the OSMOSIS OSCB, May 2006
•
•
•
•
37
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)
•
•
•
•
38
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

•
•
•
•
•
39
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)
•
•
•
•
•
40
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

•
•
•
•
•
•
41
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1)
•
•
•
•
•
•
42
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
•
•
•
•
•
•
•
43
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2)
•
•
•
•
•
•
•
44
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
•
•
•
•
•
•
•
•
45
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3)
•
•
•
•
•
•
•
•
46
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
•
•
•
•
•
•
•
•
•
47
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4)
•
•
•
•
•
•
•
•
•
48
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
•
•
•
•
•
•
•
•
•
•
49
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
Global Diff Pairs (5)
•
•
•
•
•
•
•
•
•
•
50
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
Global Diff Pairs (5) 
•
•
•
•
•
•
•
•
•
•
•
51
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
Global Diff Pairs (5) 
Single Ended (1)
•
•
•
•
•
•
•
•
•
•
•
52
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
Global Diff Pairs (5) 
Single Ended (1)

•
•
•
•
•
•
•
•
•
•
•
•
53
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
Global Diff Pairs (5) 
Single Ended (1)

Single Ended (2)
•
•
•
•
•
•
•
•
•
•
•
•
54
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
Global Diff Pairs (5) 
Single Ended (1)

Single Ended (2)

•
•
•
•
•
•
•
•
•
•
•
•
•
55
pdi, Layout of the OSMOSIS OSCB, May 2006
Flow Control Bus

Global Sync Signals

Master Clock (Diff Pairs) 
Local Diff Pairs (1)

Local Diff Pairs (2)

Global Diff Pairs (1) 
Global Diff Pairs (2) 
Global Diff Pairs (3) 
Global Diff Pairs (4) 
Global Diff Pairs (5) 
Single Ended (1)

Single Ended (2)

Tuning, Clean-Up

4th Approach – Observations (1)
- Routing passes take now
hours (instead of days)
 Now we have quick
feedback,
Can do router test runs!
- Further improvement of
placement to free routing
channels
 Marginal improvement of
routing results
 Can be easily completed
- Beyond 90% completion
autorouter leaves more
manually in most cases!
and more nets (DPs!) open
56
pdi, Layout of the OSMOSIS OSCB, May 2006
4th Approach – Observations (2)
- Router introduces many
diff pair separations
57
pdi, Layout of the OSMOSIS OSCB, May 2006
 Tedious manual repair
4th Approach – Observations (3)
- Odd FPGA routing
 Tedious manual repair
Partial blockage
58
pdi, Layout of the OSMOSIS OSCB, May 2006
4th Approach – Observations (4)
- Further improvement
 Still not the final solution
- Time has now become
the determining factor!
 manufacturing
deadline
Fallback Solution:
Partial wiring
97.7% completion (672 opens!)
– after manual effort
 Design does not fully meet requirements
 reduced performance
59
pdi, Layout of the OSMOSIS OSCB, May 2006
Board Statistics








Parts Placed:
Pins:
Nets:
Differential Pairs:
Connections:
Routed:
Total Trace Length:
Parts Placed:
Total plated holes:
60
pdi, Layout of the OSMOSIS OSCB, May 2006
3643
42142
12954
4072
29246
97.7% (672 Opens)
2.59 km (1.6 miles)
3643
50987
Tools Used
I/O Designer
Hyperlynx (pre Layout Simulation)
DesignView

CES
Expedition
1
2
3
4
Xtreme
Hyperlynx
(Post Layout Simulation)
61
pdi, Layout of the OSMOSIS OSCB, May 2006
Overall Results (1)



Our approach seems to be OK, no obvious mistakes
Extra four layers (due to poor router performance):
Symptom treatment, not fix!
 caused additional manufacturing problems
CES performance inadequate!
Especially for large designs, where CES is really
needed, it becomes almost unusable

Xtreme AR brought breakthrough!

Outstanding support from MG Switzerland!

Bare in Mind: According to MG this is one of the
most complex and demanding designs at the time
62
pdi, Layout of the OSMOSIS OSCB, May 2006
Overall Results (2)



Router has difficulties with diff pair fanout of
large BGAs, many opens can easily be routed
manually
- Our view: Diff pair fanout of large BGAs not
solved
Router introduces separated diff pairs
Lack of router control, e.g.
- cost factors,
- x/y vs. orthogonal routing,
- control script,
- fences (hard and soft)
63
pdi, Layout of the OSMOSIS OSCB, May 2006
Overall Results (3)

No "Advanced Fanout":
- Fanout traces and vias should be flagged
accordingly and considered part of device,
even after device is placed
 much less hassle when large device with
fanout needs to be moved/pushed


Spread doesn't seem to work properly with diff
pairs
Too much hidden automatism,
should be left to the user when to use auto
functions
64
pdi, Layout of the OSMOSIS OSCB, May 2006
Oh, by the Way - Some (Personal) Insights




The answer is: Yes, almost always!
When changing over to a new PCB tool – better
don't start with a design such as this one!
For designs like this: There is no quick solution!
OSCB design brought tools to the limits BUT – to make things clear – OSCB is an exception
- Tool supports newest technology 
- Mentor Graphics is committed to do their part  
65
pdi, Layout of the OSMOSIS OSCB, May 2006