Transcript ARNES experience with obtaining and lighting dark fiber Avgust Jauk Arnes

1
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
ARNES experience with obtaining and
lighting dark fiber
Avgust Jauk
Arnes
[email protected]
CEF Networks workshop, Prague, May 15th, 2009
2
Agenda
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Arnes
Requirements for the network
Tendering procedure
– Fiber
– WDM equipment
Experience with WDM and dark fiber
Plans
3
Arnes
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Arnes: Academic and Research Network of Slovenia
Public institute
Service provider for research & education in SI
– Network
– Services
Mission: to provide efficient infrastructures and services
in order to allow Slovenia equal opportunities for
cooperation in the unified European research and
educational area.
4
Arnes – User Community
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Closed user group!
1.100 connected institutions:
– Universities, institutes, research laboratories
– Museums, schools, (digital) libraries
200.000 individual users
5
Arnes Backbone, January 2003
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
6
Requirements for the new network
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Support for “ordinary” as well as “advanced” use
Advanced users
– Research experiments, Grids, …
• Physics, Chemistry, Meteorology, …
– Business Continuity, Disaster Recovery
• Data replication, remote backup, …
Hybrid network model
– IP (IPv4, IPv6)
• QoS, multicast
– Dedicated point-to-point links (Gbps, 10Gbps)
7
Requirements for the new network
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Performance:
– n * GE between smaller PoPs
– n * 10GE between main PoPs
Availability
– Minimize outages: fiber routing, equipment
– BER <= 10^-12
Affordability
– High capacity leased lines far to expensive!
8
Tendering for dark fiber
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Tenders:
– 2000: fiber in Ljubljana
– 2003, 2008: intercity links
Market failure
– Late in deregulation
– Lack of competition in intercity fiber
• Telekom Slovenije
• Electricity distribution company
• Highway company
Result
– High prices
– Limited flexibility
• fiber types
– G.652A only – water peak!
– But managed to get PMD <= 0.2 ps/sqrt(km)
• Maintenance options
– Problems with repair time in case of aerial (OPWG) cables
Redundancy is a must!
9
Leased fiber, May 2009: 1200 km
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
9
10
Requirements for the WDM network
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Specifics: Slovenia is a small country, but
–
–
–
–
–
A lot of PoPs (in 24 cities)
Many PoPs are relatively small, without advanced users
Distance between PoPs up to 100km
Expensive fiber lease!
Financial constraints
Solution:
–
–
–
DWDM only between major PoPs
CWDM between all PoPs
Operation over a single fiber strand!
11
Bidirectional use of optical fiber
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
One fiber strand used for DWDM between main PoPs
Another fiber strand to connect smaller PoPs with GigE
using passive CWDM
DWDM equip.
DWDM Equip.
10 GigE implemented with DWDM
IP router
IP router
Gigabit Ethernet
Implemented with CWDM IP router
Ljubljana
fiber pair
Celje
11
Maribor
12
Requirements for the CWDM network `03
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Performance
– At least 4 channels, 1Gbps
Price/performance
– Single fiber strand operation!
– Passive device: filters only
• CWDM pluggables in routers
– Compact size
– Affordable price
13
CWDM network topology (Sunrise POC)
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
13
14
Requirements for the DWDM network `07
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Performance
– Future proof (high traffic growth rate!)
• At least 16 channels, 10Gbps
Availability
– No protection on optical layer
• On L2 and L3 only!
– BER < 10^-12
15
Requirements for the DWDM network `07
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Price/performance
–
–
–
–
–
–
–
–
Single fiber strand operation!
Support for 10GE LAN PHY, without regeneration!
OTN was not required
Support for alien lambdas (Cisco DWDM Xenpak)
Compact size, green (low power consumption)
Easy to install, operate and maintain
Flexible, support for different applications (cards)
Avoiding “cheap initial config / expensive add-on”
sindrom
16
DWDM Network – Solution
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Single Fiber Working ADVA FSP 3000 with enhanced EDFAs
17
DWDM network topology (ADVA FSP 3000)
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
17
18
Experience with WDM / leased fiber
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
General remarks
–
–
–
–
A lot of work with tendering, but it pays off
Dealing with fiber providers can be challenging
Need for basic knowledge on optical and WDM
DWDM easier to install and run than expected
• Routers are more difficult!
– Modern metro DWDM solutions can be quite compact (7HU,
shallow: back to back)
– Alien lambda support really works (tested with DWDM Xenpaks)
– Problems with pluggable modules for routers:
• Form factors (SFP, GBIC, Xenpak, X2, XFP, SFP+)
• Vendor locking
19
Experience with WDM / leased fiber
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Operational: WDM
– WDM equipment is reliable. BER ~ 0
– Firmware upgrades even for DWDM ;-)
– History of Tx & Rx power very usefull
• SNMP support on DWDM equipment
• SNMP and SFF-8472 (Diagnostic Monitoring Interface)
support on routers
20
Experience with WDM / leased fiber
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Operational: fiber
–
–
–
–
Measuring PMD makes sense
Fiber cuts (constructions, rats), bends, …
Quite a lot of maintenance work
Aerial fiber:
• No problems with stability due to temperature and wind
• But problems with shooters and thieves
• Repair can take days
– CWDM & DWDM in parallel, Cacti -> easier troubleshooting
Excellent price/performance
21
Finally enough bandwidth!
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
22
Backbone – alternative view
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
23
Plans
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
Connect cities passed by the fiber
Add more Bandwidth when needed
– Additional C/DWDM channels
– Upgrading CWDM to 10G speeds and/or DWDM
• Concurrent use of CWDM and DWDM on the same fiber
– 40G/100G
Looking for simple technology for Point to Point links
CBF - options:
– Austria (Maribor/Graz)
– Croatia (Krško/Zagreb)
– Italy (Sežana/Trieste, Nova Gorica/Gorizia)
24
Thank You!
Akademska
Academic
and
in raziskovalna
Research Network
mrežaof
Slovenije
Slovenia
24