Optical Packet-Switched WDM Networks: a Cost and Energy Perspective Rodney S. Tucker ARC Special Research Centre for Ultra-Broadband Information Networks (CUBIN) University of Melbourne
[email protected]
Acknowledgements:
Jayant Baliga, Kerry Hinton, Rob Ayre, Gangxiang Shen, Wayne Sorin Australian Research Council, Cisco
Summary
Optical packet-switched networks
Network architectures Point-to-point WDM IP networks IP networks with optical grooming Optical burst switching Optical packet switching
CAPEX and network architectures Scaling
Energy consumption and network architectures Core and access networks
Disclaimers: Numbers given here are approximate - YMMV OPEX not included
E-mail me for copies of these slides
Impact on network growth
IP Packets
Edge Routers
Core Router
OXC
Optical Packet-Switched WDM Network 1h
op
Is this simple model realistic?
The Reality: Map of the Internet
Patent(s) Pending and Copyright © Lumeta Corporation 2007
Not Included: • Enterprise Networks • Content Distribution Networks • Data Centres
Number of Hops in the Internet 2006 Data
0.1
Pr [ H = k ]
0.08 0.06 0.04 0.02 0
0
5
15 10 Number of Hops, k
20
Source: P. Van Mieghem, “Performance Analysis of Computer Systems and Networks”, Cambridge (2006)
25
Evolution of Optical Packet-Switched Networks “Conventional” Wisdom Capacity CAPEX? Energy Consumption?
Optical Burst Switching (OBS)
Stage 1
Optical Packet Switching (OPS)
Optical Circuit Switching (OCS) Point-to-Point WDM (P2P) Time
1. Point-to-Point WDM Network Lightpath Fiber
O/E/O Converters
Core Router
O/E/O Converters Access Routers
Router provides sub-wavelength grooming by statistical multiplexing
2. Optical IP Network Core Router
Edge Routers
OXC
1h
op
2. Optical IP Network Lightpath Fiber
Core Router
O/E/O Converters
OXC/ROADM
O/E/O Converters Access Routers
• Optical bypass • Sub-wavelength grooming
3. Waveband IP Network Lightpath Waveband Fiber
Core Router
O/E/O Converters
MUX OXC/ROADM • Waveband grooming • Optical bypass • Sub-wavelength grooming Access Routers
Cost and Scalability of Optical Networks Consider three network architectures: P2P WDM
Number of users: 10 million
Parthiban et al., 2003
Optical IP
Waveband IP
Compare CAPEX
Input Parameters
Average Access rate per user: 1 - 100 Mb/s 10 million users
Capacities: Router: OXC: Lightpath: Fiber:
5 – 90 Tb/s 1000 ports 40 Gb/s 250 lightpaths
1000
1000 1 λ = 40 Gb/s
250 λ Component costs [Ferreira 02, Sengupta 03] OXC, router – chassis, port Fiber, amplifiers, lightpath terminations Ignore Access network, search engines, data centers, etc. Optical impairment cost – e.g. regenerators Protection & restoration Multiple domains Cost reductions as technology matures OPEX
Results – Network Cost 104
Cost per User ($)
P2P WDM
= pe o l S
1
Waveband IP 102 Optical IP
10 Today’s Internet (~ 100 kb/s) Parthiban et al., 2003
1
10 Average Access Rate (Mb/s)
(Access rate per user in a busy period)
100
Sanity Check Cumulative Capex $ Billions
90 80
Cumulative Internet Capex (North America) 150 million users
70
~$400 / user
60 50
Total
40
Core and Metro
30
Optical transport
20 10 0
Access 2000
2005
Year
2010
Nemertes, November 19, 2007: “User demand could outpace network capacity by 2010 $137 billion global infrastructure investment needed” Based on data from Nemertes Research, 2007
Observations •
Optical IP network – Can save costs in today’s network – Optical bypass reduces router ports
•
Waveband IP network – Eliminates the bottleneck in number of ports and lightpaths – Least costly for high access rates
•
There is no such thing as a free lunch – If you want more bandwidth, you will have to pay for it
Energy Consumption of the Network Hot spot
Why worry about energy consumption? • OPEX • Greenhouse Impact • Managing “Hot Spots” - Getting the energy in - Getting the heat out • Energy-limited capacity bottlenecks
Power In
Energy Model of Simple IP Network CRS-1 ~ 10 kW / rack Packet over Sonet
Core
Core
Core
Fibre Amps
12816 Edge ~ 4 kW
Metro
WDM
Edge
OLT - 100W
Passive Optical Network
Curb
Edge
Curb Curb
ONU ~ 5-10W
0.1 - 1000 Mb/s to the user Baliga et al., 2007
Core
Curb
Access
Power Consumption of IP Network 25 20 hops
2008 Technology 1.0
Total 15
Today’s Internet (~ 100 kb/s)
Core
10
0.5
Access +Metro 5
SDH/WDM Links 0
0
2
4
6
8
Average Access Rate (Mb/s)
Baliga et al., 2007
10
0
% of Electricity Supply
Power (W/user)
20
Power Consumption of IP Network 10
200 2008 Technology
7.5
Power (W/user)
150
Total 5.0
100
Core 2.5
50
Access +Metro 0
0
20
40
60
SDH/WDM
80
Average Access Rate (Mb/s)
Baliga et al., 2007
100
0
% of Electricity Supply
20 hops
Observations / Questions •
Access network dominates energy consumption at low rates – Standby mode?
•
Core network dominates at higher rates – Reduce hop count?
•
What is the bottleneck in the core? Speed or energy?
1
– Optical packet switching
3
•
Optical transport (WDM) consumes relatively little energy < 5% of energy > 25% of CAPEX
•
Annual CAPEX / Annual energy OPEX > 2
2
Electronic Router Forwarding Engine
1
Fibers 1
1
1
J Switch Fabric
F
F
K F Demutiplexers O/E Converters
Switch Fabrics
Buffers
Reduced bit rate Electronics
Optics
F Multiplexers
Power Consumption in Routers
Power consumption (W)
1,000,000 ?
P = C2/3
100,000
where P is in Watts where C is in Mb/s
10,000 10 nJ/bit 1,000
P ~ 10 100 100 nJ/bit 10 1
1 Mb/s
1 Gb/s
1 Tb/s
Router Throughput
Source: METI, 2006, Nordman, 2007
1 Pb/s
Energy per Bit in Routers 100,000
Energy per Bit (nJ)
10,000 1,000 100 nJ/bit 100 10 nJ/bit 10 ?
1 nJ/bit
1 0.1
1 Mb/s
1 Gb/s
1 Tb/s
Router Throughput
1 Pb/s
Heat Load Air Cooling Limit 1 nJ/bit
Year Source: K. Brill, The Uptime Institute
Energy in High-End Electronic Router Line Card Buffer O/E
Forwarding Engine Switch Fabric
Buffer
I/O
Buffer
O/E
Forwarding Engine
Switch Control
Routing Tables
Routing Engine Control Plane
Data Plane
Power supply inefficiency Fans and blowers
Energy/bit
0.7 nJ
3.2 nJ
0.5 nJ
1.0 nJ
1.1 nJ
3.5 nJ
Fraction of Total
7%
32%
5%
10%
11%
35%
Source: G. Epps, Cisco, 2007
Network Energy Consumption per Bit 10-3 20 hops Energy per bit (J)
10-4 ~1 μJ/b Total
10-5 10-6 10-7
Core Metro + Access WDM Links
10-8 0.1
1
10
Average Access Rate (Mb/s)
100
Energy in Electronic Integrated Circuits
Cwire CMOS Gates CMOS IC
Energy =
∑
Egate +
1⎡ 2⎣
∑
Cwire ⎤ V 2 ⎦
Power = Energy x Bit Rate
Diversion: Capacitance of the Core Network Source
Destination
V
C2
C1
E =1 ⎡ 2⎣
Energy per bit per user: V = 2, E =
1 μJ 2
→
∑
∑
C ⎤V 2 ⎦
C ≈ 200 nF
Capacitance per user
50% efficiency ~ 150 million users → CNetwork = 200 nF x1.5 × 108
C Network ≈ 30 F (North America)
C Network ≈ 300 F (Global)
Energy Consumption in Access Networks NEC CM7710T
Access N/W Edge Node Cisco 12816
Cabinet Splitter
NEC CM7700S
Cabinet
PON NEC VF200F6
FTTN Zyxel VES-1616F-34
Cisco 4503
NEC GM100
PtP Axxcelera ExcelMax CPE Axxcelera ExcelMax BTS
Baliga et al., OThT6
WiMAX
Energy Consumption in Access Networks WiMAX Energy per bit (J)
10-5
PtP 10-7
10-8
• • •
FTTN
10-6
PON
1
10 100 Average Access Rate (Mb/s)
Wireless access consumes more energy than optical access PON FTTH is “greener” than FTTN “Standby mode” shows significant potential (OThT6)
Baliga et al., OThT6
1000
Evolution of Optical Packet-Switched Networks Capacity Optical Packet Switching (OPS)
Stage 2 CAPEX and Energy Barrier
Stage 1 Point-toPoint WDM (P2P)
Optical Burst Switching (OBS)
All-optical sub-wavelength grooming
Optical Circuit Switching (OCS)
Time
Stage 2 Evolution – OCS to OBS
Core Router Edge Node
Optical Burst Switches Electronics WDM Link
OXC
Edge Router
Access Routers Burst Assembly Router
Optical Burst Switched Network Optical Burst Switch No Buffering
IP Packets (duration < μs)
Data Bursts (duration < ms) IP Packets t Burst Assembly Router
C. Qiao and M. Yoo, JHSN, 1999
Headers Offset
t
Optical Burst Switching Source
Header
Destination
Offset
Time
Burst length < ms
D A T A
Switch
1
2
3
4 Distance
5
Normalized Cost: OBS vs. IP 20
Optical IP
Normalized Cost
1 Gbit/s Access Rate 15
Without wavelength conversion OBS
10 With wavelength conversion 5
Waveband IP
0
10-6
10-5
10-4
10-3
10-2
Average Path Blocking Probability
Parthiban et al., OFC 2005
10-1
Why is OBS more Costly? OBS MUX
Wavelengths
Waveband Bursts
Wavelengths
Bursts
DEMX Waveband
•
Requires increased number of lightpaths for a given blocking probability
•
Switch technology requires fast reconfiguration time: More costly per port than “slow” OXC’s (MEMS etc.)
Solution: Waveband Burst Switching (WBS) Waveband Route λ
Burst
t
WBS Waveband
Waveband
Pro’s: • Requires fewer OXC ports Con’s: • OXC ports must be wideband • Requires waveband (i.e. multi-channel) wavelength conversion • Dispersion issues Y. Huang et al., OFC 2004, Pathiban et al., OFC 2006
Waveband Burst Switching (WBS) 1 Gbit/s Access Rate 7 Normalized Cost
Optical IP 5
WBS with wavelength conversion
3
Waveband IP
1
WBS (WC) + Deflection Routing + Burst Segmentation
10-5
Parthiban et al., OFC 2006
10-4
10-3
10-2-2 10-1-1 10 10 Average Path Blocking Probability
Evolution of Optical Packet-Switched Networks Capacity
Stage 3
Stage 2
Stage 1 Point-toPoint WDM (P2P)
Optical Burst Switching (OBS)
Optical Packet Switching (OPS)
Optical Circuit Switching (OCS)
Time
Optical Packet-Switched Network The “Holy Grail” of Optical Networking
Optical Packet Switches With Buffering
OXC
Access Router
• Will a viable optical buffering technology emerge? • Can OPS reduce energy consumption?
Optical Packet Switch (OPS)
Forwarding Engine
Electronics
Optics
1
1
1
F Demutiplexers
K
WavelengthInterchanging Cross Connect + Header Replacement
F
Output Buffers
Input Synchronizers
2
1
Mutiplexers
Optical Buffer Structures Control
Delay Line
Delay
Variable Delay Line
Delay
Recirculating Loop
Cross Point
Staggered Delay Line
Delay
Cross Point
Delay
Cross Point
Optical Fiber Buffers Total buffer capacity of 104 Gigabits ~ 103 RAM chips Cost < US$ 50k Buffer power dissipation < 1 kW Cisco CRS-1 with 1000 ports, 250 ms buffering per port
Total fibre length = 40 Gm 150 times distance from Earth to Moon! .
OPS with 1000 ports, 250 ms buffering per port optical fiber delay lines
Reduced Buffer Size Minimum feasible buffer size?
~5 μs (~20 packets) buffering per port
Optical fiber delay lines
Total fibre length = 1000 km Loss per port = 0.2 dB
Planar waveguide or slow light delay lines (0.1 dB/cm)
Loss per port = 33,000 dB Loss Tucker, PS 2007
Energy
Holographic Buffers Holographic Medium: Photorefractive material (e.g LiNbO3, polymers) Output image Write speeds up to 1 Gb/s Read speeds up to 10 Gb/s Reference beam Input image
Orlov et al., Proc IEEE, 2004 Psaltis, CLEO 2002 Ashley et al. IBM J. Res. Dev., May 2000
Storage density
Retention time
Access Time
Write Time
1/λ3
∞
~ 50 μs
> 500 μs
Comparison of Optical Buffer Technologies Technology
Fiber
Planar WG, Slow Light
Optical Resonator
Holographic
CMOS
Access Time
Structuredependent
Structuredependent
Small
~ 50 μs
200 ps
Retention Time
> 500 μs
< 5 μs
1-100 ns
∞
64 ms
Capacity (Packets)
> 2,000
< 20
100 pJ/bit (10 pJ/bit per gate)
Benes Array
Tucker, JLT 2005, OSN 2008
Benes
Chips (2 pJ/bit)
Observations on Optical Packet Switching •
No viable optical buffering technology in sight
•
Optical switch fabrics may become competitive with CMOS
•
Not clear whether optical packet switching will solve the energy bottleneck problem
Summary •
Optical bypass reduces CAPEX and energy consumption - Promising future for optical cross-connects and ROADMs
•
Energy bottleneck in routers is looming - More significant than the so-called “electronic speed bottleneck”
•
Can optical packet switching overcome the energy bottleneck? - Optical buffering is currently a show-stopper
•
Think “Energy per Bit”
