All Rights Reserved, ©2007 Fujitsu Network Communications 40 Gb/s and 100 Gb/s Technologies for Research & Education Networks Tom McDermott Fujitsu July 17, 2007

All Rights Reserved, ©2007 Fujitsu Network Communications 1 Agenda Growing Bandwidth Requirements Motivation for 40 Gb/s Motivation for 100 Gb/s Client Interfaces and Network Interfaces Characteristics of optical transmission systems Technical approaches for modulation Some experiments

All Rights Reserved, ©2007 Fujitsu Network Communications 2 LHC Data Collection LHC currently plans on 10GE clear optical wavelengths for world-wide grid-computing network. When LHC comes on-line, it is predicted to produce 12 to 14 petabytes / year* 14 petabytes per year  ~ 11.2 * bits /  * 10 7 seconds ~ 3.6 * 10 9 bits / second Keeps one 10GE busy pretty much full-time forever… But real data collection is usually very bursty in time 40G & 100G would offer numerous practical benefits Data set exchange and backup, Load sharing, etc. *HP News release, Jan 27, 2004

All Rights Reserved, ©2007 Fujitsu Network Communications 3 Motivation for 40 Gb/s Support 40G Packet-Over-SONET (POS) router interfaces. Avoiding inverse-multiplexing or link-aggregation issues. System Capacity Expansion: 4 x 10GE, 4 x OC-192 Existing DWDM Systems support ~ 40 channels in C-band, with 10 Gb/s per wavelength. Typical system has 100 GHz (0.8 nm) spacing between optical channels. Filter bandwidths ~ 65 gHz ( ±0.325 gHz) at half-dB down points 40 Gb/s can ‘fit’ within existing DWDM channel filters and ROADMs if the modulation bandwidth can be constrained. Permits 4x capacity expansion without additional fibers or terminals.

All Rights Reserved, ©2007 Fujitsu Network Communications 4 Motivation for 100 Gigabit Ethernet Today: Link Aggregation (802.3ag) approach for N x 10 GE. Must not re-order packets within any specific flow. How to identify flows, and force each flow to just one of the multiple 10GE lanes? Today: Equal Cost Multi Path -- A way to (almost) randomly assign each flow to one of the 10GE lanes. Dynamically it breaks: all the big flows fight for the same one lane. Performance degrades to 10GE for milliseconds to seconds.  Need a better solution. Tomorrow: Use native 100 GE link. No need to ‘hash’ flows. IEEE HSSG: 4-5 approaches being considered 802.3: Low cost, high volume, short range (300m) are key drivers 10-lane byte-striped, 1-lane x 10 colors, 5 lanes, even copper is being proposed. Low cost ~10 kM also being considered. Question: how to transport 100GE on DWDM systems?

All Rights Reserved, ©2007 Fujitsu Network Communications 5 IEEE HSSG Status Call for Interest (CFI) for Higher Speed Signalling Group [HSSG] approved by IEEE plenary in Charter is to define an IEEE standards project: Technical need and economic justification of a new standard. Successful CFI results in an IEEE Project Authorization Request (PAR). Time frame roughly 2 years to a standard (unless it gets derailed). Estimate ~4Q09. Potential detours / derailments: 40GE, 40+kM Short Range Ethernet connections that are cost-effective and producible. Typically meters. Connecting Host to Network, Data center. IEEE objective: 10x the speed at 3x the cost. Current proposals: Copper, 4x CWDM, 5x VCSEL parallel, 10x parallel, serial. Also typically specifies a medium-range interface ~ 10 kM. Current Proposals: 1x (serial), 2x, 4x, 5x parallel.

All Rights Reserved, ©2007 Fujitsu Network Communications 6 Client & Network Interfaces Ethernet Service Switch Client Interface – IEEE Proposed Standards Activity: Inter-vendor interoperable Lower performance, shorter distance Higher volume Lower cost Link status Network Interface – Not a proposed standard: High performance, longer distance Minimize wavelength consumption Manageable, FEC counts NE Service-provider’s network

All Rights Reserved, ©2007 Fujitsu Network Communications Gb/s Network Interface options Utilize multiple wavelengths, for example 10 x 10 Gb/s. Requires delay equalization across the 10 wavelengths due to differential propagation delay. Consumes ¼ of the C-band. Need to manage a band of wavelengths as a single service Lower cost T/R module, but extremely expensive in terms of fiber utilization. May be difficult for carrier to justify in a commercial network. Utilize single wavelength Single optical wavelength must carry 100 Gb/s of information Complex T/R, but extremely efficient in terms of fiber utilization. Easier for carrier to manage as a single wavelength. If the T/R cost is reasonable, much easier to justify in a commercial network.

All Rights Reserved, ©2007 Fujitsu Network Communications Gigabit Ethernet: How? Today: DWDM systems designed for 100 GHz channel spacing. Channel passband: roughly 65 GHz. Cascading ROADMs narrows the channel from there. Traditional amplitude modulation methods have upper and lower sidebands. Limits data rate to roughly 30 Gb/s per channel.  New approach needed Higher-order modulation: 2 bits / symbol Polarization multiplexing: 2 symbols at the same time. Can theoretically get us to 100~120 Gb/s per 100 GHz channel. 50 Ghz channel spacing theoretically handles 50~60 Gb/s per channel (4PSK).  Narrow channels not as well suited for 100 GE

All Rights Reserved, ©2007 Fujitsu Network Communications 9 Channel Filtering ~65 GHz Single ROADM passband Mag Cascade of ROADM passbands through multiple nodes 100 GigaSymbol/s Modulation: Spectrum too wide to fit filter 25 GigaSymbol/s Modulation: Spectral width fits OK 100 Gbit/s NRZ signal 200 GHz 25 Gbit/s NRZ signal 50 GHz ROADM passband Traveling through multiple nodes reduces equivalent filter width

All Rights Reserved, ©2007 Fujitsu Network Communications 10 Reducing the Symbol Rate Utilize multiple bits / symbol constellation QPSK, 8PSK, 16QAM, etc. Common technique for radio & wireline, where S/N can be very high. But optical devices are noisy and the channel is nonlinear, so it’s difficult to expand the constellation very much. Utilize orthogonal optical polarizations Vertical and horizontal polarizations do not crosstalk too much. Transmit two independent signal sets at the same time. Combination of the two QPSK: 2 bits / symbol Polarization Multiplexing: 2 symbols at the same time Yields 25 GigaSymbols per second on each channel. Fits within ROADM filter bandwidths Electronics is capable of operating at 25 GigaSymbol/s rate.

All Rights Reserved, ©2007 Fujitsu Network Communications Gigabit Ethernet: Transmission At 100 Gsym/s: transmission impairments are severe. Example: Chromatic Dispersion: 100 Gsym/s is 100 times worse than 10 Gb/s. 25 Gsym/sec is 6.25 times worse than 10 Gb/s. Approach: Coherently demodulate the optical signal: In-phase and Quadrature-phase components (I and Q). In essence a radio receiver with an optical front-end. Many Optical distortions then linearly map to baseband distortions.  Use baseband DSP to un-do some of these effects.

All Rights Reserved, ©2007 Fujitsu Network Communications 12 Coherent Baseband Equalization (OFC 2007 OTuA1) Optical signal is mixed with local oscillator Producing In-phase (I) and Quadrature-phase (Q) signals, for Both vertical and horizontal polarization: VI, VQ, HI, HQ. Digitize all 4 signals with high-speed AD converter. Baseband processor utilizes adaptive digital filters to remove optical distortions. VLBA telescope array uses this conversion technique at radio frequencies for mm-wave radio astronomy.

All Rights Reserved, ©2007 Fujitsu Network Communications 13 Key Technologies Key Technologies for 100GE: Multi-level modulation. Polarization Multiplexing. Coherent Receiver. DSP at baseband. Advantages to this approach: Compatibility with existing transmission systems Ease of adding 100 GE to existing services without disruption Economic use of facilities for 100GE Disadvantages: More complex T/R module than parallel optical lanes.

All Rights Reserved, ©2007 Fujitsu Network Communications 14