Dan Magorian Director of Engineering & Operations What’s MAX Production Been Up to? Presentation to MAX membership Spring 07 Member Meeting

CLPK MCLN BALT MCLN ASHB ARLG DCGW DCNE Internet2 NewNet NGIX T640 M40E Cogent ISP R&E Nets Old Abilene Qwest ISP Qwest ISP MAX Production topology 1. Original Zhone dwdm over Qwest fiber 2. Movaz dwdm over State Md fiber 3. Gige on HHMI dwdm over Abovenet fiber 4. Univ Sys Md MRV dwdm, various fiber Ring 1 Ring 2 Ring 3 Ring 4 National LambdaRail

What has the Production Side of MAX been doing since the Fall member meeting? Provisioned lambdas to help Internet2 transition to Newnet & ESnet routers in Level3 MCLN pop seamlessly. –Old Qwest router leaving for Kansas City Friday! Designing new dwdm system for Baltimore BERnet –Uses “client dwdm”, lower end-to-end transponder costs. –Inexpensive idea usable anyone w/ local fiber, eg NOAA. –Interconnecting with new MIT pop in Baltimore at Level3 DWDM ring replacement: Phase 1 this FY –listening to vendor presentations now –TAC meetings for community input next month New lab across hall MAX hdq (almost) done, tours welcome. Buying advanced 1G/10G ethernet/sonet tester

Ongoing projects (con’t) 10G backbone upgrade underway, Juniper pics here soon HHMI very generously providing 10G protected lambda to Equinix Ashburn replacing 1G. Thanks! GWU provided old Juniper M20 and M10 routers for MAX lab. Thanks! Community welcome to use MAX test lab. Provisioning Juniper VRFs for experimental service testing. Ciena CoreDirectors only took 9 months to buy (!), will be installed for NGIX & Internet2 dynamic wave (DCS) fanout. Want to convene community focus group to listen to member colo needs, mesh with what’s happening with I2. AtlanticWave: slowly going production, governance group service document work, MAX providing measurement pcs.

New Service: Layer 2 vlans MAX has traditionally run Layer 1 (optical) and Layer 3 (routed IP) service. –Only NGIX/E exchange point is Layer 2 service. –Continues to be demand for non-routed L2 service (vlans), similar to NLR’s FrameNet service. –As replace old Luxn dwdm system and put in new, quite economic to put in dwdm filters and 10G ethernet switches with “colored” optics (XFPs) This means that folks will be able to stretch private vlans from DC to Mclean to Baltimore over shared 10G channel. Also will be able to provision dedicated ethernet bandwidth –Haven’t figured out service costs yet. Will ask TAC.

New Service: Flow analysis Turns out it would be useful to be able to characterize traffic flows passing thru MAX infrastructure. –So MAX buying Juniper hardware assists to do this. –Starting to look at open-source Flow Tools analysis. Have in mind web interface where people put in AS #s – then look at flows to/from their institutions By date range (yr), ASN, source & dest, traffic type. The data is anonymized, packet contents not collected. Have the cost for this one figured out: free! Will ask TAC about details of what would like to see after have a prototype to comment on. Also just heard about a local flow analysis workshop by expert kc claffey of CAIDA to be held at Loyola Columbia.

Flash: MIT is building Nortel dwdm system into Baltimore & offering to trade lambdas They have well-funded DR initiative paying for ring build –from Boston to Albany and NYC, –spur down coast thru Philly to Baltimore Level3 pop. USM already has 2 fiber pairs 660 Redwood – 300 W Lexington, donating one for this (thanks!) So now possibility of not just: –Baltimore lambdas south to MAX and Mclean to NLR and Internet2, –But also north to NYC and Boston, eg NOX/Harvard –A bit like AtlanticWave “north” MIT costs/lambda still under discussion, but eager to trade, may give us a shared 10G in exchange.

BERnet dwdm 6 St Paul MIT Nortel dwdm Boston Long-distance high-level diagram of new dwdm system (meeting MIT in Baltimore) MIT Nortel dwdm Albany MIT Nortel dwdm NYC MIT Nortel dwdm BALT BERnet Participants BERnet Participants MAX dwdm MCLN MAX dwdm CLPK NLR and I2 lambdas Lambdas to Europe

BALT M40e MAX view of current Baltimore L1/L3 topology (both lambdas pass thru CLPK router) CLPK T640 MCLN T640 BERnet Participants Internet2 NLR East & west OC48 lambdas on USM MRV ring MAX Movaz lambdas

Proposed BERnet Production L1/L2 topology USM F10 6 St Paul USM F10 Redwood St USM MRV USM MRV BERnet Production 10G lambdas USM MRV New MAX F10 CLPK New MAX F10 MCLN CLPK T640MCLN T640 MAX Movaz Dwdm New DC ring BERnet Participants BERnet Participants Participant Production lambdas Xconnect New BERnet Res DWDM

High Level Regional Diagram of Balt Research Net MCLN NLR & I2 College Park 6 St. Paul660 RedwoodUMBC JHUJHMIMIT 300 Lex. 40 wavelength Amplified Line Side Path 40 wavelength MUX w ITU XFP’s Client Side Path One Transponder Pair to Pay for and Provision End to End

Node Sites MCLN NLR & I2 College Park 6 St. Paul Three nodes and room for Gbps channels ~ $130K per site with Mclean Level 3 and College Park nodes being paid for by MAX. Pre Amplifiers at all three sites

“Client” and “Line” dwdm provisioning example MCLN NLR & I2 College Park 6 St. Paul660 Redwood JHU Client-side: instead of “normal” 1310 optic, “colored” dwdm XFP pair on an assigned wavelength 40 km reach $6K Line side: Transponder Pair makes the 185 km reach with Amps and holds the client XFP $32K

More “Client Dwdm” examples between participants 6 St. Paul 660 Redwood JHU switch XFP pairs on an assigned wavelengths on 40 channel dwdm filters (40 km reach $6K). Red lambda is to DC, blue to NYC, green local to UMBC DWDM 300 W Lexington DWDM UMBC DWDM All each participant fiber needs is filter pair, not full dwdm chassis

MAX will replace some dwdm nodes this FY (summer install) and some next FY Phase 1: College Park, Mclean, Baltimore research net. –Since have protected ring, can cut one side, then other. –Balt research fiber is new (actually old Ciena OC48). –Will select and procure vendor soon. Phase 2: GWU and Qwest Eckington next FY –Replacing original Luxn dwdm ring –Demand for lambda services to Dc unknown Have met with Qwest for comarketing to justify DCNE upgrade –Might do client dwdm with shared 10G ethernet instead This means that MAX working to unified new dwdm system! Get involved with TAC to hear details:

Closing thought: “Lots of lambdas in your future, what are you going to do about it?” This was the title of a BOF at the I2 MM yesterday hosted by Joe St Sauver, U Oregon. Idea was to get people thinking about the sea change happening in the community –appropriate campus strategies for fanout –to researcher high-bandwidth projects. Not replacement of “regular” ip routing, but in addition Circuit-like services for high-bandwidth low-latency projects People may say, “I don’t have any of those apps right now” –But suddenly may have researchers with check in hand –Need to plan for how will deliver, just as BERnet doing –Eg, in planning phase now for DC ring, need to forecast. –Talk to us about what you’re doing and thinking!

Thanks!