100G Ethernet Technology and Applications

Zhang Yuanwang

(ZTE Corporation， Shenzhen 518057， P. R. China)

Abstract:The increasing requirements for bandwidth drive the I 00G Ethernet into use as quickly as possible.The major technologies supporting I00G Ethernet interface include the physical layer channel convergence technology,multi-fiber channel and Wavelength Division Multiplexing(WDM)technology.The key technology for high-speed optical devices at the interface requires a breakthrough.The demands driven by the increase of interface rate require better packet processing and storage,system switching,and backplane technology.In addition,in the network,the transport problem of the new interface should be solved,including defining transport standards and pushing the development of key transport technologies.As far as the cost and requirements are concerned,the commercial application of I00G Ethernet is viable in Metropolitan Area Networks(MANs).

T he basic demand for upgrading the Ethernet interface rate to I00 Gbit/s is the increase of bandwidth.

The precondition for promoting the I00G Ethernet application is the establishment of relevant standards.The standard corresponding to I00 Gbit/s Ethernet interface is IEEE802.3ba［I］,whose draft 2.3 is formed［2］.In the standard,the medium interface,rate and Physical Code Sublayer(PCS),and Media Access Control(MAC)architecture are defined.In addition,the standard organizations relevant with I00GEinclude ITUTelecommunication Standardization Sector(ITU-T)and Optical Interconnection Forum(OIF).

Upgrading Ethernet to I00 Gbit/s interface requires the support of key technology.The mature and commercial application of key technology takes certain time.Technical breakthrough and time are required for chip,system,network and standard.

1 100G Ethernet Technology and Standards

The key technologies supporting I00G Ethernet interface include physical channel convergence technology,multiple-fiber channel and WDM technology.The Physical Media Dependent(PMD)sublayer meets the I00 Gbit/s rate.New chip technology support 40 nm technique.These provide the possibility of developing next generation high-speed interface.For the interface,the parallel multiple-mode interface of fiber interface PMD has the problem of intensive encapsulation and great power consumption.The Wavelength Division Multiplexing(WDM)interface of single-mode 4X25 Gbit/s should break through the technology of serial-parallel conversion circuit(SERDES)technology and the non-cooled optical components.For the system,the increase of interface rate brings new threshold for packet processing,storage,system switching,and backplane technology.For the network,the transport problem of the new interface should be solved.New Optical Transmission Unit(OTU)frame structure should be defined.In addition,ultra-high-speed transport,the problems including signal processing,optical signal modulation,physical coding,dispersion compensation,non-linear processing,compatibility and consistency with Fast Ethernet(FE)/Gigabit Ethernet(GE)/I0GEframe structure and physical sub-layers should be solved.The I 00G transport features should meet the relevant features of I 0G transport network.Otherwise,the reconstruction will impact the progress of new technology.

The next-generation Ethernet technology standard includes 40 Gbit/s and I00 Gbit/s,which are designed for different requirements of server and network.The 40 Gbit/s rate is mainly for the calculation application but the I00 Gbit/s rate is mainly for the core and convergence application.Two rates are provided because IEEEwants to ensure that the Ethernet can meet requirements of different applications effectively and economically to promote the network convergence based on Ethernet technology.The standard defines the modules and connection interface bus of Physical Coding Sublayer(PCS),Physical Medium Access(PMA)sublayer,PMD sublayer,and Forwarding Error Correction(FEC).The inter-chip bus between MACand PHYuse XLAUI(40 Gbit/s)and CAUI(I 00 Gbit/s).The intra-chip bus uses XLGMII(40 Gbit/s)and CGMII(I00 Gbit/s).The architecture of each medium is shown in Figure I［3］.

▲Figure 1. Various medium architectures defined in IEEE802.3ba.

The standard supports only full-duplex operation.The 802.3MAC Ethernet frame format is reserved.Multiple physical medium interface regulations are defined,including I m backplane connection(definition ofI00GEinterface no-backplane connection),7 m copper cable,I00 m parallel multiplex-mode fiber and I 0 km single-mode fiber(based on WDM technology).I 00 Gbit/s interface defines 40 km transport distance at most.The standard defines the architecture of Multi-Lane Distribution(MLD)in PCS.It also defines the regulation of electrical interface for inter-chip connection.40 Gbit/s and I00 Gbit/s respectively use four and ten I0.3I25 Gbit/s channels.It uses polling mechanism to allocate data to obtain the 40G and I00G rates.In addition,the definition of virtual channel solves the problem of adapting to different physical channel of long optical wavelength.It specifies that the physical layer coding uses 64B/66B.

The standard provides the architecture and interface definition of I00 Gbit/s Ethernet,but many problems remain unsolved.First,PMD is one key part of 802.3a.40G/I00G optical module contains the parallel interface of short wavelength corresponding to 40GBASE-4SRand I00GBASE-I0SR.

The technical difficulty lies in the intensity of encapsulation.For the wavelength division interface of long wavelength,the difficulty lies in the SERDESand encapsulation technology of 25 Gbit/s corresponding to PMA.For I00GWDM optical module,non-cooled laser technology is a key technology related with the standard.The encapsulation mode is specified to CFPby the CFP Multiple Source Agreement(MSA)［4］.The definition of copper Medium Dependent Interface(MDI)standard uses SFF-8436 and SFF-8642.The detailed architecture size and pin assignment are provided.As we know,the major suppliers will not provide the I00G WDM optical module until20I0.

For the relevant chips of I 00G interface,the MAC layer is ready.The electrical interface regulation of PMA service interface requires that each channel should work at the rate of I0.3I2 5 Gbit/s.Use the Application-Specific Integrated Circuit(ASIC)after the standard is mature.The MAC implemented based on the Field Programmable Gate Array(FPGA)should support the rate of I0.3I 2 5 Gbit/s.Only a few FPGA companies support this rate［5］.In the previous evaluation system,the SERDES Mux devices［6］are added.It is transitive measure converting 8/20 5.I56 25 Gbit/s channels to 4/I 0 I0.3I2 5 Gbit/s standard interfaces［7］.

For the I00G Ethernet device system,the preceding technical problems of I00GEthernet interface are hard to remove.The relevant packet processer is required.For the distributed large-capacity switching system,the problem of chipsets of large-capacity packet switching system should be solved.

For the I00G packet processing capability,no universal solution is available in the industry,The solution in progress is to be evaluated.For the Content Address Memory(CAM)of the network processor,the bandwidth of search interface should be doubled.

There is a bottleneck for the data bus width and rate.It encourages the application of serial high-speed bus interface such as Interlaken LA.Owing to the restriction of the single chip processing capability and the pileup of multiple chips caused by bus interface conversion,the board area and power consumption are unacceptable.The solution based on FPGA-customization requires comprehensive technology.The provided service processing capability is restricted.

The packet switching system chipset,including switching network and switching network interface chips probably contain the Traffic Management(TM)chip.Previously,most systems do not support valid data bandwidth greater than I00 Gbit/s for each line.In the recent new solution,the backplane bandwidth of each line card is about I00-200 Gbit/s.The backplane SERDES bus rate supports up to 6.5 Gbit/s.Supporting I00 Gbit/s interface,the bandwidth of each line card should be upgraded to 200-500 Gbit/s bandwidth.The backplane SERDESrate should reach I0.3I 2 5 Gbit/s or higher.The requirements for backplane design,technic,material,and meeting the bus length are more critical than before.For the system meeting the carrier-class requirements,the Virtual Output Queue(VOQ)and Hierarchy Quality of Service(HQoS)should be met.It requires greater processing bandwidth,queue processing capability,and buffer,which greatly increases the difficulty of system design.

With the increase of system requirements,the power of the system is also increasing.I00 Gbit/s long wavelength PMD requires four 25 Gbit/s channels.The increase of SERDESrate and channels require greater power supply.The I 00 Gbit/s processor requires larger number of memories and greater power.The micro-processing technology also requires greater power.Consequently,we need to find a solution.

If the high-speed Ethernet wants to bring real benefits to users,the transport network must carry the transport network service.It should not only apply the technology to the large data center or small LAN.Except the modulation technology,the transport over optical transport network of high-speed Ethernet and the Operation and Maintenance(OAM)also determine the success of the technology.In the Jeju Island conference of ITU-TSGI 5 QII in September 2008,the definition of OTU mapping of 40G/I00G Ethernet interface is formed［8］.G.709 defines that 40GEis mapped to OPU3.It uses I,024B/I,027B transport codes.I00GEis mapped to ODU4/OTU4,and the bit rate is III.809973 Gbit/s(=255/227×2.488320 Gbit/s×40).The standard is expected to get mature in 20II/20I2.For the high-speed services including I 00 Gbit/s Ethernet,the virtual concatenation can implement the adaptation.But to improve the utilization of fiber,virtual concatenation is not an efficient technology.It only improves the bit rate of each wavelength.

The serial I 00G Dense Wavelength Division Multiplex(DWDM)technology is used to adapt the I0×I0GE/4×25GE I 00GEservice to the OTU4 of III.809 973 Gbit/s through ODU4.The rate single-wave I00G is quite high.The requirements for physical-damage tolerance,such as Optical Signal Noise Ratio(OSNR)and PMD,are also increased.Special technology is required to decrease the baud rate of optical signals transmitted in the fiber to increase the damage tolerance.For example,advanced code modulation technology such as Quadrature Phase Shift Keying(QPSK),8PSK,QAM,Orthogonal Frequency Division Multiplexing(OFDM),and polarization multiplexing/demultiplexing are used.The single-wave I00 GEhas more restrict requirements for PMD and Color Dispersion(CD).Therefore,the receiving end may use the receiving/electrical processing mode to increase the tolerance of physical damage,including non-linear effect suppression,PMD and CD compensation.As a result,the single-wave I00GEcan be transmitted in a mixed mode and upgraded smoothly in the I 0G/40G network.

From a long run,the I00GEDWDM transport uses an integrated solution including polarization multiplexing,high-level coding modulation,coherent reception/electrical processing,ultra FEC.Then,the 40G optical network can be upgraded to the I00G system smoothly.The I 00GEtransport requires the support of high-speed optical devices.It is estimated that the high-speed optical devices will become mature in 20I2.

For the key technologies mentioned above,I00 Gbit/s Ethernet is not only the technology for solving interface construction,the processing capability of the system should also be improved for the large-capacity switching system.High bandwidth traffic management and packet processing capability can provide line speed processing and forwarding.As a result,the carrier-class features are provided.ZXCME 9500 Metropolitan Area Network(MAN)Ethernet hardware platform developed by ZTECorporation currently supports I00 Gbit/s interface bandwidth.Sufficient redundancy and acceleration ratio are reserved.Line speed forwarding of a single I00 Gbit/s is supported without changing the switching network and system heat dissipation.Only a new line card is required.After the system upgrading,the single line card bidirectional 400 Gbit/s or above bandwidth is supported.At the same time,though the problem of a single system is solved,the application of I00G Ethernet interface is also restricted by the improvement of transport network technology.We still have a long way to go before the commercial application of I00GE.

2 Applications of 100G Ethernet

According to IEEE802.3ba Task Force(TF),the standard will be completed in the middle of 20I0.The commercial application is determined by many factors.

First,if the standard is mature,it should be driven by actual network demands.In addition,it should comply with the benefits of carriers.The major factors of bandwidth demands include:the increasing services are based on IP,like described of ALL IP.All IPpackets are encapsulated in Ethernet frames from the source to the destination.The TDM over Ethernet technology is mature.It is compatible with traditional voice service.Ethernet encapsulation is simpler than Synchronous Optical Network(SONET)/Synchronous Digital Hierarchy(SDH),and the cost is lower.These determine that the demands of upgrading Ethernet interface rate to I00 Gbit/s is urgent.In I00 Gbit/s Ethernet,the objective of"network communication is accelerated;the application efficiency is improved"can be realized.Fast access to applications stored in the data center,frequency bandwidth management,fast retrieval,compression,path optimization,and protocol acceleration can be implemented.Figure 2 illustrates the detailed scenarios［9］.For the application of convergence layer,the downlink port is switched to I0 Gbit/s,and the uplink can only use the link convergence of I 0 Gbit/s port.If the I00 Gbit/s Ethernet interface is used,the management,assignment,and efficiency of data flow can be improved.For the data center,with the increase of I0 Gbit/s interfaces,the demands for uplink and internal interconnection high-speed interfaces also exist.The efficient transport of backbone network also expects the mature of I00G high-speed interface and transport.

When the P802.3ba standard is made,the relevant standards of electrical interfaces and the mature condition of the technology are taken into consideration.

The inter-chip transport channel of I 0.3I2 5 Gbit/s is used.Multiple-mode parallel optical interface supports the distance of I00m or more in OM3 fiber.Single-mode 40GB ASE-LR4 uses Coarse Wavelength Division Multiplexing(CWDM),which is cost-effective.I00GB ASE-LR4 uses DWDM,each wavelength transmits 25.78I 25 Gbit/s,using I,295-I,3I0 nm wavelength.Therefore,the original fiber can be used.Take the technology and cost into consideration,the technologies selected by the standard is practicable,which helps the commercial application of I00G interface partially or in the MAN.

For the application in overall network,before the I00GEtransport standard and technology are mature,you can use the inverse multiplexing technology.Adapt the I00GEservice of I0×I0GEor 4×25GEinterface to OTU2/OTU3 through ODU2/ODU3.Transmit the service in the I0G/40G opticalnetwork through multiple wavelengths.The current I0G/40G DWDM optical network need not be re-designed or changed.The transport code pattern is Optical Double Binary(ODB)/Differential Return-Zero(DRZ)/Electrical Return-Zero Differential Quadrature Phase Shift Keying(eRZ-DQPSK).The mode can use the current mature optical devices of I0G/40G.The performance index of the entire system is consistent with that of I0G/40G system.Through the solution,the smooth upgrade of the network can be implemented.It meets the expectation of the carrier for cost.In addition,the devices are mature［I0］.

According to the current cost and demands,first applying the I00 Gbit/s Ethernet in the MAN is a practicable solution.In the MAN,a great deal of data should be uploaded and downloaded.A transport system requiring no compensation devices willsimplify the network design.I00 Gbit/s Ethernet can meet the demand.At the same time,the high bandwidth meets the requirements of 40%increase in traffic of the MAN.In a word,the demands for developing I00 Gbit/s Ethernet is obvious.The advantage of cost will be enhanced.But the technology of modulation mode and OAM etc.in the Ethernet transport should be improved continually.We have a long way to go before the large-scale commercial application.

In addition,in the technical upgrade,besides I00 Gbit/s Ethernet,fiber channel,InfiniBand,and SONETare also involved in the 40/I00 Gbit/s network.In the late I990s,the decreasing rate of price of Ethernet port devices is double of that of the ATM and FDDIdevices.Meanwhile,40 Gbit/s and faster network share similar FPGA,SERDES,and coding technology.As a result,the devices corresponding to any protocol are hard to obtain the cost advantage through volume production.I00 Gbit/s Ethernet will not dominate the industry like before.

3 Conclusions

Generally speaking,the I00 Gbit/s Ethernet technology is vital and widely concerned.Except the challenge of technology and commercial application,the opportunity for involved parties is great.For the study organizations,it is an opportunity of discovery and innovation;for the device and module suppliers,it is a new and high-repayment;for the system provider,it is an opportunity of dumping the tray and leading the market.

▲Figure 2. 100 Gbit/s Ethernet applications.