SERIAL CONTROL AND DATA INTERCONNECT SYSTEM USING A CROSSBAR SWITCH AND A MULTIPOINT TOPOLOGY
RELATED APPLICATIONS
This application claims the benefit of United States Provisional Application Serial No. 60/001,498, filed July 19, 1995.
TECHNICAL FIELD OF THE INVENTION
This invention relates generally to communication systems, and more specifically to a serial control and data 0 interconnect.
BACKGROUND OF THE INVENTION
A communication system includes a collection of components that communicate, manipulate, and process information in a variety of ways. The system may support different access technologies, such as frame relay, circuit services, and new and evolving connection-based or connectionless services, that communicate information, such as data, voice, and video. Switches in the communication system employ hardware and software to route information generated by access technologies to an intended destination. In an integrated services network, switches may route information among access technologies in a unified manner.
With an increasing demand for more sophisticated and higher bandwidth communication, switches in a communication system must be scalable and adaptable to the particular needs of the users. Also, switches should support existing access technologies, and provide a flexible framework for new and evolving services. Existing switches in an integrated services environment suffer from several disadvantages. Switches fail to be modular and scalable to adapt, for example, to the needs and resources of a small private network serving hundreds of users, as well as a larger public network serving tens of thousands of users. Often, switches only support one or a few number of access technologies and offer limited expansion capabilities. Also, as integrated services networks get larger and more complex, existing switches may fail to provide adequate redundancy and fault isolation.
SUMMARY OF THE INVENTION
In accordance with the present invention, the disadvantages and problems associated with switches in a communication system has been substantially reduced or eliminated. In particular, a communication device routes information generated by a variety of access technologies, and includes serial control and data interconnects among its modules to provide modularity, scalability, redundancy, and improved fault isolation. In accordance with one embodiment of the present invention, a communication device includes a switch control module having a switch fabric and a number of input/output modules having cell flow processors. Control interconnects couple cell flow processors of the input/output modules to the switch fabric of the switch control module. Data interconnects couple cell flow processors of the input/output modules to the switch fabric of the switch control module.
Important technical advantages of the present invention include an architecture of a communication device that provides modularity, scalability, redundancy, and improved fault isolation. In particular, a communication device includes a switch control module coupled to a number of input/output modules using control and data interconnects that operate in a serial fashion to reduce connection complexity and size. In a specific implementation, each input/output module is coupled to a switch control module using a dedicated seven-line serial control interconnect and a dedicated four-line serial data interconnect. This connectivity scheme avoids the complexity, inefficiency, and increased cost of a common or
shared bus architecture, while supporting additional input/output modules. A redundant switch control module can also couple to the input/output modules for increased reliability. The serial control and data interconnects improve fault isolation through parity checking. Other technical advantages are readily apparent to one skilled in the art from the following figures, descriptions, and claims.
BRIEF DESCRIPTION OF THE DRAWINGS
For a more complete understanding of the present invention, and for further features and advantages, reference is now made to the following written description taken in conjunction with the accompanying drawings, in which:
FIGURE 1 illustrates a communication device;
FIGURE 2 illustrates the connectivity scheme among components of the communication device;
FIGURE 3 illustrates in more detail a control interconnect between components in the communication device;
FIGURE 4 illustrates in more detail a data interconnect between components in the communication device; and FIGURE illustrates timing diagram for communications using the data interconnect,
DETAILED DESCRIPTION OF THE INVENTION
FIGURE 1 illustrates a communication device 10 that includes a plurality of input/output modules (IOMs) 12 coupled to a switch control module (SCM) 14. In a particular embodiment, a redundant SCM 16 is also coupled to IOMs 12. In operation, IOMs 12 receive information, such as voice, video, and data, using a variety of access technologies. This information is passed to SCM 14 and routed through a selected IOM 12 to an intended destination.
Each IOM 12 includes zero or more line interfaces 20 and a cell flow processor 22 having a to-switch port processor (TSPP) 24 and a from-switch port processor (FSPP) 26. In one embodiment, line interface 20 includes a connectivity engine, network interworking, and physical interface to receive data from and provide data to a variety of access technologies. For example, line interface 20 may support asynchronous transfer mode (ATM) cell relay (OC-12, OC-3c, 155 Mbps UTP), frame relay (Tl, El, T3, E3, V.35), circuit emulation (Tl, El, T3, E3) , internetworking using Ethernet, Fast Ethernet, Internet Protocol (IP) , or IP over ATM, or any other communications protocol or access technology. Communication device 10 contemplates line interface 20 that supports any suitable communication technique, whether connection-based or connectionless.
The structure and function of line interface 20 may vary between IOMs 12 to provide modular support for different access technologies. For example, communication device 10 may include an IOM 12 to support ATM, another IOM 12 to support frame relay, still another IOM 12 to support
an internetworking function with an Ethernet local area network (LAN) , and any other suitable IOM 12. Communication device 10 contemplates any number and arrangement of IOMs 12 to support different access technologies. Line interface 20 may include one or more components in hardware or software and, in a particular example, includes one or more application specific integrated circuits (ASICs) .
Cell flow processor 22 provides an interface between line interface 20 and SCM 14. Unlike line interface 20 which may vary for different access technologies, cell flow processor 22 has the same structure and performs the same function for all IOMs 12. In a particular embodiment, cell flow processor 22 implements a core cell transfer function using ATM with virtual channel (VC) accounting and buffer control. Each cell flow processor 22 includes TSPP 24 for communications into SCM 14 and FSPP 26 for communications from SCM 14. In a particular implementation, TSPP 24 and FSPP 26 comprise an ASIC. Both SCM 14 and redundant SCM 16 may have portions 30 and 32, respectively, that include cell flow processor 22. This allows SCM 14 and redundant SCM 16 to communicate information among components in communication device 10 in the same manner as IOMs 12. Each cell flow processor 22 corresponds to a port or switch port in communication device 10.
Cell flow processors 22 residing on IOMs 12 provide a distributed cell processing architecture that reduces the complexity of SCM 14 and enhances the modularity and scalability of communication device 10. In particular, each IOM 12 added to communication device 10 as an upgrade or to enhance capacity includes its own cell flow processor
22. The on-board cell flow processor 22 of IOMs 12 reduces complexity and cost of entry-level systems, and also establishes a common and consistent interface between IOMs 12 and SCM 14. Therefore, IOMs 12 may be developed for new or evolving access technologies by combining a different structure and function for line interface 20 with the established structure and function for cell flow processor 22 for easy integration into communication device 10.
SCM 14 includes a number of multi-point topology controllers (MTCs) 30, a bandwidth arbiter (BA) 32, and a data crossbar 34, all making up a switch fabric 35. MTC 30 may comprise an ASIC that communicates with a selected number of IOMs 12 and centralizes state information needed for multi-point topology supported by communication device 10. Since IOMs 12 provide access to both connection-based and connectionless environments, MTCs support point-to- point (P2P) , multipoint-to-point (M2P) , point-to-multipoint (P2M) , and multipoint-to-multipoint (M2M) communications. BA 32 accumulates and arbitrates transfer requests from each IOM 12. Specifically, BA 32 directs input/output mapping of data crossbar 34 on a per cell-time basis, dynamically schedules momentarily unused bandwidth of communication device 10, and resolves M2P bandwidth contention. Redundant SCM 16 includes the same components and operates in the same fashion as SCM 14.
One particular technical advantage of communication device 10 is the interconnection between IOMs 12, SCM 14, and redundant SCM 16. Each IOM 12 couples to SCM 14 using interconnect 40 and to redundant SCM 16 using redundant interconnect 42. Both interconnect 40 and redundant interconnect 42 include control interconnect 44 and data
interconnect 46, which are dedicated connections since they support communication between SCM 14 and the associated IOM 12. Each portion 30 and 32 of SCM 14 and redundant SCM 16, respectively, may also include interconnect 40 and redundant interconnect 42.
Control interconnect 44 and data interconnect 46 operate serially and, therefore, reduce the connection complexity and size between IOMs 12 and SCM 14 in communication device 10. As described below, the serial operation of control interconnect 44 and data interconnect 46 enhances the modularity and scalability of communication device 10. Control interconnect 44 and data interconnect 46 reduce or eliminate the need for an expensive and complicated common or shared bus architecture in communication device 10. Also, the reduced number of lines in control interconnect 44 and data interconnect 46 simplifies fault isolation. For example, information communicated on these serial lines may include one or more parity bits to quickly and efficiently identify faulty components in communication device 10.
FIGURE 2 illustrates a connectivity scheme used by communication device 10. This connectivity scheme represents the interconnections established by a backplane 48 or other similar device in the chassis or support structure of communication device 10. IOMs 12, SCM 14, and redundant SCM 16 may be integrated circuit boards that plug into slots in backplane 48 to effect the connectivity scheme. Backplane 48 or other similar device implements the specified connections between components in communication device 10 to establish interconnects 40 and redundant interconnects 42.
Each IOM 12 includes a connector region 50 and a redundant connector region 52. SCM 14 and redundant SCM 16 include connector regions 54 and redundant connector regions 56. Connector regions 50, 52, 54, and 56 represent a collection of contiguous or non-contiguous pins, conductors, or other matings on components in communication device 10.
Connector region 50 of each IOM 12 is coupled to and has a one-to-one correspondence with an associated connector region 54 on SCM 14 to establish interconnect 40. Likewise, redundant connection region 52 of each IOM 12 is coupled to and has a one-to-one correspondence with an associated connection region 54 of redundant SCM 16 to establish redundant interconnect 42. Therefore, each interconnect 40 establishes a dedicated communication path or link between an associated IOM 12 and SCM 14, and each redundant interconnect 42 establishes a dedicated communication path or link between an associated IOM 12 and redundant SCM 16. Throughout this description, interconnect 40, redundant interconnect 42, control interconnect 44, and data interconnect 46 may represent connection regions, lines, pins, conductors, matings, connectors, or any combination of these elements to accomplish a coupling between components in communication device 10.
Since SCM 14 and redundant SCM 16 may include a cell flow processor 22, the connectivity scheme includes two additional redundant interconnects 42a and 42b. In particular, cell flow processor 22 on portion 32 of redundant SCM 16 is coupled to a corresponding connection region 54 on SCM 14 using redundant connection region 56 on
redundant SCM 16 to establish redundant interconnect 42a. Likewise, cell flow processor 22 on portion 30 of SCM 14 is coupled to a corresponding connection region 54 on redundant SCM 16 using redundant connection region 56 on SCM 14 to establish redundant interconnect 42b.
The connectivity scheme illustrated in FIGURE 2 provides several advantages to communication device 10. SCM 14 and redundant SCM 16 may include many connection regions 54 that support a number of existing or potential IOMs 12. However, each IOM 12 maintains one connection region 50 and one redundant connection region 52 to establish interconnect 40 and redundant interconnect 42, respectively, which reduces the cost and complexity of backplane 48. This dedicated connection approach instead of a common or shared bus approach improves modularity and scalability. In particular, an entry-level communication device 10 may include SCM 14 and eight IOMs 12, but with a backplane 48 of reduced complexity and size that still accommodates future upgrades and additions. Depending upon its requirements and demands, a user of communication device 10 may add redundant SCM 16 to enhance reliability or add more IOMs 12 to improve capacity or support a different access technology.
FIGURE 3 illustrates in more detail control interconnect 44 between IOMs 12 and SCM 14. A similar control interconnect 44 exists between IOMs 12 and redundant SCM 16. Each control interconnect 44 includes two lines, conductors, couplings, connectors, matings, or connections (referred to generally as lines) between TSPP 24 of IOM 12 and an associated MTC 30 of SCM 14: a line from TSPP 24 to MTC 30 (T2M) 60 and a line from MTC 30 to
TSPP 24 (M2T) 62. Control interconnect 44 also includes two lines between FSPP 26 and MTC 30: a line from FSPP 26 to MTC 30 (F2T) 64 and a line from MTC 30 to FSPP 26 (M2F) 66. Also, control interconnect 44 includes a line from TSPP 24 to BA 32 (T2B) 68. In a particular implementation, T2M 60 and M2T 62 comprise two lines each, whereas F2M 64, M2F 66, and T2B 68 comprise one line each. Therefore, in this particular embodiment, control interconnect 44 comprises seven lines between IOM 12 and SCM 14. Another seven lines from IOM 12 may establish a similar control interconnect 44 with redundant SCM 16.
Each MTC 30 may support several IOMs 12. In a particular implementation, each MTC 30 supports four IOMs, resulting in twenty-four lines to implement four control interconnects 44. MTC 30 communicates with BA 32 using control line 70. Control line 70 comprises sixteen lines from MTC 30 to BA 32 and nine lines from BA 32 to MTC 30. In a system that includes sixteen IOMs 12 and four MTCs 20, BA 32 receives and transmits control information relating to the operation of communication device 10 on eighty input lines (twenty-four lines for each control line 70 and one line for each T2B 68) and four output lines (one line for each control line 70) .
In operation, communication device 10 using control interconnect 44 manages communications between receive IOM 12a and transmit IOM 12b. For a P2P cell transfer, line interface 20 receives information in the format or protocol used by the access technology supported by IOM 12a. Line interface 20 translates this information into the core cell transfer format supported by cell flow processor 22. TSPP 24 receives the cell and generates a request for
communication over T2B 68 for a designated output port. BA 32 grants the request and communicates the grant to TSPP 24 using control line 70, MTC 30, and M2T 62.
MTC 30 then performs any necessary translation to identify the virtual channel (VC) queue associated with transmit IOM 12b. Each TSPP 24 transmits a scheduling list number to the MTC 30 on the T2M line 60. The scheduling list number reflects the connection it is transferring a cell from. The MTC 30 uses this information to determine the output port(s) and multiQueue number(s) to which the cell will be transferred. The MTC 30 passes this information to the BA 32 on the M2B lines 70, and the BA 32 passes the multiQueue number to the MTC(s) on the B2M lines 70. The MTC 30 further passes the multiQueue number(s) to the FSPP(s) 26 using the M2F line 66. The FSPP 26 uses multiQueue number to determine which output queue (s) are to receive the cell and whether these queues are full. Flow control information, based on the queue status is communicated in the reverse direction, i.e. F2M 64, followed by M2B 70, followed by B2M 70, and lastly M2T 62. This control information happens in parallel for all port processors, and control flow is capable of multipoint transfers.
FIGURE 4 illustrates in more detail data interconnect 46 that transfers the cell after establishing the appropriate control information using control interconnect 44. Shown in more detail in IOM 12 is TSPP 24 coupled to a serializer 80 and FSPP 26 coupled to a deserializer 82. In a particular embodiment, serializer 80 and deserializer 82 support a fiber channel high speed serial interface. Serializer 80 converts an n-bit word received from TSPP 24
into a differential emitter coupled logic (ECL) signal for transmission over transmit line pair 84. Transmit line pair 84 is coupled to input port 86 of data crossbar 34. Output port 88 of data crossbar 34 is coupled to a differential ECL receive line pair 90, which in turn is coupled to deserializer 82 in IOM 12. Deserializer 82 converts information received over receive line pair 90 into an n-bit word for delivery to FSPP 26.
In this embodiment, data crossbar 34 includes input port 86 and an associated output port 88 for each IOM 12 coupled to SCM 14. Input port 86 and output port 88 each comprise a line pair connection. Data crossbar 34 may be an ECL cross-point device under the control of BA 32. Upon receiving the appropriate control information established using control interconnect 44, BA 32 configures data crossbar 34 to achieve the proper cell transfer. Data crossbar 34 maps one input port 86 to one or more output ports 88. For example, data crossbar 34 may establish a connection between input port 86 associated with IOM 12a and output port 88 associated with IOM 12b. In another example, data crossbar 34 may establish a connection between input port 86 and output port 88 associated with the same IOM 12.
In a particular implementation, TSPP 24 and FSPP 26 on IOM 12 operate using a fifty MHZ system clock. Data is transferred in a twenty bit word to serializer 80 and converted into a one GHz differential ECL signal for transfer over transmit line pair 84 to data crossbar 34. Data crossbar 34 simultaneously supports sixteen ports or IOMs 12, and additional data crossbars 34 may be added to increase the capacity of SCM 14. Data crossbar 34 performs
high speed switching between input ports 86 and associated output ports 88 under the direction of BA 32. Receive line pair 90 communicates information from output port 88 to deserializer 82, which converts the one GHz differential ECL signal into a twenty bit digital word at fifty MHZ for presentation to FSPP 26.
To accommodate high bandwidth traffic, transmit line pair 84 and receive line pair 90 of data interconnect 46 are high speed serial links. With increased data rates of one GHz or more, communication device 10 may experience a clock skew problem caused by communication delays in transmit line pair 84 and receive line pair 90. Therefore, communication device 10, and specifically deserializer 82, employs a phase lock loop and resynchronizer to account for clock skew over data interconnect 46.
FIGURE 5 illustrates a timing diagram for transfer of a cell from TSPP 24 TO FSPP 26 using data interconnect 46. A cell clock pulse 102 occurs every thirty-two cycles of system clock 104 which, in a particular implementation, operates at fifty MHZ. Each cell clock pulse 102 represents a single cell transfer and receive event using data interconnect 46.
A signal 106 generated by serializer 80 for transmission over transmit line pair 84 begins with a preamble 108 which, in a particular embodiment, comprises a serial bit stream of alternating "Is" and "0s." A phase lock loop in deserializer 82 establishes synchronization with the one GHz signal using preamble 108. After preamble 108, serializer 80 generates a sync 110 that indicates the beginning of data 112. Serializer 80 concludes the transmission during cell clock pulse 102 with a postamble
114 that, like preamble 108, includes a serial bit stream of alternating "Is" and "Os" to allow synchronization by deserializer 82.
Deserializer 82 receives a signal 116, which includes preamble 108, sync 110, data 112, and postamble 114. A sufficient length of postamble 114 from the previous cell and preamble 108 from the current cell before sync 110 ensures that deserializer 82 is in sync with the one GHz signal and ready to receive data 112. Between cells, BA 32 reconfigures data crossbar 34 to deliver the next cell of information.
Although the present invention has been described in several embodiments, a myriad of changes, variations, alterations, transformations, and modifications may be suggested to one skilled in the art, and it is intended that the present invention encompass such changes, variations, alterations, transformations, and modifications as fall within the spirit and scope of the appended claims.