CN102111403B - Method and device for acquiring transmission control protocol (TCP) connection data at high speed - Google Patents
Method and device for acquiring transmission control protocol (TCP) connection data at high speed Download PDFInfo
- Publication number
- CN102111403B CN102111403B CN201010608983.9A CN201010608983A CN102111403B CN 102111403 B CN102111403 B CN 102111403B CN 201010608983 A CN201010608983 A CN 201010608983A CN 102111403 B CN102111403 B CN 102111403B
- Authority
- CN
- China
- Prior art keywords
- tcp
- message
- data
- connection
- high speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides a method and device for acquiring transmission control protocol (TCP) connection data at a high speed. After a hardware network card receives messages and carries out TCP connection management, directly uploading ordered messages in which information headers for recording TCP connection are added to a host machine; carrying out rearrangement of TCP disordered data on disordered messages, wherein data splicing is not performed by a buffer in the rearrangement process; and carrying out data splicing, TCP data format conversion and other operations after an application program interface (API) library receives the TCP messages. According to the invention, the messages can be submitted to the host machine in time; original message data can be conveniently converted into a TCP data format, thus playing a role of data splicing; and the whole system can acquire the TCP data at a high speed.
Description
Technical field
The present invention relates to network data processing field, be specifically related to a kind of method and apparatus for obtaining TCP connection data with high speed.
Background technology
TCP Unloading Technology is that the TCP/ip network protocol stack processing in host software is cured in network interface card hardware and is realized, to reduce system cpu load, increases the technology of network I/O bandwidth.TCP uninstalling system generally comprises two parts function, and Part I is TCP connection status management, safeguards each TCP connection beginning, transfer of data, the state of closing by TCP connection table.Part II is tcp data management, is used for the data of out of order TCP message to resequence, and the data of TCP transmission are extracted and submit to main frame.
Generally in TCP uninstalling system, hardware, in the time carrying out tcp data management, use a data buffer, and the data that each TCP is connected extract, complete reordering of tcp data and splice with buffer, then take buffering area as unit is submitted to host computer system.
Application number 200680026247.4 discloses the method and system of a kind of transparent transmission control protocol (TCP) unloading, the various aspects of described method comprise: in network interface Card processor, collect TCP section, but state information is not transferred to host computer system.The TCP section of collecting is buffered in colligator.Colligator can check the network flow relevant to the TCP section of collecting to have entrance in stream look-up table (FLT).In the time that FLT expires, colligator can cut out current entrance, and network flow is assigned to available entrance.Colligator is the information in renewable FLT also.In the time stopping event that TCP section collects and occur, colligator can the TCP section based on collecting generates the TCP section of single polymerization.The TCP section of polymerization and state information can be sent to main frame to process.
Application number 03820350.2 discloses a kind of system and method for TCP unloading.On TEEC, receive input TCP grouping, and at least a portion of processing described input grouping by described TEEC once, and needn't carry out any restructuring and/or repeating transmission by described TEEC.Described at least a portion, inputting TCP grouping can be buffered at least one inner elastomeric buffer of described TEEC.Described inner elastomeric buffer can comprise reception inner elastomeric buffer and/or send inner elastomeric buffer.Therefore, described at least a portion, inputting TCP grouping can be buffered in described reception inner elastomeric buffer.The input grouping of processing described at least a portion can be put in a part for a mainframe memory, to processed by host-processor or CPU.In addition the input TCP grouping of processing described at least a portion, can be sent in a part for described mainframe memory by DMA.
By the shortcoming that general TCP uninstalling system obtains tcp data be, some applied business need to be processed in real time to tcp data, main frame need to be taken the data of TCP transmission in time, after needing hardware to splice a certain amount of data, general TCP uninstalling system submits to again main frame, although be conducive to improve system bandwidth, reduced the response speed of system.
Summary of the invention
The object of the invention is to provide a kind of system tcp data is uploaded to main frame in time, avoids tcp data to splice in hardware.
For a method for obtaining TCP connection data with high speed, comprise following steps:
A, special chip receive after TCP message by network interface, extract TCP link information from message, and original message is kept in to plate carry in internal memory;
B, special chip are according to the TCP link information of message, and inquiry plate carries the TCP connection table in internal memory, if the message newly connecting just increases a new list item to this connection in connection table; If close the message of connection, just the list item of this connection in connection table is deleted; If the data message having connected records the current state of this connection in connection list item;
C, for tcp data message, special chip, according to connecting the state recording in list item, judges whether this message has occurred out of order, if do not have out of orderly, is directly uploaded to main frame; If occur out of orderly, message is put into chained list, wait for that follow-up message resequences, after having sorted, upload main frame take message as unit;
D, network interface card, submitting to before message to main frame, increase a self-defining header structure before the encapsulation header of message, preserve the information that this TCP connects;
E, drive software carry internal memory message from network interface card plate and are transferred to host memory;
When F, application software read tcp data by api interface storehouse, interface library software reads original message from host memory, and according to the information header of COM Continuation of Message, location tcp data, original message data structure, converts the tcp data structure that application needs to.
A preferred technical solution of the present invention is: the sequencer procedure occurring after out of order uses message chained list, does not use splicing buffer, only, to message rearrangement, does not carry out the splicing of tcp data.
An optimal technical scheme more of the present invention is: the TCP link information of preserving in described D step comprises the skew in message of connection status, tcp data.
For an equipment for obtaining TCP connection data with high speed, comprise network interface card and host software part,
Described network interface card comprises that special chip, network interface and plate carry internal memory;
Described host software part comprises that network interface card drives and application software api interface storehouse.
A preferred technical solution of the present invention is: the main control chip that described special chip is network interface card, moves all message processing logics.
The present invention can submit to main frame message in time, can conveniently realize again the conversion of original message data to tcp data form, plays the effect of data splicing, and whole system can realize the tcp data that obtains at a high speed.
Accompanying drawing explanation
Fig. 1 is system construction drawing of the present invention
Embodiment
The technical program is the equipment of a software and hardware one, hardware be with special chip for the network interface card that main devices designs, comprise network interface, special chip, plate carry internal memory form.Software is driven by network interface card and application software api interface storehouse forms.
The function of the every part of system is as follows:
(1) network interface: network message data receiver network access card.
(2) special chip: the main control chip of network interface card, moves all message processing logics.
(3) plate carries internal memory: the connection table that storage TCP connection management is used, temporary message.
(4) network interface card drives: administration configuration network interface card, and for network interface card provides path to the transfer of data of main frame.
(5) api interface storehouse: connect the DLL (dynamic link library) of network interface card system and application software, the message format of network interface card being submitted to main frame converts the tcp data form that application software needs to.
Processing method and the process of this equipment are as follows:
(1) hardware network interface card receives message.
Special chip receives after TCP message by network interface, extracts TCP link information from message, and original message is kept in to plate carries in internal memory.
(2) hardware network interface card carries out TCP connection management.
Special chip is according to the TCP link information of message, inquiry plate carries the TCP connection table in internal memory, if the message newly connecting, just in connection table, increase a new list item to this connection, if close the message of connection, just the list item of this connection in connection table is deleted, if the data message having connected records the current state of this connection in connection list item.
(3) network interface card hardware carries out the rearrangement of the out of order data of TCP, but does not carry out data splicing.
To tcp data message, special chip, according to connecting the state recording in list item, judges whether this data message occurs out of order, if do not have out of orderly, is directly uploaded to main frame; If occur out of orderly, message is put into chained list, wait for that follow-up message resequences, after having sorted, upload main frame take message as unit.This sequencer procedure uses message chained list, does not use splicing buffer, does not carry out the splicing of tcp data.
(4) network interface card hardware adds information header to message, records TCP link information.
Network interface card, submitting to before message to main frame, increases a self-defining header structure before the encapsulation header of message, wherein preserves the information that this TCP connects, and comprises connection status, the tcp data skew in message etc.
(5) drive software carries internal memory message from network interface card plate and is transferred to host memory.
(6) Data Format Transform is carried out in api interface storehouse.
When application software reads tcp data by api interface storehouse, interface library software reads original message from host memory, and according to the information header of COM Continuation of Message, location tcp data, original message data structure, converts the tcp data structure that application needs to.
In the network security private server that the present invention has produced at dawn, use, obtained good effect.
Claims (3)
1. for a method for obtaining TCP connection data with high speed, it is characterized in that: comprise following steps:
A, special chip receive after TCP message by network interface, extract TCP link information from message, and original message is kept in to plate carry in internal memory;
B, special chip are according to the TCP link information of message, and inquiry plate carries the TCP connection table in internal memory, if the message newly connecting just increases a new list item to this connection in connection table; If close the message of connection, just the list item of this connection in connection table is deleted; If the data message having connected records the current state of this connection in connection list item;
C, for tcp data message, special chip, according to connecting the state recording in list item, judges whether this message has occurred out of order, if do not have out of orderly, is directly uploaded to main frame; If occur out of orderly, message is put into chained list, wait for that follow-up message resequences, after having sorted, upload main frame take message as unit;
D, network interface card, submitting to before message to main frame, increase a self-defining header structure before the encapsulation header of message, preserve the information that this TCP connects;
E, drive software carry internal memory message from network interface card plate and are transferred to host memory;
When F, application software read tcp data by api interface storehouse, interface library software reads original message from host memory, and according to the information header of COM Continuation of Message, location tcp data, original message data structure, converts the tcp data structure that application needs to.
2. a kind of method for obtaining TCP connection data with high speed as claimed in claim 1, is characterized in that: the sequencer procedure occurring after out of order uses message chained list, does not use splicing buffer, does not carry out the splicing of tcp data.
3. a kind of method for obtaining TCP connection data with high speed as claimed in claim 1, is characterized in that: the TCP link information of preserving in described D step comprises the skew in message of connection status and tcp data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010608983.9A CN102111403B (en) | 2010-12-17 | 2010-12-17 | Method and device for acquiring transmission control protocol (TCP) connection data at high speed |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010608983.9A CN102111403B (en) | 2010-12-17 | 2010-12-17 | Method and device for acquiring transmission control protocol (TCP) connection data at high speed |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102111403A CN102111403A (en) | 2011-06-29 |
| CN102111403B true CN102111403B (en) | 2014-05-21 |
Family
ID=44175433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010608983.9A Active CN102111403B (en) | 2010-12-17 | 2010-12-17 | Method and device for acquiring transmission control protocol (TCP) connection data at high speed |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102111403B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107979592B (en) | 2013-09-30 | 2020-06-30 | 腾讯科技（深圳）有限公司 | Method and device for sending service request message |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040258076A1 (en) * | 2003-06-05 | 2004-12-23 | Jha Ashutosh K. | Setting up a delegated TCP connection |
| CN1679282A (en) * | 2002-08-30 | 2005-10-05 | 美国博通公司 | System and method for TCP offload |
| CN101253745A (en) * | 2005-07-18 | 2008-08-27 | 博通以色列研发公司 | Method and system for transparent TCP offload |
| CN101601256A (en) * | 2007-01-30 | 2009-12-09 | 国际商业机器公司 | Network interface card transmission control protocol acceleration offload fault detect and Restoration Mechanism |
| US7738500B1 (en) * | 2005-12-14 | 2010-06-15 | Alacritech, Inc. | TCP timestamp synchronization for network connections that are offloaded to network interface devices |
-
2010
- 2010-12-17 CN CN201010608983.9A patent/CN102111403B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1679282A (en) * | 2002-08-30 | 2005-10-05 | 美国博通公司 | System and method for TCP offload |
| US20040258076A1 (en) * | 2003-06-05 | 2004-12-23 | Jha Ashutosh K. | Setting up a delegated TCP connection |
| CN101253745A (en) * | 2005-07-18 | 2008-08-27 | 博通以色列研发公司 | Method and system for transparent TCP offload |
| US7738500B1 (en) * | 2005-12-14 | 2010-06-15 | Alacritech, Inc. | TCP timestamp synchronization for network connections that are offloaded to network interface devices |
| CN101601256A (en) * | 2007-01-30 | 2009-12-09 | 国际商业机器公司 | Network interface card transmission control protocol acceleration offload fault detect and Restoration Mechanism |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102111403A (en) | 2011-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104820701B (en) | Data record and synchronous method and system | |
| CN109067733B (en) | Method and apparatus for transmitting data, and method and apparatus for receiving data | |
| US10043220B2 (en) | Method, device and storage medium for data processing | |
| US10708378B2 (en) | Data processing method and apparatus, server, and controller | |
| CN107818120A (en) | Data processing method and device based on big data | |
| CN103607476A (en) | System and method for distributed multi-stage persistence of huge number of messages | |
| CN104462562A (en) | Data migration system and method based on data warehouse automation | |
| EP2431939A1 (en) | Distributed image processing method and image processing system | |
| CN114201421A (en) | Data stream processing method, storage control node and readable storage medium | |
| US20220366089A1 (en) | Method, System, Electronic Device, and Storage Medium For Storing and Collecting Temperature Data | |
| CN107196992A (en) | A kind of file data management system of law-enforcing recorder | |
| CN102111403B (en) | Method and device for acquiring transmission control protocol (TCP) connection data at high speed | |
| WO2014117585A1 (en) | System and method for audio signal collection and processing | |
| CN111083408B (en) | Method, system and equipment for processing video storage service | |
| CN110417892B (en) | Message analysis-based data replication link optimization method and device | |
| CN202121599U (en) | Equipment for obtaining TCP connection data with high speed | |
| CN115344633A (en) | Data processing method, device, equipment and storage medium | |
| CN111401819B (en) | Intersystem data pushing method and system | |
| CN113315809B (en) | High-speed data transmission delay tolerance method and system for medical equipment | |
| CN106412044A (en) | Storm-based real-time massive cloud application access statistical implementation method | |
| CN106033367A (en) | Scripts game server engine | |
| CN113507499B (en) | Smart campus dormitory-checking system based on big data | |
| CN111581244B (en) | Method, system and equipment for orderly synchronizing service transaction data of heterogeneous system | |
| US20150180777A1 (en) | Data processing apparatus and method | |
| CN101141299A (en) | Collection system and implementing method of network management communication |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20171213 Address after: 300384 Tianjin city Xiqing District Huayuan Industrial Zone (outer ring) Haitai Huake Street No. 15 1-3 Patentee after: Sugon Information Industry Co., Ltd. Address before: 100084 Beijing Haidian District City Mill Street No. 64 Patentee before: Dawning Information Industry (Beijing) Co., Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20180409 Address after: 430040 Wuhuan Road No. 666 (10), economic and Technological Development Zone, Wuhan, Hubei Province Patentee after: Dawning Network Technology Co., Ltd. Address before: 300384 Tianjin city Xiqing District Huayuan Industrial Zone (outer ring) Haitai Huake Street No. 15 1-3 Patentee before: Sugon Information Industry Co., Ltd. |