WO2012126229A1

WO2012126229A1 - Distributed cache system data access method and device

Info

Publication number: WO2012126229A1
Application number: PCT/CN2011/077589
Authority: WO
Inventors: 李豪伟; 韩银俊; 郭斌
Original assignee: 中兴通讯股份有限公司
Priority date: 2011-03-23
Filing date: 2011-07-25
Publication date: 2012-09-27
Also published as: CN102694828A; CN102694828B

Abstract

Disclosed are a distributed cache system data access method, comprising: a server node receives a data access request sent by a client terminal; determines the storage mode of the data to be accessed according to the data access request, accesses the data according to the storage mode; and returns the access result to the client terminal. Also disclosed is a distributed cache system data access device. The data access efficiency is improved and the price/performance ratio of the distributed cache system is increased using the method and device.

Description

Method and device for data access of distributed cache system

The present invention relates to the field of distributed caching, and more particularly to a method and apparatus for data access in a distributed caching system. Background technique

Cloud Computing is Grid Computing, Distributed Computing, Parallel Computing, Utility Computing, Network Storage Technologies, Virtualization. And load balancing (Load Balance) and other traditional computer technology and network technology development and fusion products. Cloud computing aims to integrate multiple relatively low-cost computing entities into a system with powerful computing power through the network. Distributed caching is a domain in the field of cloud computing. Its role is to provide distributed storage services for massive data and high-speed read and write access.

A distributed cache system consists of several server nodes and clients interconnected. The server node is responsible for storing data, and the client can read, update, and delete data of the server node. The data in the server node is composed of a key (Key) and a value (Value). The Key is equivalent to the data index, and the Value is the data content represented by the Key. Key and Value are one-to-one relationships.

In the distributed cache system, how to improve the access efficiency of large-volume data is a difficult problem to solve. The memory cache is quickly exhausted by repeatedly reading, updating, and deleting a large amount of different data. In order to increase storage capacity, it is often necessary to combine memory and hard disk to store data. In this way, a part of the data in the distributed cache must be stored on the hard disk. If the data on the hard disk is to be read, it will take a long time on the disk 10, and the read operation will become very slow, which is the large quantity. Data access efficiency issues. In the prior art, access efficiency of large amounts of data There are generally two solutions:

Option 1, using all memory in a distributed cache system to store large amounts of data, can ensure access efficiency, but this requires a high economic cost, only in some projects that are very important or can create higher economic value. Use, it is difficult to apply in ordinary projects;

Option 2, using the combination of memory and hard disk to store data, the data is all stored on the hard disk, when reading data, the data is read from the hard disk and cached into the memory. The main disadvantage of this method is that some Data that requires higher access efficiency also needs to be read from the hard disk first, cached to memory, and shared with a common data to store data. Data access can only obtain common access efficiency, resulting in better access efficiency. low. Summary of the invention

In view of this, the main object of the present invention is to provide a method and apparatus for data access of a distributed cache system, improve data access efficiency, and improve the cost performance of the distributed cache system.

In order to solve the above technical problem, the technical solution of the present invention is implemented as follows:

The present invention provides a method for data access of a distributed cache system, including: a server node receives a data access request sent by a client, determines a storage mode of the accessed data according to the data access request, and accesses the data according to the storage mode. And return the access result to the client.

In the above solution, the determining, by the data access request, the storage mode of the accessed data includes: reading an application identifier ID in the data access request, and acquiring the accessed according to the corresponding relationship between the locally configured application ID and the storage mode. The storage mode of the data.

In the above solution, the received data access request is a data update request, and the accessing the data according to the storage mode includes: determining the memory storage mode, and updating the data in the memory according to the key KEY and the value Value in the data update request. Or, determine the memory plus normal hard disk storage mode, update the data in the hard disk according to the KEY and Value in the data update request, and then update the data in the memory; or, determine the memory plus SSD mode, determine the remaining memory Space, update the data in memory, otherwise, free up memory space, and then update the data in memory. In the above solution, the received data access request is a data read request, and the accessing the data according to the storage mode includes: determining to be a memory storage mode, and reading data in the memory according to a KEY in the data read request; Or, determine the memory plus normal hard disk storage mode, read the data in the memory according to the KEY, if there is no data to be read in the memory, read the data from the hard disk; or, determine the memory plus solid state hard disk mode, The data to be read is determined according to the index information in the memory, and then the storage location of the data is determined according to the index information, and the data is read.

In the above solution, the received data access request is a data deletion request, and the accessing the data according to the storage mode includes: determining the memory storage mode, deleting the data in the memory according to the KEY in the data deletion request; or determining Add the normal hard disk storage mode to the memory, delete the data in the hard disk according to the KEY in the data deletion request, and then delete the data in the memory; or, determine the memory plus the solid state hard disk storage mode, and determine the deletion according to the index information in the memory. The data exists, and the data is deleted according to the storage location of the data in the index information.

The invention also provides a device for data access of a distributed cache system, comprising: a data storage module and an information feedback module;

a data access module, configured to receive a data access request sent by the client, obtain a storage mode of the data to be accessed according to the data access request, access the data according to the storage mode, and send the access result to the information feedback module;

The feedback module is set to return the access result to the client.

In the above solution, the data access module specifically includes an access mode unit, a data update unit, a data read unit, and a data deletion unit. The access mode unit is configured to read an application ID in the data access request according to the local Corresponding relationship between the configured application ID and the storage mode, obtaining a storage mode of the accessed data, receiving the data access request as a data update request, and transmitting the acquired storage mode and data update request to the data update unit; The request is a data read request, and the data read request and the acquired storage mode are sent to the data read unit; the received data access request is a data delete request, and the data deletion request and the acquired storage are performed. The mode is sent to the data deletion unit.

In the above solution, the data update unit is configured to: when the acquired storage mode is the memory storage mode, update the data in the memory according to the KEY and the value in the data update request; or, when the memory is added to the normal hard disk storage mode, Update the data in the hard disk according to the KEY and Value of the data in the data update request, and then update the data in the memory; or, when adding the solid state hard disk storage mode to the memory, determine that there is remaining space in the memory, and update the data in the memory; otherwise, Free up memory space and then update the data in memory.

In the above solution, the data reading unit is configured to: when the acquired storage mode is the memory storage mode, read the data in the memory according to the KEY in the data read request; or, when the memory is added to the normal hard disk storage mode According to the KEY, according to the KEY in the data read request, the data is first read from the memory, and if there is no data to be read in the memory, the data is read from the hard disk and cached in the memory; or, is the memory When the SSD storage mode is added, the data to be read exists according to the index information in the memory, and then the storage location of the data is determined according to the index information, and the data is read.

In the above solution, the data deletion unit is configured to: when the stored storage mode is the memory storage mode, delete the data in the memory according to the KEY in the data deletion request; or, when adding the normal hard disk storage mode to the memory, according to the KEY, first delete the data in the ordinary hard disk, and further delete the data in the memory; or, when adding the solid state hard disk storage mode to the memory, determine the data to be deleted according to the index information in the memory, according to the storage location of the data in the index information , delete data.

It can be seen that the method and device for data access of the distributed cache system according to the present invention use different access modes for different data storage modes, thereby improving data access efficiency and improving distributed The cost performance of the cache system. DRAWINGS

1 is a schematic diagram of an embodiment of a method for implementing data access in a distributed cache system according to the present invention; 2 is a schematic diagram of an embodiment of an apparatus for implementing data access in a distributed cache system according to the present invention. detailed description

The basic idea of the present invention is: the client sends a data access request to the server node, the server node acquires a storage mode to access the data according to the data access request, and then accesses the data according to the storage mode, and returns the access result to the client. .

The invention will now be described in detail by means of the drawings and specific embodiments.

The present invention provides a method for data access of a distributed cache system, as shown in FIG. 1, including the following steps:

Step 101: The client sends a data access request to the server node.

The client sends a data access request to the server node, where the data access request is: a data update request, or a data read request, or a data delete request; wherein the data update request is data write or replace, including an application Identification (ID), and the KEY, Value of the updated data. The application ID is used to identify the service application system. The embodiment of the present invention divides the storage mode of the corresponding service application data according to the service application system corresponding to the application ID, and may also use other methods. The data read request includes an application ID and a KEY of the read data; the data deletion request includes an application ID, and a KEY of the deleted data.

Step 102: The server node acquires a storage mode of the data to be stored according to the data access request, and then accesses the data according to the storage mode.

The server node reads the application ID in the data access request, and obtains the storage mode corresponding to the application ID in the data access request according to the corresponding relationship between the locally configured application ID and the storage mode, that is, the storage mode corresponding to the data to be accessed. The data to be accessed is the data to be updated, read or deleted.

The accessing data according to the access mode includes:

Receiving a data update request, if the data to be updated is in the memory storage mode, then according to the data The KEY and Value in the new request update the data in the memory, and the memory storage mode stores and manages the data in the memory, and the update is specifically: when replacing the data, querying the memory according to the KEY in the data update request The saved data, if the KEY of the data in the memory matches the KEY in the data update request, it can be determined that the data exists in the memory, and the Value of the corresponding KEY data in the memory is overwritten by the Value in the data update request, if there is no data and data update If the KEY in the request matches, the data does not exist in the memory, and the update fails. When the data is written, the data is directly written into the memory; or, if the data to be updated is the memory plus the normal hard disk storage mode, the data in the request is updated according to the data. KEY and Value update the data in the hard disk, and then determine that the data exists in the memory, and update the data in the memory, the process of determining that the data in the memory is the same as the process of determining the data in the memory during the update process of the memory storage mode, The description is not repeated here; the update is the same as the update process of the memory storage mode, and details are not described herein again; The memory plus normal hard disk storage mode is a storage mode of data stored independently by the memory and the hard disk, and the data stored in the memory may be hot data, that is, data that is frequently accessed; or, the data to be updated is memory plus solid state. The hard disk storage mode determines whether there is any remaining space in the memory according to the memory capacity information stored in the memory. If the memory capacity information indicates that there is remaining space in the memory, the data in the memory is updated, and if there is no remaining space, the memory space is released, and then Updating the data in the memory, and updating the storage location of the data moved to the SSD, the data in the update memory is the same as the update in the memory storage mode, and details are not described herein again. The least recently used (LRU) algorithm, according to the time record saved in the data saved in the memory, moves the earliest data to the solid state hard disk, and the memory plus solid state hard disk storage mode is an index information stored in the memory, the memory There is space left, memory access data, no memory The remaining space accesses the data in the solid state hard disk, and manages the data in the memory and the solid state hard disk through the index information stored in the memory, and the access speed of the solid state hard disk data is greater than the access speed of the ordinary hard disk data, as the memory Extended, the data is only saved in one place in the memory or SSD. The index information is the update or delete operation of the memory according to the memory and the SSD, and the package is updated in real time. The information of the KEY of the data and the storage location of the data.

Or, receiving a data read request, the data to be read is a memory storage mode, and reading the data in the memory according to the KEY in the data read request, the read data includes: according to the KEY in the data read request Query the KEY of the data in the memory. If the KEY in the data read request matches, the data is read. If there is no data matching the KEY in the data read request, there is no data to be read, and the read fails. Or, the data to be read is the memory plus normal hard disk storage mode. According to the KEY in the data read request, the data is read from the memory first. If there is no data to be read in the memory, the data is read from the hard disk. At the same time, the data is cached into the memory, and the reading process is the same as the reading process of the memory storage mode, and will not be described here; or, the data to be read is the memory plus solid state hard disk storage mode, according to the index information in the memory. Determine whether the data to be read exists, the data to be read exists, the data to be read according to the storage location of the data in the index information, the data to be read If there is no read operation, the message that the data does not exist is returned to the client, and determining whether the data to be read exists according to the index information in the memory includes: KEY in the data read request and KEY in the index information In comparison, if the KEY in the data read request matches, the data to be read exists.

Or, receiving the data deletion request, the data to be deleted is the memory storage mode, and the data in the memory is deleted according to the KEY in the data deletion request, where the deleting comprises: querying the data in the memory according to the KEY in the data deletion request KEY, if there is a KEY in the data deletion request, it is determined that the data to be deleted exists, and then the data in the memory is deleted. Otherwise, the data to be deleted does not exist, and the deletion fails. Alternatively, the data to be deleted is a memory plus normal hard disk storage mode. According to the KEY in the data deletion request, the data in the hard disk is deleted first, and then the data in the memory is deleted. The deletion process is the same as the deletion process of the memory storage mode. The data to be deleted is the memory plus SSD storage mode. According to the index information in the memory, it is determined whether the data to be deleted exists. If the deleted data exists, the data is deleted according to the storage location of the data in the index information. Data, and delete the index information corresponding to the data, to delete If the data does not exist, the deletion will not be performed. The process of determining whether the data to be deleted exists, and the process of receiving the data read request and the data to be read is the memory plus SSD storage mode, the process of determining whether the data to be read exists is the same, and details are not described herein again.

Step 103: The server node replies to the client with an access result.

The access result includes: when receiving the data update request, returning the updated message to the client, and when receiving the data read request, returning the read data to the client, if the data to be read does not exist, The message that the data does not exist is returned to the client; when the data deletion request is received, the message that the data has been deleted is returned to the client, and if the data does not exist, the message that the data does not exist is returned to the client.

Based on the above method, the present invention further provides a device for data access of a distributed cache system. As shown in FIG. 2, the device includes: a data access module 201, an information feedback module 202;

The data access module 201 is configured to receive a data access request sent by the client, acquire an access mode of the data to be accessed according to the data access request, access the data according to the access mode, and send the access result to the information. Feedback module 202;

The information feedback module 202 is configured to send the access result to the client.

The data access module 201 includes: an access mode unit 2011, a data update unit 2012, a data reading unit 2013, and a data deletion unit 2014;

The access mode unit 2011 is configured to receive and read an application ID in the data access request, acquire a storage mode of the data to be accessed according to the corresponding relationship between the configured application ID and the storage mode, and receive the data access. The request is a data update request, and the obtained storage mode and data update request is sent to the data update unit 2012; or the received data access request is a data read request, and the data read request and the acquired storage mode are sent to The data reading unit 2013; or, the received data access request is a data deletion request, and the data deletion request and the acquired storage mode are sent to the data deletion unit 2014.

The data update unit 2012, when the storage mode for acquiring is the memory storage mode, according to the number Update the data in the memory according to the KEY and Value in the update request; or, when adding the normal hard disk storage mode to the memory, update the data in the hard disk according to the KEY and Value of the data in the data update request, and then update the data in the memory; or When adding the SSD storage mode to the memory, determine the remaining space in the memory, update the data in the memory, otherwise, release the memory space, and then update the data in the memory.

The data reading unit 2013, when the storage mode for acquiring is the memory storage mode, reads the data in the memory according to the KEY in the data read request; or, when the memory is added to the normal hard disk storage mode, according to the data read request The KEY in the first reading data from the memory, if there is no data to be read in the memory, the data is read from the hard disk and cached into the memory; or, when the memory is added to the solid state hard disk storage mode, according to the index in the memory The information determines that the data to be read exists, and then determines the storage location of the data based on the index information, and reads the data.

The data deletion unit 2014, when the storage mode used for obtaining is the memory storage mode, deletes the data in the memory according to the KEY in the data deletion request; or, when the memory is added to the normal hard disk storage mode, according to the KEY, Delete the data in the normal hard disk, and further delete the data in the memory; or, when adding the SSD storage mode to the memory, determine the data to be deleted according to the index information in the memory, and delete the data according to the storage location of the data in the index information. And delete the index information corresponding to the data.

The data update unit 2012 is specifically configured to determine, according to the memory capacity information stored in the memory, whether there is any remaining space in the memory, and if there is no remaining space, move the data with the earliest storage time in the memory to the fast solid state hard disk by using the LRU algorithm. To free up memory space.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included. Within the scope of protection of the present invention.

Claims

Claim

A method for data access of a distributed cache system, comprising:

The server node receives the data access request sent by the client, determines the storage mode of the accessed data according to the data access request, accesses the data according to the storage mode, and returns the access result to the client.

2. The method according to claim 1, wherein the determining, according to the data access request, the storage mode of the accessed data comprises:

The application identifier ID in the data access request is read, and the storage mode of the accessed data is obtained according to the correspondence between the locally configured application ID and the storage mode.

The method according to claim 1 or 2, wherein the received data access request is a data update request, and the accessing the data according to the storage mode comprises:

Determined to be the memory storage mode, and update the data in the memory according to the key KEY and the value Value in the data update request; or,

Determine the memory plus normal hard disk storage mode, update the data in the hard disk according to the KEY and Value in the data update request, and then update the data in the memory; or,

Determine the memory plus SSD mode, determine the remaining space in the memory, update the data in the memory, otherwise, free up the memory space, and then update the data in the memory.

The method according to claim 1 or 2, wherein the received data access request is a data read request, and the accessing the data according to the storage mode comprises:

Determined to be the memory storage mode, read the data in the memory according to the KEY in the data read request; or,

Determine to add the normal hard disk storage mode to the memory, read the data in the memory according to the KEY, and if there is no data to be read in the memory, read the data from the hard disk; or,

It is determined that the memory plus SSD mode determines the existence of the data to be read according to the index information in the memory, and then determines the storage location of the data according to the index information, and reads the data.

The method according to claim 1 or 2, wherein the received data access request is a data deletion request, and the accessing the data according to the storage mode comprises:

Determined to be in the memory storage mode, according to the KEY in the data deletion request, delete the data in the memory; or,

Determine the memory plus normal hard disk storage mode, delete the data in the hard disk according to the KEY in the data deletion request, and then delete the data in the memory; or,

Determine the memory plus SSD storage mode, determine the data to be deleted according to the index information in the memory, and delete the data according to the storage location of the data in the index information.

6. A device for data access of a distributed cache system, comprising: a data access module and an information feedback module;

The feedback module is set to return the access result to the client.

The device according to claim 6, wherein the data access module comprises an access mode unit, a data update unit, a data reading unit, and a data deletion unit;

The access mode unit is configured to read an application ID in the data access request, acquire a storage mode of the accessed data according to a mapping relationship between the locally configured application ID and the storage mode, and receive the data access request as a data update request. And sending the obtained storage mode and data update request to the data update unit; the received data access request is a data read request, and the data read request and the acquired storage mode are sent to the data reading unit; the received data The access request is a data deletion request, and the data deletion request and the acquired storage mode are sent to the data deletion unit.

The device according to claim 7, wherein the data updating unit is configured to: when the acquired storage mode is the memory storage mode, according to the KEY in the data update request and

Value, update the data in memory; or, When the normal hard disk storage mode is added to the memory, the data in the hard disk is updated according to the KEY and Value of the data in the data update request, and then the data in the memory is updated; or,

When adding the SSD storage mode to the memory, make sure there is free space in the memory, update the data in the memory, otherwise, free up the memory space, and then update the data in the memory.

The device according to claim 7, wherein the data reading unit is configured to: when the acquired storage mode is the memory storage mode, read the data in the memory according to the KEY in the data read request; or ,

When the normal hard disk storage mode is added to the memory, according to the KEY, according to the KEY in the data read request, the data is first read from the memory, and if there is no data to be read in the memory, the data is read from the hard disk and cached. Into the memory; or,

When the SSD storage mode is added to the memory, the data to be read is determined according to the index information in the memory, and then the storage location of the data is determined according to the index information, and the data is read.

The device according to claim 7, wherein the data deleting unit is configured to: when the acquired storage mode is the memory storage mode, delete the data in the memory according to the KEY in the data deletion request; or

When the normal hard disk storage mode is added to the memory, according to the KEY, the data in the ordinary hard disk is deleted first, and the data in the memory is further deleted; or

When the SSD storage mode is added to the memory, the data to be deleted is determined according to the index information in the memory, and the data is deleted according to the storage location of the data in the index information.