WO2016186457A1

WO2016186457A1 - Information providing system and method using image recognition technology

Info

Publication number: WO2016186457A1
Application number: PCT/KR2016/005283
Authority: WO
Inventors: 김동주
Original assignee: 주식회사 넥스비즈코리아
Priority date: 2015-05-18
Filing date: 2016-05-18
Publication date: 2016-11-24

Abstract

The present invention relates to an information providing method using an image recognition technology, which is connected through a mobile terminal and a network and includes an image recognition server, a webserver, and a DB server. The method comprises the steps of: transmitting, by the mobile terminal, an obtained image or an image pattern value extracted from the obtained image to the image recognition server; extracting the image pattern value when having received the image transmitted from the mobile terminal, and searching for an image pattern value similar to a pre-registered image pattern value when having directly received the image pattern value transmitted thereto, by the image recognition server; transmitting an image code value, which is a searching result value, either through the mobile terminal or directly to a webserver by the image recognition server; and searching for a promotional code value and a shopping code value, which are registered in the received image code value, and transmitting promotion information registered in the promotional code value, shopping mall URL information registered in the shopping mall code value, or the like to the mobile terminal by the webserver.

Description

Information provision system and method using image recognition technology

The present invention relates to a method for providing related information registered by multiple information providers in the same image using image recognition technology in a mobile environment.

Image recognition technology is applied in various ways according to the field of use, one of which is to find the same image among multiple images. As an example, when a mobile app (Application) is driven in a mobile device equipped with a camera such as a smartphone, and a user photographs an advertisement image or a label of a product using a smartphone, the photographed image is the image. It is a technology that delivers the information registered in the image by finding the same image or the most similar image that is delivered to the recognition server.

However, there has been a shortcoming when such a technique is used for product promotion sales. For example, in advertising, an advertiser focuses on product promotion rather than linking to a specific purchase site because it does not matter which shopping mall is sold through or offline. In contrast, direct sellers, such as distributors of the product, want customers to connect to their product purchase site in order to have their products buy from their sales site.

In this case, a number of vendors are trying to link the same image to their purchase site, but there is a problem that does not exist at present.

The present invention has been made to solve the above-described problems, the present invention is a mobile user acquires an image from his mobile terminal, the seller of the image through the APP and the server using the image recognition technology registered by the seller An object of the present invention is to provide a means for connecting a user terminal to a purchase site (such as an internet shopping mall) of a product.

As a technical means for solving the above problems, an aspect of the present invention is connected to a mobile terminal via a network, and in the information providing method using an image recognition technology comprising an image recognition server, a web server, and a DB server Transmitting the image pattern value extracted from the acquired image or the acquired image to the image recognition server; Extracting an image pattern value when the image recognition server receives an image from the mobile terminal, and retrieving an image pattern value similar to a registered image pattern value when the image pattern value is directly transmitted; The image recognition server transmitting an image code value, which is a search result value, through the mobile terminal or directly to a web server; The web server finds a promotional code value and a shopping code value registered in the received image code value, and transmits the promotional material information registered in the promotional code value or the shopping mall URL information registered in the shopping mall code value to a mobile terminal. It provides an information providing method using an image recognition technology comprising a.

Preferably, the image and the corresponding image code value is registered in advance in the DB server, the registration method is to register a new image that has not been previously registered or to find a desired image from the pre-registered image to the desired image You can register additional information.

In addition, a method of finding a desired image from the previously registered image is a method of searching through a keyword search by registering a keyword corresponding to a registered image when registering an image in advance, or by searching the same or most through an image search. The same image may be regarded as a method of searching for and selecting a similar image, or registering an image used for the search, or having an extremely small difference between the pattern value of an image to be registered and an image to be registered and the same as the registered pattern value. .

Another aspect of the present invention is an information providing system using an image recognition technology connected through a mobile terminal and a network, the image recognition to search for a similar image with respect to the received image, the image code value corresponding to the retrieved similar image server; And a DB server connected to the image recognition server, the DB server storing a plurality of promotional and / or shopping related information in an image code value corresponding to the image in order to provide a plurality of information in one image code value. Including, but if the mobile terminal wants to provide information through the acquired image, it is configured to connect to select from a plurality of promotional and / or shopping-related information for the image.

The effect by the present invention is as follows.

First, the number of images registered on the server is greatly reduced. In order to speed up the search, the server searches in a state where all image pattern values are resident in RAM. Instead of registering the same number of images, only one image is registered and multiple link addresses or related information are searched. Linking information not only reduces the data capacity of the server, but also speeds up the search.

Second, by linking multiple purchase site link addresses to a single image, consumers can purchase products at a lower price purchase site, and the information displayed at the top, such as a premium link on an Internet portal, can be viewed through auctions. Higher profits are possible because they can be provided at higher prices.

1 illustrates an information providing system using an image recognition technology according to an exemplary embodiment of the present invention.

2 is a reference diagram for explaining a process of identifying whether or not a similar image is identical by using an image recognition technology according to an embodiment of the present invention.

Figure 3 relates to a method for automatically finding the same image using image recognition technology in accordance with an embodiment of the present invention.

4 is an example of an image connection DB and an information registration DB table for each promotion and shopping information provider according to an exemplary embodiment of the present invention.

5 is a flowchart illustrating a method of finding an image to be registered from a pre-registered image through keyword search according to an embodiment of the present invention.

6 is a flowchart illustrating a method of determining whether an image to be registered is a registered image through a search according to an embodiment of the present invention, and selecting and registering an image or a newly registered image according to the result. to be.

FIG. 7 is a flowchart for providing related information registered by a plurality of information providers using image recognition technology according to an exemplary embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will be apparent from the following detailed description with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms. It is provided to fully convey the scope of the invention to those skilled in the art, the invention being defined only by the scope of the claims.

1 illustrates an information providing system using an image recognition technology according to an embodiment of the present invention.

The information providing system using the image recognition technology of FIG. 1 is connected to the mobile terminal 100 through a network, and includes a web server 200, an image recognition server 300, and the like. The image recognition server 300 is connected to the image server connection DB 310, information registration DB 320 for each information provider. The information provider DB information for each provider 320 may be configured to include a promotion DB (320A), shopping DB (320B). Various DBs are classified for convenience of description and may be collectively named as DB server 305.

According to this embodiment, when only one information is provided in one image, the image code connected to the image recognition server 300 may be configured as a single code, but the information provided by a plurality of information providers is provided in one image. In order to do this, it is preferable that there is a separate information registration DB for each information provider separately from the image server connection DB 310 of the image recognition server 300. The image server connection DB 310 stores a code value connected to at least one information provider-specific information registration DB 320 in one image information.

Each image may be registered by the mobile terminal 100 or registered by the system operator. In addition, a method of registering an image may be classified into a promotional material registration and a shopping mall registration, and may be registered in the promotion DB 320A and the shopping DB 320B.

Here, the term "public relations" refers to various types of promotional materials such as introduction articles, photo albums, and videos, and a purchase site for sale refers to an Internet shopping mall.

The mobile terminal 100 installs and holds an application APP for providing information using the image recognition technology of the present invention. Looking at the function of the mobile terminal 100, and transmits the image or image pattern value obtained from the mobile terminal to the image recognition server (300). This image may be an image taken through an APP or an internal camera function installed in a mobile terminal, may be a recognized image, or may be an image secured by a method such as down, scan, or the like separately. The preferred method is camera shot or recognized image. In response, the mobile terminal 100 receives the search result value (image code value) from the image recognition server 300 and transmits it to the web server 200.

The image recognition server 300 extracts an image pattern value when the image is received from the mobile terminal 100, and retrieves the most similar image pattern value among the registered image pattern values when the image pattern value is directly transmitted. The search result value (image code value) is transmitted to the mobile terminal 100. Alternatively, the search result value may be transmitted to the web server 200 without transmitting the search result value to the mobile terminal 100.

The web server 200 finds the promotional code value and the shopping code value registered in the image code value received from the information registration DB 320 for each information provider, and the promotional material (introduction, photo album, video) registered in the promotional code value. ) Or the shopping mall URL information registered in the shopping mall code value to the mobile terminal 100, and the mobile terminal 1000 receiving the received information is displayed on the mobile screen. Icon), one or more shopping mall information appears, and a user selects a shopping mall among them.

In this embodiment, the image recognition server 300 plays a role of delivering the image code value by finding the same (most similar) image, and the web server 200 stores the promotional code value and the shopping mall in the DB server where the image code value is found. It is described as playing an intermediary role to find the code value and again to find the information registered in the promotional code value and the shopping mall code value. However, if necessary, the image recognition server 300 and the web server 200 may be configured as a single server.

Hereinafter, the image registration method will be described in detail.

The image registration method may include i) a method of registering a new image not previously registered, and ii) a method of finding a desired image from the registered image and additionally registering information desired by the user.

As a method of searching for a desired image from a registered image, ii-1) when registering an image in advance, the keyword corresponding to the registered image is also registered, so that the keyword can be found through keyword search. Ii-2) There is a method of searching for and selecting the same or most similar image through image search, or registering an image used in the search. In addition to ii-3), there may be a method in which the image having the same or identical error value between the registered pattern value and the pattern value of the registered image may be regarded as the same image. However, according to this method, a completely different image may be regarded as the same image depending on the image. Therefore, when using this method, it may be effective to configure a standard.

How to determine whether the same image to register the image will be described in more detail.

In the first method, if the pattern value of the image to be newly registered is equal to or larger than a predetermined value with the pattern value of the previously registered image, it is regarded as the same image. In other words, if more pattern values are extracted, the higher the value of Query Image Score compared to the comparison image, the less likely it is to consider another image as the same. If it is high, it is regarded as the same image. In the above method, it is necessary to additionally find the same image below a certain level.

As a second method, there is a method of judging based on a comparison ratio of a predetermined level or more between a pre-registered image and a newly registered image. According to this method, in order to calculate a matching rate, a reference value (Resxult Image Score) may be extracted from a pattern value of a pre-registered image to calculate a comparison value (Query Image Score) and a comparison rate of the comparison image.

The method of calculating the reference value from the pattern value of the pre-registered image may be based on the number of pattern values extracted from the image engine server 300, but the same image as the pre-registered image is searched and compared with the pre-registered image. The comparison value calculated by reference may be used as a reference value. Similar to the first method, the above method can regard different images as the same image if the comparison value is below a certain level.

After all, whether compared with a comparison value or calculated with a comparison rate, a method of borrowing human judgment may be effective to determine whether or not the same image is determined below a predetermined comparison value. The reason for this is, for example, when an image of a specific product logo is registered and the same product logo is searched, and when an image including a product name is included in the product logo, both comparison values may be identical. Because it can.

Referring to Figure 2, Figure (a) and Figure (b) in the product, except for the letters G1, G1FE are all the same, but the two products are different model name. On the other hand, Figure (c) is the same product as Figure (a), with the letters GS Caltex engraved at the bottom.

Combined with the existing image recognition technology, the pattern value is much larger than the difference between Fig. (A) and Fig. (B), and the error of Fig. (A) and Fig. (C) is much larger. ) Is more likely to be perceived as an image similar to Figure (b). For this reason, the method of using the human cognitive ability may be effective to solve this problem. In this case, borrowing the human cognitive ability does not apply the human cognitive ability as part of the system, but means that a step of selecting one of the images displayed on the terminal screen is added.

As a result, when an information provider registers an image, it searches for whether an image similar to that image is registered in the existing image recognition server 300, and considers the same image if the comparison image (Query Image Score) of the comparison image is above a certain level. It may be desirable to enable automatic registration, and below a certain level, to expose at least one or more of the most similar images on the screen, and to allow the information provider to select which image to use between the pre-registered image and the image that he or she would like to initially register. have.

According to another method, the user selects and registers an image to be registered as the same image even though the same image is extracted. In this case, two identical images are registered in the image recognition server 100, but a solution in this case is also required. The solution is that the comparison values of the comparison images are searched for a plurality of comparison images in ascending order, so that the comparison value (Query Image Score) of the highest comparison image or the reference value (Result Image Score) of the registered image and the plurality of retrieved images are compared. By calculating the comparison rate with the comparison value, the comparison rate is displayed to a number of images or simple related information of a certain level or more, and then the service user is encouraged to view the image or simple related information and select the desired information. You can do it.

In the above method, the method of extracting a plurality of image codes having a predetermined level or more may be extracted from the image recognition server 300 and sent to the mobile terminal 100, or the image recognition server 300 in the mobile terminal 100. There is a method of selecting only image codes having a certain level of comparison rate among a predetermined number of image codes received from.

In the above method, it is preferable that a plurality of pieces of information registered in a corresponding image are arranged first, and the remaining ones are arranged at the rear, rather than having a high comparison ratio of image information. The reason is that most information providers select the same image and register the information, but even though the information is the same image, the information that the information provider wants to register has already been subtracted from the information of the information provider who selected the image that they wanted to register. If it is registered, it is effective to induce selection of the image.

3 is a reference diagram for explaining a method of automatically finding the same image using an image recognition technique according to an exemplary embodiment of the present invention.

Referring to FIG. 3, the reference value of an image previously registered in an image recognition server, a reference value of a new image, and a comparison value with the previously registered image are identical through an image search. It's a way of thinking of images.

In other words, the method extracts a comparison value and determines the same value between the reference value of the pre-registered image, the reference value of the new image, and the comparison value of comparing the new image to the pre-registered image.

In more detail, the reference value of the image is extracted through the image recognition server or the test image server to register the image and the image is registered in the image recognition server. Next, the reference value is extracted from the new image through the image recognition server or the test image recognition server. The new image is extracted by the image recognition server. The new image is registered as a new image in the image recognition server when the comparison value for the new image and the reference value of the pre-registered image that is the target of the comparison value are out of a specific error range. If the comparison value of the new image is within the same or a specific error range as the reference value of the previously registered image, and the comparison value of the new image is within the same or specific error range compared to the reference value of the new image, the same image is processed. When the comparison value of the new image is out of a specific error range compared to the reference value of the new image, it is possible to configure to register as a new image.

The procedure and method of the steps can be variously modified, but the basic concept is that the reference value of the image registered in the image recognition server, the reference value of the new image, and the comparison value with the previously registered image through the new image search are the same. Or, it is based on a method that is regarded as the same image when it is within an error range, and various methods that can be generated therein are all included in the method.

On the other hand, the method of extracting the reference value is not a problem if the reference value can be extracted immediately when registering the image to the image recognition server, but if the image recognition server extracts and stores only the image pattern value of the image In order to extract the reference value, there is a method of using the comparison value of the image as a reference value by searching the image recognition server again with the same image.

In addition, the procedure in this step is regarded as the same if the result value is the same even if it is partially changed.

4 is an example of an image server connection DB and an information registration DB table for each promotion and shopping information provider according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the promotional code and the shopping mall code are matched for each image code. For example, if an image is registered, an image code of R1512224000100 is generated, and is matched with the image and stored. Promotional code for each information provider 1, 2, 3 is registered in the information registration DB table for promotional information providers, and shopping code is registered in the shopping mall DB.

In addition, the promotional code is provided in various forms such as introduction, video, photo album, etc., and the shopping code is linked to one or more shopping mall link addresses. For example, if an information provider 1 registers a promotion and registers a shopping mall, a promotional code may be assigned and a shopping mall code may also be assigned.

And the promotional code is to register the information you want to register in the introduction, videos, photo albums, etc. If you have one shopping mall in the code, only one URL can be registered, multiple shopping malls can be registered.

Hereinafter, an information providing method using an image recognition technology according to an embodiment of the present invention will be described in detail.

In this method, multiple information providers register different information in the same image. First, a PC or mobile terminal 100 connected to the Internet network accesses a web server 200 that provides a service using image recognition technology, and performs keyword search for a specific image (A01). At least one image registered with the keyword is transmitted from the web server 200 and displayed on the screen of the PC or mobile terminal (A02). One of the images displayed on the screen of the PC or mobile terminal 100 is selected and the related information is registered (A03). The selected image and the related information are transmitted to the web server 200 (A04).

In the present embodiment, when there is no image corresponding to the search keyword or the image searched by the keyword does not have the desired image, the step of registering a new image through the PC or mobile terminal 100 is added, or another process Can be added.

First, a PC or mobile terminal 100 connected to the Internet network accesses a web server 200 that provides a service using image recognition technology, uploads and searches a specific image (B01). The web server 200 transmits the received image to the image recognition server 300 (B02). The image recognition server 300 analyzes characteristic pattern values of the transmitted image (B03). The image recognition server 300 compares the pattern value of the transmitted image with a pre-registered image pattern value (B04). The image recognition server 300 transmits at least one or more image information corresponding to the same or most similar pattern value to the web server 200 (B05). The web server 200 transmits the received image information to the PC or the mobile terminal 100 and appears on the screen of the PC or the mobile terminal 100 (B06). At least one of the image or images displayed on the screen of the PC or the mobile terminal 100 and the image used for the search is selected and related information is registered (B07). The selected image and related information are transmitted to the web server 100 (B08).

If a comparison value (Query Image Score) of the image comparing with the previously registered image pattern value is a predetermined level or more may be added to the step is automatically considered to be the same image.

Alternatively, the step of storing the comparison image score (Resxult Image Score), which is obtained by searching for the same image as the registered image instead of the above step, as the reference image score or storing a reference value of a predetermined level or more, and If the comparison value is equal to or higher than the reference value or more than a predetermined comparison rate, the step of automatically registering the same image may be added.

In addition, in the above embodiment, the image transmitted from the PC or mobile terminal 100 has a structure that is transmitted to the image recognition server 300 via the web server 200, but the image recognition directly from the PC or mobile terminal 100 The server 300 may be configured to transmit or transmit the result to the web server 200 after the result is transmitted from the image recognition server 300 to the PC or the mobile terminal 100.

In addition, analyzing the characteristic pattern value of the transmitted image may be transmitted to the image recognition server 300 and the image recognition server 300, but it may be analyzed, but the image on the PC or mobile terminal 100 or a separate server Contains a logic to analyze the pattern value of the they may be made to the structure to transmit the pattern value of the image to the image recognition server 300.

First, an app that provides a service using the image recognition technology of the mobile terminal 100 is driven to capture a specific image (C01). As described above, this image may be an image taken through an APP or an internal camera function installed in a mobile terminal, may be a recognized image, or may be an image secured by a method such as down, scan, or the like. Thereafter, the image is transmitted to the image recognition server 300 (C02). The image recognition server 300 analyzes characteristic pattern values of the transmitted image (C03). Next, the image recognition server 300 compares the pattern value of the transmitted image and the registered image pattern value (C04). At least one image information corresponding to the same or most similar pattern value is transmitted to the mobile terminal 100 (C05).

The mobile terminal 100 transmits the received at least one image information to the web server (C06). The web server 200 searches for relevant information registered by at least one or more information providers on the at least one or more image information through a DB server (C07). The web server 200 transmits the at least one or more related information respectively retrieved to the at least one or more image information to the mobile terminal 100 (C08). When there are a plurality of searched related information, the searched list of a plurality of related information is displayed on the screen of the mobile device (C09). Promotional information or shopping mall information may be displayed on the mobile screen. In particular, when the shopping mall UI (icon) is selected, one or more shopping mall information appears, and the shopping mall information may be implemented to select a shopping mall desired by the user.

In the above embodiment, the image transmitted from the PC or the mobile device is transmitted to the image recognition server, but the mobile device is transmitted to the image recognition server via a web server, and the resulting image information is transmitted from the image recognition server to the web server. The web server may be configured to search for relevant information registered by a plurality of information providers corresponding to the image information and transmit the related information to the mobile device.

In addition, analyzing the characteristic pattern value of the transmitted image may be transmitted to the image recognition server and the image recognition server may be analyzed, but the mobile device or a separate server contains the logic to analyze the pattern value of the image You can also make a structure that analyzes the pattern value of the image and sends it to the image recognition server.

In another variation, the above-described embodiment may be provided by additionally classifying into promotion and shopping according to the information type provided by the information provider. When providing the relevant information corresponding to the image information, only the information related to the promotion is initially provided, and the UI (icon) of shopping related information is selected on the screen of the mobile device, and the mobile device transmits the information to a web server. The web server searches for shopping related information registered by at least one information provider through the DB server, and the web server transmits the searched at least one shopping related information to the mobile device. When there are a plurality of searched related information, a step of displaying the searched plurality of related information lists on the screen of the mobile device may be further provided.

Although the above-described present invention has been described with respect to the preferred embodiments, the present invention is not limited thereto, and it is possible to carry out various modifications within the scope of the claims and the detailed description of the invention and the appended drawings, and this invention also. Belongs to.

Claims

In the information providing method using an image recognition technology connected to a mobile terminal and a network, comprising an image recognition server, a web server, and a DB server,

Transmitting the image pattern value extracted from the image acquired from the mobile terminal or the acquired image to the image recognition server;

Extracting an image pattern value when the image recognition server receives an image from the mobile terminal, and retrieving an image pattern value similar to a registered image pattern value when the image pattern value is directly transmitted;

The image recognition server transmitting an image code value, which is a search result value, through the mobile terminal or directly to a web server;

The web server finds a promotional code value and / or a shopping code value registered in the received image code value, and transmits promotional material information registered in the promotional code value or shopping mall URL information registered in the shopping mall code value to a mobile terminal. Information providing method using an image recognition technology comprising the step of.
According to claim 1,

The image and the corresponding image code value are registered in advance in the DB server,

The registration method is a method of registering a new image that has not been previously registered, or an information providing method using image recognition technology that finds a desired image in a pre-registered image and adds the desired information to the image.
The method of claim 2,

How to find the desired image from the registered image,

When registering an image in advance, the keyword corresponding to the registered image is also registered so as to find it through keyword search, or by searching for and selecting the same or most similar image through image search, or registering the image used in the search. An information providing method using an image recognition technology using a method or a method in which an error of the pattern value of the registered image and the registered image and the image to be registered and the registered image is extremely small may be regarded as the same image.
The method of claim 2,

In the method of registering a new image not previously registered in the image recognition server, the reference value of the previously registered image is extracted, and the reference value, the reference value of the new image, and the comparison value between the previously registered image through the new image search are all error. Information providing method using image recognition technology to determine the same image within the range.
In the information providing system using the image recognition technology connected via a mobile terminal and a network,

An image recognition server searching for a similar image with respect to the received image and searching for an image code value corresponding to the found similar image; And

A DB server connected to the image recognition server, the DB server storing a plurality of promotional and / or shopping related information in an image code value corresponding to an image in order to provide a plurality of information in one image code value; But

When providing information through the image obtained by the mobile terminal, an information providing system configured to select and connect from a plurality of promotional and / or shopping-related information for the image.
The method of claim 5,

The image and the corresponding information are registered in advance in the DB server,

The registration method is a method of registering a new image that has not been previously registered or an information providing system that finds a desired image in a pre-registered image and additionally registers desired information in the image.
The method of claim 6,

How to find the desired image from the registered image,

When registering an image in advance, the keyword corresponding to the registered image is also registered so as to find it through keyword search, or by searching for and selecting the same or most similar image through image search, or registering the image used in the search. An information providing system using a method or a method in which an error of the pattern value of an image to be registered and a registered image that is identical to or registered with an already registered pattern is extremely small can be regarded as the same image.
The method of claim 5,

Further includes a web server,

The image recognition server searches for a similar image and transmits an image code value, which is information about a similar image, to the web server, and the web server searches for a promotional code value or a shopping mall code value matching the image code value in the DB server. Information provision system.