CN103440332A - Image searching method based on relation matrix regularization enhancement representation - Google Patents

Abstract

The invention discloses a method for searching an image from an image case library based on a relation matrix regularization enhancement method. The method comprises the following steps of: step 1, inputting an image to be searched; step 2, extracting characteristics of the image to be searched and images in the image case library; step 3, selecting P image classes from the image case library and forming sample data X by selecting n images from each image class; step 4, constructing three matrixes by the sample data X based on a manifold learning algorithm of a spectral graph theory; step 5, primarily establishing an enhancement relation matrix W'; step 6, calculating a regularization enhancement relation matrix W*; step 7, calculating a generalized characteristic matrix A; step 8, calculating final image representation; step 9, calculating image representation of the image to be searched; and step 10, calculating the similarity of the image to be searched and all the images in the image case library by adopting an Euclidean distance; outputting the images in the image case library, which are the most similar with the image to be searched, according to the similarity from large to small.

Description

A kind of image search method meaned that strengthens based on the relational matrix regularization

Technical field

The invention belongs to field of image search, particularly a kind of image search method based on relational matrix regularization Enhancement Method.

Background technology

The today reached increasingly in Science and Technology Day, fast development and popularization and application along with image acquisition process equipment and Internet technology, the generation information resource that the image of take is representative has become the strategic resource that has status of equal importance with material, the energy, its data volume has also reached the magnanimity scale, becomes the main body of current information processing and construction of information resources.The advantage such as contain much information because image has, abundant in content, expressive force is strong, therefore carry out effective information processing and application to the image of magnanimity scale, become the key problem of numerous practical application area.

Because current image date has been the magnanimity scale, and, in continuous growth, traditional technological means can't adapt to this demand, and the technology such as this tissue to image, analysis, retrieval and management have all proposed brand-new challenge.Although CBIR research at present makes great progress, effectively overcome the limitation of carrying out image retrieval based on the manual text message marked, but also have certain distance from the real practical stage, especially the high-level semantic of image is understood to aspect.Major part method also only rests on around the low-level image feature of image carries out semantic description and this level of study, the colourful semantic concept that can understand and use with respect to the mankind, the ability to express of bottom data feature still has very large limitation, therefore exist larger gap between low-level image feature and high-level semantic, i.e. so-called " semantic gap " (semantic gap), thereby cause also not reaching the needs of practical application far away on the accuracy rate of image retrieval and efficiency, especially the multiple abundant semanteme of image is understood and the retrieval aspect accurately and effectively.Even to this day, " semantic gap " problem in image retrieval still is not well solved, and remains one of key difficult problem of puzzlement researcher.In the middle of the numerous technology that solve this difficult problem, the image retrieval technologies based on relevant feedback provides a kind of feasible solution.Early stage Relevance Feedback mainly concentrates on the information based on relevant feedback, and revising query vector is characteristics of image, for example every one dimensional numerical of query vector is redistributed to weights, adjusts the position of query vector etc.In recent years, due to the rise of manifold learning, many researchers turn to by the manifold learning technology, the image data space dimensionality reduction of higher-dimension are sought to the immanent structure of image feature space, its main theory hypothesis is to regard image as a kind of stream shape, and target is exactly to find its inherent structural information.The low n-dimensional subspace n that discovery is embedded in high dimensional data is the important means of the potential stream of learning data shape, and in manifold learning, the learning method of subspace all is based on partial analysis.Learn the semantic subspace of its corresponding low-dimensional by the method for manifold learning, this and manifold learning suppose that whole data set only meets Euclidean distance in part and matches, therefore by the local message of analysis of image data, excavate local semantic manifold structure more meaningful concerning image retrieval.

Summary of the invention

Goal of the invention: the present invention, in order to solve the problems of the prior art, has proposed a kind of image search method meaned that strengthens based on the relational matrix regularization, effectively solves under large-scale data the quick and precisely search problem of image.

Summary of the invention: the invention discloses a kind of image search method based on relational matrix regularization Enhancement Method, the method is retrieving images from the image case library, comprises following steps:

Step 1, input image to be retrieved;

Step 2, extract the feature of image in image to be retrieved and image case library, with the N dimensional vector, describes every width image, and N=112, obtain image example aspects storehouse U=(u ₁..., u _m), u _ifeature for image case library i width image, i=1, M, M is the picture number comprised in the image case library, and the feature v of image to be retrieved, described image case library comprises the image class more than 50, and each image class means a semantic category, and each image class comprises the image that 600 width are above;

Step 3 is chosen P image class from image example aspects storehouse, and P span 20～50, choose n width image from each image class, n span 100～500, and P the total n * P of image class opens image construction sample data X; For example, in an embodiment of invention, therefrom choose 30 image classes, each class has meaned different semantic categories, and each class has 100 width images, has 3000 image construction sample data X, X=(x ₁..., x _q), q=n * P, x _ifor the feature of i width image in sample data, q is the sample data size, the matrix that X is 112 * q dimension;

Step 4, the manifold learning arithmetic based on spectral graph theory, build and strengthen relational matrix W, positive example relational matrix W sample data X ^pwith counter-example relational matrix W ⁿ;

Step 5, strengthened the relational matrix W built, and preliminary foundation strengthens relational matrix W ';

Step 6, strengthen relational matrix W ' by the probability transfer matrix regularization and obtain regularization enhancing relational matrix W ^*;

Step 7, strengthen relational matrix W according to regularization ^*the establishing target equation, calculate generalized characteristic matrix A;

Step 8, utilize generalized characteristic matrix A to carry out dimensionality reduction to all images in image example aspects storehouse, i.e. AU=A* (u ₁..., u _m)=(A*u ₁..., A*u _m), note y _i=A*x _i, i=1 ... M, obtain final image representation Y=(y ₁..., y _m), y _ifor the feature after image case library i width characteristics of image dimensionality reduction;

Step 9, utilize generalized characteristic matrix A to treat retrieving images feature v dimensionality reduction, obtains the image representation f=A*v of image to be retrieved;

Step 10, calculate the similarity of all images in image to be retrieved and image case library according to the Euclidean distance of the image representation of the image to be retrieved of the final image representation of step 8 and step 9, calculate the Euclidean distance of feature after image dimensionality reduction feature f to be retrieved and image example aspects storehouse every width characteristics of image dimensionality reduction || f-y _i|| ², i=1 ... M, y _ifor the feature after image case library i width characteristics of image dimensionality reduction, according to the image the most similar to image to be retrieved in the descending output image case library of similarity.

In step 2, characteristics of image comprises color moment, Tamura textural characteristics, Gabor textural characteristics, color histogram.

Step 4 specifically comprises the steps: to choose at random piece image in sample data X, calculate the Euclidean distance of other images in this image and sample data X, utilize the relevance feedback retrieval technology, set up positive example set and counter-example set according to the similar image in returning results and inhomogeneity image correspondence, and adopt simple k near neighbor method opening relationships matrix, belong to the k neighbour and be that weights between two images of same image class are 1, otherwise be 0.

In step 4, adopt the imbeding relation based on feedback technique to widen the manifold learning arithmetic of ARE method as spectral graph theory, comprise the following steps:

(1) at first sample data X is built to relational matrix W, randomly draw piece image I from sample data X, image I be characterized as x _i, adopt the k near neighbor method to calculate x _iwith the Euclidean distance of other characteristics of image in sample data X, obtain the k width image the most similar to image I, wherein the k span 5～10;

Take out arbitrarily piece image T and belong to from k width image, image T is characterized as x _t, the weights W between image I and image T _itbe 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ N ^k(x _t) or x _t∈ N ^k(x _i), W _it=1, N wherein ^k(x _i) presentation video x _ik neighbour set, N ^k(x _t) presentation video x _tk neighbour set; Obtain relational matrix W, the value of the capable t row of relational matrix W i is W _it;

Formula is:

The image that belongs to same image class with image I in k width image is designated as to positive example set Pos, and the image of different images class is designated as counter-example set Neg;

(2) build positive example relational matrix W ^pif image R and image I belong to same image class and all belong to k width image, and image R be characterized as x _r, the weights between image I and image R are 1, the weights between the image beyond image I and k width image are 0; That is,

for the weights between image I and image R, x _i, x _r∈ Pos is representation feature x _i, x _rbelong to positive example set Pos, positive example relational matrix W ^pthe value of the capable r of i row be

formula is:

(3) build counter-example relational matrix W ⁿif image H and image I belong to the different images class and all belong to k width image, image H is characterized as x _h, the weights between image I and image H are 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ Pos and x _h∈ Neg or x _h∈ Pos and

representation feature x _ibelong to positive example set Pos, x _h∈ Neg representation feature x _hbelong to counter-example set Neg, x _h∈ Pos representation feature x _hbelong to positive example set Pos, x _i∈ Neg representation feature x _ibelong to counter-example set Neg,

for the weights between image I and image H, counter-example relational matrix W ⁿi h classify as

formula is:

Finally build and obtain three relational matrix W, W ^pand W ⁿ, the relational matrix that its neutralization need to be used for calculating generalized characteristic matrix.

Step 5 specifically comprises the steps: from relational matrix W, if image z is neighbour's image of image i, and image z is also neighbour's image of image j, adopts following formula to calculate the weights W ' strengthened between image i and image j _ij: W ' _ij=Σ _zw _izw _jz

W wherein _izfor the weights of image i and image z, W _jzfor the weights of image j and image z, W ' _ijbe the capable j train value of i that strengthens relational matrix W '.

Step 6 specifically comprises the steps:

The neighbor relationships of repeatedly propagating between image obtains new enhancing relational matrix W ", formula is W "=W ' * W ';

Utilize the transfer relationship between the transition probability matrix presentation video, corresponding transition matrix is P=[P _ij] _{n * n}, P _ij=p (j|i) is that in sample data X, arbitrary image i, to the transition probability of arbitrary image j, selects the n width image the most similar to image i according to Euclidean distance, and image j is characterized as x _j, the computing formula of transition probability P (j|i) is:

D wherein _ij=|| x _i-x _j|| ², the Euclidean distance of presentation video i and image j feature.

The model W that adopts the regularization of following formula calculated relationship matrix to strengthen ^r:

W ^R＝ηP+(1-η)ge ^T

Wherein, η is that image i transfers to the probability that this event of image j occurs, and (1-η) is the probability of the random redirect of image i, g=(1/n) e, wherein g is an even stochastic distribution vector, and e is n dimension unit column vector, and n is the picture number of each image class, e=(1,1 ...) ^t, the capable j of i of matrix P classifies P (j|i) as;

The new weights that concern between image i and image j

computing formula is:

$w_{\mathrm{ij}}^{*}=w_{\mathrm{ij}}^{\″}\·w_{\mathrm{ij}}^R$

W " _ijfor the weights of image i and image j, w " _ijfor W " the value of the capable j of i row,

for image i jumps to the probability weights of image j,

for W ^rthe value of the capable j of i row;

Finally obtain regularization and strengthen relational matrix W ^*, W ^*the capable j of i classify as

In step 7, comprise the steps:

At first choose the feature x of any two width images from sample data X _iand x _j, the weights that concern of two width images are W _ij, the positive example of two width images concerns that weights are the counter-example of two width images concerns that weights are

calculate generalized characteristic matrix A according to following target equation:

X(L ^N-γL ^P)X ^TA=λXLX ^TA，

The Laplacian Matrix that L is relational matrix W, L ⁿfor counter-example relational matrix W ⁿlaplacian Matrix, L ^pfor positive example relational matrix W ^plaplacian Matrix, γ is the constant that is directly proportional to the ratio of counter-example image number and positive example image number, X ^tmean the transposed matrix of sample data X, λ means the eigenwert of equation solution.

In the present invention, ARE concerns embedding grammar (Augmented Relation Embedding) for widening, a kind of manifold learning dimension-reduction algorithm of widening the graph of a relation embedding, ARE mainly utilizes positive example relational matrix and counter-example relational matrix to embed in the holotopy matrix, find projection matrix, be generalized characteristic matrix, thereby realize the dimensionality reduction to the data feature.

The principle of the invention is, sample data X=(x ₁..., x _n), x _i∈ R ^m, the relational matrix W ∈ R between data point ^{n * N}mean, the entry of a matrix element has been weighed the similarity between every pair of data point.Diagonal matrix D and corresponding Laplacian Matrix L are defined by following formula:

$D_{\mathrm{ii}}={\mathrm{\Σ}}_j{W}_{\mathrm{ij}},\∀i,$

L＝D-W

D _iifor the capable i row of i of diagonal matrix D, suppose that generalized characteristic matrix is A, the low-dimensional that completes original data space by projection embeds, and A can be minimized and be tried to achieve by following formula:

$\underset{\mathrm{ij}}{\mathrm{\Σ}}{(A^T{x}_i-A^T{x}_j)}^2{W}_{\mathrm{ij}}$

Every row a of matrix A _jindependent role, therefore above formula can be write as argmin _aΣ _ij(a ^tx _i-a ^tx _j) ²w _ij, wherein a is proper vector to be asked.Make y _i=a ^tx _i, have:

$\underset{\mathrm{ij}}{\mathrm{\Σ}}{(y_i-y_j)}^2{w}_{\mathrm{ij}}=\underset{\mathrm{ij}}{\mathrm{\Σ}}{y}_i^2{w}_{\mathrm{ij}}-2\underset{\mathrm{ij}}{\mathrm{\Σ}}{y}_i{y}_j{w}_{\mathrm{ij}}+\underset{\mathrm{ij}}{\mathrm{\Σ}}{y}_j^2{W}_{\mathrm{ij}}$

$=2{\mathrm{\Σ}}_i{y}_i^2{D}_{\mathrm{ii}}-2{\mathrm{\Σ}}_{\mathrm{ij}}{y}_i{y}_j{W}_{\mathrm{ij}}$

$={2y}^T(D-W)y={2y}^T\mathrm{Ly}$

Wherein, y means the projection of all data on this projection vector of a, and y=a ^tx.To the restriction of the coordinate after conversion, D _iimean to put with i the number be connected, this importance degree has been described in a way, and then can increase to retrain and make y ^tdy=1.After this constraint can make the conversion of the high point of importance, its coordinate figure approaches the territory initial point more, allows close quarters be positioned at initial point, and the objective function equation finally solved becomes:

$a^{*}=\arg \underset{a}{\min }{a}^T{\mathrm{XLX}}^{T}a,s.t.a^T{\mathrm{XDX}}^{T}a=1$

From derivation, relational matrix W plays leading role in whole process, and the data point y after projection also has close relationship with W, for example works as W _ijwhen larger, mean x _iand x _jsimilarity is larger, y after dimensionality reduction _iand y _jbetween the distance also should be the smaller the better; If W _ijless, mean x _iand x _jsimilarity is less, y after dimensionality reduction _iand y _jbetween the distance also should be the bigger the better.The similarity relation here can mean between data whether to belong to same classification, and the similarity between homogeneous data is naturally very high; For the data that there is no classification information, the similarity between data is just weighed by neighbor relationships, and the similarity between neighbour's data point should be higher; For neither homogeneous data, the similarity also do not had between the data point of neighbor relationships can be lower, generally makes W _ij=0.

Beneficial effect: the present invention utilizes relational matrix regularization enhancing to mean the image example aspects is carried out to dimensionality reduction, the method can effectively be strengthened the relation between similar image, merged the classification information of data in the process of structure relational matrix, make in its framework that expands to easily semi-supervised learning, thereby take full advantage of flag data and Unlabeled data, effectively improve the stability of algorithm and reduce computation complexity, make image querying there is higher accuracy rate simultaneously, therefore the image search method that relational matrix regularization enhancing means has higher use value.

The accompanying drawing explanation

Fig. 1 is process flow diagram of the present invention.

Fig. 2 bit image case library Feature Dimension Reduction process flow diagram.

Fig. 3 is characteristics of image dimensionality reduction process flow diagram to be retrieved.

Fig. 4 is that the image relation strengthens schematic diagram.

Fig. 5 is image random walk model schematic diagram.

The regularization enhancing of Fig. 6 position concerns schematic diagram.

Fig. 7 is the image searching result schematic diagram.

Embodiment

As shown in Figure 1, the invention discloses a kind of image search method meaned based on regularization enhancing relational matrix; Comprise following steps:

Step 1: input image to be retrieved;

As shown in Fig. 2～3, build image regulation enhancing relational matrix and mainly undertaken by step 2～step 6, image example aspects storehouse dimensionality reduction is undertaken by step 8, treat the retrieving images Feature Dimension Reduction and undertaken by step 9:

Step 2, extract the characteristics of image of image to be retrieved and image case library image, feature comprises color moment, Tamura textural characteristics, Gabor textural characteristics and color histogram, with the vector of N dimension, describes every width image, N=112, image to be retrieved is v, and image example aspects storehouse is U=(u ₁..., u _m), M is image case library total number of images, U is that N * M ties up matrix;

Step 3, the every width image of the character representation after extraction is chosen 30 image classes from the image case library, and each class means a semantic category, and each class has 100 width images, has 3000 images, and using it as sample data X, X=(x ₁..., x ₃₀₀₀), matrix X is 112 * 3000 dimensions;

Step 4, the manifold learning arithmetic based on spectral graph theory, build and strengthen relational matrix W, positive example relational matrix W sample data X ^pwith counter-example relational matrix W ⁿ;

Step 5, strengthen relational matrix W, and preliminary foundation strengthens relational matrix W ';

Step 6, strengthen relational matrix W ' by the probability transfer matrix regularization and obtain regularization enhancing relational matrix W ^*;

Step 7, strengthen relational matrix W according to regularization ^*with positive example relational matrix W ^pwith counter-example relational matrix W ⁿthe establishing target function, solve generalized characteristic matrix A;

Step 8, utilize generalized characteristic matrix A to carry out dimensionality reduction to all images in the image case library, i.e. AU=A* (u ₁..., u _m)=(A*u ₁..., A*u _m), note y _i=A*x _i, i=1 ... M, obtain final image representation Y=(y ₁..., y _m);

Step 9, as shown in Figure 3, utilize generalized characteristic matrix A to treat retrieving images feature v and carry out dimensionality reduction, obtains the image representation f=A*v of image to be retrieved;

Step 10, adopt Euclidean distance to calculate the similarity of all images in image to be retrieved and image case library, calculates || f-y _i|| ², i=1 ... M, according to the image the most similar to image to be retrieved in the descending output image case library of similarity.

Step 2 specifically comprises the steps:

Extract every width characteristics of image, i.e. iamge description aspect is by color moment (RGB color space): 9 dimensions; Color moment (LUV color space): 9 dimensions; Tamura textural characteristics: 6 dimensions; Gabor textural characteristics: 24 dimensions; Color histogram (hsv color space): 64 dimensions form.

Step 4 specifically comprises the steps: to choose at random piece image in sample data X, calculate the Euclidean distance of other images in this image and sample data X, utilize the relevance feedback retrieval technology, set up positive example set and counter-example set according to the similar image in returning results and inhomogeneity image correspondence, and adopt simple k near neighbor method opening relationships matrix, belong to the k neighbour and be that weights between two images of same image class are 1, otherwise be 0.

In step 4, adopt the imbeding relation based on feedback technique to widen the manifold learning arithmetic of ARE method as spectral graph theory, comprise the following steps:

(1) at first sample data X is built to relational matrix W, randomly draw piece image I from sample data X, image I be characterized as x _i, adopt the k near neighbor method to calculate x _iwith the Euclidean distance of other characteristics of image in sample data X, obtain the k width image the most similar to image I, wherein the k span 5～10;

Take out arbitrarily piece image T and belong to from k width image, image T is characterized as x _t, the weights W between image I and image T _itbe 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ N ^k(x _t) or x _t∈ N ^k(x _i), W _it=1, N wherein ^k(x _i) presentation video x _ik neighbour set, N ^k(x _t) presentation video x _tk neighbour set; Obtain relational matrix W, the value of the capable t row of relational matrix W i is W _it;

Formula is:

The image that belongs to same image class with image I in k width image is designated as to positive example set Pos, and the image of different images class is designated as counter-example set Neg;

(2) build positive example relational matrix W ^pif image R and image I belong to same image class and all belong to k width image, and image R be characterized as x _r, the weights between image I and image R are 1, the weights between the image beyond image I and k width image are 0; That is,

for the weights between image I and image R, x _i, x _r∈ Pos is representation feature x _i, x _rbelong to positive example set Pos, positive example relational matrix W ^pthe value of the capable r of i row be

formula is:

(3) build counter-example relational matrix W ⁿif image H and image I belong to the different images class and all belong to k width image, image H is characterized as x _h, the weights between image I and image H are 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ Pos and x _h∈ Neg or x _h∈ Pos and

x _i∈ Pos representation feature x _ibelong to positive example set Pos, x _h∈ Neg representation feature x _hbelong to counter-example set Neg, x _h∈ Pos representation feature x _hbelong to positive example set Pos, x _i∈ Neg representation feature x _ibelong to counter-example set Neg, for the weights between image I and image H, counter-example relational matrix W ⁿi h classify as

formula is:

Finally build and obtain three relational matrix W, W ^pand W ⁿ, the relational matrix that its neutralization need to be used for calculating generalized characteristic matrix.

Step 5 specifically comprises the steps: from relational matrix W, if image z is neighbour's image of image i, and image z is also neighbour's image of image j, adopts following formula to calculate the weights W ' strengthened between image i and image j _ij:

W′ _ij＝∑ _zW _izW _jz

W wherein _izfor the weights of image i and image z, W _jzfor the weights of image j and image z, W ' _ijbe the capable j train value of i that strengthens relational matrix W '.

Step 6 specifically comprises the steps:

The neighbor relationships of repeatedly propagating between image obtains new enhancing relational matrix W ", formula is W "=W ' * W ';

Utilize the transfer relationship between the transition probability matrix presentation video, corresponding transition matrix is P=[P _ij] _{n * n}, P _ij=p (j|i) is that in sample data X, arbitrary image i, to the transition probability of arbitrary image j, selects the n width image the most similar to image i according to Euclidean distance, and image j is characterized as x _j, the computing formula of transition probability P (j|i) is:

D wherein _ij=|| x _i-x _j|| ², the Euclidean distance of presentation video i and image j feature.

The model W that adopts the regularization of following formula calculated relationship matrix to strengthen ^r:

W ^r=η P+ (1-η) ge ^twherein, η is that image i transfers to the probability that this event of image j occurs, and (1-η) is the probability of the random redirect of image i, g=(1/n) e, wherein g is an even stochastic distribution vector, and e is n dimension unit column vector, and n is the picture number of each image class, e=(1,1 ...) ^t, the capable j of i of matrix P classifies P (j|i) as;

The new weights that concern between image i and image j

computing formula is:

$w_{\mathrm{ij}}^{*}=w_{\mathrm{ij}}^{\″}\·w_{\mathrm{ij}}^R$

W " _ijfor the weights of image i and image j, w " _ijfor W " the value of the capable j of i row,

for image i jumps to the probability weights of image j,

for W ^rthe value of the capable j of i row; Finally obtain regularization and strengthen relational matrix W ^*, W ^*the capable j of i classify as

comprise the steps: at first to choose the feature x of any two width images in step 7 from sample data X _iand x _j, the weights that concern of two width images are W _ij, the positive example of two width images concerns that weights are

the counter-example of two width images concerns that weights are

calculate generalized characteristic matrix A:X (L according to following target equation ⁿ-γ L ^p) X ^ta=λ XLX ^ta,

The Laplacian Matrix that L is relational matrix W, L ⁿfor counter-example relational matrix W ⁿlaplacian Matrix, L ^pfor positive example relational matrix W ^plaplacian Matrix, γ is the constant that is directly proportional to the ratio of counter-example image number and positive example image number, X ^tmean the transposed matrix of sample data X, λ means the eigenwert of equation solution.

Embodiment 1

The present embodiment comprises following part:

1. input an image I to be retrieved;

2. the characteristics of image of abstract image case library and image to be retrieved, each feature and its corresponding dimension are as follows:

Color moment (RGB color space): 9 dimensions; Color moment (LUV color space): 9 dimensions; Tamura textural characteristics: 6 dimensions; Gabor textural characteristics: 24 dimensions; Color histogram (hsv color space): 64 dimensions.Every like this width image will be described with the vector of 112 dimensions, and image to be retrieved is v, and image example aspects storehouse is U=(u ₁..., u _m), M is image case library total number of images, U is that N * M ties up matrix;

3. choose the training sample data from the U of characteristics of image storehouse, extraction character representation for every width image, and therefrom choose 30 image classes, and each class means a semantic category, each class has 100 width images, has 3000 images, and using it as sample data X, X=(x ₁..., x ₃₀₀₀), matrix X is 112 * 3000 dimensions; .

4. choose at random piece image in sample data X, calculate the Euclidean distance of other images in this image and sample data X, utilize the relevance feedback retrieval technology, set up positive example set and counter-example set according to the similar image in returning results and inhomogeneity image correspondence, and adopt simple k near neighbor method opening relationships matrix, belong to the k neighbour and be that weights between two images of same image class are 1, otherwise be 0.

In step 4, adopt the imbeding relation based on feedback technique to widen the manifold learning arithmetic of ARE method as spectral graph theory, comprise the following steps:

(1) at first sample data X is built to relational matrix W, randomly draw piece image I from sample data X, image I be characterized as x _i, adopt the k near neighbor method to calculate x _iwith the Euclidean distance of other characteristics of image in sample data X, obtain the k width image the most similar to image I, wherein the k value 5;

Take out arbitrarily piece image T and belong to from k width image, image T is characterized as x _t, the weights W between image I and image T _itbe 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ N ^k(x _t) or x _t∈ N ^k(x _i), W _it=1, N wherein ^k(x _i) presentation video x _ik neighbour set, N ^k(x _t) presentation video x _tk neighbour set; Obtain relational matrix W, the value of the capable t row of relational matrix W i is W _it;

Formula is:

The image that belongs to same image class with image I in k width image is designated as to positive example set Pos, and the image of different images class is designated as counter-example set Neg;

(2) build positive example relational matrix W ^pif image R and image I belong to same image class and all belong to k width image, and image R be characterized as x _r, the weights between image I and image R are 1, the weights between the image beyond image I and k width image are 0; That is,

for the weights between image I and image R, x _i, x _r∈ Pos is representation feature x _i, x _rbelong to positive example set Pos, positive example relational matrix W ^pthe value of the capable r of i row be

formula is:

(3) build counter-example relational matrix W ⁿif image H and image I belong to the different images class and all belong to k width image, image H is characterized as x _h, the weights between image I and image H are 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ Pos and x _h∈ Neg or x _h∈ Pos and x _i∈ pos representation feature x _ibelong to positive example set Pos, x _h∈ Neg representation feature x _hbelong to counter-example set Neg, x _h∈ Pos representation feature x _hbelong to positive example set Pos, x _i∈ Neg representation feature x _ibelong to counter-example set Neg,

for the weights between image I and image H, counter-example relational matrix W ⁿi h classify as

formula is:

Finally build and obtain three relational matrix W, W ^pand W ⁿ, the relational matrix that need to use for calculating generalized characteristic matrix.

5. set up initial relation strengthen matrix W ', from relational matrix W, if image z is neighbour's image of image i, and image z is also neighbour's image of image j, adopts following formula to calculate the weights W ' strengthened between image i and image j _ij:

W′ _ij＝Σ _zW _izW _jz

W wherein _izfor the weights of image i and image z, W _jzfor the weights of image j and image z, W ' _ijbe the capable j train value of i that strengthens relational matrix W '.Instantiation as shown in Figure 4, image 3 is neighbour's images of image 1, and image 3 is neighbour's images of image 2, between image, with the solid line that arrow is arranged, connects and represents neighbor relationships, between image 1 and image 2, with dotted line, be connected, the relation between representative image 1 and image 2 needs to strengthen.

6. build probability transfer matrix W ^rand carry out regularization to strengthening relational matrix W ',

The neighbor relationships of repeatedly propagating between image obtains new enhancing relational matrix W ", formula is w "=w ' * w ';

Utilize the transfer relationship between the transition probability matrix presentation video, corresponding transition matrix is P=[P _ij] _{n * n}, P _ij=P(j|i) be that in sample data X, arbitrary image i, to the transition probability of arbitrary image j, selects the n width image the most similar to image i according to Euclidean distance, image j is characterized as x _j, the computing formula of transition probability P (j|i) is:

D wherein _ij=|| x _i-x _j|| ², the Euclidean distance of presentation video i and image j feature.

The model W that adopts the regularization of following formula calculated relationship matrix to strengthen ^r:

W ^R＝ηP+(1-η)ge ^T

Wherein, η is that image i transfers to the probability that this event of image j occurs, η is taken as 0.85, and (1-η) is the probability of the random redirect of image i, g=(1/n) e, wherein g is an even stochastic distribution vector, e is n dimension unit column vector, and n is the picture number of each image class, and e=(1,1 ...) ^t, the capable j of i of matrix P classifies P (j|i) as;

The new weights that concern between image i and image j

computing formula is:

$w_{\mathrm{ij}}^{*}=w_{\mathrm{ij}}^{\″}\·w_{\mathrm{ij}}^R$

W " _ijfor the weights of image i and image j, w " _ijfor W " the value of the capable j of i row,

for image i jumps to the probability weights of image j, for W ^rthe value of the capable j of i row;

Finally obtain regularization and strengthen relational matrix W ^*, W ^*the capable j of i classify as

instantiation is as shown in Fig. 5～6, probability between Fig. 5 representative image shifts the weights relation, enhancing relational matrix W in Fig. 6 between upper left Fig. 1 presentation video "; what between two width images, with solid line, connect is neighbour's image; it is the enhancing relation between two width images that dotted line connects representative, the transition probability matrix W between top right plot 2 presentation videos ^r, there is transfer relationship with realizing connecting between representative image between image, the regularization between lower Fig. 3 representative image strengthens relational matrix W ^*, " and the W by W ^rmultiply each other and obtain;

7. strengthen matrix W according to the relation after regularization ^*the establishing target function, solve generalized characteristic matrix A,

At first choose the feature x of any two width images from sample data X _iand x _j, the weights that concern of two width images are W _ij, the positive example of two width images concerns that weights are

the counter-example of two width images concerns that weights are

calculate generalized characteristic matrix A according to following target equation:

X(L ^N-γL ^P)X ^TA=λXLX ^TA，

The Laplacian Matrix that L is relational matrix W, L ⁿfor counter-example relational matrix W ⁿlaplacian Matrix, L ^pfor positive example relational matrix W ^plaplacian Matrix, γ is the constant that is directly proportional to the ratio of counter-example image number and positive example image number, X ^tmean the transposed matrix of sample data X, λ means the eigenwert of equation solution.

8. mainly utilize generalized characteristic matrix A to carry out dimensionality reduction to view data in image example aspects storehouse and obtain final image representation, be i.e. AU=A* (u ₁..., u _m)=(A*u _i..., A*u _m), note y _i=A*x _i, i=1 ... M, final image representation is Y=(y ₁..., y _m);

9. mainly utilize generalized characteristic matrix A to treat retrieving images feature v and carry out dimensionality reduction, obtain the image representation f of image to be retrieved, f=A*v;

10. calculate image similarity in image to be retrieved and image case library:

Adopt Euclidean distance to calculate the similarity of all images in image to be retrieved and image case library, calculate || f-y _i|| ², i=1 ... M, || f-y _i|| ²less similarity is larger, according to the image the most similar to image to be retrieved in the descending output image case library of similarity.As shown in Figure 7, calculate the similarity of image to be retrieved and all images of image case library according to Euclidean distance, according to the most similar image of descending output 4 width of similarity.

Embodiment 2

Fig. 1 is embodiment 2 retrieval flow figure, and picture in picture picture source is public Corel5k database.In figure, 2 is that original image is carried out to pre-service, mean piece image with color moment, Tamura textural characteristics, Gabor textural characteristics and color histogram, 3 selected characteristic samples in figure are chosen 30 image classes from the image case library, and each class has meaned a semantic category, each class has 100 width images, have 3000 width images, in order to improve computing velocity, in a use result set, front 400 width images are as overall data set, for the opening relationships matrix W, positive example relational matrix W ^p, counter-example relational matrix W ⁿ.Then relational matrix W is strengthened and obtains W ', and utilize probability transfer matrix W ^rregularization strengthens relational matrix, obtains W ^*, then according to the enhancing relational matrix W of regularization ^*solve the generalized characteristic matrix A of objective function, finally utilize generalized characteristic matrix A to carry out dimensionality reduction to characteristics of image in the image case library and characteristics of image to be retrieved, treating retrieving images is retrieved, utilize Euclidean distance to calculate the similarity of image in image to be retrieved and image case library, according to the image the most similar to image to be retrieved in the descending output image case library of similarity.

The invention provides a kind of regularization and strengthen the image search method that relational matrix means; method and the approach of this technical scheme of specific implementation are a lot; the above is only the preferred embodiment of the present invention; should be understood that; for those skilled in the art; under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.In the present embodiment not clear and definite each ingredient all available prior art realized.

Claims (7)

1. the image search method based on relational matrix regularization Enhancement Method, is characterized in that, the method is retrieving images from the image case library, comprises following steps:

Step 1, input image to be retrieved;

Step 2, extract the feature of image in image to be retrieved and image case library, with the N dimensional vector, every width image is described, N=112, obtain image example aspects storehouse and with the feature of retrieving images, described image case library comprises the image class more than 50, and each image class means a semantic category, and each image class comprises the image that 600 width are above;

Step 3 is chosen P image class from image example aspects storehouse, and P span 20～50, choose n width image from each image class, n span 100～500, and P the total n * P of image class opens image construction sample data X; Step 4, the manifold learning arithmetic based on spectral graph theory, build and strengthen relational matrix W, positive example relational matrix W sample data X ^pwith counter-example relational matrix W ⁿ; Step 5, strengthened the relational matrix W built, and preliminary foundation strengthens relational matrix W '; Step 6, strengthen relational matrix W ' by the probability transfer matrix regularization and obtain regularization enhancing relational matrix W ^*step 7, strengthen relational matrix W according to regularization ^*the establishing target equation, calculate generalized characteristic matrix A;

Step 8, utilize generalized characteristic matrix A to carry out dimensionality reduction to all images in image example aspects storehouse, obtains final image representation;

Step 9, utilize generalized characteristic matrix A to treat the retrieving images dimensionality reduction, obtains the image representation of image to be retrieved;

Step 10, calculate the similarity of all images in image to be retrieved and image case library according to the Euclidean distance of the image representation of the image to be retrieved of the final image representation of step 8 and step 9, according to the image the most similar to image to be retrieved in the descending output image case library of similarity.

2. a kind of image search method meaned that strengthens based on the relational matrix regularization according to claim 1, is characterized in that, in step 2, characteristics of image comprises color moment, Tamura textural characteristics, Gabor textural characteristics, color histogram.

3. a kind of image search method strengthen meaned based on the relational matrix regularization according to claim 2, it is characterized in that, step 4 specifically comprises the steps: to choose at random piece image in sample data X, calculate the Euclidean distance of other images in this image and sample data X, utilize the relevance feedback retrieval technology, set up positive example set and counter-example set according to the similar image in returning results and inhomogeneity image correspondence, and adopt simple k near neighbor method opening relationships matrix, belong to the k neighbour and be that weights between two images of same image class are 1, otherwise be 0.

4. a kind of image search method strengthen meaned based on the relational matrix regularization according to claim 3, it is characterized in that, in step 4, adopt the imbeding relation based on feedback technique to widen the manifold learning arithmetic of ARE method as spectral graph theory, comprise the following steps:

(1) at first sample data X is built to relational matrix W, randomly draw piece image I from sample data X, image I be characterized as x _i, adopt the k near neighbor method to calculate x _iwith the Euclidean distance of other characteristics of image in sample data X, obtain the k width image the most similar to image I, wherein the k span 5～10;

Take out arbitrarily piece image T and belong to from k width image, image T is characterized as x _t, the weights W between image I and image T _itbe 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ N ^k(x _t) or x _t∈ N ^k(x _i), W _it=1, N wherein ^k(x _i) presentation video x _ik neighbour set, N ^k(x _t) presentation video x _tk neighbour set; Obtain relational matrix W, the value of the capable t row of relational matrix W i is W _it;

The image that belongs to same image class with image I in k width image is designated as to positive example set Pos, and the image of different images class is designated as counter-example set Neg;

(2) build positive example relational matrix W ^pif image R and image I belong to same image class and all belong to k width image, and image R be characterized as x _r, the weights between image I and image R are 1, the weights between the image beyond image I and k width image are 0; That is,

for the weights between image I and image R, x _i, x _r∈ Pos is representation feature xi, x _rbelong to positive example set Pos, positive example relational matrix W ^pthe value of the capable r of i row be

formula is:

(3) build counter-example relational matrix W ⁿif image H and image I belong to the different images class and all belong to k width image, image H is characterized as x _h, the weights between image I and image H are 1, the weights between the image beyond image I and k width image are 0; Be x _i∈ Posand x _h∈ negor x _h∈ Posand x _i∈ neg,

x _i∈ pos representation feature x _ibelonging to positive example set Pos is, x _h∈ Neg representation feature x _hbelong to counter-example set Neg, x _h∈ Pos representation feature x _hbelong to positive example set Pos, x _i∈ Neg representation feature x _ibelonging to counter-example set Neg is,

for the weights between image I and image H, counter-example relational matrix W ⁿi h classify as

formula is:

finally build and obtain three relational matrix W, W ^pand W ⁿ, the relational matrix that its neutralization need to be used for calculating generalized characteristic matrix.

5. a kind of image search method strengthen meaned based on the relational matrix regularization according to claim 4, it is characterized in that, step 5 specifically comprises the steps: from relational matrix W, if image z is neighbour's image of image i, and image z is also neighbour's image of image j, adopt following formula to calculate the weights W strengthened between image i and image j ^' _ij: W ^' _ij∑ _zw _izw _jz, W wherein _izfor the weights of image i and image z, W _jzfor the weights of image j and image z, W ' _ijbe the capable j train value of i that strengthens relational matrix W '.

6. a kind of image search method meaned that strengthens based on the relational matrix regularization according to claim 5, is characterized in that, step 6 specifically comprises the steps:

The neighbor relationships of repeatedly propagating between image obtains new enhancing relational matrix W ", formula is W "=W ' * W ';

Utilize the transfer relationship between the transition probability matrix presentation video, corresponding transition matrix is P=[P _ij] _{n * n}, P _ij=p (j|i) is that in sample data X, arbitrary image i, to the transition probability of arbitrary image j, selects the n width image the most similar to image i according to Euclidean distance, and image j is characterized as x _j, the computing formula of transition probability P (j|i) is:

D wherein _ij=|| x _i-x _j|| ², the Euclidean distance of presentation video i and image j feature;

The model W that adopts the regularization of following formula calculated relationship matrix to strengthen ^r:

W ^R＝ηP+(1-η)ge ^T

Wherein, η is that image i transfers to the probability that this event of image j occurs, and (1-η) is the probability of the random redirect of image i, g=(1/n) e, wherein g is an even stochastic distribution vector, and e is n dimension unit column vector, and n is the picture number of each image class, e=(1,1 ...) ^t, the capable j of i of matrix P classifies P (j|i) as;

The new weights that concern between image i and image j

computing formula is:

$w_{\mathrm{ij}}^{*}=w_{\mathrm{ij}}^{\″}\·w_{\mathrm{ij}}^R$

W " _ijfor the weights of image i and image j, w " _ijfor W " the value of the capable j of i row,

for image i jumps to the probability weights of image j, for W ^rthe value of the capable j of i row;

Finally obtain regularization and strengthen relational matrix W ^*, W ^*the capable j of i classify as

7. a kind of image search method meaned that strengthens based on the relational matrix regularization according to claim 6, is characterized in that, in step 7, comprises the steps:

At first choose the feature x of any two width images from sample data X _iand x _j, the weights that concern of two width images are W _ij, the positive example of two width images concerns that weights are

the counter-example of two width images concerns that weights are

calculate generalized characteristic matrix A according to following target equation:

X(L ^N-γL ^P)X ^TA＝λXLX ^TA

The Laplacian Matrix that L is relational matrix W, L ⁿfor counter-example relational matrix W ⁿlaplacian Matrix, L ^pfor positive example relational matrix W ^plaplacian Matrix, γ is the constant that is directly proportional to the ratio of counter-example image number and positive example image number, X ^tmean the transposed matrix of sample data X, λ means the eigenwert of equation solution.

Images