US20080065627A1 - Method and system for identifying image relatedness using link and page layout analysis - Google Patents
Method and system for identifying image relatedness using link and page layout analysis Download PDFInfo
- Publication number
- US20080065627A1 US20080065627A1 US11/936,029 US93602907A US2008065627A1 US 20080065627 A1 US20080065627 A1 US 20080065627A1 US 93602907 A US93602907 A US 93602907A US 2008065627 A1 US2008065627 A1 US 2008065627A1
- Authority
- US
- United States
- Prior art keywords
- block
- page
- image
- indicators
- link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
Definitions
- the described technology relates generally to analyzing web pages and particularly to relatedness of images of web pages.
- search engine services such as Google and Overture, provide for searching for information that is accessible via the Internet. These search engine services allow users to search for display pages, such as web pages, that may be of interest to users. After a user submits a search request that includes search terms, the search engine service identifies web pages that may be related to those search terms. To quickly identify related web pages, the search engine services may maintain a mapping of keywords to web pages. This mapping may be generated by “crawling and indexing” the web (i.e., the World Wide Web) to identify the keywords of each web page. To crawl the web, a search engine service may use a list of root web pages to identify all web pages that are accessible through those root web pages.
- the keywords of any particular web page can be identified using various well-known information retrieval techniques, such as identifying the words of a headline, the words supplied in the metadata of the web page, the words that are highlighted, and so on.
- the search engine service then ranks the web pages of the search result based on the closeness of each match, web page popularity (e.g., Google's PageRank), and so on.
- the search engine service may also generate a relevance score to indicate how relevant the information of the web page may be to the search request.
- the search engine service displays to the user links to those web pages in an order that is based on their rankings.
- web pages are graphically oriented in that they may contain many images
- conventional search engine services typically search based on only the textual content of a web page.
- Some attempts have been made, however, to support image-based searching of web pages. For example, a user viewing a web page may want to identify other web pages that contain images related to an image on that web page.
- the image-based search techniques are typically either content-based or link-based and additionally use surrounding text to aid in analyzing images.
- the content-based techniques use low-level visual information for image indexing. Because the content-based search techniques are very computationally expensive, they are not practical for image searching on the web.
- the link-based search techniques typically assume that images on the same web page are likely to be related and that images on web pages that are each linked to by the same web page are related.
- a web page for a news web site may contain content relating to an international political event and content relating to a national sporting event. In such a case, it is unlikely that a picture of a sports team relating to the national sporting event is related to a web page linked to by the content relating to the international political event.
- a system for determining relatedness of images of pages based on link and page layout analysis determines relatedness between images by first identifying blocks within pages, and then analyzing the importance of the blocks to pages, pages to blocks, and images to blocks. Based on this analysis, the link analysis system determines the degree to which each image is related to each other image. Because the relatedness of an image to another image is based on block-level importance, which is a smaller unit than a page, rather than page-level importance, this relatedness is a more accurate representation of relatedness than conventional link-based search techniques.
- FIG. 1 is a block diagram illustrating blocks, images, and links in a sample collection of web pages.
- FIG. 2 is a block diagram illustrating components of the link analysis system in one embodiment.
- FIG. 3 is a flow diagram that illustrates processing of a generate image-to-image matrix component in one embodiment.
- FIG. 4 is a flow diagram that illustrates the processing of a generate block-to-page matrix component in one embodiment.
- FIG. 5 is a flow diagram that illustrates the processing of a generate page-to-block matrix component in one embodiment.
- FIG. 6 is a flow diagram that illustrates the processing of a generate block-to-image matrix component in one embodiment.
- a link analysis system determines relatedness between images by first identifying blocks within web pages, and then analyzing the importance of the blocks to web pages, web pages to blocks, and images to blocks. Based on this analysis, the link analysis system determines the degree to which each image is related to each other image.
- a block of a web page represents an area of the web page that appears to relate to a similar topic. For example, a news article relating to an international political event may represent one block, and a news article relating to a national sporting event may represent another block.
- the importance of a block to a page may indicate a probability that a user will focus on that block when viewing that page.
- the importance of a page to a block may indicate the probability that a user will select from that block a link to that page.
- the importance of an image to a block may indicate the probability that a user will focus on that image when viewing that block.
- the link analysis system After calculating a numeric indicator of these importances for pairs of pages and blocks and pairs of images and blocks, the link analysis system generates an indicator of the relatedness of each image to each other image by combining the calculated importance of a block to a page, the calculated importance of a page to a block, and the calculated importance of an image to a block. Because the relatedness of an image to another image is based on block-level importance rather than on page-level importance, this relatedness is a more accurate representation of relatedness than conventional link-based search techniques.
- the link analysis system may also use the relatedness of images to generate a ranking of the images.
- the ranking may be based on a probability that a user who starts viewing an arbitrary image will transition to another image after an arbitrarily large number of transitions between images.
- the link analysis system may also generate a vector representation of the images based on their relatedness and apply a clustering algorithm to the vector representations to identify clusters of related images.
- FIG. 1 is a block diagram illustrating blocks, images, and links in a sample collection of web pages.
- This collection of web pages includes web pages 1 - 4 .
- the blocks within the web pages are represented as rectangles, the images within blocks are represented as circles, and the links within blocks are represented as directed arrows from a block to a linked-to web page.
- Web page 1 contains block 1 , which contains images 1 and 2 and links 1 and 2 .
- Web page 2 contains block 2 , which contains image 3 and link 3 , and block 3 , which contains image 4 and link 4 .
- Web page 3 contains block 4 , which contains image 5 and links 5 and 6 , and block 5 , which contains image 6 and link 7 .
- Web page 4 contains block 5 , which contains images 7 , 8 , 9 , and 10 and link 8 . Because the link analysis system bases image relatedness on blocks rather than entire web pages, the relatedness of an image to other images is likely based on a more accurate representation of the topic of an image. For example, web page 2 contains blocks 2 and 3 , which may be directed to different topics such as an international political event and a national sporting event, respectively. The link analysis system may identify that image 4 is more closely related to the images of web page 4 than to the images of web page 3 , because block 3 , which contains image 4 , has a link 4 to web page 4 .
- web page 4 is more likely sport-related than is web page 3 because block 3 contains a link to web page 4 , but not to web page 3 .
- image 4 is more likely related to images 7 , 8 , 9 , and 10 than to images 5 and 6 of web page 3 .
- Techniques that are not based on block-level analysis may identify that image 4 is equally related to web page 3 and web page 4 because those techniques do not distinguish block 2 from block 3 on web page 2 .
- the link analysis system calculates the importance of a page to a block, for each block and page combination, as the probability that a user who selects a link of that block will select a link to that page. If a block does not have a link to a page, then the probability is zero. If a block has a link to a page, then the link analysis system may assume a user will select each of the links of the block with equal probability.
- Z ij represents the probability that a user who selects a link of block i will select the link to page j and s i is the number of links in block i.
- the block-to-page matrix Z for the web pages of FIG. 1 is shown in Table 1.
- the rows of Table 1 represent the blocks and the columns represent the pages.
- the probability that a user who selects of link of block 4 will select a link to web page 2 is 0.5. TABLE 1 1 2 3 4 1 .5 .5 2 1 3 1 4 .5 .5 5 1 6 1
- the link analysis system calculates, for each page and block combination, the importance of a block to a page as the probability of that block being the most important block of the page.
- the probability of a block not contained on a page being the most important block of that page is zero.
- the link analysis system may assume that each block contained on a page is most important with equal probability.
- the link analysis system calculates a probability that a block is the most important block of a page based on position, size, font, color, and other physical attributes of the block. For example, a large block that is centered in the middle of a page may be more important than a small block in the lower left corner of the page.
- a technique for calculating block importance and the degree of coherency of blocks is described in U.S. patent application No. ______, entitled “Method and System for Calculating Importance of a Block Within a Display Page” and filed on Apr. 29, 2004, which is hereby incorporated by reference.
- f pi is a function representing the probability that block j is the most important block of page i.
- the function f pi is defined as the size of block j divided by the distance of the center of the block from the center of the screen when page i is displayed.
- ⁇ is a normalization factor that ensures that the sum of the values of the function for a block is 1.
- the function f can be considered to be the probability that a user is focused on block j when viewing page i.
- the page-to-block matrix X for the web pages of FIG. 1 is shown in Table 2.
- the rows of Table 2 represent the pages and the columns represent the blocks.
- the probability that block 4 is the most important block of web page 3 is 0.8. TABLE 2 1 2 3 4 5 6 1 1 2 .5 .5 3 .8 .2 4 1
- the link analysis system calculates, for each block and image combination, the importance of an image to a block as the probability of that image being the most important image of that block. If a block does not contain a certain image, then the probability of that image being the most important of that block is zero.
- the link analysis system may assume that each image of a block is most important with equal probability.
- the link analysis system could use other measures of importance of an image to a block, such as based on the relative sizes of the images, the location of the images within the blocks, and so on.
- Y ij represents the probability that image j is the most important image of block i and s i is the number of images in block i.
- the block-to-image matrix Y for the web pages of FIG. 1 is shown in Table 3.
- the rows of Table 3 represent blocks and the columns represent the images.
- the probability that image 2 is the most important image of block 1 is 0.5.
- TABLE 3 1 2 3 4 5 6 7 8 9 10 1 .5 .5 2 1 3 1 4 1 5 1 6 .25 .25 .25 .25
- the link analysis system calculates the importance of one page to another page, for each ordered pair of pages, as the probability that a user viewing the first page of the pair will select a link to the second page of the pair.
- the link analysis system calculates the probability for each pair by summing for each block of the first page the probability of that block being the most important block of the first page times the probability that the second page is the most important page to that block.
- the importance of a page to another page thus factors in that users may prefer to select links within the most important blocks of a page.
- the probability of W can alternately be represented as: Prob( ⁇
- ⁇ ) ⁇ b ⁇ Prob( ⁇
- the link analysis system calculates, for each ordered pair of blocks, the importance of one block to another block as the probability that a user viewing the first block of the pair will select a link to the page containing the second block of the pair and will find that second block to be the most important of its page.
- the link analysis system calculates the probability for each pair by summing the probabilities that a user who selects a link of the first block will select a link for the page that contains the second block times the probability of that second block being the most important block of its page.
- the importance of one block to another block represents that a user viewing the first block will select a link to the page containing the second block and focus their attention on the second block.
- W B ZX (8) where W B represents the block-to-block matrix.
- a ) ⁇ ⁇ ⁇ P ⁇ ⁇ Prob ⁇ ( ⁇
- ⁇ ) Prob ⁇ ( ⁇
- ⁇ ) Z ⁇ ( a , ⁇ ) ⁇ X ⁇ ( ⁇ , b ) , ⁇ a , b ⁇ B ( 9 )
- the block-to-block matrix W B for the web pages of FIG. 1 is shown in Table 5.
- the probability that a user viewing block 4 will jump to page 2 and focus their attention on block 3 is 0.25.
- TABLE 5 1 2 3 4 5 6 1 0 .25 .25 .4 .1 0 2 0 0 .8 .2 0 0 3 0 0 0 0 0 0 1 4 0 .25 .25 0 0 .5 5 1 0 0 0 0 0 0 0 0 0 6 0 0 .8 .2 0 0 0
- the link analysis system factors into the block-to-block matrix the probability that two blocks on the same page may be related.
- the weighting factor t may typically be set to a small value (e.g., less than 0.1) because in most instances different blocks on the same page relate to different topics.
- the link analysis system calculates for each ordered pair of images the probability that the first image of the pair is related to the second image of the pair.
- the link analysis system calculates the probability by summing the block-to-block probabilities for the combination of each block that contains the first image to each block that contains the second image.
- W I represents the image-to-image matrix.
- the image-to-image matrix W I for the web pages of FIG. 1 is shown in Table 6.
- the probability that a user viewing block 10 will next view page 3 and focus on block 5 is 0.05.
- TABLE 6 1 2 3 4 5 6 7 8 9 10 1 0 0 .125 .125 .2 .05 0 0 0 0 2 0 0 .125 .125 .2 .05 0 0 0 0 3 0 0 0 .8 .2 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .25 .25 .25 5 0 0 .25 .25 0 0 .125 .125 .125 6 .5 .5 0 0 0 0 0 0 0 0 0 0 7 0
- the link analysis system factors into the image-to-image matrix the probability that two blocks on the same page may be related.
- the weighting factor t may be set to a large value (e.g., 0.7-0.9) because two images in the same block are likely to be related.
- the link analysis system generates a vector representation of each image from the image-to-image matrix.
- the link analysis system generates the vectors using a least-squares approach that factors in the similarity between a pair of images as indicated by the image-to-image matrix.
- the link analysis system selects eigenvectors I through K to represent the images in a k-dimensional Euclidean space.
- the vector for an image is represented as follows: image j ⁇ (y 1 (j), . . . , y k (j)) (19) where y i (j) denotes the j th element of y i .
- the link analysis system identifies clusters of related images by representing each image by a vector such that the distance between the image vectors represents their semantic similarity.
- Various clustering algorithms may be applied to the image vectors to identify clusters of semantically related images. These clustering algorithms may include a Fiedler vector from spectral graph theory, a k-means clustering, and so on.
- the clustering of images can be used to assist in browsing. For example, when browsing to a web page, a user can select an image and request to see related images. The web pages that contain the images that are clustered together with the selected image can then be presented as the result of the request. In one embodiment, the web pages can be presented in an order that is based on the distance between the image vector of each image and the image vector of the selected image.
- the clustering of images can also be used to provide a multidimensional visualization of images that are semantically related.
- the image vectors can be generated for the images of a collection of web pages. Once the clusters are identified, the system can display an indication of each cluster on a two-dimensional grid representing clusters based on different eigenvectors.
- the link analysis system can rank images based on the image-to-image matrix.
- FIG. 2 is a block diagram illustrating components of the link analysis system in one embodiment.
- the link analysis system 200 includes a web page store 201 , a calculate image rank component 202 , an identify image clusters component 203 , and a generate image-to-image matrix component 211 .
- the generate image-to-image matrix component 211 uses an identify blocks component 212 , a generate block-to-page matrix component 213 , a generate page-to-block matrix component 214 , and a generate block-to-image matrix component 215 to generate a matrix that indicates the image-to-image relatedness.
- the web page store contains the collection of web pages.
- the calculate image rank component uses the generate image-to-image component to calculate the relatedness of the images and then uses those calculations of relatedness to rank the images.
- the identify image clusters component uses the generate image-to-image matrix component to calculate the relatedness of the images, generates a vector representation of the images based on the matrix, and identifies clusters of images using the generated vectors.
- the link analysis system may also include a component to calculate ranking elements of a web page other than the images. For example, the link analysis system may apply the rankings of Equations 20 and 21 to the block-to-block matrix to rank the blocks and to the page-to-page matrix to rank the pages themselves.
- the computing device on which the link analysis system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives).
- the memory and storage devices are computer-readable media that may contain instructions that implement the link analysis system.
- the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communications link.
- Various communications links may be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection.
- FIG. 2 illustrates an example of a suitable operating environment in which the link analysis system may be implemented.
- the operating environment is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality of the link analysis system.
- Other well-known computing systems, environments, and configurations that may be suitable for use include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
- the link analysis system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices.
- program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types.
- functionality of the program modules may be combined or distributed as desired in various embodiments.
- FIG. 3 is a flow diagram that illustrates processing of a generate image-to-image matrix component in one embodiment.
- the component identifies the blocks within the web pages stored in the web page store.
- the component invokes the generate block-to-page matrix component.
- the component invokes the generate page-to-block matrix component.
- the component invokes the generate block-to-image matrix component.
- the component generates the block-to-block matrix.
- the component generates the image-to-image matrix and then completes.
- FIG. 4 is a flow diagram that illustrates the processing of a generate block-to-page matrix component in one embodiment.
- the component loops selecting each page, each block within each page, and each link within each block and sets the importance of the pages linked to by that link, to that block.
- the component selects the next page.
- decision block 402 if all the pages have already been selected, then the component returns the block-to-page matrix, else the component continues at block 403 .
- block 403 the component selects the next block of the selected page.
- decision block 404 if all the blocks of the selected page have already been selected, then the component loops to block 401 to select the next page, else the component continues at block 405 .
- the component counts the number of links within the selected block.
- the component selects the linked-to page of the next link of the selected block.
- decision block 407 if all the linked-to pages of the selected block have already been selected, then the component loops to block 403 to select the next block, else the component continues at block 408 .
- block 408 the component sets the importance of the linked-to page to the block and then loops to block 406 to select the linked-to page of the next link of the selected block.
- FIG. 5 is a flow diagram that illustrates the processing of a generate page-to-block matrix component in one embodiment.
- the component loops selecting each page and each block within each page and setting the importance of that block to the selected page.
- the component selects the next page of the web page store.
- decision block 502 if all the pages have already been selected, then the component returns the page-to-block matrix, else the component continues at block 503 .
- block 503 the component selects the next block of the selected page.
- decision block 504 if all the blocks of the selected page have already been selected, then the component loops to block 501 to select the next page, else the component continues at block 505 .
- the component calculates the importance of the selected block to the selected page.
- the component sets the importance of the selected block to the selected page and then loops to block 503 to select the next block of the selected page.
- FIG. 6 is a flow diagram that illustrates the processing of a generate block-to-image matrix component in one embodiment.
- the component loops selecting each page, each block within each page, and each image within each block and setting the importance of the image to the selected block.
- the component selects the next page of the web page store.
- decision block 602 if all the pages have already been selected, then the component returns the block-to-image matrix, else the component continues at block 603 .
- the component selects the next block of the selected page.
- decision block 604 if all the blocks of the selected page have already been selected, then the component loops to block 601 to select the next page, else the component continues at block 605 .
- the component counts the number of images of the selected block.
- the component selects the next image of the selected block.
- decision block 607 if all the images of the selected block have already been selected, then the component loops to block 603 to select the next block, else the component continues at block 608 .
- block 608 the component sets the importance of the selected image to the selected block and then loops to block 606 to select the next image of the selected block.
Abstract
A method and system for determining relatedness of images of pages based on link and page layout analysis. A link analysis system determines relatedness between images by first identifying blocks within web pages, and then analyzing the importance of the blocks to web pages, web pages to blocks, and images to blocks. Based on this analysis, the link analysis system determines the degree to which each image is related to each other image. The link analysis system may also use the relatedness of images to generate a ranking of the images. The link analysis system may also generate a vector representation of the images based on their relatedness and apply a clustering algorithm to the vector representations to identify clusters of related images.
Description
- This application is a divisional of U.S. patent application Ser. No. 10/834,483, filed Apr. 29, 2004, now U.S. Pat. No. 7,293,007, issued Nov. 6, 2007, entitled “METHOD AND SYSTEM FOR IDENTIFYING IMAGE RELATEDNESS USING LINK AND PAGE LAYOUT ANALYSIS,” which is hereby incorporated by reference in its entirety.
- The described technology relates generally to analyzing web pages and particularly to relatedness of images of web pages.
- Many search engine services, such as Google and Overture, provide for searching for information that is accessible via the Internet. These search engine services allow users to search for display pages, such as web pages, that may be of interest to users. After a user submits a search request that includes search terms, the search engine service identifies web pages that may be related to those search terms. To quickly identify related web pages, the search engine services may maintain a mapping of keywords to web pages. This mapping may be generated by “crawling and indexing” the web (i.e., the World Wide Web) to identify the keywords of each web page. To crawl the web, a search engine service may use a list of root web pages to identify all web pages that are accessible through those root web pages. The keywords of any particular web page can be identified using various well-known information retrieval techniques, such as identifying the words of a headline, the words supplied in the metadata of the web page, the words that are highlighted, and so on. The search engine service then ranks the web pages of the search result based on the closeness of each match, web page popularity (e.g., Google's PageRank), and so on. The search engine service may also generate a relevance score to indicate how relevant the information of the web page may be to the search request. The search engine service then displays to the user links to those web pages in an order that is based on their rankings.
- Although many web pages are graphically oriented in that they may contain many images, conventional search engine services typically search based on only the textual content of a web page. Some attempts have been made, however, to support image-based searching of web pages. For example, a user viewing a web page may want to identify other web pages that contain images related to an image on that web page. The image-based search techniques are typically either content-based or link-based and additionally use surrounding text to aid in analyzing images. The content-based techniques use low-level visual information for image indexing. Because the content-based search techniques are very computationally expensive, they are not practical for image searching on the web. The link-based search techniques typically assume that images on the same web page are likely to be related and that images on web pages that are each linked to by the same web page are related. Unfortunately, these assumptions are incorrect in many situations primarily because a single web page may have content relating to many different topics. For example, a web page for a news web site may contain content relating to an international political event and content relating to a national sporting event. In such a case, it is unlikely that a picture of a sports team relating to the national sporting event is related to a web page linked to by the content relating to the international political event.
- It would be desirable to have an image-based search technique that would not be computationally as expensive as conventional content-based search techniques and that, unlike conventional link-based search techniques, would account for the diverse topics that can occur on a single web page.
- A system for determining relatedness of images of pages based on link and page layout analysis is provided. A link analysis system determines relatedness between images by first identifying blocks within pages, and then analyzing the importance of the blocks to pages, pages to blocks, and images to blocks. Based on this analysis, the link analysis system determines the degree to which each image is related to each other image. Because the relatedness of an image to another image is based on block-level importance, which is a smaller unit than a page, rather than page-level importance, this relatedness is a more accurate representation of relatedness than conventional link-based search techniques.
-
FIG. 1 is a block diagram illustrating blocks, images, and links in a sample collection of web pages. -
FIG. 2 is a block diagram illustrating components of the link analysis system in one embodiment. -
FIG. 3 is a flow diagram that illustrates processing of a generate image-to-image matrix component in one embodiment. -
FIG. 4 is a flow diagram that illustrates the processing of a generate block-to-page matrix component in one embodiment. -
FIG. 5 is a flow diagram that illustrates the processing of a generate page-to-block matrix component in one embodiment. -
FIG. 6 is a flow diagram that illustrates the processing of a generate block-to-image matrix component in one embodiment. - A method and system for determining relatedness of images of pages based on link and page layout analysis is provided. In one embodiment, a link analysis system determines relatedness between images by first identifying blocks within web pages, and then analyzing the importance of the blocks to web pages, web pages to blocks, and images to blocks. Based on this analysis, the link analysis system determines the degree to which each image is related to each other image. A block of a web page represents an area of the web page that appears to relate to a similar topic. For example, a news article relating to an international political event may represent one block, and a news article relating to a national sporting event may represent another block. The importance of a block to a page may indicate a probability that a user will focus on that block when viewing that page. The importance of a page to a block may indicate the probability that a user will select from that block a link to that page. The importance of an image to a block may indicate the probability that a user will focus on that image when viewing that block. After calculating a numeric indicator of these importances for pairs of pages and blocks and pairs of images and blocks, the link analysis system generates an indicator of the relatedness of each image to each other image by combining the calculated importance of a block to a page, the calculated importance of a page to a block, and the calculated importance of an image to a block. Because the relatedness of an image to another image is based on block-level importance rather than on page-level importance, this relatedness is a more accurate representation of relatedness than conventional link-based search techniques.
- The link analysis system may also use the relatedness of images to generate a ranking of the images. The ranking may be based on a probability that a user who starts viewing an arbitrary image will transition to another image after an arbitrarily large number of transitions between images. The link analysis system may also generate a vector representation of the images based on their relatedness and apply a clustering algorithm to the vector representations to identify clusters of related images.
-
FIG. 1 is a block diagram illustrating blocks, images, and links in a sample collection of web pages. This collection of web pages includes web pages 1-4. The blocks within the web pages are represented as rectangles, the images within blocks are represented as circles, and the links within blocks are represented as directed arrows from a block to a linked-to web page.Web page 1 containsblock 1, which containsimages links Web page 2 containsblock 2, which containsimage 3 andlink 3, andblock 3, which containsimage 4 andlink 4.Web page 3 containsblock 4, which containsimage 5 andlinks block 5, which containsimage 6 andlink 7.Web page 4 containsblock 5, which containsimages link 8. Because the link analysis system bases image relatedness on blocks rather than entire web pages, the relatedness of an image to other images is likely based on a more accurate representation of the topic of an image. For example,web page 2 containsblocks image 4 is more closely related to the images ofweb page 4 than to the images ofweb page 3, becauseblock 3, which containsimage 4, has alink 4 toweb page 4. For example,web page 4 is more likely sport-related than isweb page 3 becauseblock 3 contains a link toweb page 4, but not toweb page 3. As such,image 4 is more likely related toimages images web page 3. Techniques that are not based on block-level analysis may identify thatimage 4 is equally related toweb page 3 andweb page 4 because those techniques do not distinguishblock 2 fromblock 3 onweb page 2. - In one embodiment, the link analysis system calculates the importance of a page to a block, for each block and page combination, as the probability that a user who selects a link of that block will select a link to that page. If a block does not have a link to a page, then the probability is zero. If a block has a link to a page, then the link analysis system may assume a user will select each of the links of the block with equal probability. A block-to-page matrix of probabilities is defined by the following equation:
- where Zij represents the probability that a user who selects a link of block i will select the link to page j and si is the number of links in block i. The block-to-page matrix Z for the web pages of
FIG. 1 is shown in Table 1. The rows of Table 1 represent the blocks and the columns represent the pages. In this example, the probability that a user who selects of link ofblock 4 will select a link toweb page 2 is 0.5.TABLE 1 1 2 3 4 1 .5 .5 2 1 3 1 4 .5 .5 5 1 6 1 - In one embodiment, the link analysis system calculates, for each page and block combination, the importance of a block to a page as the probability of that block being the most important block of the page. The probability of a block not contained on a page being the most important block of that page is zero. The link analysis system may assume that each block contained on a page is most important with equal probability. A page-to-block matrix of probabilities is defined by the following equation:
where Xij represents the probability that block j is the most important block of page i and si is the number of blocks on page i. - In one embodiment, the link analysis system calculates a probability that a block is the most important block of a page based on position, size, font, color, and other physical attributes of the block. For example, a large block that is centered in the middle of a page may be more important than a small block in the lower left corner of the page. A technique for calculating block importance and the degree of coherency of blocks is described in U.S. patent application No. ______, entitled “Method and System for Calculating Importance of a Block Within a Display Page” and filed on Apr. 29, 2004, which is hereby incorporated by reference. The page-to-block matrix X may be more generally represented as:
where fpi is a function representing the probability that block j is the most important block of page i. In one embodiment, the function fpi is defined as the size of block j divided by the distance of the center of the block from the center of the screen when page i is displayed. The function f may be defined by the following: - where α is a normalization factor that ensures that the sum of the values of the function for a block is 1. The function f can be considered to be the probability that a user is focused on block j when viewing page i. The page-to-block matrix X for the web pages of
FIG. 1 is shown in Table 2. The rows of Table 2 represent the pages and the columns represent the blocks. In this example, the probability that block 4 is the most important block ofweb page 3 is 0.8.TABLE 2 1 2 3 4 5 6 1 1 2 .5 .5 3 .8 .2 4 1 - In one embodiment, the link analysis system calculates, for each block and image combination, the importance of an image to a block as the probability of that image being the most important image of that block. If a block does not contain a certain image, then the probability of that image being the most important of that block is zero. The link analysis system may assume that each image of a block is most important with equal probability. The link analysis system could use other measures of importance of an image to a block, such as based on the relative sizes of the images, the location of the images within the blocks, and so on. A block-to-image matrix of the probabilities is defined by the following equation:
- where Yij represents the probability that image j is the most important image of block i and si is the number of images in block i. The block-to-image matrix Y for the web pages of
FIG. 1 is shown in Table 3. The rows of Table 3 represent blocks and the columns represent the images. In this example, the probability thatimage 2 is the most important image ofblock 1 is 0.5.TABLE 3 1 2 3 4 5 6 7 8 9 10 1 .5 .5 2 1 3 1 4 1 5 1 6 .25 .25 .25 .25 - In one embodiment, the link analysis system calculates the importance of one page to another page, for each ordered pair of pages, as the probability that a user viewing the first page of the pair will select a link to the second page of the pair. The link analysis system calculates the probability for each pair by summing for each block of the first page the probability of that block being the most important block of the first page times the probability that the second page is the most important page to that block. The importance of a page to another page thus factors in that users may prefer to select links within the most important blocks of a page. A page-to-page matrix of these probabilities is represented by the following:
WP=XZ (6)
where WP represents the page-to-page matrix. The probability of W can alternately be represented as:
Prob(β|α)=ΣbεαProb(β|b)Prob(b|α) (7) - where α represents the first page of the pair and β represents the second page of the pair. The page-to-page matrix WP for the web pages of
FIG. 1 is shown in Table 4. In this example, the probability that auser viewing page 3 will transition topage 2 is 0.4.TABLE 4 1 2 3 4 1 0 .5 .5 0 2 0 0 .5 .5 3 .2 .4 0 .4 4 0 0 1 0 - The link analysis system calculates, for each ordered pair of blocks, the importance of one block to another block as the probability that a user viewing the first block of the pair will select a link to the page containing the second block of the pair and will find that second block to be the most important of its page. The link analysis system calculates the probability for each pair by summing the probabilities that a user who selects a link of the first block will select a link for the page that contains the second block times the probability of that second block being the most important block of its page. Thus, the importance of one block to another block represents that a user viewing the first block will select a link to the page containing the second block and focus their attention on the second block. A block-to-block matrix of these probabilities is represented by the following:
WB=ZX (8)
where WB represents the block-to-block matrix. The probabilities of W can alternately be represented as: - The block-to-block matrix WB for the web pages of
FIG. 1 is shown in Table 5. In thus example, the probability that auser viewing block 4 will jump topage 2 and focus their attention onblock 3 is 0.25.TABLE 5 1 2 3 4 5 6 1 0 .25 .25 .4 .1 0 2 0 0 .8 .2 0 0 3 0 0 0 0 0 1 4 0 .25 .25 0 0 .5 5 1 0 0 0 0 0 6 0 0 .8 .2 0 0 - In one embodiment, the link analysis system factors into the block-to-block matrix the probability that two blocks on the same page may be related. The revised block-to-block matrix is represented by the following:
W B=(1−t)ZX+tDU (10)
where D is a diagonal matrix Dii=ΣjUij, U is a coherence matrix, and t is a weighting factor. The matrix U is defined as follows:
where DOC is the degree of coherency of the smallest block containing both block i and block j. The weighting factor t may typically be set to a small value (e.g., less than 0.1) because in most instances different blocks on the same page relate to different topics. - The link analysis system calculates for each ordered pair of images the probability that the first image of the pair is related to the second image of the pair. The link analysis system calculates the probability by summing the block-to-block probabilities for the combination of each block that contains the first image to each block that contains the second image. An image-to-image matrix of these probabilities is represented by the following:
Wt=YTWBY (12) - where WI represents the image-to-image matrix. The image-to-image matrix WI for the web pages of
FIG. 1 is shown in Table 6. In this example, the probability that auser viewing block 10 will next viewpage 3 and focus onblock 5 is 0.05.TABLE 6 1 2 3 4 5 6 7 8 9 10 1 0 0 .125 .125 .2 .05 0 0 0 0 2 0 0 .125 .125 .2 .05 0 0 0 0 3 0 0 0 .8 .2 0 0 0 0 0 4 0 0 0 0 0 0 .25 .25 .25 .25 5 0 0 .25 .25 0 0 .125 .125 .125 .125 6 .5 .5 0 0 0 0 0 0 0 0 7 0 0 0 .2 .05 0 0 0 0 0 8 0 0 0 .2 .05 0 0 0 0 0 9 0 0 0 .2 .05 0 0 0 0 0 10 0 0 0 .2 .05 0 0 0 0 0 - In one embodiment, the link analysis system factors into the image-to-image matrix the probability that two blocks on the same page may be related. The revised image-to-image matrix is represented by the following:
W=tDY T Y+(1−t)Y T W B Y (13)
where t is a weighting factor and D is a diagonal matrix representing
D ii =E j(Y T Y)ij (14)
The weighting factor t may be set to a large value (e.g., 0.7-0.9) because two images in the same block are likely to be related. - In one embodiment, the link analysis system generates a vector representation of each image from the image-to-image matrix. The link analysis system generates the vectors using a least-squares approach that factors in the similarity between a pair of images as indicated by the image-to-image matrix. The link analysis system initially converts the image-to-image matrix to a similarity matrix represented by the following:
S=(W I +W I T)/2 (15)
where S represents the similarity matrix. If yi is a vector representation of image i, then the optimal set of image vectors is y=(y1, . . . , ym) obtained using the following objective function:
If D is a diagonal matrix such that Dii is the sum of the values of the ith row of the similarity matrix S, then the minimization problem reduces to the following:
where L is equal to D−S. The solution is given by the minimum eigenvalue solution to the general eigenvalue problem:
Ly=λy (18)
If (y0, λ0), (y1, λ1), . . . , (ym−1, λm−1) are solutions to Equation 16, and λ0<λ1< . . . <λm−1, then λ0=0 and y0=(1, 1, . . . , 1). The link analysis system selects eigenvectors I through K to represent the images in a k-dimensional Euclidean space. The vector for an image is represented as follows:
image j←(y1(j), . . . , yk(j)) (19)
where yi(j) denotes the jth element of yi. - The link analysis system identifies clusters of related images by representing each image by a vector such that the distance between the image vectors represents their semantic similarity. Various clustering algorithms may be applied to the image vectors to identify clusters of semantically related images. These clustering algorithms may include a Fiedler vector from spectral graph theory, a k-means clustering, and so on.
- The clustering of images can be used to assist in browsing. For example, when browsing to a web page, a user can select an image and request to see related images. The web pages that contain the images that are clustered together with the selected image can then be presented as the result of the request. In one embodiment, the web pages can be presented in an order that is based on the distance between the image vector of each image and the image vector of the selected image.
- The clustering of images can also be used to provide a multidimensional visualization of images that are semantically related. The image vectors can be generated for the images of a collection of web pages. Once the clusters are identified, the system can display an indication of each cluster on a two-dimensional grid representing clusters based on different eigenvectors.
- The link analysis system can rank images based on the image-to-image matrix. The image-to-image matrix represents the probability of transitioning from image to image. It is possible that a user will transition to an image randomly. To account for this, the link analysis system generates a probability transition matrix that factors this randomness into the image-to-image matrix as follows:
P=εW+(1−ε)U (20)
where P is a probability transition matrix, ε is a weighting factor (e.g., 0.1-0.2), and U is a transition matrix of uniform transition probabilities (Uij=1/m for all i, j). Because of the introduction of U, the graph is connected and a stationary distribution of a random walk of the graph exists. The rank of an image can be represented as follows:
PTπ=π (21)
where π is an eigenvector of pT witheigenvalue 1 representing the image rank. π=(π1, π1, . . . , πm) represents a stationary probability distribution and πi represents the rank of image i. -
FIG. 2 is a block diagram illustrating components of the link analysis system in one embodiment. Thelink analysis system 200 includes aweb page store 201, a calculateimage rank component 202, an identifyimage clusters component 203, and a generate image-to-image matrix component 211. The generate image-to-image matrix component 211 uses anidentify blocks component 212, a generate block-to-page matrix component 213, a generate page-to-block matrix component 214, and a generate block-to-image matrix component 215 to generate a matrix that indicates the image-to-image relatedness. The web page store contains the collection of web pages. The calculate image rank component uses the generate image-to-image component to calculate the relatedness of the images and then uses those calculations of relatedness to rank the images. The identify image clusters component uses the generate image-to-image matrix component to calculate the relatedness of the images, generates a vector representation of the images based on the matrix, and identifies clusters of images using the generated vectors. Although not shown inFIG. 2 , the link analysis system may also include a component to calculate ranking elements of a web page other than the images. For example, the link analysis system may apply the rankings of Equations 20 and 21 to the block-to-block matrix to rank the blocks and to the page-to-page matrix to rank the pages themselves. - The computing device on which the link analysis system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives). The memory and storage devices are computer-readable media that may contain instructions that implement the link analysis system. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communications link. Various communications links may be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection.
-
FIG. 2 illustrates an example of a suitable operating environment in which the link analysis system may be implemented. The operating environment is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality of the link analysis system. Other well-known computing systems, environments, and configurations that may be suitable for use include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. - The link analysis system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
-
FIG. 3 is a flow diagram that illustrates processing of a generate image-to-image matrix component in one embodiment. Inblock 301, the component identifies the blocks within the web pages stored in the web page store. Inblock 302, the component invokes the generate block-to-page matrix component. Inblock 303, the component invokes the generate page-to-block matrix component. Inblock 304, the component invokes the generate block-to-image matrix component. Inblock 305, the component generates the block-to-block matrix. Inblock 306, the component generates the image-to-image matrix and then completes. -
FIG. 4 is a flow diagram that illustrates the processing of a generate block-to-page matrix component in one embodiment. In blocks 401-408, the component loops selecting each page, each block within each page, and each link within each block and sets the importance of the pages linked to by that link, to that block. Inblock 401, the component selects the next page. Indecision block 402, if all the pages have already been selected, then the component returns the block-to-page matrix, else the component continues atblock 403. Inblock 403, the component selects the next block of the selected page. Indecision block 404, if all the blocks of the selected page have already been selected, then the component loops to block 401 to select the next page, else the component continues atblock 405. Inblock 405, the component counts the number of links within the selected block. Inblock 406, the component selects the linked-to page of the next link of the selected block. Indecision block 407, if all the linked-to pages of the selected block have already been selected, then the component loops to block 403 to select the next block, else the component continues atblock 408. Inblock 408, the component sets the importance of the linked-to page to the block and then loops to block 406 to select the linked-to page of the next link of the selected block. -
FIG. 5 is a flow diagram that illustrates the processing of a generate page-to-block matrix component in one embodiment. In blocks 501-506, the component loops selecting each page and each block within each page and setting the importance of that block to the selected page. Inblock 501, the component selects the next page of the web page store. Indecision block 502, if all the pages have already been selected, then the component returns the page-to-block matrix, else the component continues atblock 503. Inblock 503, the component selects the next block of the selected page. Indecision block 504, if all the blocks of the selected page have already been selected, then the component loops to block 501 to select the next page, else the component continues atblock 505. Inblock 505, the component calculates the importance of the selected block to the selected page. Inblock 506, the component sets the importance of the selected block to the selected page and then loops to block 503 to select the next block of the selected page. -
FIG. 6 is a flow diagram that illustrates the processing of a generate block-to-image matrix component in one embodiment. In blocks 601-607, the component loops selecting each page, each block within each page, and each image within each block and setting the importance of the image to the selected block. Inblock 601, the component selects the next page of the web page store. Indecision block 602, if all the pages have already been selected, then the component returns the block-to-image matrix, else the component continues atblock 603. Inblock 603, the component selects the next block of the selected page. Indecision block 604, if all the blocks of the selected page have already been selected, then the component loops to block 601 to select the next page, else the component continues atblock 605. Inblock 605, the component counts the number of images of the selected block. Inblock 606, the component selects the next image of the selected block. Indecision block 607, if all the images of the selected block have already been selected, then the component loops to block 603 to select the next block, else the component continues atblock 608. Inblock 608, the component sets the importance of the selected image to the selected block and then loops to block 606 to select the next image of the selected block. - One skilled in the art will appreciate that although specific embodiments of the link analysis system have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention. Accordingly, the invention is not limited except by the appended claims.
Claims (20)
1. A method in a computer system for determining relatedness between blocks of pages, the method comprising:
calculating value indicators of a page to a block;
calculating value indicators of a block to a page; and
calculating block-to-block indicators of relatedness of one block to another block by combining the value indicators of a block to a page and the value indicators of importance of a page to a block.
2. The method of claim 1 wherein the value indicators of a page to a block are probabilities that a user will select a link from each block that will lead to each other page.
3. The method of claim 1 wherein the value indicators of a block to a page are probabilities that a user will focus on each block of the page.
4. The method of claim 1 wherein the value indicators of a page to a block are probabilities that a user will select a link from each block that will lead to each other page and the value indicators of a block to a page are probabilities that a user will focus on each block of the page.
5. The method of claim 1 including calculating a rank of the blocks from the block-to-block indicators.
6. The method of claim 5 wherein the calculated rank is based on a probability that a user starting at an arbitrary block will transition to another block.
7. The method of claim 1 wherein the block-to-block indicators are calculated as follows:
WBZX
where X is a matrix of the value indicators of a block to a page and Z is a matrix of the value indicators of a page to a block.
8. A computer-readable storage medium containing instructions for controlling a computer system to determine relatedness between page elements, the method comprising:
calculating value indicators of a first element to a second element;
calculating value indicators of a second element to a first element; and
calculating indicators of relatedness of a first element to another first element by combining the value indicators of a first element to a second element and the value indicators of a second element to a first element.
9. The computer-readable storage medium of claim 8 wherein the first element is an image of a block of a page and the second element is a block.
10. The computer-readable storage medium of claim 8 wherein the first element is a block of a page and the second element is a page.
11. The computer-readable storage medium of claim 8 wherein the value indicators of a first element to a second element are probabilities that a user will select from the first element information relating to each other second element.
12. The computer-readable storage medium of claim 8 wherein the value indicators of a second element to a first element are probabilities that a user will focus on each second element within the first element.
13. The computer-readable storage medium of claim 8 wherein the value indicators of a first element to a second element are probabilities that a user will select from the first element information relating to each other second element and wherein the value indicators of a second element to a first element are probabilities that a user will focus on each second element within the first element.
14. The computer-readable storage medium of claim 8 including calculating a rank of the first elements from the indicators of relatedness of the first element to another first element.
15. The computer-readable storage medium of claim 14 wherein the calculated rank is based on a probability that a user starting at a first element will transition to another first element.
16. The computer-readable storage medium of claim 8 wherein the first element is a page and the second element is a block of the page.
17. A computer system for determining relatedness between blocks of pages, comprising:
for each combination of a page and a block,
a probability of the page to the block indicating that a user will select a link from the block that will lead to the page; and
a probability of the block to the page indicating that a user will focus on the block of that page; and
a component that combines the probabilities of a block to a page and the probabilities of a page to a block to calculate block-to-block indicators of relatedness of one block to another block.
18. The computer system of claim 17 including a component that ranks the block based on the block-to-block indicators of relatedness.
19. The computer system of claim 18 wherein the ranking is based on a probability that a user starting at an arbitrary block will transition to another block.
20. The computer system of claim 17 wherein in the block-to-block indicators are calculated as follows:
WBZX
where WB is a matrix of block-to-block indicators, Z is a matrix of the probabilities of pages to blocks, and X is a matrix of the probabilities of blocks to pages.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/936,029 US20080065627A1 (en) | 2004-04-29 | 2007-11-06 | Method and system for identifying image relatedness using link and page layout analysis |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/834,483 US7293007B2 (en) | 2004-04-29 | 2004-04-29 | Method and system for identifying image relatedness using link and page layout analysis |
US11/936,029 US20080065627A1 (en) | 2004-04-29 | 2007-11-06 | Method and system for identifying image relatedness using link and page layout analysis |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/834,483 Division US7293007B2 (en) | 2004-04-29 | 2004-04-29 | Method and system for identifying image relatedness using link and page layout analysis |
Publications (1)
Publication Number | Publication Date |
---|---|
US20080065627A1 true US20080065627A1 (en) | 2008-03-13 |
Family
ID=34939562
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/834,483 Expired - Fee Related US7293007B2 (en) | 2004-04-29 | 2004-04-29 | Method and system for identifying image relatedness using link and page layout analysis |
US11/936,029 Abandoned US20080065627A1 (en) | 2004-04-29 | 2007-11-06 | Method and system for identifying image relatedness using link and page layout analysis |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/834,483 Expired - Fee Related US7293007B2 (en) | 2004-04-29 | 2004-04-29 | Method and system for identifying image relatedness using link and page layout analysis |
Country Status (10)
Country | Link |
---|---|
US (2) | US7293007B2 (en) |
EP (1) | EP1591921B1 (en) |
JP (1) | JP4634214B2 (en) |
KR (1) | KR101130429B1 (en) |
CN (1) | CN1694105B (en) |
AU (1) | AU2005201771A1 (en) |
BR (1) | BRPI0501448A (en) |
CA (1) | CA2505311A1 (en) |
MX (1) | MXPA05004679A (en) |
RU (1) | RU2390833C2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8660378B2 (en) | 2010-02-10 | 2014-02-25 | Panasonic Corporation | Image evaluating device for calculating an importance degree of an object and an image, and an image evaluating method, program, and integrated circuit for performing the same |
CN109670072A (en) * | 2018-11-01 | 2019-04-23 | 广州企图腾科技有限公司 | A kind of trade mark similarity-rough set method extracted based on interval |
WO2023136605A1 (en) * | 2022-01-11 | 2023-07-20 | Samsung Electronics Co., Ltd. | Method and electronic device for intelligently reading displayed contents |
Families Citing this family (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040193698A1 (en) * | 2003-03-24 | 2004-09-30 | Sadasivuni Lakshminarayana | Method for finding convergence of ranking of web page |
US7293007B2 (en) * | 2004-04-29 | 2007-11-06 | Microsoft Corporation | Method and system for identifying image relatedness using link and page layout analysis |
US8081827B2 (en) * | 2006-02-28 | 2011-12-20 | Ricoh Co., Ltd. | Compressed data image object feature extraction, ordering, and delivery |
CN100392656C (en) * | 2006-05-10 | 2008-06-04 | 南京大学 | Graphic and text cooperation method in web search |
US7877384B2 (en) * | 2007-03-01 | 2011-01-25 | Microsoft Corporation | Scoring relevance of a document based on image text |
US7676520B2 (en) * | 2007-04-12 | 2010-03-09 | Microsoft Corporation | Calculating importance of documents factoring historical importance |
US8949214B1 (en) * | 2007-04-24 | 2015-02-03 | Wal-Mart Stores, Inc. | Mashup platform |
US7870130B2 (en) * | 2007-10-05 | 2011-01-11 | International Business Machines Corporation | Techniques for identifying a matching search term in an image of an electronic document |
US8051083B2 (en) * | 2008-04-16 | 2011-11-01 | Microsoft Corporation | Forum web page clustering based on repetitive regions |
US20090288034A1 (en) * | 2008-05-19 | 2009-11-19 | International Business Machines Corporation | Locating and Identifying Controls on a Web Page |
US20100205176A1 (en) * | 2009-02-12 | 2010-08-12 | Microsoft Corporation | Discovering City Landmarks from Online Journals |
US7945668B1 (en) * | 2009-08-21 | 2011-05-17 | Narus, Inc. | System and method for content-aware co-clustering algorithm based on hourglass model |
US9323426B2 (en) | 2009-10-05 | 2016-04-26 | Google Inc. | System and method for selecting information for display based on past user interactions |
US9218322B2 (en) * | 2010-07-28 | 2015-12-22 | Hewlett-Packard Development Company, L.P. | Producing web page content |
US9325804B2 (en) * | 2010-11-08 | 2016-04-26 | Microsoft Technology Licensing, Llc | Dynamic image result stitching |
US9384216B2 (en) * | 2010-11-16 | 2016-07-05 | Microsoft Technology Licensing, Llc | Browsing related image search result sets |
US8503769B2 (en) | 2010-12-28 | 2013-08-06 | Microsoft Corporation | Matching text to images |
JP5990105B2 (en) * | 2011-01-26 | 2016-09-07 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | Image management apparatus, image management method, program, recording medium, integrated circuit |
KR101428538B1 (en) * | 2012-05-11 | 2014-08-12 | (주)다음소프트 | Method of providing contents block, server performing the same and storage media storing the same |
CN103593344B (en) * | 2012-08-13 | 2016-09-21 | 北大方正集团有限公司 | A kind of information collecting method and device |
US10089394B2 (en) * | 2013-06-25 | 2018-10-02 | Google Llc | Personal search result identifying a physical location previously interacted with by a user |
US11250203B2 (en) | 2013-08-12 | 2022-02-15 | Microsoft Technology Licensing, Llc | Browsing images via mined hyperlinked text snippets |
US10055433B2 (en) * | 2014-09-18 | 2018-08-21 | Microsoft Technology Licensing, Llc | Referenced content indexing |
CN117438053A (en) * | 2018-08-27 | 2024-01-23 | 卡西欧计算机株式会社 | Display control device, display control system, display control method, and recording medium |
Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5546517A (en) * | 1994-12-07 | 1996-08-13 | Mitsubishi Electric Information Technology Center America, Inc. | Apparatus for determining the structure of a hypermedia document using graph partitioning |
US6112202A (en) * | 1997-03-07 | 2000-08-29 | International Business Machines Corporation | Method and system for identifying authoritative information resources in an environment with content-based links between information resources |
US6154767A (en) * | 1998-01-15 | 2000-11-28 | Microsoft Corporation | Methods and apparatus for using attribute transition probability models for pre-fetching resources |
US6182133B1 (en) * | 1998-02-06 | 2001-01-30 | Microsoft Corporation | Method and apparatus for display of information prefetching and cache status having variable visual indication based on a period of time since prefetching |
US6195622B1 (en) * | 1998-01-15 | 2001-02-27 | Microsoft Corporation | Methods and apparatus for building attribute transition probability models for use in pre-fetching resources |
US20030005038A1 (en) * | 2001-06-29 | 2003-01-02 | International Business Machines Corporation | Method and system for predictive directional data caching |
US20030004966A1 (en) * | 2001-06-18 | 2003-01-02 | International Business Machines Corporation | Business method and apparatus for employing induced multimedia classifiers based on unified representation of features reflecting disparate modalities |
US20030004995A1 (en) * | 2001-06-29 | 2003-01-02 | International Business Machines Corporation | Graphical web browsing interface for spatial data navigation and method of navigating data blocks |
US20030149704A1 (en) * | 2002-02-05 | 2003-08-07 | Hitachi, Inc. | Similarity-based search method by relevance feedback |
US20030208482A1 (en) * | 2001-01-10 | 2003-11-06 | Kim Brian S. | Systems and methods of retrieving relevant information |
US20030225763A1 (en) * | 2002-04-15 | 2003-12-04 | Microsoft Corporation | Self-improving system and method for classifying pages on the world wide web |
US6665837B1 (en) * | 1998-08-10 | 2003-12-16 | Overture Services, Inc. | Method for identifying related pages in a hyperlinked database |
US20050105775A1 (en) * | 2003-11-13 | 2005-05-19 | Eastman Kodak Company | Method of using temporal context for image classification |
US6901411B2 (en) * | 2002-02-11 | 2005-05-31 | Microsoft Corporation | Statistical bigram correlation model for image retrieval |
US7293007B2 (en) * | 2004-04-29 | 2007-11-06 | Microsoft Corporation | Method and system for identifying image relatedness using link and page layout analysis |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3694149B2 (en) * | 1997-07-07 | 2005-09-14 | 株式会社リコー | Image search apparatus, image search key text generation method, program for causing a computer to function as the apparatus, and computer-readable recording medium on which a program for causing the computer to execute the method is recorded |
US6446095B1 (en) | 1998-06-09 | 2002-09-03 | Matsushita Electric Industrial Co., Ltd. | Document processor for processing a document in accordance with a detected degree of importance corresponding to a data link within the document |
JP2002304402A (en) * | 2001-04-06 | 2002-10-18 | Foundation For The Promotion Of Industrial Science | Information retrieval system |
JP2003036267A (en) * | 2001-07-23 | 2003-02-07 | Canon Inc | Information retrieval device, and information retrieval method |
JP2004038253A (en) * | 2002-06-28 | 2004-02-05 | Fuji Photo Film Co Ltd | Database system |
-
2004
- 2004-04-29 US US10/834,483 patent/US7293007B2/en not_active Expired - Fee Related
-
2005
- 2005-04-26 CA CA002505311A patent/CA2505311A1/en not_active Abandoned
- 2005-04-27 AU AU2005201771A patent/AU2005201771A1/en not_active Abandoned
- 2005-04-27 JP JP2005129809A patent/JP4634214B2/en not_active Expired - Fee Related
- 2005-04-28 EP EP05103485.8A patent/EP1591921B1/en not_active Not-in-force
- 2005-04-28 RU RU2005113002/09A patent/RU2390833C2/en not_active IP Right Cessation
- 2005-04-29 KR KR1020050036128A patent/KR101130429B1/en not_active IP Right Cessation
- 2005-04-29 MX MXPA05004679A patent/MXPA05004679A/en active IP Right Grant
- 2005-04-29 BR BR0501448-4A patent/BRPI0501448A/en not_active IP Right Cessation
- 2005-04-29 CN CN2005100792213A patent/CN1694105B/en not_active Expired - Fee Related
-
2007
- 2007-11-06 US US11/936,029 patent/US20080065627A1/en not_active Abandoned
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5546517A (en) * | 1994-12-07 | 1996-08-13 | Mitsubishi Electric Information Technology Center America, Inc. | Apparatus for determining the structure of a hypermedia document using graph partitioning |
US6112202A (en) * | 1997-03-07 | 2000-08-29 | International Business Machines Corporation | Method and system for identifying authoritative information resources in an environment with content-based links between information resources |
US6154767A (en) * | 1998-01-15 | 2000-11-28 | Microsoft Corporation | Methods and apparatus for using attribute transition probability models for pre-fetching resources |
US6195622B1 (en) * | 1998-01-15 | 2001-02-27 | Microsoft Corporation | Methods and apparatus for building attribute transition probability models for use in pre-fetching resources |
US6182133B1 (en) * | 1998-02-06 | 2001-01-30 | Microsoft Corporation | Method and apparatus for display of information prefetching and cache status having variable visual indication based on a period of time since prefetching |
US6665837B1 (en) * | 1998-08-10 | 2003-12-16 | Overture Services, Inc. | Method for identifying related pages in a hyperlinked database |
US20030208482A1 (en) * | 2001-01-10 | 2003-11-06 | Kim Brian S. | Systems and methods of retrieving relevant information |
US20030004966A1 (en) * | 2001-06-18 | 2003-01-02 | International Business Machines Corporation | Business method and apparatus for employing induced multimedia classifiers based on unified representation of features reflecting disparate modalities |
US20030004995A1 (en) * | 2001-06-29 | 2003-01-02 | International Business Machines Corporation | Graphical web browsing interface for spatial data navigation and method of navigating data blocks |
US20030005038A1 (en) * | 2001-06-29 | 2003-01-02 | International Business Machines Corporation | Method and system for predictive directional data caching |
US20030149704A1 (en) * | 2002-02-05 | 2003-08-07 | Hitachi, Inc. | Similarity-based search method by relevance feedback |
US6901411B2 (en) * | 2002-02-11 | 2005-05-31 | Microsoft Corporation | Statistical bigram correlation model for image retrieval |
US20030225763A1 (en) * | 2002-04-15 | 2003-12-04 | Microsoft Corporation | Self-improving system and method for classifying pages on the world wide web |
US20050105775A1 (en) * | 2003-11-13 | 2005-05-19 | Eastman Kodak Company | Method of using temporal context for image classification |
US7293007B2 (en) * | 2004-04-29 | 2007-11-06 | Microsoft Corporation | Method and system for identifying image relatedness using link and page layout analysis |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8660378B2 (en) | 2010-02-10 | 2014-02-25 | Panasonic Corporation | Image evaluating device for calculating an importance degree of an object and an image, and an image evaluating method, program, and integrated circuit for performing the same |
CN109670072A (en) * | 2018-11-01 | 2019-04-23 | 广州企图腾科技有限公司 | A kind of trade mark similarity-rough set method extracted based on interval |
WO2023136605A1 (en) * | 2022-01-11 | 2023-07-20 | Samsung Electronics Co., Ltd. | Method and electronic device for intelligently reading displayed contents |
Also Published As
Publication number | Publication date |
---|---|
KR101130429B1 (en) | 2012-03-27 |
CA2505311A1 (en) | 2005-10-29 |
CN1694105B (en) | 2011-11-23 |
EP1591921A1 (en) | 2005-11-02 |
EP1591921B1 (en) | 2013-04-10 |
RU2390833C2 (en) | 2010-05-27 |
US7293007B2 (en) | 2007-11-06 |
MXPA05004679A (en) | 2005-11-17 |
JP4634214B2 (en) | 2011-02-16 |
US20050246623A1 (en) | 2005-11-03 |
BRPI0501448A (en) | 2006-01-10 |
JP2005332385A (en) | 2005-12-02 |
AU2005201771A1 (en) | 2005-11-17 |
CN1694102A (en) | 2005-11-09 |
RU2005113002A (en) | 2006-11-10 |
KR20060047643A (en) | 2006-05-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1591921B1 (en) | Method and system for identifying page elements relatedness using link and page layout analysis | |
US7363279B2 (en) | Method and system for calculating importance of a block within a display page | |
US7849089B2 (en) | Method and system for adapting search results to personal information needs | |
US7548936B2 (en) | Systems and methods to present web image search results for effective image browsing | |
US7647331B2 (en) | Detecting duplicate images using hash code grouping | |
US8024332B2 (en) | Clustering question search results based on topic and focus | |
US8001130B2 (en) | Web object retrieval based on a language model | |
US8019763B2 (en) | Propagating relevance from labeled documents to unlabeled documents | |
US7698332B2 (en) | Projecting queries and images into a similarity space | |
US7529735B2 (en) | Method and system for mining information based on relationships | |
US20070143279A1 (en) | Identifying important news reports from news home pages | |
US8612453B2 (en) | Topic distillation via subsite retrieval | |
US20070005588A1 (en) | Determining relevance using queries as surrogate content | |
US8484193B2 (en) | Look-ahead document ranking system | |
US20050256833A1 (en) | Method and system for determining similarity of objects based on heterogeneous relationships | |
US7774340B2 (en) | Method and system for calculating document importance using document classifications | |
Singh et al. | Multiple perspective interactive search: a paradigm for exploratory search and information retrieval on the web | |
JPH09114847A (en) | Information processor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034766/0001 Effective date: 20141014 |