US20160299967A1 - Document retrieval using internal dictionary-hierarchies to adjust per-subject match results - Google Patents
Document retrieval using internal dictionary-hierarchies to adjust per-subject match results Download PDFInfo
- Publication number
- US20160299967A1 US20160299967A1 US15/180,702 US201615180702A US2016299967A1 US 20160299967 A1 US20160299967 A1 US 20160299967A1 US 201615180702 A US201615180702 A US 201615180702A US 2016299967 A1 US2016299967 A1 US 2016299967A1
- Authority
- US
- United States
- Prior art keywords
- documents
- subject
- subjects
- terms
- dictionary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
-
- G06F17/30675—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24575—Query processing with adaptation to user needs using context
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
- G06F16/322—Trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3334—Selection or weighting of terms from queries, including natural language queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3335—Syntactic pre-processing, e.g. stopword elimination, stemming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3338—Query expansion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/93—Document management systems
-
- G06F17/30011—
-
- G06F17/30625—
-
- G06F17/30707—
Definitions
- the present invention is related to document retrieval and categorization, as well as information searches, and more specifically to a computer-performed method, computer system and computer program product for document retrieval using per-subject dictionaries, and in which an internal hierarchy of the dictionaries is used to adjust per-subject match results.
- Information storage and retrieval in computer systems is an ever-evolving technology as collections of data become progressively larger and more complex. So-called “big data” involves collection of large amounts of data that may be essentially unfiltered and uncategorized. While businesses, government and other entities would like to capitalize on information that can be gleaned from such large collections of data, techniques to efficiently retrieve a manageable amount of information in response to a query are needed.
- Retrieval of information from present-day databases and other more loosely-coupled information sources such as the Internet is typically performed by either crawler-based indexing, in which software engines obtain indexing information from stored documents, or from human-built directories that categorize the stored documents.
- crawler-based indexing in which software engines obtain indexing information from stored documents, or from human-built directories that categorize the stored documents.
- the data source becomes sufficiently large, the size of the response to a query also grows.
- the invention is embodied in a computer-performed method, computer program product and computer system that can efficiently categorize and retrieve documents.
- the method is a method of operation of the computer system, which executes the computer program product to carry out the steps of the method.
- the method selects dictionaries from a set of multiple dictionaries that are each associated with a different subject according to a specified one or more subjects.
- the one or more subjects are matched to documents in collection of documents according to a strength of relationship between the documents and the one or more subjects along with hierarchical information that indicates an affinity between the selected dictionaries to obtain a subset of the collection of documents that is relevant to the one or more subjects.
- FIG. 1 is a block diagram illustrating a networked computer system in which techniques according to an embodiment of the present invention are practiced.
- FIG. 2 is an organizational diagram showing an organization of a document tagging and retrieval system in accordance with an embodiment of the present invention.
- FIG. 3 is a pictorial diagram showing organization of dictionaries as may be used in the document tagging and retrieval system of FIG. 2 .
- FIG. 4 is a flowchart illustrating a dictionary hierarchy information generation method in accordance with an embodiment of the present invention.
- FIGS. 5A-5D are pictorial diagrams depicting organization of affinity groups as can be used in methods according to the present invention.
- FIG. 6 is a flowchart illustrating a document tagging methodology in accordance with an embodiment of the present invention.
- FIG. 7 is a flowchart illustrating a document retrieval methodology in accordance with an embodiment of the present invention.
- the present invention relates to document retrieval, and in particular to techniques for identifying and retrieving files in big data collections.
- Multiple dictionaries each having a corresponding subject, contain entries corresponding to descriptive terms that are associated with the subject, i.e., the terms that ordinarily occur in association with the subject in written documents. Thus, a term may occur across multiple entries in multiple dictionaries, but have a different meaning or descriptive power with respect to different subjects.
- the entries are also organized in a hierarchy of sub-classification that is used during document retrieval to weight multiple matching terms in a dictionary according to whether they have strong affinity (e.g., terms that are in the same sub-classification) or a weak affinity (e.g., terms that have no descriptive relationship at all other than being in the same subject dictionary).
- the entries in each dictionary also have a score value associated with their corresponding term and stored in the dictionary along with the term.
- the score value is a “subject-determining-power score” (SDP score) that is an indicator of the power of the term to determine the subject of a query, a document, or other item associated with the term.
- SDP score subject-determining-power score
- an SDP score may be used to weight terms used to tag a document, according to how strongly they indicate that the document concerns a particular subject.
- a tag is stored information that is descriptive in some manner of an associated document.
- the tag can be stored in the document itself, e.g., as metadata in a header, or the tag may be stored separately from the document, e.g., in a database containing a link to the document.
- the process of tagging is generating or selecting the tag information and storing it in a manner that associates the tag(s) with the document. Tagging can occur when a document is first added to a collection, which may be storage of the document in a particular storage location, or may be insertion of a link to the document in a database, or tagging may occur subsequently.
- Documents are retrieved by identifying documents from a collection and returning the documents to a requesting entity.
- the particular documents returned and the particular order of the documents can be determined by the quality of a match of the documents to one or more subjects determined from the contents of a query.
- the query is first matched to the dictionaries to determine the subject(s) of the query.
- the hierarchy information is used to adjust the results of a match between search terms and the multiple subjects, by adjusting the documents returned for each matching dictionary according to affinity between pairs of search terms for each subject as indicated by the hierarchy within the dictionary associated with that subject.
- the affinities between search terms can also be used to adjust the match scores used to select the order of the matching dictionaries.
- the hierarchical information describing the organization of the entries in a dictionary is used to determine the affinity between pairs of terms that match entries in the dictionary, and the SDP scores for those terms is adjusted based on the affinity of the pair, which strengthens the contribution to the quality of the match more for pairs of terms having higher affinities than for pairs of terms having lower affinities.
- the response to a query can differ.
- One possible response includes copying the documents to a predetermined location, such as a directory that has been specified or created to receive the results of the query.
- Another possible response is generation of a file that contains a list of document identifiers, e.g., file pathnames or links, in order of priority, and optionally including a match-score associated with each document.
- a third option is generation of an html document, e.g., html browser page that provides links to the documents in the order of priority, e.g. ordered by quality of the match of the individual documents to the query.
- dictionaries are subject-specific lists of terms along with differentiating SDP scores for the terms.
- a term can be a single word or multiple words, and can potentially include letters, numbers, punctuation, etc. The same term may appear in the dictionaries for different subjects with different SDP scores.
- an SDP score for a particular term for a particular subject indicates how strongly the appearance of the term suggests the term concerns the particular subject.
- Terms can be single words or multi-words, e.g., War of 1812.
- standard text pre-processing can be performed before any of the analytical steps, such as phrase detection using punctuation or detection of separators such as the, and, more, etc., which can be removed from the text.
- stemming can be performed to reduce or expand words of a single root to a single term, e.g., the word “acted” may be stemmed to the word “act.”
- a first workstation computer system 10 A includes a processor CPU coupled to a memory MEM that contains program instructions for execution by processor CPU, including a document collection interface 11 A, which may be a dedicated engine for performing tasks in a collection of documents as described herein, or which may be a generic browser interacting with a dedicated engine located in another system, such as document collection interface 11 B of workstation 10 B.
- the tasks performed by one or both of document collection interfaces 11 A and 11 B include query processing, document tagging, directory creation and other tasks associated with managing the collection of documents and dictionaries used by and operated on by the embodiments of the invention disclosed herein.
- Workstation computer 10 A is also depicted as including a graphical display Display and input devices Input Devices, such as mice and keyboards, for interacting with user interfaces including login screens and other user interfaces for interacting with other computers connected to the network, for example, administration screens for administering identification and authorization profiles used by the techniques of the present invention.
- a graphical display Display and input devices Input Devices such as mice and keyboards, for interacting with user interfaces including login screens and other user interfaces for interacting with other computers connected to the network, for example, administration screens for administering identification and authorization profiles used by the techniques of the present invention.
- Workstation computer system 10 A also includes a hard disc controller HDC 14 that interfaces processor CPU to local storage device 17 A and a network interface NWI that couples workstation computer system 10 A to network 15 , which may be fully wireless, fully wired or any type of hybrid network.
- Network interface NWI provides access to network resources, such as remote storage provided by networked storage devices 17 B and 17 C, which are coupled to network 15 by network disc controller (NWDC) 18 .
- NWDC network disc controller
- An external database DB may provide storage for documents, dictionaries, query results and other information discussed herein, alternatively document collection interfaces 11 A and 11 B may perform database organization, with the above-listed items stored as files in local storage device 17 A or networked storage devices 17 B and 17 C.
- Workstation computer system 10 B has an internal organization similar to that depicted in workstation computer system 10 A and is also coupled to network 15 .
- Network 15 may include wireless local area networks (WLANs), wired local-area networks (LANs), wide-area networks (WANs) or any other suitable interconnection that provides communication between workstation computer systems 10 A and 10 B, storage devices 17 A- 17 C, external database DB and any other systems and devices coupled to network 15 .
- WLANs wireless local area networks
- LANs wired local-area networks
- WANs wide-area networks
- the present invention concerns document storage and retrieval functionality that is not limited to a specific computer system or network configuration.
- the specification workstation computer systems 10 A and 10 B and the location of their specific memory MEM and document collection interfaces 11 A and 11 B does not imply a specific client-server relationship or hierarchical organization, as the techniques of the present invention may be employed in distributed systems in which no particular machine is identified as a server.
- At least one of the machines provides an instance and functionality of an object or interface that performs document storage and retrieval in accordance with an embodiment of the present invention.
- the objects or interfaces implementing document collection interfaces 11 A and 11 B process information according to methods and structures of the present invention, as described in further detail below.
- a search interface 22 which may be a dedicated search interface, or a browser-based search portal as described above, provides access to a dictionary manager 20 that processes the terms in the queries supplied by search interface 22 in order to determine the subject of the query and other targeted information that will be used to retrieve one or more of documents 26 .
- Per-subject dictionaries 24 A- 24 C contain lists of terms and their corresponding SDP scores for the containing dictionary.
- Document tag information 28 which may be stored in documents 26 , but is generally stored in a separate database or file for faster parsing of document tag information 28 , is also accessed by dictionary manager 20 to discover documents having the closest match to the subject and particular search terms provided in a query from search interface 22 .
- Dictionaries including SDPs can be created in different ways including: human-generated lists of terms with human-estimated SDPs, human-generated lists of terms with automatically calculated SDPs.
- a first example dictionary 24 A has a corresponding subject of Farm Supplies, and contains the terms FEED, FERTILIZER, IRRIGATION and PENS, with associated SDP scores given in an exemplary range of 1 through 10, of 6, 9, 7 and 2, respectively.
- FERTILIZER in the query indicates a high probability that the subject of the search corresponds to Farm Supplies
- PENS in the query indicates only a low probability that the subject of the search corresponds to Farm Supplies.
- Another example dictionary 24 B having a corresponding subject of Office Supplies, contains the terms PENS, STAPLES, PAPER and FOLDERS, with associated SDP scores of 8, 6, 5, and 7, respectively.
- the pair of dictionaries 24 A and 24 B illustrates that a term, e.g., PENS, may be present in multiple dictionaries and have different SDP scores in each of the different per-subject dictionaries, in the exemplary case, illustrating that the term PEN is a much stronger indication that a search concerns Office Supplies than Farm Supplies.
- exemplary document tags that may be used to implement document tag information 28 as illustrated in FIG. 2 .
- the tag information which is just a single document tag that may be included in a header of a document, stored in a tag database, or otherwise organized in a tag store file, contains an optional document title and summary, a link to the actual document, and lists of terms by subject for each of multiple subjects that the document concerns.
- the title and summary information are optional, but may be included to provide a quick resource for displaying search results.
- the link to the document is not needed if the tag information is itself stored in the document header.
- Exemplary document tag 28 B includes only subject names without terms, as dictionary manager 20 can use lists of subjects without specified lists of terms to perform matching within the set of dictionaries.
- hierarchy information to enhance dictionary-assisted tagging and retrieval, exploits the existence of sub-classifications that may exist within the dictionary subjects.
- a hierarchy strategy can used to find sub-classifications that represent an organization more granular than that given by just the original labels in the data set.
- the techniques described herein may use clustering to group terms in sub-classifications when the terms are more strongly related to each other than to other terms. Referring to FIG.
- a method for generating hierarchical dictionary information can be implemented by scoring a set of documents across a set of per-subject dictionaries (step 30 ), optionally generating a full-text index of all of the documents (step 32 ) and generating a hierarchy for each dictionary that has some affinity property for associated descriptive terms (step 34 ).
- the hierarchy information discovered during dictionary creation can be stored in a number of ways, i.e., an array, a table or a linked-list, but in general the information can be understood to map to a tree such as those shown in FIGS. 5A-5D .
- the tree can be the direct output of a hierarchical categorization algorithm.
- the top-level group in row R 1 contains all of the terms in the dictionary.
- the second row is split into two sub-classified portions R 2 A and R 2 B.
- Each of sub-classified portions R 2 A and R 2 B is again bifurcated in the next row down.
- Sub-classification terminates when sub-classified portion has a single member.
- the hierarchical classification information may be generated using techniques such as hierarchical clustering.
- the distance metrics used to determine which terms belong in which clusters can be determined from co-occurrence of the terms across documents, with each bifurcation of the sub-classified groups minimizing the distance metric within the groups. Documents that match entries in a dictionary in the scoring in Step 30 of FIG.
- the criteria for a match between a document and a dictionary can be, for example, that a selected dictionary is the dictionary having the best match score for the document.
- the criteria may be, for example, that the match score between a document and the dictionary exceeds a threshold.
- the influence of a document on the hierarchy calculations can be weighted according to the match score.
- the “transistor dictionary” exemplified in FIG. 5A may be augmented to store dictionary term locations in the classification hierarchy description as illustrated below in Table 1.
- One exemplary application of the affinity information encoded in the hierarchy trees illustrated above is to weight the entry SDP scores obtained from the dictionaries according to the affinity group membership of the descriptive terms in the entries.
- the weighting can be applied when matching search terms to dictionaries to determine a subject for the search. Search terms with close affinity group membership increase the confidence that a dictionary matched to the search term(s) is a dictionary that has a subject with which the search terms are strongly related.
- the affinity information can be exploited by increasing the weight on SDPs during search-text to dictionary matching. For example, if the search terms are “drain” and “trap”, the following procedure can be applied:
- the weighting factor is 1.75. 4. Adjust the SDPs according to the weighting factors.
- FIG. 5C another example of hierarchical classification in a dictionary is shown.
- the hierarchy resembles an ontology. Terms are organized in a downward branching tree, in which each branch represents a different hierarchical sub-classification. Leaf nodes at the bottom of the tree have strong affinity to each other (siblings) and less affinity to nodes above the leaf nodes (parents) and even less affinity to leaf nodes under affinity to other leaf nodes not under the same parent (cousins).
- terms Crawdad and Worm have a high affinity value with respect to each other, lower affinity values with respect to their parent term Bait, and still lower affinity values with respect to cousin terms Weights and Line.
- the full-text index of the documents generated in step 32 of FIG. 4 can also generate and store information describing the frequency of occurrence of terms in proximity to other terms, the average distance (in words) between pairs of terms in each document, or other indicators of affinity between the terms.
- the statistics of term proximities can be used to determine a distance between terms, which in turn may be used to determine which terms are grouped together at each level of the hierarchy. Terms that are adjacent most frequently, while not frequently appearing adjacent to other terms, can be collected to form groups or clusters, which then are placed in the lowest-level (bottom) row of the hierarchy. Terms that occur less frequently proximate the terms in a group, but occur equally frequently proximate the group and other groups, are placed at a next higher level in the hierarchy, and so forth.
- Step 30 of FIG. 4 can be performed by a method as described below with reference to FIG. 6 , which describes tagging of documents, while in step 30 of FIG. 4 the documents are scored for best match with respect to the dictionaries.
- a method of tagging documents prior to processing search queries is shown in a flowchart.
- the method is used to associate document with classifications, i.e., the per-subject dictionaries, so that once a subject or subjects of a query is discovered, the appropriate documents can be retrieved.
- An initial/next document is retrieved (step 40 ) and an initial/next dictionary is accessed (step 41 ).
- the document terms are matched with the dictionary entries (step 42 ). If a term matches a term in one of the dictionaries (decision 43 ), then the dictionary name or subject (or other suitable identifier for the dictionary), and optionally the SDP score and/or term is added to the collection (step 45 ).
- the next dictionary is accessed (step 41 ).
- the dictionaries are ranked by match score (step 46 ), and the dictionary name and optionally confidence score and/or terms with SDP scores are added to the document tag (step 47 ).
- the next document is retrieved and processed by the process of steps 40 - 48 .
- the first step in tagging a document is matching the document against the set of dictionaries.
- the document can then be tagged with information such as: the names of the n top-matching dictionaries, a match score of the document against all dictionaries and optionally, the top m words from the n top-matching dictionaries.
- a dictionary match annotation in the tag may include a confidence score. If dictionary words are included in the tags, they can include the SDP of the word in the dictionary. The choice of information with which to tag depends on storage space and ease of use for retrieval of the document.
- Dictionary-matching may be performed by finding all terms that exist in both the document and a dictionary of interest, generating a document-occurrence vector with one entry per common term, generating a dictionary-vector with one entry per common term in the same order as the vector above, and multiplying the two vectors to get a dictionary-match score between the document and the dictionary.
- the vector multiplication sums the term-by-term products of the two vectors to produce a single number.
- the values of the dictionary-vector are the SDP scores. For a given document, the process can be repeated for all dictionaries.
- tags can be chosen for the document as described above. The tags selected provide a structure for storing and reporting the document-to-dictionary match results produced by step 30 of FIG. 4 .
- One manner in which the tagging information associated with a single document may be organized is to include the search terms in the tagging information along with the subject and SDP scores, such as illustrated in Table 3 below:
- the full text index optionally prepared in step 32 keeps a record of each document and the counts of all occurrences of dictionary terms for each document. For example if document A contained the word tackle 20 times the full text index might contain the record ⁇ docA, tackle, 20 ⁇ , which maps the term tackle and a number of its occurrence to document A. Table 4 below gives an example of a partial dictionary with SDPs for the subject “Fishing.”
- a search request is received with search terms consisting of one or more words (step 60 ), and the dictionaries containing the search terms are matched (step 61 ).
- the dictionaries are ordered by strength of the match to the search terms (step 62 ).
- the affinity values are optionally used to adjust the SDP scores when matching the search terms to the dictionaries.
- the documents matching the dictionaries are collected and ordered according to the pre-matched tag information that associates the dictionaries with the documents (step 63 ).
- the dictionaries containing the search terms are retrieved (step 64 ) as well as the documents containing the search terms (step 65 ).
- Affinity values are computed for pairs of the search terms in each dictionary (step 66 ).
- the SDP scores for the terms are retrieved from the dictionaries and adjusted according to the affinity values (step 67 ).
- the documents are re-ordered for each dictionary according to the adjusted SDP scores (step 68 ), and finally the documents are returned in order of the match, first between the search terms and the dictionary and second between the document and the dictionary using adjusted SDP scores (step 69 ).
- Step 64 as described above locates the dictionaries containing the terms and step 65 locates the documents containing the query terms.
- the method identifies the affinity groups to which the query terms belong and calculates the distances between the terms, i.e., the affinity values.
- the distance function is defined by the number of nodes that traverse a path connecting two nodes in FIG. 5C and FIG. 5D , which correspond to terms. For example if a query specifies the terms “crawdad” and “spinner” the distance would be 1. In the example, the shortest distance is 1 not 0, indicating both terms are in the same affinity group, i.e., are connected beneath a single node at a single level of the classification hierarchy. Table 6 below gives a set of examples along with distance counts.
- weighting factors are applied according to:
- SDP new SDP (1 +w ), where SDP new is the adjusted SDP score.
- a new SDP score for each possible set of pairs of the terms “crawdad”, “worm” and “spincast” can be computed according to the distance weighting equation above. Then, the average, maximum or minimum of the scores for each term may be used as the new (adjusted) SDP.
- Step 68 in the method of FIG. 7 re-orders the document-to-dictionary match by using the adjusted SDPs to change the document scoring calculation. If hierarchy were not applied, documents would be returned in the order of the match between a raw score of the documents to the relevant dictionary, independent of the search terms. Referring to Table 7 below which lists documents that match the fishing dictionary of FIG. 5C and Table 8 below, which lists documents that match the gardening dictionary of FIG. 5D , a contrast between hierarchically-adjusted matching and non-adjusted matching will be demonstrated.
- the Fishing dictionary appears first in the order because the Fishing dictionary is the best match to the search terms. Documents tagged with the Fishing dictionary are then ordered from highest-to-lowest match score against the Fishing dictionary.
- the Gardening dictionary appears next, and again the documents are ordered from highest-to-lowest match score against the gardening dictionary.
- the order of the document within a given dictionary is independent of the search terms. Only the set of dictionaries and the order in which they are presented changes as the search terms change. For example, for search terms “worm” and “shovel” the results are:
- the hierarchical reordering of the per-dictionary results in step 68 of FIG. 7 modifies the order of the documents for the corresponding dictionaries to improve the result of the search.
- DOC score sum( SPD *percent of occurrence)
- portions of the present invention may be embodied in a computer program product, which may include firmware, an image in system memory or another memory/cache, or stored on a fixed or re-writable media such as an optical disc having computer-readable code stored thereon. Any combination of one or more computer-readable medium(s) may store a program in accordance with an embodiment of the invention.
- the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
- a computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
- the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
- a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
- a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof.
- a computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
- Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Abstract
Techniques for managing big data include retrieval using per-subject dictionaries having and hierarchical information indicating affinity between the dictionaries. A retrieval request for one or more documents pertaining to one or more subjects is performed by specifying the subjects, selecting dictionaries associated with the subjects, matching the subjects to documents in the collection according to a strength of relationship between the documents and the subjects and also using the hierarchical information, to obtain a subset of the collection of documents.
Description
- The present application is a Continuation of U.S. patent application Ser. No. 14/854,767, filed on Sep. 15, 2015 and published as U.S. Patent Publication No. 20160004704 on Jan. 7, 2016, which is a Continuation of U.S. patent application Ser. No. 14/077,305, filed on Nov. 12, 2013 and issued as U.S. Pat. No. 9,235,638 on Jan. 12, 2016, and claims priority to the above-listed U.S. patent applications under 35 U.S.C. §120. U.S. patent application Ser. No. 14/077,305 is related to U.S. patent application Ser. No. 13/891,610 entitled “DOCUMENT TAGGING AND RETRIEVAL USING PER-SUBJECT DICTIONARIES INCLUDING SUBJECT-DETERMINING-POWER SCORES FOR ENTRIES” as filed on May 10, 2013 and issued as U.S. Pat. No. 9,262,510 on Feb. 16, 2016, and U.S. patent application Ser. No. 14/055,379, entitled “DOCUMENT TAGGING AND RETRIEVAL USING ENTITY SPECIFIERS”, as filed on Oct. 16, 2013 and issued as U.S. Pat. No. 9,251,136 on Feb. 2, 2016, both having at least one common inventor with the present U.S. patent application. The Disclosures of the above-referenced U.S. patent applications are incorporated herein by reference.
- 1. Field of the Invention
- The present invention is related to document retrieval and categorization, as well as information searches, and more specifically to a computer-performed method, computer system and computer program product for document retrieval using per-subject dictionaries, and in which an internal hierarchy of the dictionaries is used to adjust per-subject match results.
- 2. Description of Related Art
- Information storage and retrieval in computer systems is an ever-evolving technology as collections of data become progressively larger and more complex. So-called “big data” involves collection of large amounts of data that may be essentially unfiltered and uncategorized. While businesses, government and other entities would like to capitalize on information that can be gleaned from such large collections of data, techniques to efficiently retrieve a manageable amount of information in response to a query are needed.
- Retrieval of information from present-day databases and other more loosely-coupled information sources such as the Internet is typically performed by either crawler-based indexing, in which software engines obtain indexing information from stored documents, or from human-built directories that categorize the stored documents. However, once the data source becomes sufficiently large, the size of the response to a query also grows.
- Therefore, it would be desirable to provide a method, computer system and computer program that can more efficiently handle categorization of documents and retrieval of documents in response to queries.
- The invention is embodied in a computer-performed method, computer program product and computer system that can efficiently categorize and retrieve documents. The method is a method of operation of the computer system, which executes the computer program product to carry out the steps of the method.
- The method selects dictionaries from a set of multiple dictionaries that are each associated with a different subject according to a specified one or more subjects. The one or more subjects are matched to documents in collection of documents according to a strength of relationship between the documents and the one or more subjects along with hierarchical information that indicates an affinity between the selected dictionaries to obtain a subset of the collection of documents that is relevant to the one or more subjects.
- The foregoing and other objectives, features, and advantages of the invention will be apparent from the following, more particular, description of the preferred embodiment of the invention, as illustrated in the accompanying drawings.
- The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of the invention when read in conjunction with the accompanying Figures, wherein like reference numerals indicate like components, and:
-
FIG. 1 is a block diagram illustrating a networked computer system in which techniques according to an embodiment of the present invention are practiced. -
FIG. 2 is an organizational diagram showing an organization of a document tagging and retrieval system in accordance with an embodiment of the present invention. -
FIG. 3 is a pictorial diagram showing organization of dictionaries as may be used in the document tagging and retrieval system ofFIG. 2 . -
FIG. 4 is a flowchart illustrating a dictionary hierarchy information generation method in accordance with an embodiment of the present invention. -
FIGS. 5A-5D are pictorial diagrams depicting organization of affinity groups as can be used in methods according to the present invention. -
FIG. 6 is a flowchart illustrating a document tagging methodology in accordance with an embodiment of the present invention. -
FIG. 7 is a flowchart illustrating a document retrieval methodology in accordance with an embodiment of the present invention. - The present invention relates to document retrieval, and in particular to techniques for identifying and retrieving files in big data collections. Multiple dictionaries, each having a corresponding subject, contain entries corresponding to descriptive terms that are associated with the subject, i.e., the terms that ordinarily occur in association with the subject in written documents. Thus, a term may occur across multiple entries in multiple dictionaries, but have a different meaning or descriptive power with respect to different subjects. The entries are also organized in a hierarchy of sub-classification that is used during document retrieval to weight multiple matching terms in a dictionary according to whether they have strong affinity (e.g., terms that are in the same sub-classification) or a weak affinity (e.g., terms that have no descriptive relationship at all other than being in the same subject dictionary). The entries in each dictionary also have a score value associated with their corresponding term and stored in the dictionary along with the term. The score value is a “subject-determining-power score” (SDP score) that is an indicator of the power of the term to determine the subject of a query, a document, or other item associated with the term. For example, an SDP score may be used to weight terms used to tag a document, according to how strongly they indicate that the document concerns a particular subject. A tag is stored information that is descriptive in some manner of an associated document. The tag can be stored in the document itself, e.g., as metadata in a header, or the tag may be stored separately from the document, e.g., in a database containing a link to the document. The process of tagging is generating or selecting the tag information and storing it in a manner that associates the tag(s) with the document. Tagging can occur when a document is first added to a collection, which may be storage of the document in a particular storage location, or may be insertion of a link to the document in a database, or tagging may occur subsequently.
- Documents are retrieved by identifying documents from a collection and returning the documents to a requesting entity. The particular documents returned and the particular order of the documents can be determined by the quality of a match of the documents to one or more subjects determined from the contents of a query. The query is first matched to the dictionaries to determine the subject(s) of the query. The hierarchy information is used to adjust the results of a match between search terms and the multiple subjects, by adjusting the documents returned for each matching dictionary according to affinity between pairs of search terms for each subject as indicated by the hierarchy within the dictionary associated with that subject. The affinities between search terms can also be used to adjust the match scores used to select the order of the matching dictionaries. The hierarchical information describing the organization of the entries in a dictionary is used to determine the affinity between pairs of terms that match entries in the dictionary, and the SDP scores for those terms is adjusted based on the affinity of the pair, which strengthens the contribution to the quality of the match more for pairs of terms having higher affinities than for pairs of terms having lower affinities.
- The response to a query can differ. One possible response includes copying the documents to a predetermined location, such as a directory that has been specified or created to receive the results of the query. Another possible response is generation of a file that contains a list of document identifiers, e.g., file pathnames or links, in order of priority, and optionally including a match-score associated with each document. A third option is generation of an html document, e.g., html browser page that provides links to the documents in the order of priority, e.g. ordered by quality of the match of the individual documents to the query.
- As mentioned above, dictionaries, as referred to herein, are subject-specific lists of terms along with differentiating SDP scores for the terms. A term can be a single word or multiple words, and can potentially include letters, numbers, punctuation, etc. The same term may appear in the dictionaries for different subjects with different SDP scores. In essence, an SDP score for a particular term for a particular subject indicates how strongly the appearance of the term suggests the term concerns the particular subject. Terms can be single words or multi-words, e.g., War of 1812. When processing queries or documents to discover terms, standard text pre-processing can be performed before any of the analytical steps, such as phrase detection using punctuation or detection of separators such as the, and, more, etc., which can be removed from the text. Similarly, stemming can be performed to reduce or expand words of a single root to a single term, e.g., the word “acted” may be stemmed to the word “act.”
- Referring to
FIG. 1 , a networked computer system in which an embodiment of the present invention is practiced is depicted in a block diagram. A firstworkstation computer system 10A includes a processor CPU coupled to a memory MEM that contains program instructions for execution by processor CPU, including adocument collection interface 11A, which may be a dedicated engine for performing tasks in a collection of documents as described herein, or which may be a generic browser interacting with a dedicated engine located in another system, such asdocument collection interface 11B ofworkstation 10B. The tasks performed by one or both ofdocument collection interfaces Workstation computer 10A is also depicted as including a graphical display Display and input devices Input Devices, such as mice and keyboards, for interacting with user interfaces including login screens and other user interfaces for interacting with other computers connected to the network, for example, administration screens for administering identification and authorization profiles used by the techniques of the present invention. -
Workstation computer system 10A also includes a harddisc controller HDC 14 that interfaces processor CPU tolocal storage device 17A and a network interface NWI that couplesworkstation computer system 10A to network 15, which may be fully wireless, fully wired or any type of hybrid network. Network interface NWI provides access to network resources, such as remote storage provided bynetworked storage devices collection interfaces local storage device 17A ornetworked storage devices Workstation computer system 10B has an internal organization similar to that depicted inworkstation computer system 10A and is also coupled tonetwork 15. -
Network 15 may include wireless local area networks (WLANs), wired local-area networks (LANs), wide-area networks (WANs) or any other suitable interconnection that provides communication betweenworkstation computer systems storage devices 17A-17C, external database DB and any other systems and devices coupled tonetwork 15. The present invention concerns document storage and retrieval functionality that is not limited to a specific computer system or network configuration. Finally, the specificationworkstation computer systems document collection interfaces document collection interfaces - Referring now to
FIG. 2 , an organization of a document tagging and retrieval system in accordance with an embodiment of the present invention is illustrated. Asearch interface 22, which may be a dedicated search interface, or a browser-based search portal as described above, provides access to adictionary manager 20 that processes the terms in the queries supplied bysearch interface 22 in order to determine the subject of the query and other targeted information that will be used to retrieve one or more ofdocuments 26. Per-subject dictionaries 24A-24C contain lists of terms and their corresponding SDP scores for the containing dictionary.Document tag information 28, which may be stored indocuments 26, but is generally stored in a separate database or file for faster parsing ofdocument tag information 28, is also accessed bydictionary manager 20 to discover documents having the closest match to the subject and particular search terms provided in a query fromsearch interface 22. Dictionaries including SDPs can be created in different ways including: human-generated lists of terms with human-estimated SDPs, human-generated lists of terms with automatically calculated SDPs. - Referring now to
FIG. 3 , organization of dictionaries as may be used in the document tagging and retrieval system ofFIG. 2 , is shown. Afirst example dictionary 24A has a corresponding subject of Farm Supplies, and contains the terms FEED, FERTILIZER, IRRIGATION and PENS, with associated SDP scores given in an exemplary range of 1 through 10, of 6, 9, 7 and 2, respectively. Thus, discovery of the term FERTILIZER in the query indicates a high probability that the subject of the search corresponds to Farm Supplies, but the discovery of the term PENS in the query indicates only a low probability that the subject of the search corresponds to Farm Supplies. Anotherexample dictionary 24B, having a corresponding subject of Office Supplies, contains the terms PENS, STAPLES, PAPER and FOLDERS, with associated SDP scores of 8, 6, 5, and 7, respectively. The pair ofdictionaries - Also illustrated in
FIG. 3 , are exemplary document tags that may be used to implementdocument tag information 28 as illustrated inFIG. 2 . Inexemplary document tag 28A, the tag information, which is just a single document tag that may be included in a header of a document, stored in a tag database, or otherwise organized in a tag store file, contains an optional document title and summary, a link to the actual document, and lists of terms by subject for each of multiple subjects that the document concerns. The title and summary information are optional, but may be included to provide a quick resource for displaying search results. The link to the document is not needed if the tag information is itself stored in the document header.Exemplary document tag 28B includes only subject names without terms, asdictionary manager 20 can use lists of subjects without specified lists of terms to perform matching within the set of dictionaries. - As pointed out above, the use of hierarchy information to enhance dictionary-assisted tagging and retrieval, exploits the existence of sub-classifications that may exist within the dictionary subjects. When a dictionary is created using a labeled data set, a hierarchy strategy can used to find sub-classifications that represent an organization more granular than that given by just the original labels in the data set. The techniques described herein may use clustering to group terms in sub-classifications when the terms are more strongly related to each other than to other terms. Referring to
FIG. 4 , a method for generating hierarchical dictionary information can be implemented by scoring a set of documents across a set of per-subject dictionaries (step 30), optionally generating a full-text index of all of the documents (step 32) and generating a hierarchy for each dictionary that has some affinity property for associated descriptive terms (step 34). The hierarchy information discovered during dictionary creation can be stored in a number of ways, i.e., an array, a table or a linked-list, but in general the information can be understood to map to a tree such as those shown inFIGS. 5A-5D . The tree can be the direct output of a hierarchical categorization algorithm. - Referring now to
FIG. 5A , an example hierarchical classification is shown. The top-level group in row R1 contains all of the terms in the dictionary. The second row is split into two sub-classified portions R2A and R2B. Each of sub-classified portions R2A and R2B is again bifurcated in the next row down. Sub-classification terminates when sub-classified portion has a single member. The hierarchical classification information may be generated using techniques such as hierarchical clustering. The distance metrics used to determine which terms belong in which clusters can be determined from co-occurrence of the terms across documents, with each bifurcation of the sub-classified groups minimizing the distance metric within the groups. Documents that match entries in a dictionary in the scoring inStep 30 ofFIG. 4 are used to generate the hierarchy for each dictionary inStep 34. The criteria for a match between a document and a dictionary can be, for example, that a selected dictionary is the dictionary having the best match score for the document. Alternatively, the criteria may be, for example, that the match score between a document and the dictionary exceeds a threshold. In the latter example, the influence of a document on the hierarchy calculations can be weighted according to the match score. In the alternative example, the “transistor dictionary” exemplified inFIG. 5A may be augmented to store dictionary term locations in the classification hierarchy description as illustrated below in Table 1. -
TABLE 1 Subject = Transistor Dictionary Term SDP Score Affinity Groups Gate 6 R1, R2A, R3A Source 6 R1, R2A, R3B, R4A Drain 6 R1, R2A, R3B, R4B Lithography 5 R1, R2B, R3C, R4C Etch 5 R1, R2B, R3C, R4D Electromigration 7 R1, R2B, R3D, R4F, R5D Trap 4 R1, R2B, R3D, R4E, R5B Metal 3 R1, R2B, R3D, R4F, R5C Oxide 4 R1, R2B, R3D, R4E, R5A - An additional example is given for a plumbing dictionary in Table 2, which is illustrated in graphical form in
FIG. 5B . -
TABLE 2 Subject = Plumbing Dictionary Term SDP Score Affinity Groups Faucet 7 T1, T2A, T3A, T4B Mixer 3 T1, T2A, T3A, T4A Drain 5 T1, T2A, T3B, T4C Trap 2 T1, T2A, T3B, T4D Union 2 T1, T2B, T3C, T4E, T5A Copper 6 T1, T2B, T3D Tee 4 T1, T2B, T3C, T4E, T5B Elbow 4 T1, T2B, T3C, T4F - One exemplary application of the affinity information encoded in the hierarchy trees illustrated above is to weight the entry SDP scores obtained from the dictionaries according to the affinity group membership of the descriptive terms in the entries. In particular, the weighting can be applied when matching search terms to dictionaries to determine a subject for the search. Search terms with close affinity group membership increase the confidence that a dictionary matched to the search term(s) is a dictionary that has a subject with which the search terms are strongly related. The affinity information can be exploited by increasing the weight on SDPs during search-text to dictionary matching. For example, if the search terms are “drain” and “trap”, the following procedure can be applied:
- 1. Calculate the maximum distance in the affinity tree needed to travel from the leaf node (a node containing only one term) that contains one of the search terms to a group containing both terms. For the example Transistor dictionary, the maximum distance is 4, from leaf node R5B to group R1. For the example Plumbing dictionary, the maximum distance for the same search terms is 1.
2. Determine the maximum distance as defined above between any two of the search terms in each affinity tree. For the example Transistor dictionary the result is 4. For the example Plumbing dictionary the result is also 4.
3. Calculate a weighting factor from the affinity values determined for each dictionary as 1+(max−x)/max, where x is the value calculated instep 1 above and max is the value calculated instep 2 above. For the example Transistor dictionary, the weighting factor is 1.0. For the example Plumbing dictionary, the weighting factor is 1.75.
4. Adjust the SDPs according to the weighting factors. In the example given above, the search-text match to the Transistor Dictionary is (6*0.5+4*0.5)*1=5. For the Plumbing dictionary the result is (5*0.5+2*0.5)*1.75=6.1.
Note that without the affinity-group enhancement the top match for the search-text would have been the Transistor dictionary, but with the affinity-group enhancement, the top-matching dictionary is the Plumbing dictionary. This change in ranking of the per-subject dictionaries is reflective of the principle that while words “drain” and “trap” are associated with both the Transistor and Plumbing subjects, when words “drain” and “trap” are used together they are more suggestive of the subject/classification Plumbing than the subject/classification Transistors. - Referring now to
FIG. 5C , another example of hierarchical classification in a dictionary is shown. InFIG. 5C , the hierarchy resembles an ontology. Terms are organized in a downward branching tree, in which each branch represents a different hierarchical sub-classification. Leaf nodes at the bottom of the tree have strong affinity to each other (siblings) and less affinity to nodes above the leaf nodes (parents) and even less affinity to leaf nodes under affinity to other leaf nodes not under the same parent (cousins). In the exemplary dictionary ofFIG. 5C , terms Crawdad and Worm have a high affinity value with respect to each other, lower affinity values with respect to their parent term Bait, and still lower affinity values with respect to cousin terms Weights and Line. The full-text index of the documents generated instep 32 ofFIG. 4 can also generate and store information describing the frequency of occurrence of terms in proximity to other terms, the average distance (in words) between pairs of terms in each document, or other indicators of affinity between the terms. The statistics of term proximities can be used to determine a distance between terms, which in turn may be used to determine which terms are grouped together at each level of the hierarchy. Terms that are adjacent most frequently, while not frequently appearing adjacent to other terms, can be collected to form groups or clusters, which then are placed in the lowest-level (bottom) row of the hierarchy. Terms that occur less frequently proximate the terms in a group, but occur equally frequently proximate the group and other groups, are placed at a next higher level in the hierarchy, and so forth. The classification process continues until the most generic term that is, on average, equally related to each of the highest sub-classifications is placed at the highest level of the hierarchy. The most generic term can be used as a descriptor of the subject of the dictionary. The steps taken to add hierarchy information to the dictionaries are described in further detail below.Step 30 ofFIG. 4 can be performed by a method as described below with reference toFIG. 6 , which describes tagging of documents, while instep 30 ofFIG. 4 the documents are scored for best match with respect to the dictionaries. - Referring now to
FIG. 6 , a method of tagging documents prior to processing search queries is shown in a flowchart. The method is used to associate document with classifications, i.e., the per-subject dictionaries, so that once a subject or subjects of a query is discovered, the appropriate documents can be retrieved. An initial/next document is retrieved (step 40) and an initial/next dictionary is accessed (step 41). The document terms are matched with the dictionary entries (step 42). If a term matches a term in one of the dictionaries (decision 43), then the dictionary name or subject (or other suitable identifier for the dictionary), and optionally the SDP score and/or term is added to the collection (step 45). Until the last dictionary is processed (decision 44) the next dictionary is accessed (step 41). After the last dictionary is processed (decision 44), the dictionaries are ranked by match score (step 46), and the dictionary name and optionally confidence score and/or terms with SDP scores are added to the document tag (step 47). Until the last document is processed (decision 48), the next document is retrieved and processed by the process of steps 40-48. In the example, the first step in tagging a document is matching the document against the set of dictionaries. The document can then be tagged with information such as: the names of the n top-matching dictionaries, a match score of the document against all dictionaries and optionally, the top m words from the n top-matching dictionaries. A dictionary match annotation in the tag may include a confidence score. If dictionary words are included in the tags, they can include the SDP of the word in the dictionary. The choice of information with which to tag depends on storage space and ease of use for retrieval of the document. Dictionary-matching may be performed by finding all terms that exist in both the document and a dictionary of interest, generating a document-occurrence vector with one entry per common term, generating a dictionary-vector with one entry per common term in the same order as the vector above, and multiplying the two vectors to get a dictionary-match score between the document and the dictionary. The vector multiplication sums the term-by-term products of the two vectors to produce a single number. The values of the dictionary-vector are the SDP scores. For a given document, the process can be repeated for all dictionaries. Once the dictionary-match step is completed, tags can be chosen for the document as described above. The tags selected provide a structure for storing and reporting the document-to-dictionary match results produced bystep 30 ofFIG. 4 . - One manner in which the tagging information associated with a single document may be organized is to include the search terms in the tagging information along with the subject and SDP scores, such as illustrated in Table 3 below:
-
TABLE 3 Confidence value Subject of Dictionary Term/SDP score pairs 0.5 Bicycle Bottom- Bracket 8,Schwinn 10, Raleigh 10,Frame 3 . . .0.3 Home Construction Brick 5, Frame 3.5, . . . 0.1 Fishing Hook 7, Line 2
In the Example given above, once a candidate subject has selected, the terms having the top SDP scores (e.g., top 100 terms) may be inserted into the document tagging information in order to generalize the intersection between potential query terms and the document tag information. The end result is a set of documents scored to per-subject dictionaries. The next steps (steps FIG. 4 ) prepare the hierarchical information within (or stored alongside) the per-subject dictionaries. The full text index optionally prepared instep 32 keeps a record of each document and the counts of all occurrences of dictionary terms for each document. For example if document A contained the word tackle 20 times the full text index might contain the record {docA, tackle, 20}, which maps the term tackle and a number of its occurrence to document A. Table 4 below gives an example of a partial dictionary with SDPs for the subject “Fishing.” -
TABLE 4 Fishing Term SDP Baitcase 9 Crawdad 9 Spincast 9 Reel 8 Bait 7
Table 5 below is a list of documents with that match the fishing dictionary and some selected words in the document that lead to a match to the fishing dictionary. -
TABLE 5 Selected terms in document that Documents Score matched fishing dictionary docA 100 Tackle, bait, worm, line docB 90 Crawdad, Spinner docC 60 Spinner, weights, fiberglass - Referring now to
FIG. 7 , a method of document retrieval in accordance with an embodiment of the invention is shown in a flowchart. A search request is received with search terms consisting of one or more words (step 60), and the dictionaries containing the search terms are matched (step 61). The dictionaries are ordered by strength of the match to the search terms (step 62). The affinity values are optionally used to adjust the SDP scores when matching the search terms to the dictionaries. For each matching dictionary, the documents matching the dictionaries are collected and ordered according to the pre-matched tag information that associates the dictionaries with the documents (step 63). The dictionaries containing the search terms are retrieved (step 64) as well as the documents containing the search terms (step 65). Affinity values are computed for pairs of the search terms in each dictionary (step 66). The SDP scores for the terms are retrieved from the dictionaries and adjusted according to the affinity values (step 67). The documents are re-ordered for each dictionary according to the adjusted SDP scores (step 68), and finally the documents are returned in order of the match, first between the search terms and the dictionary and second between the document and the dictionary using adjusted SDP scores (step 69). -
Step 64 as described above locates the dictionaries containing the terms and step 65 locates the documents containing the query terms. For example if the query is “Crawdad, Spinner” the method would identify that the fishing dictionary and documents docB and docC contain the query terms as noted in Table 5. In step 66, the method identifies the affinity groups to which the query terms belong and calculates the distances between the terms, i.e., the affinity values. In the examples given herein, the distance function is defined by the number of nodes that traverse a path connecting two nodes inFIG. 5C andFIG. 5D , which correspond to terms. For example if a query specifies the terms “crawdad” and “spinner” the distance would be 1. In the example, the shortest distance is 1 not 0, indicating both terms are in the same affinity group, i.e., are connected beneath a single node at a single level of the classification hierarchy. Table 6 below gives a set of examples along with distance counts. -
TABLE 6 Query Terms Path Distance (Affinity) Worm, Bait Worm → Bait 2 Worm, Line Worm → Bait → Tackle→ Line 4 Worm, Spincast Worm→Bait→Tackle→Fish- 6 ing→Reel→Spincast
Alternatively, other methods can be used to compute the affinity values. One alternative method is to compute the difference in “height” in the tree between the terms. A term that is located in the same level of classification hierarchy is assigned a distance of 1, and any levels of classification that are traversed to reach the other term increment the distance for each such level traversed. - In the example, once the affinity values are obtained, in order to adjust the SDP scores, weighting factors are applied according to:
-
w=α/D, where D is the distance and α is a weighting factor and -
SDP new =SDP(1+w), where SDP new is the adjusted SDP score. - The above computations are performed for all matched search terms. For example if the search terms are “worm” and “bait”, if α=1 then the new SDP for worm is 3*(1+1*1/2)=4.5 and Bait is 7*(1+1*1/2)=10.5. When 3 or more query terms match in a dictionary a number of strategies can be employed. For example, if “crawdad”, “worm” and “spincast” are query terms, since they are all present in the fishing dictionary, the strong affinity of “crawdad” and “worm” must be managed in view of the low affinity of “spincast.” An average resulting SDP score may be computed, or the maximum or minimum SDP score might be used, depending on the needs of the system. For example, a new SDP score for each possible set of pairs of the terms “crawdad”, “worm” and “spincast” can be computed according to the distance weighting equation above. Then, the average, maximum or minimum of the scores for each term may be used as the new (adjusted) SDP.
-
Step 68 in the method ofFIG. 7 re-orders the document-to-dictionary match by using the adjusted SDPs to change the document scoring calculation. If hierarchy were not applied, documents would be returned in the order of the match between a raw score of the documents to the relevant dictionary, independent of the search terms. Referring to Table 7 below which lists documents that match the fishing dictionary ofFIG. 5C and Table 8 below, which lists documents that match the gardening dictionary ofFIG. 5D , a contrast between hierarchically-adjusted matching and non-adjusted matching will be demonstrated. -
TABLE 7 Terms in document that match with initial SDP and Score of document to percentage of occurrence in Documents fishing dictionary the document docA 2.3 Tackle (6) (0.1), bait (7) (0.1), worm (3) (0.2), line (4) (0.1) docB 2.1 Crawdad (9) (0.1), Spinner (6) (0.1), fiberglass (4) (0.1), weights (2) (0.1), docC 1.5 Crawdad (9) (0.1), Spinner (6) (0.1) docD 0.3 Worm (3) (0.1) -
TABLE 8 Terms in document that match gardening dictionary with initial SDP and Score of document to percentage of occurrence in Documents gardening dictionary the document docD 1 Worm (5)(0.1), grubs (5)(0.1) docE 1.2 Squash (4)(0.1), Carrots 2 (0.4) docF 0.3 Shovel (3)(0.1)
Without hierarchical adjustment of the SDP scores in the search, search terms “Crawdad” and “Worm” would return the result: -
Fishing<docA,docB,docC,docD>,Gardening<docE,docD,docF> - The Fishing dictionary appears first in the order because the Fishing dictionary is the best match to the search terms. Documents tagged with the Fishing dictionary are then ordered from highest-to-lowest match score against the Fishing dictionary. The Gardening dictionary appears next, and again the documents are ordered from highest-to-lowest match score against the gardening dictionary. In the non-hierarchical retrieval example, the order of the document within a given dictionary is independent of the search terms. Only the set of dictionaries and the order in which they are presented changes as the search terms change. For example, for search terms “worm” and “shovel” the results are:
-
Gardening<docE,docD,docF>,Fishing<docA,docB,docC,docD> - The order of the dictionaries differs from the results for “Crawdad” and “Worm” in the example above, but the order of the documents for the associated dictionaries is the same.
- The hierarchical reordering of the per-dictionary results in
step 68 ofFIG. 7 modifies the order of the documents for the corresponding dictionaries to improve the result of the search. A newly calculated SDP score SDPNEW is computed from the original SDP score using the affinity values described above. For example, if the search terms are “Crawdad” and “Worm” the SDP scores for these terms in the dictionaries in which they appear will be increased according to the affinity between the terms “Crawdad” and “Worm” in the hierarchy of the particular dictionary. In the instant example, since only one term matches in the Gardening dictionary there will be no adjustment of match score for any document against the gardening dictionary. However, in the Fishing dictionary the search terms will be adjusted using the calculation shown above, yielding the following result (assuming α=1): -
Crawdad SDP NEW=18 -
Worm SDP NEW=6 - For the subject of Fishing the scores of the documents can be calculated as follows:
-
DOC score=sum(SPD*percent of occurrence) -
docA=6*0.1+7*0.1+6*0.2+4*(0.1)=2.9 -
docB=18*0.1+6*0.1+4*0.1+2(0.1)=3.0 -
docC=18*0.1+6*0.1=2.4 -
docD=6*0.1=0.6 - Therefore, for search terms “Crawdad” and “Worm” using hierarchical retrieval the result would be:
-
Fishing<docB,docA,docC,docD>,Gardening<docE,docD,docF> - In the above result, documents docB and docA have changed order demonstrating the effect of hierarchical retrieval. The score for docD has changed only against the fishing dictionary, not the gardening dictionary, since the gardening dictionary does not contain both search terms and thus the SDP scores for the term Worm is not adjusted.
- As noted above, portions of the present invention may be embodied in a computer program product, which may include firmware, an image in system memory or another memory/cache, or stored on a fixed or re-writable media such as an optical disc having computer-readable code stored thereon. Any combination of one or more computer-readable medium(s) may store a program in accordance with an embodiment of the invention. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
- In the context of the present application, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
- While the invention has been particularly shown and described with reference to the preferred embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form, and details may be made therein without departing from the spirit and scope of the invention.
Claims (18)
1. A computer-performed method of retrieving documents pertaining to one or more subjects from a collection of documents, the method comprising:
within a computer system, specifying the one or more subjects;
by at least one processor of the computer system, selecting dictionaries from a set of dictionaries uniquely associated with corresponding ones of the one or more subjects;
by the at least one processor, matching the one or more subjects to documents in the collection of documents according to strength of a relationship between the documents and the one or more subjects and hierarchical information that indicates an affinity between the selected dictionaries, to obtain a subset of the collection of documents relevant to the one or more subjects; and
returning, by the at least one processor, the subset of the collection of documents.
2. The method of claim 1 , wherein the strength of the relationship between the documents and the one or more subjects is determined from entries within the dictionaries having associated subject-determining power scores corresponding to descriptive terms within the entries, wherein an individual subject-determining-power score indicates the relative strength or weakness of the corresponding descriptive term with respect to the subject associated with a particular dictionary containing the entry, and wherein the matching generates the document scores using the subject-determining power scores.
3. The method of claim 2 , wherein the adjusting adjusts an order of the subset of documents returned by the matching according to the change in document scores.
4. The method of claim 2 , wherein the adjusting is performed by multiplying the subject-determining power scores by a factor of (1+α/D), where α is a tuning parameter and D is a distance indicating a degree of decreasing affinity within the hierarchy of the corresponding dictionary.
5. The method of claim 1 , further comprising computing, by the at least one processor, the affinity values according to a traversal distance between pairs of nodes of the tree structure corresponding to pairs of the terms or a difference in hierarchical levels of the terms within the tree structure.
6. The method of claim 1 , wherein the hierarchical information is organized as a tree structure having hierarchical levels corresponding to levels of classification, wherein a highest level of the hierarchy of classification is the subject associated with the dictionary and wherein the lowest level of the hierarchy of classification contains descriptive terms that do not describe a further sub-classification, with terms corresponding to sub-classifications of the subject disposed between the highest level of the hierarchy of classification and the lowest level of the hierarchy of classification.
7. A computer system comprising a processor for executing program instructions coupled to a memory for storing the program instructions, wherein the program instructions are program instructions for retrieving documents pertaining to one or more subjects from a collection of documents, and wherein the program instructions comprise:
program instructions that specify the one or more subjects;
program instructions that select from a set of dictionaries uniquely associated with corresponding ones of the one or more subjects;
program instructions that match the one or more subjects to documents in the collection of documents according to strength of a relationship between the documents and the one or more subjects and hierarchical information that indicates an affinity between the selected dictionaries, to obtain a subset of the collection of documents relevant to the one or more subjects; and
program instructions that return the subset of the collection of documents obtained by the matching.
8. The computer system of claim 7 , wherein the strength of the relationship between the documents and the one or more subjects is determined from entries within the dictionaries having associated subject-determining power scores corresponding to descriptive terms within the entries, wherein an individual subject-determining-power score indicates the relative strength or weakness of the corresponding descriptive term with respect to the subject associated with a particular dictionary containing the entry, and wherein the matching generates the document scores using the subject-determining power scores.
9. The computer system of claim 8 , wherein the program instructions that adjust change an order of the subset of documents returned by the program instructions that match according to the change in document scores.
10. The computer system of claim 8 , wherein the program instructions for that adjust multiply the subject-determining power scores by a factor of (1+α/D), where α is a tuning parameter and D is a distance indicating a degree of decreasing affinity within the hierarchy of the corresponding dictionary.
11. The computer system of claim 7 , further comprising program instructions that compute the affinity values according to a traversal distance between pairs of nodes of the tree structure corresponding to pairs of the terms or a difference in hierarchical levels of the terms within the tree structure.
12. The computer system of claim 7 wherein the hierarchical information is organized as a tree structure having hierarchical levels corresponding to levels of classification, wherein a highest level of the hierarchy of classification is the subject associated with the dictionary and wherein the lowest level of the hierarchy of classification contains descriptive terms that do not describe a further sub-classification, with terms corresponding to sub-classifications of the subject disposed between the highest level of the hierarchy of classification and the lowest level of the hierarchy of classification.
13. A computer program product comprising a computer-readable storage device storing program instructions for retrieving documents pertaining to one or more subjects from a collection of documents, wherein the program instructions comprise program instructions for:
specifying the one or more subjects;
selecting dictionaries from a set of dictionaries uniquely associated with corresponding ones of the one or more subjects;
matching the one or more subjects to documents in the collection of documents according to strength of a relationship between the documents and the one or more subjects and hierarchical information that indicates an affinity between the selected dictionaries, to obtain a subset of the collection of documents relevant to the one or more subjects; and
returning, by at least one processor, the subset of the collection of documents.
14. The computer program product of claim 13 , wherein the strength of the relationship between the documents and the one or more subjects is determined from entries within the dictionaries having associated subject-determining power scores corresponding to descriptive terms within the entries, wherein an individual subject-determining-power score indicates the relative strength or weakness of the corresponding descriptive term with respect to the subject associated with a particular dictionary containing the entry, and wherein the matching generates the document scores using the subject-determining power scores.
15. The computer program product of claim 14 , wherein the program instructions for adjusting change an order of the subset of documents returned by the program instructions for second matching according to the change in document scores.
16. The computer program product of claim 14 , wherein the program instructions for adjusting multiply the subject-determining power scores by a factor of (1+α/D), where α is a tuning parameter and D is a distance indicating a degree of decreasing affinity within the hierarchy of the corresponding dictionary.
17. The computer program product of claim 13 , further comprising program instructions for computing the affinity values according to a traversal distance between pairs of nodes of the tree structure corresponding to pairs of the terms or a difference in hierarchical levels of the terms within the tree structure.
18. The computer program product of claim 13 , wherein the hierarchical information is organized as a tree structure having hierarchical levels corresponding to levels of classification, wherein a highest level of the hierarchy of classification is the subject associated with the dictionary and wherein the lowest level of the hierarchy of classification contains descriptive terms that do not describe a further sub-classification, with terms corresponding to sub-classifications of the subject disposed between the highest level of the hierarchy of classification and the lowest level of the hierarchy of classification.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/180,702 US20160299967A1 (en) | 2013-11-12 | 2016-06-13 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/077,305 US9235638B2 (en) | 2013-11-12 | 2013-11-12 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
US14/854,767 US9430559B2 (en) | 2013-11-12 | 2015-09-15 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
US15/180,702 US20160299967A1 (en) | 2013-11-12 | 2016-06-13 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/854,767 Continuation US9430559B2 (en) | 2013-11-12 | 2015-09-15 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
Publications (1)
Publication Number | Publication Date |
---|---|
US20160299967A1 true US20160299967A1 (en) | 2016-10-13 |
Family
ID=53044717
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/077,305 Expired - Fee Related US9235638B2 (en) | 2013-11-12 | 2013-11-12 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
US14/854,767 Expired - Fee Related US9430559B2 (en) | 2013-11-12 | 2015-09-15 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
US15/180,702 Abandoned US20160299967A1 (en) | 2013-11-12 | 2016-06-13 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/077,305 Expired - Fee Related US9235638B2 (en) | 2013-11-12 | 2013-11-12 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
US14/854,767 Expired - Fee Related US9430559B2 (en) | 2013-11-12 | 2015-09-15 | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
Country Status (1)
Country | Link |
---|---|
US (3) | US9235638B2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106649563A (en) * | 2016-11-10 | 2017-05-10 | 新华三技术有限公司 | Method and device for constructing lexicon of website classification |
US9971828B2 (en) | 2013-05-10 | 2018-05-15 | International Business Machines Corporation | Document tagging and retrieval using per-subject dictionaries including subject-determining-power scores for entries |
US9971782B2 (en) | 2013-10-16 | 2018-05-15 | International Business Machines Corporation | Document tagging and retrieval using entity specifiers |
WO2018121153A1 (en) * | 2016-12-29 | 2018-07-05 | 北京国双科技有限公司 | Written judgment retrieval method and device |
Families Citing this family (107)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8930331B2 (en) | 2007-02-21 | 2015-01-06 | Palantir Technologies | Providing unique views of data based on changes or rules |
US9348499B2 (en) | 2008-09-15 | 2016-05-24 | Palantir Technologies, Inc. | Sharing objects that rely on local resources with outside servers |
US9547693B1 (en) | 2011-06-23 | 2017-01-17 | Palantir Technologies Inc. | Periodic database search manager for multiple data sources |
US9092482B2 (en) | 2013-03-14 | 2015-07-28 | Palantir Technologies, Inc. | Fair scheduling for mixed-query loads |
US8799240B2 (en) | 2011-06-23 | 2014-08-05 | Palantir Technologies, Inc. | System and method for investigating large amounts of data |
US8732574B2 (en) | 2011-08-25 | 2014-05-20 | Palantir Technologies, Inc. | System and method for parameterizing documents for automatic workflow generation |
US8504542B2 (en) | 2011-09-02 | 2013-08-06 | Palantir Technologies, Inc. | Multi-row transactions |
US9348677B2 (en) | 2012-10-22 | 2016-05-24 | Palantir Technologies Inc. | System and method for batch evaluation programs |
US9123086B1 (en) | 2013-01-31 | 2015-09-01 | Palantir Technologies, Inc. | Automatically generating event objects from images |
US10037314B2 (en) | 2013-03-14 | 2018-07-31 | Palantir Technologies, Inc. | Mobile reports |
US9965937B2 (en) | 2013-03-15 | 2018-05-08 | Palantir Technologies Inc. | External malware data item clustering and analysis |
US8788405B1 (en) | 2013-03-15 | 2014-07-22 | Palantir Technologies, Inc. | Generating data clusters with customizable analysis strategies |
US8868486B2 (en) | 2013-03-15 | 2014-10-21 | Palantir Technologies Inc. | Time-sensitive cube |
US10275778B1 (en) | 2013-03-15 | 2019-04-30 | Palantir Technologies Inc. | Systems and user interfaces for dynamic and interactive investigation based on automatic malfeasance clustering of related data in various data structures |
US8909656B2 (en) | 2013-03-15 | 2014-12-09 | Palantir Technologies Inc. | Filter chains with associated multipath views for exploring large data sets |
US8917274B2 (en) | 2013-03-15 | 2014-12-23 | Palantir Technologies Inc. | Event matrix based on integrated data |
US8937619B2 (en) | 2013-03-15 | 2015-01-20 | Palantir Technologies Inc. | Generating an object time series from data objects |
US8799799B1 (en) | 2013-05-07 | 2014-08-05 | Palantir Technologies Inc. | Interactive geospatial map |
US8713467B1 (en) | 2013-08-09 | 2014-04-29 | Palantir Technologies, Inc. | Context-sensitive views |
US8938686B1 (en) | 2013-10-03 | 2015-01-20 | Palantir Technologies Inc. | Systems and methods for analyzing performance of an entity |
US8924872B1 (en) | 2013-10-18 | 2014-12-30 | Palantir Technologies Inc. | Overview user interface of emergency call data of a law enforcement agency |
US9116975B2 (en) | 2013-10-18 | 2015-08-25 | Palantir Technologies Inc. | Systems and user interfaces for dynamic and interactive simultaneous querying of multiple data stores |
US9021384B1 (en) | 2013-11-04 | 2015-04-28 | Palantir Technologies Inc. | Interactive vehicle information map |
US8868537B1 (en) | 2013-11-11 | 2014-10-21 | Palantir Technologies, Inc. | Simple web search |
US9235638B2 (en) | 2013-11-12 | 2016-01-12 | International Business Machines Corporation | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
US9105000B1 (en) | 2013-12-10 | 2015-08-11 | Palantir Technologies Inc. | Aggregating data from a plurality of data sources |
US9727622B2 (en) | 2013-12-16 | 2017-08-08 | Palantir Technologies, Inc. | Methods and systems for analyzing entity performance |
US10356032B2 (en) | 2013-12-26 | 2019-07-16 | Palantir Technologies Inc. | System and method for detecting confidential information emails |
US8832832B1 (en) | 2014-01-03 | 2014-09-09 | Palantir Technologies Inc. | IP reputation |
US9483162B2 (en) | 2014-02-20 | 2016-11-01 | Palantir Technologies Inc. | Relationship visualizations |
US9727376B1 (en) | 2014-03-04 | 2017-08-08 | Palantir Technologies, Inc. | Mobile tasks |
US8935201B1 (en) | 2014-03-18 | 2015-01-13 | Palantir Technologies Inc. | Determining and extracting changed data from a data source |
US9857958B2 (en) | 2014-04-28 | 2018-01-02 | Palantir Technologies Inc. | Systems and user interfaces for dynamic and interactive access of, investigation of, and analysis of data objects stored in one or more databases |
US9009171B1 (en) | 2014-05-02 | 2015-04-14 | Palantir Technologies Inc. | Systems and methods for active column filtering |
US9619557B2 (en) | 2014-06-30 | 2017-04-11 | Palantir Technologies, Inc. | Systems and methods for key phrase characterization of documents |
US9535974B1 (en) | 2014-06-30 | 2017-01-03 | Palantir Technologies Inc. | Systems and methods for identifying key phrase clusters within documents |
US9256664B2 (en) * | 2014-07-03 | 2016-02-09 | Palantir Technologies Inc. | System and method for news events detection and visualization |
US9202249B1 (en) | 2014-07-03 | 2015-12-01 | Palantir Technologies Inc. | Data item clustering and analysis |
US9454281B2 (en) | 2014-09-03 | 2016-09-27 | Palantir Technologies Inc. | System for providing dynamic linked panels in user interface |
US9767172B2 (en) | 2014-10-03 | 2017-09-19 | Palantir Technologies Inc. | Data aggregation and analysis system |
US9501851B2 (en) | 2014-10-03 | 2016-11-22 | Palantir Technologies Inc. | Time-series analysis system |
US9785328B2 (en) | 2014-10-06 | 2017-10-10 | Palantir Technologies Inc. | Presentation of multivariate data on a graphical user interface of a computing system |
US9984133B2 (en) | 2014-10-16 | 2018-05-29 | Palantir Technologies Inc. | Schematic and database linking system |
US9229952B1 (en) | 2014-11-05 | 2016-01-05 | Palantir Technologies, Inc. | History preserving data pipeline system and method |
US9043894B1 (en) | 2014-11-06 | 2015-05-26 | Palantir Technologies Inc. | Malicious software detection in a computing system |
US20160162576A1 (en) * | 2014-12-05 | 2016-06-09 | Lightning Source Inc. | Automated content classification/filtering |
US10362133B1 (en) | 2014-12-22 | 2019-07-23 | Palantir Technologies Inc. | Communication data processing architecture |
US10552994B2 (en) | 2014-12-22 | 2020-02-04 | Palantir Technologies Inc. | Systems and interactive user interfaces for dynamic retrieval, analysis, and triage of data items |
US9367872B1 (en) | 2014-12-22 | 2016-06-14 | Palantir Technologies Inc. | Systems and user interfaces for dynamic and interactive investigation of bad actor behavior based on automatic clustering of related data in various data structures |
US9348920B1 (en) | 2014-12-22 | 2016-05-24 | Palantir Technologies Inc. | Concept indexing among database of documents using machine learning techniques |
US9817563B1 (en) | 2014-12-29 | 2017-11-14 | Palantir Technologies Inc. | System and method of generating data points from one or more data stores of data items for chart creation and manipulation |
US9335911B1 (en) | 2014-12-29 | 2016-05-10 | Palantir Technologies Inc. | Interactive user interface for dynamic data analysis exploration and query processing |
US9870205B1 (en) | 2014-12-29 | 2018-01-16 | Palantir Technologies Inc. | Storing logical units of program code generated using a dynamic programming notebook user interface |
US10372879B2 (en) | 2014-12-31 | 2019-08-06 | Palantir Technologies Inc. | Medical claims lead summary report generation |
US9727560B2 (en) | 2015-02-25 | 2017-08-08 | Palantir Technologies Inc. | Systems and methods for organizing and identifying documents via hierarchies and dimensions of tags |
EP3070622A1 (en) | 2015-03-16 | 2016-09-21 | Palantir Technologies, Inc. | Interactive user interfaces for location-based data analysis |
US9886467B2 (en) | 2015-03-19 | 2018-02-06 | Plantir Technologies Inc. | System and method for comparing and visualizing data entities and data entity series |
US9392008B1 (en) | 2015-07-23 | 2016-07-12 | Palantir Technologies Inc. | Systems and methods for identifying information related to payment card breaches |
US9454785B1 (en) | 2015-07-30 | 2016-09-27 | Palantir Technologies Inc. | Systems and user interfaces for holistic, data-driven investigation of bad actor behavior based on clustering and scoring of related data |
US9996595B2 (en) | 2015-08-03 | 2018-06-12 | Palantir Technologies, Inc. | Providing full data provenance visualization for versioned datasets |
US9456000B1 (en) | 2015-08-06 | 2016-09-27 | Palantir Technologies Inc. | Systems, methods, user interfaces, and computer-readable media for investigating potential malicious communications |
US10489391B1 (en) | 2015-08-17 | 2019-11-26 | Palantir Technologies Inc. | Systems and methods for grouping and enriching data items accessed from one or more databases for presentation in a user interface |
US9600146B2 (en) | 2015-08-17 | 2017-03-21 | Palantir Technologies Inc. | Interactive geospatial map |
US10853378B1 (en) | 2015-08-25 | 2020-12-01 | Palantir Technologies Inc. | Electronic note management via a connected entity graph |
US11150917B2 (en) | 2015-08-26 | 2021-10-19 | Palantir Technologies Inc. | System for data aggregation and analysis of data from a plurality of data sources |
US9485265B1 (en) | 2015-08-28 | 2016-11-01 | Palantir Technologies Inc. | Malicious activity detection system capable of efficiently processing data accessed from databases and generating alerts for display in interactive user interfaces |
US10706434B1 (en) | 2015-09-01 | 2020-07-07 | Palantir Technologies Inc. | Methods and systems for determining location information |
US9576015B1 (en) | 2015-09-09 | 2017-02-21 | Palantir Technologies, Inc. | Domain-specific language for dataset transformations |
US10296617B1 (en) | 2015-10-05 | 2019-05-21 | Palantir Technologies Inc. | Searches of highly structured data |
US9424669B1 (en) | 2015-10-21 | 2016-08-23 | Palantir Technologies Inc. | Generating graphical representations of event participation flow |
US20170116194A1 (en) * | 2015-10-23 | 2017-04-27 | International Business Machines Corporation | Ingestion planning for complex tables |
US10613722B1 (en) | 2015-10-27 | 2020-04-07 | Palantir Technologies Inc. | Distorting a graph on a computer display to improve the computer's ability to display the graph to, and interact with, a user |
US10726075B2 (en) * | 2015-11-09 | 2020-07-28 | Imi: Intelligence & Management Of Information Inc. | Streamlining and searching document text |
US9542446B1 (en) | 2015-12-17 | 2017-01-10 | Palantir Technologies, Inc. | Automatic generation of composite datasets based on hierarchical fields |
US9823818B1 (en) | 2015-12-29 | 2017-11-21 | Palantir Technologies Inc. | Systems and interactive user interfaces for automatic generation of temporal representation of data objects |
US10268735B1 (en) | 2015-12-29 | 2019-04-23 | Palantir Technologies Inc. | Graph based resolution of matching items in data sources |
US9612723B1 (en) | 2015-12-30 | 2017-04-04 | Palantir Technologies Inc. | Composite graphical interface with shareable data-objects |
CN105683967A (en) * | 2016-01-30 | 2016-06-15 | 深圳市博信诺达经贸咨询有限公司 | Web page grabbing method and web page grabbing system based on big data |
US10698938B2 (en) | 2016-03-18 | 2020-06-30 | Palantir Technologies Inc. | Systems and methods for organizing and identifying documents via hierarchies and dimensions of tags |
CN105874458A (en) * | 2016-03-30 | 2016-08-17 | 马岩 | Method and system for analyzing network information |
US10650558B2 (en) | 2016-04-04 | 2020-05-12 | Palantir Technologies Inc. | Techniques for displaying stack graphs |
US10007674B2 (en) | 2016-06-13 | 2018-06-26 | Palantir Technologies Inc. | Data revision control in large-scale data analytic systems |
US10628522B2 (en) * | 2016-06-27 | 2020-04-21 | International Business Machines Corporation | Creating rules and dictionaries in a cyclical pattern matching process |
CN106126711B (en) * | 2016-06-30 | 2019-11-01 | 北京奇虎科技有限公司 | Encyclopaedia entry classification method and device |
US10719188B2 (en) | 2016-07-21 | 2020-07-21 | Palantir Technologies Inc. | Cached database and synchronization system for providing dynamic linked panels in user interface |
US10324609B2 (en) | 2016-07-21 | 2019-06-18 | Palantir Technologies Inc. | System for providing dynamic linked panels in user interface |
US10437840B1 (en) | 2016-08-19 | 2019-10-08 | Palantir Technologies Inc. | Focused probabilistic entity resolution from multiple data sources |
US9881066B1 (en) | 2016-08-31 | 2018-01-30 | Palantir Technologies, Inc. | Systems, methods, user interfaces and algorithms for performing database analysis and search of information involving structured and/or semi-structured data |
US10318630B1 (en) | 2016-11-21 | 2019-06-11 | Palantir Technologies Inc. | Analysis of large bodies of textual data |
US10460602B1 (en) | 2016-12-28 | 2019-10-29 | Palantir Technologies Inc. | Interactive vehicle information mapping system |
EP3343403A1 (en) | 2016-12-28 | 2018-07-04 | Palantir Technologies Inc. | Systems and methods for retrieving and processing data for display |
US10475219B1 (en) | 2017-03-30 | 2019-11-12 | Palantir Technologies Inc. | Multidimensional arc chart for visual comparison |
CN108733686A (en) * | 2017-04-17 | 2018-11-02 | 伊姆西Ip控股有限责任公司 | Information processing method and equipment |
US11182443B2 (en) * | 2017-06-07 | 2021-11-23 | Sap Se | Context-sensitive document search based on hierarchical tags |
US10956406B2 (en) | 2017-06-12 | 2021-03-23 | Palantir Technologies Inc. | Propagated deletion of database records and derived data |
US10455087B2 (en) * | 2017-06-15 | 2019-10-22 | Microsoft Technology Licensing, Llc | Information retrieval using natural language dialogue |
US10403011B1 (en) | 2017-07-18 | 2019-09-03 | Palantir Technologies Inc. | Passing system with an interactive user interface |
US10929476B2 (en) | 2017-12-14 | 2021-02-23 | Palantir Technologies Inc. | Systems and methods for visualizing and analyzing multi-dimensional data |
US11016959B2 (en) * | 2018-01-31 | 2021-05-25 | Salesforce.Com, Inc. | Trie-based normalization of field values for matching |
US11599369B1 (en) | 2018-03-08 | 2023-03-07 | Palantir Technologies Inc. | Graphical user interface configuration system |
US10754822B1 (en) | 2018-04-18 | 2020-08-25 | Palantir Technologies Inc. | Systems and methods for ontology migration |
US10885021B1 (en) | 2018-05-02 | 2021-01-05 | Palantir Technologies Inc. | Interactive interpreter and graphical user interface |
US10884644B2 (en) * | 2018-06-28 | 2021-01-05 | Amazon Technologies, Inc. | Dynamic distributed data clustering |
US11120052B1 (en) | 2018-06-28 | 2021-09-14 | Amazon Technologies, Inc. | Dynamic distributed data clustering using multi-level hash trees |
US10860809B2 (en) * | 2019-04-09 | 2020-12-08 | Sas Institute Inc. | Word embeddings and virtual terms |
CN112017759A (en) * | 2019-05-31 | 2020-12-01 | 通用电气精准医疗有限责任公司 | Method and system for selecting scanning protocol and medium storing corresponding program |
CN111209378B (en) * | 2019-12-26 | 2024-03-12 | 航天信息股份有限公司企业服务分公司 | Ordered hierarchical ordering method based on business dictionary weights |
Family Cites Families (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6038561A (en) | 1996-10-15 | 2000-03-14 | Manning & Napier Information Services | Management and analysis of document information text |
US6098066A (en) | 1997-06-13 | 2000-08-01 | Sun Microsystems, Inc. | Method and apparatus for searching for documents stored within a document directory hierarchy |
US6473730B1 (en) | 1999-04-12 | 2002-10-29 | The Trustees Of Columbia University In The City Of New York | Method and system for topical segmentation, segment significance and segment function |
US6711585B1 (en) | 1999-06-15 | 2004-03-23 | Kanisa Inc. | System and method for implementing a knowledge management system |
WO2001090921A2 (en) | 2000-05-25 | 2001-11-29 | Kanisa, Inc. | System and method for automatically classifying text |
US8396859B2 (en) | 2000-06-26 | 2013-03-12 | Oracle International Corporation | Subject matter context search engine |
US7185001B1 (en) | 2000-10-04 | 2007-02-27 | Torch Concepts | Systems and methods for document searching and organizing |
US6961723B2 (en) | 2001-05-04 | 2005-11-01 | Sun Microsystems, Inc. | System and method for determining relevancy of query responses in a distributed network search mechanism |
JP4215425B2 (en) | 2001-11-21 | 2009-01-28 | 日本電気株式会社 | Text management system, management method thereof, and program thereof |
US7676462B2 (en) | 2002-12-19 | 2010-03-09 | International Business Machines Corporation | Method, apparatus, and program for refining search criteria through focusing word definition |
US20060167930A1 (en) | 2004-10-08 | 2006-07-27 | George Witwer | Self-organized concept search and data storage method |
CN100462961C (en) | 2004-11-09 | 2009-02-18 | 国际商业机器公司 | Method for organizing multi-file and equipment for displaying multi-file |
EP1846815A2 (en) * | 2005-01-31 | 2007-10-24 | Textdigger, Inc. | Method and system for semantic search and retrieval of electronic documents |
US20080082486A1 (en) | 2006-09-29 | 2008-04-03 | Yahoo! Inc. | Platform for user discovery experience |
US20090006391A1 (en) | 2007-06-27 | 2009-01-01 | T Reghu Ram | Automatic categorization of document through tagging |
US9081852B2 (en) | 2007-10-05 | 2015-07-14 | Fujitsu Limited | Recommending terms to specify ontology space |
US8019748B1 (en) | 2007-11-14 | 2011-09-13 | Google Inc. | Web search refinement |
US7895181B2 (en) | 2008-02-01 | 2011-02-22 | Intuit Inc. | Configuration-based search |
US8914363B2 (en) | 2008-05-22 | 2014-12-16 | International Business Machines Corporation | Disambiguating tags in network based multiple user tagging systems |
US8214346B2 (en) | 2008-06-27 | 2012-07-03 | Cbs Interactive Inc. | Personalization engine for classifying unstructured documents |
KR101045955B1 (en) | 2008-11-14 | 2011-07-04 | 한국과학기술정보연구원 | Method for extracting semantic correlation of context, and recording device storing device and program source thereof |
US8639682B2 (en) | 2008-12-29 | 2014-01-28 | Accenture Global Services Limited | Entity assessment and ranking |
US20100198816A1 (en) | 2009-01-30 | 2010-08-05 | Yahoo! Inc. | System and method for presenting content representative of document search |
KR101048540B1 (en) | 2009-03-24 | 2011-07-11 | 엔에이치엔(주) | Apparatus and method for classifying search keywords using clusters according to related keywords |
US8719249B2 (en) | 2009-05-12 | 2014-05-06 | Microsoft Corporation | Query classification |
US8180783B1 (en) | 2009-05-13 | 2012-05-15 | Softek Solutions, Inc. | Document ranking systems and methods |
US20110125791A1 (en) | 2009-11-25 | 2011-05-26 | Microsoft Corporation | Query classification using search result tag ratios |
US8756231B2 (en) | 2010-01-28 | 2014-06-17 | International Business Machines Corporation | Search using proximity for clustering information |
GB201010545D0 (en) * | 2010-06-23 | 2010-08-11 | Rolls Royce Plc | Entity recognition |
US8600979B2 (en) | 2010-06-28 | 2013-12-03 | Yahoo! Inc. | Infinite browse |
US9798800B2 (en) | 2010-09-24 | 2017-10-24 | International Business Machines Corporation | Providing question and answers with deferred type evaluation using text with limited structure |
US9928296B2 (en) * | 2010-12-16 | 2018-03-27 | Microsoft Technology Licensing, Llc | Search lexicon expansion |
US9665643B2 (en) | 2011-12-30 | 2017-05-30 | Microsoft Technology Licensing, Llc | Knowledge-based entity detection and disambiguation |
US9262510B2 (en) | 2013-05-10 | 2016-02-16 | International Business Machines Corporation | Document tagging and retrieval using per-subject dictionaries including subject-determining-power scores for entries |
US9251136B2 (en) | 2013-10-16 | 2016-02-02 | International Business Machines Corporation | Document tagging and retrieval using entity specifiers |
US9235638B2 (en) | 2013-11-12 | 2016-01-12 | International Business Machines Corporation | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results |
-
2013
- 2013-11-12 US US14/077,305 patent/US9235638B2/en not_active Expired - Fee Related
-
2015
- 2015-09-15 US US14/854,767 patent/US9430559B2/en not_active Expired - Fee Related
-
2016
- 2016-06-13 US US15/180,702 patent/US20160299967A1/en not_active Abandoned
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9971828B2 (en) | 2013-05-10 | 2018-05-15 | International Business Machines Corporation | Document tagging and retrieval using per-subject dictionaries including subject-determining-power scores for entries |
US9971782B2 (en) | 2013-10-16 | 2018-05-15 | International Business Machines Corporation | Document tagging and retrieval using entity specifiers |
CN106649563A (en) * | 2016-11-10 | 2017-05-10 | 新华三技术有限公司 | Method and device for constructing lexicon of website classification |
WO2018121153A1 (en) * | 2016-12-29 | 2018-07-05 | 北京国双科技有限公司 | Written judgment retrieval method and device |
CN108255862A (en) * | 2016-12-29 | 2018-07-06 | 北京国双科技有限公司 | The search method and device of a kind of judgement document |
US11288326B2 (en) * | 2016-12-29 | 2022-03-29 | Beijing Gridsum Technology Co., Ltd. | Retrieval method and device for judgment documents |
Also Published As
Publication number | Publication date |
---|---|
US9430559B2 (en) | 2016-08-30 |
US9235638B2 (en) | 2016-01-12 |
US20160004704A1 (en) | 2016-01-07 |
US20150134666A1 (en) | 2015-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9430559B2 (en) | Document retrieval using internal dictionary-hierarchies to adjust per-subject match results | |
US7672943B2 (en) | Calculating a downloading priority for the uniform resource locator in response to the domain density score, the anchor text score, the URL string score, the category need score, and the link proximity score for targeted web crawling | |
US7984035B2 (en) | Context-based document search | |
KR101793222B1 (en) | Updating a search index used to facilitate application searches | |
KR101076894B1 (en) | System and method for incorporating anchor text into ranking search results | |
US8095539B2 (en) | Taxonomy-based object classification | |
US8725732B1 (en) | Classifying text into hierarchical categories | |
JP4944406B2 (en) | How to generate document descriptions based on phrases | |
US6738678B1 (en) | Method for ranking hyperlinked pages using content and connectivity analysis | |
US8266144B2 (en) | Techniques to perform relative ranking for search results | |
US20170371923A1 (en) | Template-driven structured query generation | |
Park et al. | Keyword search in relational databases | |
US9396188B2 (en) | Assigning tags to digital content | |
US20120117050A1 (en) | Creation and enrichment of search based taxonomy for finding information from semistructured data | |
US20100114855A1 (en) | Method and system for automatic objects classification | |
US9971828B2 (en) | Document tagging and retrieval using per-subject dictionaries including subject-determining-power scores for entries | |
US9971782B2 (en) | Document tagging and retrieval using entity specifiers | |
Kaur et al. | IHWC: intelligent hidden web crawler for harvesting data in urban domains | |
US20110295861A1 (en) | Searching using taxonomy | |
Zhang | Start small, build complete: Effective and efficient semantic table interpretation using tableminer | |
Kaur et al. | SmartCrawler: A Three-Stage Ranking Based Web Crawler for Harvesting Hidden Web Sources. | |
Ramya et al. | Automatic extraction of facets for user queries [AEFUQ] | |
Kahng et al. | Exploiting paths for entity search in rdf graphs | |
Phinitkar et al. | Personalization of search profile using ant foraging approach | |
Parimala et al. | Enhanced Performance of Search Engine with Multi-Type Feature Co-Selection for Clustering Algorithm |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GATTIKER, ANNE E.;GEBARA, FADI H.;HYLICK, ANTHONY N.;AND OTHERS;SIGNING DATES FROM 20131106 TO 20131108;REEL/FRAME:038897/0749 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |