DE102014204830A1 - Computer-implemented systems and methods for comparing and associating objects - Google Patents

Abstract

Computer implemented systems and methods for comparing and associating objects are disclosed. In some embodiments, a method is provided to associate a first object with one or more objects within a plurality of objects, each object having a first plurality of properties, each property having data reflecting an identifier of an entity represented by the object wherein the associated objects have matching data in corresponding properties for a second plurality of properties. The method may include performing, for each object within the plurality of objects and for the first object: creating a slug for the object, the slug having the second plurality of properties of the object; and entering the slug for the object into a bloom filter. The method can further include that, for a position in the Bloom filter that corresponds to the slug for the first object, an association is generated between objects whose slugs correspond to the position, if the slugs for these objects match.

Description

This application claims the benefit of US Provisional Patent Application No. 61 / 801,297, filed on Mar. 15, 2013, and US Patent Application No. 14 / 099,661, filed on Dec. 6, 2013, the disclosures of which are incorporated herein by reference in their entireties this document.

Numerous organizations, including industry and government, recognize that important conclusions can be drawn when analyzing huge data sets to identify patterns of behavior that threaten public safety or prove illegal activity. These analyzes often involve aligning data associated with a person or thing of interest with other data associated with the same person or thing to determine that the same person or thing was involved in multiple acts that have safety concerns or raise criminal concerns.

However, the quality of the analytical result resulting from the use of technically advanced analysis tools may be limited by the quality of the data the tool uses. For some types of analysis, an acceptable error rate must be literal or near zero, so that an analytical conclusion drawn from the data is well-founded. Achieving this zero or near-zero error rate for records that include a two or three digit million value of individual records may be problematic. Current data comparison tools are not well suited to solve these problems.

The problems discussed above are particularly severe for analyzes involving data related to identifying persons or things for public safety investigations. For example, with analysis tools for identifying potential security threats, the acceptable error rate is generally no greater than zero, since the price that is erroneously recognized as having a security threat (ie, "false-positive") or allowed to fail to detect a security threat (ie "False-negative"), unacceptably high. Therefore, it is necessary that tools serving public safety tools correctly relate data associated with persons or things of interest to data related to the same person or thing.

There are some tools to do a precise data comparison, but with datasets that contain millions of individual entries, they are virtually unusable for computer evaluation. For example, one solution to determining whether two particular objects are associated with the same person or thing of interest is to compare each element of an object to a corresponding element in the second object. For example, for objects containing M elements, a first element in the first object may be compared to a corresponding first element in the second object, and corresponding comparisons may be made for each of the remaining M-1 elements that are common to the first and second objects are. If the elements in each object are altogether suitable for unambiguously identifying the represented person or thing, and corresponding elements in the first and second objects agree, it can reasonably be inferred that the objects reflect the same person or thing. Alternatively, each object could be converted (serialized) into a single string reflecting the contents of each item to be compared. Thereafter, a string generated from the one object could be compared with a string generated from another object as a form of object comparison.

For some data sets, it is possible that the approaches described above consume little memory or system resources, since the objects or their serialized strings can be stored on disk instead of main memory. However, the solution approaches described above can quickly no longer be used with large or non-trivial data sets. As the number of objects to compare increases, the number of comparisons, and hence the processing time of the comparisons, increases exponentially; ie proportional to n ^2/2 , where n represents the number of objects to be compared. Thus, a comparison of 500 objects using a serialized approach whose processing time can be approximately estimated as the time to perform 125,000 string comparisons can be accomplished by computer evaluation. However, a comparison of 100 million (100M) data sets using this approach, whose computation time may be approximated as the time to perform 5 quadrillion (5e15) string comparisons, can be difficult to accomplish by computer evaluation. In addition, reading strings from a disk instead of reading them from memory may add extra computation time.

Another solution for identifying matching objects in a corpus of objects is to store each object in a multi-map. This multimap is an associative data field that stores multiple values for each key. Importing the objects into the Multimap results in objects with the same element data being stored in a single entry of the Multimap. Thus, by using a multi-map, an association of identical objects occurs.

A disadvantage of using a multimap for object comparisons is that the multimap is typically stored in main memory, due to algorithm-related considerations concerning organizing keys in the multimap, and therefore an object comparator must have sufficient main memory to hold the memory entire body to hold comprehensive multimap in memory. Therefore, a multimap solution may be impossible for records of 100M objects or more. Similar disadvantages exist for any approach when applied to other object matching problems, such as efficiently identifying unique (unique) objects in a corpus of objects, and efficiently comparing a single object to all objects in a corpus of objects.

None of the solutions is viable for datasets that include approximately or more than 100M objects. However, object data sets that include 100M or more objects are not uncommon nowadays. Therefore, the problems described here are quite real, and there is a need for improved object comparators.

The present invention is set forth in the independent claims. The dependent claims relate to optional features of some embodiments of the invention.

Reference will now be made to the accompanying drawings, which illustrate exemplary embodiments of the present application, wherein:

1 FIG. 10 is a flowchart of an example process for comparing a target object to at least some objects in a corpus, according to some embodiments of the present disclosure.

2 FIG. 12 shows a flow chart of an example process for comparing all objects in a corpus with all other objects in the corpus to determine matches within the corpus according to some embodiments of the present disclosure.

3 FIG. 5 is a flow chart of an example process for comparing all objects in a corpus with all other objects in the corpus to determine unique objects within the corpus, in accordance with some embodiments of the present disclosure.

4 FIG. 10 illustrates an exemplary computing environment within which embodiments of the present disclosure may be implemented.

Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Whenever possible, like reference numbers will be used throughout the drawings to refer to the same or like parts.

Embodiments of the present disclosure can avoid the disadvantages of conventional object comparators by providing computer-implemented systems and methods to compare objects in a manner that allows for greater computational throughput and storage consumption without compromising comparison accuracy, and for dataset sizes that previously not or not at an acceptable level of computational throughput.

Embodiments of the present disclosure relate to a class of computational problems related to object comparison. An element of this class includes an efficient object comparison of a particular object with a corpus of objects. Another element of this class involves an efficient comparison of each object in a corpus with all other objects in the corpus. An additional element of this class involves efficient identification of unique objects in a corpus of objects.

The following detailed description begins with a general overview of an object comparison. Some examples of objects to be compared or analyzed are provided. The description then illustrates an exemplary embodiment that addresses the first problem class discussed above (ie, efficiently comparing a single object to all objects in a corpus). The description then extends the solution to the first problem class to address the second problem class discussed above (ie, an efficient comparison of each object in a corpus with all other objects in the corpus). The detailed description then discloses a solution to the third problem class (ie efficient identification of unique objects in a corpus of objects). An introduction to objects and an overview of object comparisons follows.

Several types of objects exist in the field of computer science. An object type well known in the field of computer science is an object in the object-oriented sense. Wikipedia describes an object of this type as a set of elements (i.e., data structures) and methods that are similar to functions. Without necessarily following this simplistic description, embodiments implementing the object comparison solutions discussed herein are compatible with comparing objects of this type.

Another object type in the field of computer science is a data structure that reflects the characteristics of a person or thing that is relevant to a particular task or computing environment. In some embodiments, these properties are reflected by strings. In further embodiments, properties may be reflected by strings, integers, real numbers, time or dates, binary values, structures in the sense of C programming, variable enumerations, and / or other data forms. In some embodiments, properties of any object type may be converted to strings before being compared. In other embodiments, some properties may be strings or may possibly be converted to strings, however, other properties may not be strings and may not be converted into strings. The embodiments of the present disclosure may operate with string or non-string properties.

Moreover, the notion of a "data structure" in this context is very flexible. The term "data structure" may reflect any type of structured data, namely information stored in a database (with table columns reflecting elements in an object or data structure and table rows reflecting instances of the object or data structure), and to formatted text in a text file (for example, data in an XML structure), to data stored in a running computer program. Accordingly, since a data structure roughly includes the above-described types of structured data, objects also roughly include these types of structured data. In addition, the object comparison solutions discussed herein are also compatible with comparing objects of these types.

In some embodiments, an effective object comparison involves taking into account which properties of the objects to be compared are relevant to performing the comparison, since the entities (e.g., persons or things) reflected by those objects are different Environments may have different relevant properties. For example, an object may store properties of an automobile that may be relevant to a state vehicle authority by storing the following information: vehicle identification number (VIN), year of manufacture, make, model, expiration date of the vehicle registration, and a direct or indirect indication of the person; the owner of the vehicle is.

However, for automobiles sold on an auction site such as eBay, the relevant characteristics of an automobile may differ from those relevant to the state's motor vehicle office. For example, a data structure for storing properties of an automobile listed for sale on eBay may include: VIN, year, make, model, mileage, condition of the automobile, minimum auction bid, and a direct or indirect indication of the person providing the Has stopped selling vehicle. Thus, properties of an entity (eg, a person or thing) that is relevant to the one environment may differ from properties of the entity that are relevant to another environment. Accordingly, properties of an object considered during an object comparison in one environment may be different than those considered during object comparison in a second environment.

In some embodiments, effective data comparison may also include taking into account which properties are intended to distinguish an entity (eg, a person or thing) from other instances of the entity. For example, as designed, a VIN for an automobile should be unique to this automobile. However, occasionally situations may arise in which a VIN is not unique to a particular automobile. Such situations may arise from deliberate errors or accidental errors. An example of a deliberate error is an attempt to fraudulently register a stolen vehicle under a purported VIN. An example of a random error occurs when a smog-scoped employee improperly enters a VIN into a computer at a smog review station, resulting in a smog-check record with incorrect VIN, which is subsequently passed to a state database becomes. data error However, some embodiments of the present disclosure perform minimization or elimination of errors by identifying objects using a combination of multiple object properties, rather than identifying objects using a single object property.

In some embodiments, one or more identifying characteristics of an object are extracted from the object and stored in a data structure. This data structure is called a "slug"; it contains information that may be sufficient to uniquely identify an entity (eg, a person or thing) with some degree of information redundancy to enable detection of errors in the properties in the slug. In some embodiments, the slug has a concatenation of strings separated by a delimiter. In some embodiments, the separator is a NULL character, however, in other embodiments, the separator may be a character that otherwise does not exist in the concatenated string. In some embodiments, the concatenated strings may be delimited by a separator string (eg, "-") instead of a separator. In embodiments that use a separator string, the separator string may be any string that does not otherwise appear in the strings that were concatenated. In further embodiments, the slug has a data structure such as an object, a data array, a structure, or an associative array.

For example, in one embodiment, a slug for an automobile may include features that include a VIN, a brand, a model, and a year for the automobile. Because the brand, model, and year properties for the automobile are included in the slug, an ability to detect errors in the VIN property is provided because the VIN property is not the only object property that is compared. In order for slugs associated with two automobiles to match in the presence of a defect in the VIN property of the one automobile object, an automotive object having the same VIN property as the defective VIN must also have the same make, model, and model. and year features.

The chances of this coincident coincidence of several properties between two or more objects can be negligible. Therefore, including some level of information redundancy should avoid, or at least significantly reduce, erroneous matches in an object comparison with respect to object comparisons in which only a single property is compared between objects, despite the fact that the individual property was intended to be its own Entity (eg person or thing) clearly identified.

Exemplary embodiments will be described below which solve the first problem explained above, that is, the first problem described above. H. efficiently comparing a specific object (hereinafter referred to as "target object") with all objects in a corpus. The disclosed embodiments use a bloom filter to identify slugs associated with objects in the corpus that do not match the slug for the target object. This quick detection is performed by discarding slugs that are associated with a position in the bloom filter that is different from the position associated with the slug for the target object.

Bloom filters have the property that two slugs that fall into different positions in the bloom filter, certainly have different properties and thus reflect different objects. Therefore, if the slug for the target does not fall into the same position as the slug for a particular object in the body, the target does not match the particular object in the body and thus may be excluded from future consideration in such embodiments.

1 shows a flowchart of an exemplary process 100 for comparing a target object with at least some objects in a body, in accordance with some embodiments of the present disclosure. In some embodiments, the target object to be compared with at least some objects in the corpus is part of the corpus. In these embodiments, a comparison is made between the target object and all other objects in the corpus. In further embodiments, the object to be compared with at least some objects in the body is not part of the body. In these further embodiments, a comparison is made between the target object and all objects in the corpus.

As shown, at step 102 determines the size of a bloom filter and creates it taking into account the error rate that results for the body size to be processed. For example, increasing the number of positions in a bloom filter may aim to reduce the error rate for a specific body size, while decreasing the number of locations in a bloom filter may aim to increase the error rate for a specific body size. Methods for sizing a bloom filter to achieving a desired error rate for a specific body size is well known in the art, and therefore these methods are not discussed here.

At step 104 a slug is created for the target object (ie the object against which all objects in the corpus are compared). Considerations as to which properties to include in a slug for an object have been discussed previously. At step 106 a Bloom Filter position corresponding to the slug for the target object is determined. In some embodiments, a bloom filter position for a slug may be determined by entering the slug into a bloom filter and instructing the bloom filter to reveal the position in which the slug was added.

In further embodiments, a bloom filter position for a slug may be determined by presenting the slug as an input to a software function associated with the bloom filter without storing the slug in the bloom filter. In further embodiments, a position for a slug may be determined by entering the slug into a software function that reflects a position selection algorithm for a bloom filter, in the absence of using an actual and / or full bloom filter, and the bloom filter position as the output thereof Software function is obtained. In other embodiments, other approaches may be used to provide a bloom position from a slug. These approaches to identifying a bloom filter for a slug, according to the previously discussed embodiments, will be described in total in steps 106 . 108 Referenced. The particular Bloom Filter position is used to identify matches of the Slug Comparison, some of which may be "false positive" using the Bloom Filter discussed below.

At step 108 a slug is created for each object in the corpus. At step 110 a Bloom Filter position is determined for each object in the corpus. In some embodiments, a bloom filter position for an object may be determined by entering the slug of the object into the bloom filter and instructing the bloom filter to reveal the position to which the slug was added.

After completing step 110 reflect slugs that step by step 108 Identified position, matches with the slug for the target object. However, some of these matches may be false-positive matches rather than true matches. Therefore, steps filter 112 and 114 these false-positive matches by using a multimap.

In step 112 is added for each slug associated with an object in the corpus and whose position in the bloom filter is the same position as the slug for the target object, the slug associated with an object in the corpus and its associated object in the corpus of a multimap. When adding the slug and its associated object to the multimap, the slug represents the key to the multimap, and the object in the corpus represents the value to the multimap. This multimap is used to remove false-positive results from processing. In step 114 For example, the method is completed by selecting the true-positive matches identified in the multi-map. These non-false-positive matches can be retrieved from the multimap by reading out data from the multimap with the slug for the target object as a key.

In some embodiments, the process may 100 distributed across multiple processors. For example, a bloom filter may be present on each of multiple processors, and steps 102 to 114 can be run on any of the multiple processors. The corpus of objects can be distributed among the various processors so that all objects are processed by a processor, but no object is processed by more than one processor. In such embodiments, each of the plurality of processors performs outputting of a portion of the object matching objects in the corpus.

Exemplary embodiments will be described below which solve the second problem explained above, that is, the second embodiment of the present invention. H. an efficient comparison of all objects with all objects in a corpus. These embodiments use a counting Bloom filter to quickly identify slugs that are associated with objects in the corpus that do not match the slug for the target object. Counting bloom filters are well known in the art, and therefore their structure and construction will not be discussed here.

Specifically, if a position in the counting Bloom filter has a value of zero or one, after slugs have been entered into the Bloom filter for all objects in the corpus, no object whose slug is associated with that position could match another Slug, and therefore These slugs are excluded from further consideration. These slugs can be excluded because it is clear to those skilled in the art that bloom filters can have false-positive results, but they are not false-positive. can have negative results. Therefore, a counting Bloom filter whose count value is less than two reflects an accurate determination that there is no match between slugs associated with that position since any match would produce a count of at least two. However, false-positive results can occur among objects whose slugs are associated with the same bloom-filter position, and therefore false-positive results can be removed by additional processing, as will be explained later.

2 shows a flowchart of an exemplary process 200 compare all objects in a body with all other objects in the body to determine matches within the body, in accordance with some embodiments of the present disclosure. As shown, in step 202 Sets the size of a scoring Bloom filter and creates it taking into account the error rate that results for the body size to be processed. For example, increasing the number of positions in a counting bloom filter may aim to reduce the error rate for a specific body size, while decreasing the number of positions in a counting bloom filter may aim to increase the error rate for a specific body size. Methods for sizing a scoring bloom filter to achieve a desired error rate for a specific body size are well known in the art, and therefore, these methods are not discussed here.

In some embodiments, the counting bloom filter may include an N-bit counter, and these counters may be implemented as a two-bit counter (i.e., N = 2). In further embodiments, these counters may be one-bit counters or more than two-bit counters. In still other embodiments, these counters may be saturation counters; d. H. these counters count up to a maximum value and then do not exceed this value.

In step 204 a slug is created for each object in the corpus. In step 206 Each slug is entered into the counting bloom filter which causes a counter to increment in a position corresponding to a slug. After completing step 206 Positions whose counters have a value greater than one will reflect one or more matching slugs. However, some of these matches may be false-positive matches rather than true matches. Therefore, steps filter 208 and 210 the false-positive matches using a multimap.

In step 208 For slugs associated with positions in the counting bloom filter whose counters have a value greater than one, the slug and its associated object are added to a multi-map. When adding the slug and its associated object to the multimap, the slug represents the key to the multimap, and the object in the corpus represents the value to the multimap. This multimap is used to remove false-positive results from processing. In step 210 becomes the process 200 completed by outputting a value for any key in the multi-map that has two or more values. The output values reflect objects whose slugs matched slugs from at least one other object in the corpus. Thus, the output objects identify those objects in which selected properties, as reflected in a slug of an object, uniquely match at least one other object in the corpus.

In some embodiments, the process may 200 distributed across multiple processors. For example, there may be a counting bloom filter on each of several processors, and steps 202 . 204 and 206 can be run on any of the multiple processors. The corpus of objects can be distributed among the various processors so that all objects are processed by a processor, but no object is processed by more than one processor. In such embodiments, prior to performing step 208 , summing up counters for each position in the counting bloom filter with counters for the same position in counting bloom filters on other processors. After that, the process continues 200 with a running steps 208 and 210 on a single processor.

Exemplary embodiments will be described below that solve the third problem discussed above, that is, the third aspect of the invention. H. an efficient identification of unique objects in a corpus. These embodiments use a counting bloom filter and a multi-map to quickly identify unique objects. When slugs are entered into the counting bloom filter for all objects in the corpus, any position with a count of one reflects a unique object, since bloom filters do not produce false-negative results. Additionally, to the extent that positions have counts of two or more, these counts could reflect false-positive results. Therefore, a multi-map allows a determination of whether the matches reflected in the counts are false or true-positive.

3 shows a flowchart of an exemplary process 300 compare all objects in a corpus with all other objects in the corpus to determine unique objects within the corpus, in accordance with some embodiments of the present disclosure. As shown, in step 302 Sets the size of a scoring Bloom filter and creates it taking into account the error rate that results for the body size to be processed. For example, increasing the number of positions in a counting bloom filter may aim to reduce the error rate for a specific body size, while decreasing the number of positions in a counting bloom filter may aim to increase the error rate for a specific body size. Methods for sizing a scoring bloom filter to achieve a desired error rate for a specific body size are well known in the art, and therefore, these methods are not discussed here.

In some embodiments, the counting bloom filter may include an N-bit counter, and these counters may be implemented as a two-bit counter (i.e., N = 2). In further embodiments, these counters may be one-bit counters or more than two-bit counters. In still other embodiments, these counters may be saturation counters; d. H. these counters count up to a maximum value and then do not exceed this value.

In step 304 a slug is created for each object in the corpus. In step 306 Each slug is entered into the counting bloom filter which causes a counter to increment in a position corresponding to a slug. As previously discussed, after slugs have been entered into the scoring Bloom filter for all objects in the corpus, any position with a count of one reflects a unique object in the corpus since the scoring Bloom filter does not produce false-negative results. Therefore, in step 308 for each slug whose count in the counting bloom filter is one, output the object associated with the slug as a unique object in the corpus.

After completing step 308 positions whose counts have a value greater than one reflect one or more matching slugs; ie slugs that are not unique. However, some of these matches may be false-positive matches instead of true matches due to the nature of bloom filters, as discussed previously. Therefore, in steps 310 and 312 filtered out the false-positive matches using a multimap.

In steps 310 and 312 determines whether the scoring Bloom filter masks the presence of other unique objects, since the Bloom filter allows false-positive results. In step 310 For each slug whose associated position has a count greater than one, the slug is entered as a key to a multimap, and the object associated with the slug is entered as the value to that key. In step 312 The process ends after each value in the multimap is output for keys that have only one value. Unique objects in the corpus are created by the collection of objects by step 308 are output, and the collection of objects by step 312 because the former reflects objects whose slugs were the sole slug in a position in the scoring Bloom filter, and which were therefore uniquely slug-associated with objects in the corpus, whereas the latter reflects slugs, the false-positive results in the counting Bloom filters, but for which the Multimap uniqueness was

In some embodiments, the process may 300 distributed across multiple processors. For example, there may be a counting bloom filter on each of several processors, and steps 302 . 304 and 306 can be run on any of the multiple processors. The corpus of objects can be distributed among the various processors so that all objects are processed by a processor, but no object is processed by more than one processor. In such embodiments, prior to performing step 308 , summing up counters for each position in the counting bloom filter with counters for the same position in counting bloom filters on other processors. After that, the process continues 300 with a running steps 308 . 310 and 312 on a single processor.

4 FIG. 10 illustrates an example computing environment in which the embodiments of the present disclosure may be implemented.

computer system 400 includes a bus 402 or another communication mechanism for forwarding information, and a hardware processor 404 with the bus 402 for processing information. In some embodiments, the hardware processor may 404 for example, a general purpose microprocessor, or it may be a reduced instruction set microprocessor.

The computer system 400 also includes a main memory 406 , for example a RAM (Random Access Memory) or another dynamic one Storage device connected to the bus 402 is connected to store information and instructions by the processor 404 are to be executed. The main memory 406 can also be used to store temporary variables or other intermediate information while executing instructions by the processor 404 , Such instructions, when stored in non-transitory storage media, make the processor 404 accessible, the computer system 400 to a specialized machine customized to perform the operations specified in the instructions.

In some embodiments, the computer system includes 400 further a ROM (read only memory) 408 or another static storage device connected to the bus 402 is connected to static information and instructions for the processor 404 save. A storage device 410 , such as a magnetic disk or an optical disk, is provided and to the bus 402 connected to store information and instructions.

The computer system 400 can over the bus 402 with an ad 412 be connected, for example, a cathode ray tube (CRT) or an LCD screen to display information to a user of a computer. An input device 414 , which includes alphanumeric and other keys, is by bus 402 connected to information and selected commands to the processor 404 to convey. Another type of user input device is cursor control 416 such as a mouse, trackball, or cursor direction keys to the processor 404 Direction information and selected commands and a cursor movement on the display device 412 to control. This input device typically has two degrees of freedom in two axes, a first axis (eg x) and a second axis (eg y), which allows the device to designate positions in a plane.

In the computer system 400 For example, the processes and methods described herein may be implemented using custom hardwired logic circuits, one or more ASICs or FPGAs, firmware, and / or program logic that, in combination with the computer system, effect or program the computer system 400 a special machine is. In some embodiments, the processes and methods described herein are implemented by the computer system 400 Responding to that the processor 404 one or more sequences of one or more in main memory 406 executes contained instructions. Such instructions may be in main memory 406 from another storage medium, for example the storage device 410 , are read. An execution of the main memory 406 contained instruction sequences causes the processor 404 to execute the process steps described here. In other embodiments, hardwired circuitry may be used instead of or in combination with software instructions.

The term "storage media" as used herein refers to any non-transitory media that stores data and / or instructions that cause a machine to work in a specific manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as the storage device 410 , Volatile media include dynamic memory such as main memory 406 one. Conventional forms of storage media include, for example, a floppy disk, a flexible disk, a hard disk, a semiconductor disk, a magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any hole-patterned physical media, a RAM PROM and an EPROM, a flash EPROM, an NVRAM, and any other memory chip or cartridge.

Storage media are different from, but may be used in conjunction with, transmission media. Transmission media participate in a transfer of information between storage media. For example, transmission media include coaxial cable, copper wire and optical fiber, including the wires that the bus 402 includes. Transmission media may also take the form of sound or light waves, such as those generated in radio and infrared data communications.

Various forms of media may be involved in the execution of one or more sequences of one or more instructions issued by the processor 404 are to be executed. For example, the instructions may initially be stored on a magnetic disk or a solid state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. One at the computer system 400 The modem in question may receive the data over the telephone line and use an infrared transmitter to convert the data into an infrared signal. An infrared detector can receive the data carried in the infra-red signal and appropriate circuitry can transfer the data to the bus 402 lay. The bus 402 conveys the data to the main memory 406 from which the processor 404 retrieves and executes the instructions. The main memory 406 Instructions received may optionally be in the storage device 410 stored either before or after being executed by the processor 404 ,

The computer system 400 also includes a communication interface 418 by bus 402 connected is. The communication interface 418 provides a two-way data communication connection to a network link 420 ready with a local network 422 connected is. For example, a communication interface 418 an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, a communication interface 418 a Local Area Network (LAN) card to provide a data communication connection to a compatible LAN. Wireless connections can also be implemented. In any such implementation, a communication interface sends and receives 418 electrical, electromagnetic or optical signals carrying digital data streams representing various types of information.

The network link 420 typically provides data communication to other data devices over one or more networks. For example, the network link 420 over a local network 422 a connection to a host computer 424 or to data facilities provided by an Internet Service Provider (ISP) 426 operate. The ISP 426 in turn provides data communication services over the worldwide packet data communications network, now commonly referred to as the "Internet" 428 referred to as. Both the local network 422 as well as the internet 428 use electrical, electromagnetic or optical signals that can carry digital data streams. The signals passing through the various networks and the signals passing through the network link 420 and via the communication interface 418 run the digital data to the computer system 400 to and from this are exemplary forms of transmission media.

The computer system 400 can send and receive messages and data, including code, through the network (s), the network link 420 and the communication interface 418 , In the example of the Internet could be a server 430 a requested code for an application program over the Internet 428 , the ISP 426 , the local network 422 and the communication interface 418 send. The received code can be processed by the processor 404 be executed, as it is received, and / or it may be in the storage device 410 or other non-volatile memory for later execution.

Claims (16)

A method of associating a first object with one or more objects within a plurality of objects, wherein each object has a first plurality of properties, each characteristic having data reflecting an identifier of an entity represented by the object, the associated objects having matching data in have corresponding properties for a second plurality of properties, the method comprising the following operations performed by one or more processors: Execute, for each object within the plurality of objects and for the first object, the following: Generating a slug for the object, the slug having the second plurality of properties of the object; and Entering the slug for the object into a bloom filter; and Generating, for a position in the bloom filter corresponding to the slug for the first object, an association between objects whose slugs match the position if the slugs match for those objects. The method of claim 1, further comprising: Determining the size of the bloom filter for a predetermined error rate and number of objects within the plurality of objects. The method of claim 1 or claim 2, further comprising: Reading the plurality of objects from at least one database. The method of claim 1, further comprising determining whether the slugs in the position corresponding to the slug for the first object match the slug for the first object, the following operations being performed by a slug or multiple processors: input, for each slug in the position corresponding to the slug for the first object, the respective slug and its associated object into a multimap, the slug for the respective object being a key to the multimap and its associated object is a value to the multimap; and associating objects in the multimap whose key matches the slug for the first object. A method of associating objects within one or more groups of objects within a plurality of objects, wherein each object has a first plurality of properties, each property having data reflecting an identifier of an entity represented by the object, the associated objects being within a plurality of objects Group of objects have matching data in corresponding properties for a second plurality of properties, the method comprising the following operations performed by one or more processors: Execute, for each object within the plurality of objects, the following: Generating a slug for the object, the slug having the second plurality of properties of the object; and Entering the slug for the object into a counting bloom filter; Entering, for each slug created, the slug and its associated object into a multi-map if a position in the counting bloom filter corresponding to the slug has a count greater than 1, the slug being a key to the multi-map and the object a value to which Multimap is; and Associate the objects stored as values for each key of the multimap with two or more associated values. The method of claim 5, further comprising: Determining the size of the counting bloom filter for a predetermined error rate and number of objects within the plurality of objects. The method of claim 5 or claim 6, further comprising: Reading the plurality of objects from at least one database. The method of any of claims 5 to 7, wherein each entry in the counting bloom filter comprises a 2-bit counter. The method of claim 8, wherein each 2-bit counter is a saturation counter. The method of any one of claims 1 to 9, wherein the slug comprises a concatenation of two or more character strings separated by a delimiter between each concatenated character string. The method of claim 10, wherein the delimiter comprises a character otherwise not present in the strings that have been concatenated. The method of claim 10, wherein the delimiter has a sequence of two or more characters and the sequence of two or more characters is not present in any of the two or more strings that have been concatenated. The method of any one of claims 10 to 12, wherein the number of properties in the first plurality of properties is equal to the number of properties in the second plurality of properties. The method of any of claims 10 to 12, wherein the number of properties in the first plurality of properties is greater than the number of properties in the second plurality of properties. System comprising: a storage device that stores a set of instructions; and at least one processor executing the set of instructions to perform operations involving the operations of any one of claims 1 to 14. A computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations involving the operations of any one of claims 1 to 14.

Images