DE10319427A1 - Method for creating short data records characteristic of data records from a database, in particular from the World Wide Web, method for determining data records relevant to a specifiable search query from a database and search system for carrying out the method - Google Patents

Abstract

A method for creating short data records (10) which are characteristic of data records (4) from a database, in particular from the World Wide Web, for storage on a memory module (6) as a basis for determining the data records (4) relevant to a predefinable search query. should be suitable for generating a search index that is particularly suitable for obtaining high-quality information from the database or from the World Wide Web. For this purpose, according to the invention, the system resources provided for creating a short data record (10) from a data record (4) are selected taking into account determined empirical values from previous search queries.

Description

The Invention relates to a method of creating for records a database, particularly from the World Wide Web Short records for storage on a memory module as a basis for determination the one Predefinable search query relevant data records. It continues to affect one Procedure for determining for a predefinable search query relevant data records from a database, in particular from the World Wide Web, where such short records are based on their relevance for the respective search query can be searched. Furthermore concerns The invention is a search system for determining a predefinable Search query relevant records from a database, especially from the World Wide Web.

In complex databases or even in the global computer network ("World Wide Web") will be enormous Amount of information held by a user for research purposes more or less targeted. To get information from the big To be able to use the amount of information in a targeted manner so-called search engines are used, which are particularly useful for gathering information experience considerable spread from the World Wide Web to have. In such search engines, the user is usually informed about a An input / output module provides an inquiry window via which targeted search or research terms can be specified. Then searches the Search engine the information base of the database or the World Wide Web for suitable key or keywords. The response data records found thereupon are usually from the respective Search engine with regard to its relevance for the specified search order categorized and the user in the manner of a hit list in order according to their relevance.

Indeed prepares the increasing complexity of the databases used and in particular the constantly growing enormous amount of information in the World-wide-web increasing Problems with the structured and targeted procurement of information. The for The search engines used are therefore increasing in terms of the search algorithms used improved, in addition in the Type of pre-sorting or pre-filtering other aids for the classification of data records can be used from the database. Especially on the World Wide Web are the records usually structured and organized in the form of so-called domains, whereby a domain is typically maintained by an operator and in turn a variety of sub-records, text documents or the like may include.

Around especially when it comes to obtaining information from the World Wide Web the enormous variety of data sets available there or Domains with one in turn possibly comparatively large Sub-record content or information carriers a suitable pre-selection of a given search query domains to consider to be able to meet a so-called domain ranking can be used. Doing so a characteristic value is assigned to each domain, based on accessible secondary information the importance of consideration in the manner of a relative relevance the respective domain for characterizes the search query. When assigning this characteristic value to the respective domain is usually an information base in the manner of a so-called static approach used, for example due to the degree of networking the respective domain with other domains on the relative importance of the respective domain is closed. As a measure of such meaning can be the number of links or cross references from others Domains are used for the respective domain, whereby the Assumption is based on a variety of cross references the respective domain is an indication that this domain is suitable for a large number of Users of particular importance when processing their search queries is.

Indeed has been shown that when assigning such a static Parameter as a relevance indicator for the respective domain scope for Manipulations exist, regardless of the actual user interests due to economic decisions by making a Plenty of factually unjustified links or cross-references artificially the impression of a comparatively high relevance or significance of the respective domain can be generated. The use of such static relevance assignments to improve the search results Internet research is therefore of decreasing interest.

The enormous amounts of information available on the World Wide Web or the Internet make it impossible to actually search all domains, including the sub-data sets or text modules contained therein, in real time for the presence of the search query or of individual elements of the respective search query when carrying out an information search. Instead, so-called "crawlers" or searcher modules are used in search systems or search engines for obtaining information from the Internet or the World Wide Web, which are constantly used for the domains or search records from the World Wide Web or a complex database for their textual content or other information deemed relevant. Within the framework of predefined system resources (for example processing time, storage capacity or computing power), the respective search module searches the currently selected domain or the data record and sets a limit for the domain or the data record based on the information found in the respective domain up to a limit specified by the assigned system resources characteristic short data record, for example in the form of a text file with possibly assigned headings or other indicators.

This The short data record is then stored on a memory module and for one subsequent investigation held ready. The entirety of that at all considered in this approach records or domains created, stored on the memory module also as a so-called "index" of the respective search engine denotes and serves as an information basis for the research carried out below: The short data records contained in the index are created usually continuously, whereby individual domains are selected cyclically, so that the index is continuously updated. At a subsequent research, i.e. when determining the one for a given one Query relevant records, is then formed by the total of the short data records stored Index to the presence of keywords or keywords the respective search query or individual elements of it, based on the search results or hits obtained the short data records found associated records or domains as relevant for the respective search query can be determined.

by virtue of the multitude of domains or data records available on the World Wide Web cannot all Domains are taken into account when creating the short data records. The decision, which domains are taken into account when creating the index usually based on this the above relevance criteria, in particular based on information about a recognized or supposed meaning of the respective domain for the User. Especially because of the variety of information available a particularly targeted pre-sorting of the information and especially the as for a search query relevant recognized data records for the subsequent evaluation the search results can be very significant is already at the creation of the so-called index a particularly high quality and care when evaluating the considered Information desirable.

The The invention is therefore based on the object of creating a method from for records characteristic short data sets of the type mentioned above, with one for a high quality Obtaining information from the database or from the World Wide Web particularly suitable search index can be generated. Furthermore are said to be a particularly suitable one using this method Procedure for determining for a predefinable search query relevant data records from a database, in particular from the world-wide-web, and a search system to do this Procedure are specified.

Regarding the Procedure for creating the for the records characteristic short data sets this object is achieved according to the invention by those provided for creating a short data record from a data record System resources under consideration from determined empirical values from previous search queries to get voted.

The Invention is based on the consideration that for generating one for particularly high quality information gathering suitable information basis on the basis of the short data records characteristic of the data records on the one hand, information about the data available in the form of static parameters individual records or domains themselves are taken into account can, on the other hand in the form of a dynamic element for the Characteristic information taken into account for user interests should be. The underlying understanding is that the result Obtaining information from the database or the World Wide Web is considered to be of particularly high quality if it in if possible far-reaching scope correctly reflects user interest. It should therefore take action to be taken for the user interest characteristic information in the further Increase information gathering allow. A starting point for this is already creating the information base for processing the Search queries so that information about visitor interest is already there in generating the for the records or domains characteristic short data records in the index should. In order to make this possible, are already used in the allocation of system resources for the creation a short data record from an assigned data record can be and thus the completeness the in the short record for the information provided is crucial co-determine, taking into account the user interest based on past experience from previous searches.

In particularly simple and concise Way can user interests already taken into account when creating the index be by advantageously in the allocation of system resources the frequency of the same or similar searches in the search query recent past is taken into account as empirical value. In a further advantageous embodiment, the frequency of hits can also be achieved of the records or domains related to those of users in the recent past frequently given search queries become. The empirical values therefore expediently include one for the number of similar search queries in a predefinable time interval characteristic key figure.

Around the user interests are particularly effective as early as the creation stage of the index for consider the search engine to be able are advantageously the resources of a system resources to create the for the records each characteristic short data sets provided browser module or crawler taking into account from determined empirical values from previous search queries selected.

In In a particularly advantageous embodiment, the user interests particularly taken into account when allocating system resources, by the particularly complex when determining the empirical values Structure of search queries used by users is taken into account becomes. This is based on the knowledge that a particularly precise image general user interest not only by relative frequency individual elements or terms used in search queries, but additional or additionally also by considering of specific correlations between individual terms or Elements of search queries it can be targeted. In particular considered, that individual elements or components correspond to a search query in combination with the currently widespread user interest with specific other individual elements or components of search queries Can be requested. For example, the current user interest generally currently go in a direction that prefers free Multimedia files are to be downloaded from the Internet. In such a constellation, search queries are increased with a combination of the search terms "MP3", "free" and "download" the specific evaluation and consideration of past searches can therefore combine these three individual elements of a search query as a particularly important indicator of increased user interest get ranked. In order to make this possible, correlations are preferably used when determining the empirical values Individual elements of the search queries are taken into account.

Around in the manner of a first pre-filtering a comparatively simple obtainable input information for the evaluation of search queries and their relevance to the records To provide, is advantageous in determining the empirical values the relative frequency of search queries and / or of individual elements of the search queries taken into account. This can be done in a particularly simple manner directly during creation of the index are taken into account by those records the for a given search query or for a given combination are identified as relevant from individual elements of search queries, advantageously depending on the relative frequency the search query or the combination of individual elements of search queries additional System resources for the Creation of the assigned short data record can be assigned.

advantageously, are created in the manner mentioned for the records the short data records characteristic of the database for the determination of for a specifiable Search query relevant records used from the database, in particular from the World Wide Web, by doing so created short data records stored in a memory module their relevance for the respective search query can be searched. As a criterion for the determination this relevance can be used, for example, by the frequency with which a key word or keyword Search query can be found in the respective short data record, with a differentiation according to the location of each Finding, for example in a heading or in full text, can be made.

Regarding the Search system for determining for a predefinable search query relevant data records from a database, in particular from the world wide web, the stated task is solved, by in a memory module for the records characteristic short data sets are stored, whereby to create a short data set system resources provided under consideration of stored empirical values from previous search queries chosen are.

The empirical values advantageously include a characteristic number that is characteristic of the number of similar search queries in a predefinable time interval. In an additional or alternative advantageous embodiment, the system resources are the resources of a browser module provided for creating the short data records that are characteristic of the data records, taking into account stored empirical values previous searches.

The Advantages achieved with the invention are in particular that by considering from empirical values from previous search queries when awarding the system resources when creating the index or those characteristic of the data records Short records already in a particularly early Stage, namely in the preparation phase of a database or internet search, a far taking into account the currently existing user interest is enabled. Just through that consideration of user interest to or instead of specific database characteristics used so far such as frequency of the respective cross references is one of the users as special high quality viewed information gathering possible. A particularly specific image of user interest and thus is a particularly high level of accuracy when allocating resources thereby achievable through consideration of correlations between individual elements of search queries, being particularly common Combinations of specific individual elements used and the conclusion to the data records found as a result with such combined search queries or Domains a particularly that user interest can expect customized hit generation.

On embodiment the invention is explained in more detail with reference to a drawing. In it the figure shows schematically a search system for determining a search query that can be specified relevant records or domains from the World Wide Web.

The search system 1 according to the figure is about by means of the double arrows 2 indicated data lines of the Internet or World-Wide-Web with a variety of domains 4 connected, each domain 4 in turn typically includes a large number of sub-data sets, text modules, multimedia information elements or the like.

Because of the wide variety of information available on the World Wide Web, the search system is 1 for processing a search query not for searching the domains 4 or the information contained in them for the presence of certain keywords or key words, but instead for searching one in a memory module 6 deposited so-called index 8th designed. The index 8th includes a variety of short data sets 10 each of which is for a record or domain 4 of the World Wide Web is characteristic. Every short data record 10 contains a part of the information content of the assigned domain that is recognized as relevant 4 , where in the short data record 10 especially those in the respective domain 4 contained text information is reproduced. To process a search query, it is as shown by the arrow 12 indicated, an input / output module 14 of the search system 1 from where a search of the short data records is carried out using key words or keywords characteristic of the search query 10 is started. Depending on the number of results or hits with which the presence of keywords or key words in the short records 10 is determined, that with the respective short data record 10 corresponding domain 4 recognized as relevant for the search request and the corresponding domain address communicated to the user in a result list.

To create the for the domains 4 characteristic short data sets 10 which in their entirety are the index 8th form, includes the search system 1 a browser module also referred to as a "crawler" 16 , The browser module 16 contacts the respective domains at regular, preferably cyclical, time intervals 4 and searches for their information content. It can in particular be provided that on the respective domain 4 to record stored text information and compress it appropriately. Type and scope of analysis of the content of each domain 4 through the browser module 16 are determined by specifying specific system resources of the browser module 16 for the respective domain 4 established. System resources can depend on the respective domain 4 For example, the time period provided for the search, the computer power used and / or the allocated storage capacities can be specified. In particular, it can also be specified whether the respective domain 4 from the browser module 16 to be addressed at all or to be ignored from the outset. Using the information base determined during the search for the respective domain 4 is then used by the browser module 16 the assigned short data record in the manner of a short version 10 created and as part of the index 8th in the memory module 6 stored.

The allocation of system resources for the search of the respective domain 4 can, for example, depend on domain-specific relevance parameters. So-called static relevance parameters can also be provided, based on predefined criteria such as the degree of networking of a domain 4 with other domains 4 characterize how high the degree of acceptance of the respective domain 4 among users. These relevance parameters can be used to determine whether a domain 4 is taken into account at all during the search, and if so, how carefully the search of the respective domain 4 when creating the assigned Short record 10 should be done.

In addition, the search system 1 but also designed for the creation of short data sets 10 Experience values and insights from the previous search queries must also be taken into account and thus the current user interest shown therein, particularly in the creation or cyclical renewal of the index 8th let it flow in. This is the memory module 6 another memory module 18 assigned in which the incoming search queries are stored for further evaluation in the manner of a logbook. The contents of the memory module 18 become an analysis module 20 made accessible, which subjects the received search queries to an evaluation and uses the knowledge gained to redistribute the system resources to the domains to be taken into account 4 on the next search cycle. The analysis module transmits the corresponding allocation of system resources 20 how about the arrow 22 shown, then to the browser module 16 ,

The analysis module takes into account when assigning the system resources 20 thus empirical values from previous search queries. This can be done, for example, by determining the frequency of a search query or a key word or keyword as a single element of a search query, in the case of frequently used search queries or individual elements of search queries suggesting that users are currently comparatively high in popularity. Accordingly, it is assumed that the data records or domains found in comparatively popular search queries and identified as relevant 4 reflect the current user interest to a comparatively high degree. In this embodiment, the analysis module 20 thus those domains 4 , which are listed as the result for comparatively frequently used search queries, a correspondingly increased share of system resources during the next search by the search module 16 assign.

In addition, the search system 1 but also designed for comparatively complex structures in the profile of the search queries when allocating the system resources through the analysis module 20 mitzuberücksichtigen. When determining the empirical values, correlations between individual elements of search queries are also taken into account. If, for example, it is found that individual elements or search words in search queries are combined particularly frequently with certain other individual elements or search words, then a high intrinsic correlation between these two search elements is concluded, so that on the one hand those domains 4 , in which complete or approximate combinations are found, can be recognized as particularly relevant, and on the other hand, when evaluating the relative frequencies of individual search elements, the relative frequencies of the further search elements that are particularly correlated with them can also be taken into account.

The statistical analysis of the search queries is carried out in the analysis module 20 creates a correlation matrix, the matrix elements of which provide a quantitative measure for the correlation between two individual elements of search queries. In particular, the relative frequency with which the two respective individual elements of search queries are queried in combination can be provided as a quantitative measure. This correlation matrix is then diagonized by a main axis transformation, the eigenvalues of the original correlation matrix being indicated on the main diagonal of the diagonalized matrix. With this main axis transformation, the eigenvectors of the correlation matrix are also determined.

The Eigenvalues and eigenvectors of the correlation matrix can then be used for further evaluation of search queries are used. Those eigenvectors of the Correlation matrix that has a comparatively large eigenvalue correspond to a mix of individual elements of search queries, which according to the linear coefficients of the individual elements of the search queries comparatively frequently in typical search queries occurs and thus particularly the current user interest reflects. In a next step, those eigenvectors will become selected the correlation matrix, which is a comparatively large one Eigenvalue is assigned. Through the eigenvectors determined in this way receives the result is a mix of search queries in the respective combination have been particularly likely in the recent past are.

The analysis module acts with the comparatively large eigenvalues assigned to the correlation matrix 20 in the form of a test request for the index 8th and thus determines the data records or domains identified as relevant to this self-inquiry 4 , Because the domains determined in this way 4 correspond to the current user interest in particular for these domains 4 system resources when rerouting the World Wide Web increased proportionally compared to the previous run. This can be done, for example, by assigning a weighting factor when providing the system resources for the respective domain 4 after the relationship R VPA (D k ) = (1 + α · λ β k ) α, β> 0 take place, where λ _{k is} the eigenvalue of the associated self-inquiry D _k a domain displayed as a hit on this self-inquiry 4 and α can be a suitably chosen constant> 0.

1

search system

2

double arrows

4

domain

6

memory module

8th

index

10

Short records

12

arrow

14

Input / Output Module

16

Search form module

18

memory module

20

analysis module

22

arrow

Claims (10)

Procedure for creating for records ( 4 ) from a database, in particular from the World Wide Web, characteristic short data sets ( 10 ) for storage on a memory module ( 6 ) as a basis for determining the data records relevant for a specifiable search query ( 4 ), which is used to create a short data record ( 10 ) from a data record ( 4 ) provided system resources are selected taking into account the empirical values determined from previous search queries. The method of claim 1, wherein the empirical values one for the number of similar search queries in a predefinable time interval include characteristic indicator. Method according to Claim 1 or 2, in which, as system resources, the resources one for creating the data records ( 4 ) each characteristic short data sets ( 10 ) provided browser module ( 16 ) can be selected taking previous experience values from previous search queries into account. Method according to one of claims 1 to 3, wherein in the Determination of empirical values Correlations between individual elements of search queries become. Method according to one of claims 1 to 4, wherein in the Determination of the empirical values, the relative frequency of search queries and / or of individual elements of the search queries is taken into account. Method according to Claim 5, in which those data records ( 4 ), which are recognized as relevant for a given search query or for a given combination of individual elements of search queries, proportional to the relative frequency of the search query or the combination of individual elements of search queries, additional system resources for the creation of the assigned short data record ( 10 ) be assigned to. Procedure for the determination of data records relevant for a specifiable search query ( 4 ) from a database, in particular from the World Wide Web, in which in a memory module ( 6 ) for the records ( 4 ) characteristic short data sets created by the method according to one of claims 1 to 6 ( 10 ) are searched for their relevance for the respective search query. Search system ( 1 ) for the determination of data records relevant for a specifiable search query ( 4 ) from a database, in particular from the World Wide Web, in particular for carrying out the method according to claim 7, in which in a memory module ( 6 ) Short data sets characteristic of the data sets ( 10 ) are stored, whereby those for the creation of a short data record ( 10 ) from a data record ( 4 ) system resources provided are selected taking into account stored empirical values from previous search queries. Search system ( 1 ) according to Claim 8, in which the empirical values comprise a characteristic number which is characteristic of the number of similar search queries in a predeterminable time interval. Search system ( 1 ) according to Claim 8 or 9, in which, as system resources, the resources one for creating the data records ( 4 ) each characteristic short data sets ( 10 ) provided browser module ( 16 ) are selected taking into account stored empirical values from previous search queries.