US20030120670A1 - Method and system for parsing navigation information - Google Patents
Method and system for parsing navigation information Download PDFInfo
- Publication number
- US20030120670A1 US20030120670A1 US10/320,171 US32017102A US2003120670A1 US 20030120670 A1 US20030120670 A1 US 20030120670A1 US 32017102 A US32017102 A US 32017102A US 2003120670 A1 US2003120670 A1 US 2003120670A1
- Authority
- US
- United States
- Prior art keywords
- information
- data
- dimension
- entry
- routine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99936—Pattern matching access
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99937—Sorting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99952—Coherency, e.g. same view to multiple users
- Y10S707/99954—Version management
Definitions
- the described technology relates to analyzing of data relating to events generated by a computer program.
- HTTP Hypertext Transfer Protocol
- each HTTP request message contains a universal resource identifier (a “URI”), which specifies to which network resource the request is to be applied.
- a URI is either a Uniform Resource Locator (“URL”) or Uniform Resource Name (“URN”), or any other formatted string that identifies a network resource.
- URLs as an example of URIs, are discussed in detail in T. Berners-Lee, et al., Uniform Resource Locators ( URI ), RFC 1738, CERN, Xerox PARC, Univ. of Minn., December 1994.
- FIG. 1 illustrates how a browser application enables users to navigate among nodes on the web network by requesting and receiving web pages.
- a web page is any type of document that abides by the HTML format. That is, the document includes an “ ⁇ HTML>” statement.
- a web page is also referred to as an HTML document.
- the HTML format is a document mark-up language, defined by the Hypertext Markup Language (“HTML”) specification. HTML defines tags for specifying how to interpret the text and images stored in an HTML document. For example, there are HTML tags for defining paragraph formats and for emboldening and underlining text.
- the HTML format defines tags for adding images to documents and for formatting and aligning text with respect to images. HTML tags appear between angle brackets, for example, ⁇ HTML>. Further details of HTML are discussed in T. Berners-Lee and D. Connolly, Hypertext Markup Language— 2.0, RFC 1866, MIT/W3C, November 1995.
- a web browser application 101 is shown executing on a client computer 102 , which communicates with a server computer 103 by sending and receiving HTTP packets (messages). HTTP messages may also be generated by other types of computer programs, such as spiders and crawlers.
- the web browser “navigates” to new locations on the network to browse (display) what is available at these locations. In particular, when the web browser “navigates” to a new location, it requests a new document from the new location (e.g., the server computer) by sending an HTTP-request message 104 using any well-known underlying communications wire protocol.
- the HTTP-request message follows the specific layout discussed above, which includes a header 105 and a URI field 106 , which specifies the network location to which to apply the request.
- URI URI
- the server computer specified by URI receives the HTTP-request message, it interprets the message packet and sends a return message packet to the source location that originated the message in the form of an HTTP-response message 107 . It also stores a copy of the request and basic information about the requesting computer in a log file.
- the HTTP-response message contains the requested HTML document 109 .
- HTTP can also be used to transfer other media types, such as the Extensible Markup Language (“XML”) and graphics interchange format (“GIF”) formats.
- XML Extensible Markup Language
- GIF graphics interchange format
- the World Wide Web is especially conducive to conducting electronic commerce (“e-commerce”).
- E-commerce generally refers to commercial transactions that are at least partially conducted using the World Wide Web.
- numerous web sites are available through which a user using a web browser can purchase items, such as books, groceries, and software.
- a user of these web sites can browse through an electronic catalog of available items to select the items to be purchased.
- To purchase the items a user typically adds the items to an electronic shopping cart and then electronically pays for the items that are in the shopping cart.
- the purchased items can then be delivered to the user via conventional distribution channels (e.g., an overnight courier) or via electronic delivery when, for example, software is being purchased.
- Many web sites are also informational in nature, rather than commercial in nature. For example, many standards organizations and governmental organizations have web sites with a primary purpose of distributing information. Also, some web sites (e.g., a search engine) provide information and derive revenue from advertisements that are displayed.
- Web sites typically collect extensive information on how its users use the site's web pages. This information may include a complete history of each HTTP request received by and each HTTP response sent by the web site. The web site may store this information in a navigation file, also referred to as a log file or click stream file. By analyzing this navigation information, a web site operator may be able to identify trends in the access of the web pages and modify the web site to make it easier to use and more useful.
- FIG. 1 illustrates how a browser application enables users to navigate among nodes on the web network by requesting and receiving web pages.
- FIG. 2A is a block diagram illustrating components of the data warehouse system in one embodiment.
- FIG. 2B is a block diagram illustrating details of the components of the data warehouse system in one embodiment.
- FIG. 3 is a block diagram illustrating the sub-components of the data processor component in one embodiment.
- FIG. 4 is a block diagram illustrating some of the tables of the local data warehouse and the main data warehouse in one embodiment.
- FIG. 5 is a flow diagram illustrating the parse log data routine that implements the parser in one embodiment.
- FIG. 6 is a flow diagram of the filter log entry routine in one embodiment.
- FIG. 7 is a flow diagram illustrating the normalize log entry routine.
- FIG. 8 is a flow diagram of the generate dimensions routine in one embodiment.
- FIG. 9 is a flow diagram of the identify logical site routine in one embodiment.
- FIG. 10 is a flow diagram of the identify user routine in one embodiment.
- FIG. 11 is a flow diagram of the identify page type routine in one embodiment.
- FIG. 12 is a flow diagram illustrating the identify events routine in one embodiment.
- FIG. 13 is a flow diagram illustrating the identify sessions routine in one embodiment.
- FIG. 14 is a flow diagram of the generate aggregate statistics routine in one embodiment.
- FIG. 15 is a flow diagram of the import log data routine implementing the importer in one embodiment.
- FIG. 16 is a flow diagram of the load dimension table routine and one embodiment.
- FIG. 17 is a flow diagram of the load fact table routine in one embodiment.
- FIG. 18 is a flow diagram illustrating the identify user aliases routine in one embodiment.
- a method and system for providing customers with access to and analysis of event data (e.g., navigation data collected at customer web sites) is provided.
- the event data may be stored in log files and supplemented with data from other sources, such as product databases and customer invoices.
- a data warehouse system collects customer data from the customer web sites and stores the data at a data warehouse server.
- the customer data may include application event data (e.g., click stream log files), user attribute data of users of the customer web site (e.g., name, age, and gender), product data (e.g., catalog of products offered for sale by the customer), shopping cart data (i.e., identification of the products currently in a user's shopping cart), and so on.
- the data warehouse server interacts with the customer servers to collect to the customer data on a periodic basis.
- the data warehouse server may provide instructions to the customer servers identifying the customer data that is to be uploaded to the data warehouse server. These instructions may include the names of the files that contains the customer data and the name of the web servers on which the files reside. These instructions may also indicate the time the day when the customer data is to be uploaded to the data warehouse server.
- the data warehouse server receives customer data, it converts the customer data into a format that is more conducive to processing by decision support system applications by which customers can analyze their data. For example, the data warehouse server may analyze low-level navigation events (e.g., each HTTP request that is received by the customer web site) to identify high-level events (e.g., a user session).
- the data warehouse server then stores the converted data into a data warehouse.
- the data warehouse server functions as an application service provider that provides various decision support system applications to the customers.
- the data warehouse server provides decision support system applications to analyze and graphically display the results of the analysis for a customer.
- the decision support system applications may be accessed through a web browser.
- the customer servers are connected to the data warehouse server via the Internet and the data warehouse server provides data warehousing services to multiple customers.
- the data warehouse system may provide a data processor component that converts the log files into a format that is more conducive to processing by the decision support system applications.
- the converted data is stored in a data warehouse that includes fact and dimension tables.
- Each fact table contains entries corresponding to a type of fact derived from the log files.
- a web page access fact table may contain an entry for each web page access identified in the log files.
- Each entry may reference attributes of the web page access, such as the identity of the web page and identity of the accessing user.
- the values for each attribute are stored in a dimension table for that attribute.
- a user dimension table may include an entry for each user and the entries of the web access fact table may include a user field that contains an index (or some other reference) to the entry of the user dimension table for the accessing user.
- the user dimension table may contain the names of the users and other user-specific information.
- the user dimension table may itself also be a fact table that includes references to dimension tables for the attributes of users.
- the data warehouse may also include fact tables and dimension tables that represent high-level facts and attributes derived from the low-level facts and attributes of the log files. For example, high-level facts and attributes may not be derivable from only the data in a single log entry.
- the higher level category (e.g., shoes or shirts) of a web page may be identified using a mapping of web page URIs to categories. These categories may be stored in a category dimension table. Also, certain facts, such as the collection of log entries that comprise a single user web access session or visit, may only be derivable by analyzing a series of log entries.
- the data processor component may have a parser component and a loader component.
- the parser of the data processor parses and analyzes a log file and stores the resulting data in a local data warehouse that contains information for only that log file.
- the local data warehouse may be similar in structure (e.g., similar fact and dimension is tables) to the main data warehouse used by decision support system applications.
- the local data warehouse may be adapted to allow efficient processing by the parser.
- the local data warehouse may be stored in primary storage (e.g., main memory) for speed of access, rather than in secondary storage (e.g., disks).
- the parser may use parser configuration data that defines, on a customer-by-customer basis, the high-level data to be derived from the log entries.
- the parser configuration data may specify the mapping of URIs to web page categories.
- the loader of the data processor transfers the data from the local data warehouse to the main data warehouse.
- the loader may create separate partitions for the main data warehouse. These separate partitions may hold the customer data for a certain time period (e.g., a month's worth of data).
- the loader adds entries to the main fact tables (i.e., fact tables of the main data warehouse) for each fact in a local fact table (i.e., fact table of the local data warehouse).
- the loader also adds new entries to the main dimension tables to represent attribute values of the local dimension tables that are not already in the main dimension tables.
- the loader also maps the local indices (or other references) of the local dimension tables to the main indices used by the main dimension tables.
- FIG. 2A is a block diagram illustrating components of the data warehouse system in one embodiment.
- the data warehouse system includes customer components that execute on the customer servers and data warehouse components that execute on the data warehouse server.
- the customer servers 210 and the data warehouse server 260 are interconnected via the Internet 250 .
- Customer components executing on a customer server includes a data collection component 220 and a data viewer 230 .
- the data viewer may reside on a client computer of the customer, rather than a server.
- the data collection component collects the customer data from the storage devices 240 of the customer servers.
- the data viewer provides access for viewing of data generated by the decision support system applications of the data warehouse server.
- the data viewer may be a web browsed
- the data warehouse server includes a data receiver component 270 , the data processor component 280 , the data warehouse 290 , and decision support system applications 291 .
- the data receiver component receives customer data sent by the data collection components executing at the various customer web sites.
- the data processor component processes the customer data and stores it in the data warehouse.
- the decision support system application provides the customer with tools for analyzing and reviewing the customer data that is stored in the main data warehouse. Analysis performed on and reports generated from are described in U.S. patent application Ser. No. ______ (Attorney Ref. No. 34821-8010US), entitled “Identifying and Reporting on Combinations of Events in Usage Data,” and U.S. patent application Ser. No.
- each customer has its own set of dimension and fact tables so that the information of multiple customers are not intermingled.
- FIG. 2B is a block diagram illustrating details of the components of the data warehouse system in one embodiment.
- the data collection component 220 includes a monitor sub-component 221 and a pitcher sub-component 222 .
- the data collection component is described in more detail in U.S. patent application Ser. No. ______ (Attorney Ref. No. 34821-800/US), entitled “Method and System for Monitoring Resource via the Web,” which is being filed concurrently and which is hereby incorporated by reference.
- the pitcher is responsible for retrieving instructions from the data warehouse server, collecting the customer data in accordance with the retrieved instructions, and uploading the customer data to the data warehouse server.
- the monitor is responsible for monitoring the operation of the pitcher and detecting when the pitcher may have problems in collecting and uploading the customer data.
- the monitor detects that a problem may occur, it notifies the data warehouse server so that corrective action may be taken in advance of the collecting and uploading of the customer data.
- the pitcher may use certain log on information (e.g., user ID and password) to access a customer web server that contains customer data to be uploaded.
- the monitor may use that log on information to verify that the log on information will permit access to the customer data. Access may be denied if, for example, a customer administrator inadvertentry deleted from the customer web server the user ID used by the pitcher.
- the monitor provides advance notification of a problem, the problem might be corrected before the pitcher attempts to access the customer data.
- the monitor also periodically checks the pitcher to ensure that the pitcher is executing and, if executing, executing correctly.
- the data receiver component of the data warehouse server includes a status receiver sub-component 271 , a catcher sub-component 272 , an FTP server 273 , a status database 274 , and a collected data database 275 .
- the status receiver receives status reports from the customer servers and stores the status information in the status database.
- the catcher receives and processes the customer data that is uploaded from the customer web sites and stores the data in the collected data database.
- the data processor component includes a parser sub-component 281 and a loader sub-component 282 . The parser analyzes the low-level events of the customer data and identifies high-level events and converts the customer data into a format that facilitates processing by the decision support system applications.
- the loader is responsible for storing the identified high-level events in the data warehouse 290 .
- a customer may decide not to have the data collection component executing on its computer systems.
- the customer server may include an FTP client 245 that is responsible for periodically transferring the customer data to the FTP server 273 of the data warehouse server.
- the data receiver may process this customer data at the data warehouse server in the same way as the pitcher processes the data at the customer servers. The processed data is then stored in the collected data database.
- FIG. 3 is a block diagram illustrating the sub-components of the data processor component in one embodiment.
- the data processor component 300 includes a parser 310 , data storage area 370 , and a loader 330 .
- the data processor component inputs parser configuration data 340 and a log file 350 and updates the main data warehouse 360 .
- the parser configuration data may include a mapping of actual web sites to logical sites and a mapping of a combination of Uniform Resource Identifiers (“URls”) and query strings of the log entries to page definitions (e.g., categories) and event definitions.
- URLs Uniform Resource Identifiers
- the parser processes the entries of the log file to generate facts and dimensions to eventually be stored in the main data warehouse.
- the parser identifies events in accordance with the parser configuration data.
- the parser includes a filter log entry component 311 , a normalize log entry component 312 , a generate dimensions component 313 , an identify sessions component 314 , and a generate aggregate statistics component 315 .
- the filter log entry component identifies which log entries should not be included in the main data warehouse. For example, a log entry that has an invalid format should not be included.
- the normalize log entry component normalizes the data in a log entry. For example, the component may convert all times to Greenwich Mean Time (“GMT”).
- GTT Greenwich Mean Time
- the generate dimensions component identifies the various dimensions related to a log entry. For example, a dimension may be the Uniform Resource Identifier of the entry or the logical site identifier.
- the identify sessions component processes the parsed log file data stored in the local data warehouse to identify user sessions.
- a user session generally refers to the concept of a series of web page accesses that may be related in some way, such as by temporal proximity.
- the generate aggregate statistics component aggregates data for the log file being processed as each log entry is processed or after the log file is parsed.
- the data storage area 320 includes a local data warehouse 321 .
- the local data warehouse is stored non-persistently (or temporarily) in main memory of the computer system.
- the local data warehouse may contain fact tables and dimension tables that correspond generally to the tables of the main data warehouse 360 .
- the loader retrieves the information from the local data warehouse and stores the information in the main data warehouse.
- the loader includes a create partitions component 331 , a load dimension table component 332 , and a load fact table component 333 .
- the create partitions components creates new partitions for the main data warehouse.
- a partition may correspond to a collection of information within a certain time range.
- the main data warehouse may have a partition for each month, which contains all the data for that month.
- the load dimension table component and the load fact table component are responsible for loading the main data warehouse with the dimensions and facts that are stored in the local data warehouse.
- the log file is a web server log file of a customer.
- the log file may be in the “Extended Log File Foimat” as described in the document “http://www.w3.org/TR/WD-logfile-960323” provided by the World Wide Web Consortium, which is hereby incorporated by reference.
- the log file contains lines that are either directives or entries.
- An entry corresponds to a single HTTP transaction (e.g., HTTP request and an HTTP response) and consists of a sequence of fields (e.g., integer, fixed, URI, date, time, and string).
- the meaning of the fields in an entry is specified by a field directive specified in the log file.
- a field directive may specify that a log entry contains the fields date, time, client IP address, server IP address, and success code. Each entry in the log file would contain these five fields.
- the parser configuration data defines logical sites, page definitions, and event definitions.
- a logical site is a collection of one or more IP addresses and ports that should be treated as a single web site. For example, a web site may actually have five web servers with different IP addresses that handle HTTP requests for the same domain. These five IP addresses may be mapped to the same logical site to be treated as a single web site.
- FIG. 4 is a block diagram illustrating some of the tables of the local data warehouse and the main data warehouse in one embodiment.
- These data warehouses are databases that include fact tables and dimension tables.
- a fact table contains an entry for each instance of fact (e.g., web page access).
- a dimension table contains an entry for each possible attribute value of an attribute (e.g., user).
- the entries of a fact table contain dimension fields that refer to the entries into the dimension tables for their attribute values.
- a table may be both a fact table and a dimension table.
- a user dimension table with an entry for each unique user may also be a fact table that refers to attributes of the users that are stored in other dimension tables.
- the data warehouses contain a log entry table 401 , a user table 402 , a logical site table 403 , a URI table 404 , a referrer URI table 405 , a page type table 406 , event type tables 407 , a query string table 408 , and a referrer query string table 409 .
- the log entry table is a fact table that contains an entry for each log entry that is not filtered out by the parser.
- the other tables are dimension tables for the log entry table.
- the user table contains an entry for each unique user identified by the parser.
- the logical site table contains an entry for each logical site as defined in the parser configuration data.
- the URI table contains an entry for each unique URI of an entry in the log entry table.
- the referred URI table contains an entry for each referrer URI of the log entry table.
- the page type table contains an entry for each page type identified by the parser as defined in the parser configuration data.
- the data warehouse contains an event table for each type of event defined in the parser configuration data. Each event table contains an entry for each event value of that event type specified in an entry of the log entry table.
- the query string table contains an entry for each unique query string identified in an entry of the log entry table.
- the referrer query string contains an entry for each unique referrer query string identified in an entry of the log entry table.
- Table 1 is an example portion of a log file.
- the “#fields” directive specifies the meaning of the fields in the log entries. Each field in a log entry is separated by a space and an empty field is represented by a hyphen.
- the #fields directive in this example indicates that each entry includes the date and time when the transaction was completed (i.e., “date” and “time”), the client IP address (i.e., “c-ip”), and so on.
- the first log entry has a data and time of “2000-06-01 07:00:04” and a client IP address of “165.21.83.61.”
- Table 2 is an example portion of parser configuration data.
- ⁇ NewsItemID># ⁇ Uri ⁇ indicates that a page type of “news item” is specified for logical site 1 by a URI pattern of “/homepage_include/industrynews_detail.asp.”
- the definition also indicates that the event value is “ ⁇ NewsItemID># ⁇ Uri ⁇ ,” where the URI of the log entry is substituted for “ ⁇ Uri ⁇ and the value of NewsItemID in the query string is substituted for “ ⁇ NewsItemID>.”
- the event type definitions map a site identifier, URI pattern, and query string pattern to an event type and value.
- the definitions also specify the name of the event type and the name of the dimension table for that event type.
- FIGS. 5 - 14 are flow diagrams of components of the parser in one embodiment.
- FIG. 5 is a flow diagram illustrating the parse log data routine that implements the main routine of parser in one embodiment.
- the routine processes each entry in the log file based on the parser configuration data.
- the routine filters out certain log entries, normalizes the attribute values of the log entries, and generates entries in the dimension tables for the attributes of the log entries.
- the parser identifies user sessions and generates various statistics.
- the routine loops selecting and processing each log entry.
- the routine selects the next log entry of the log file starting with the first log entry.
- the routine may also pre-process the header information of the log file to identify the fields of the log entries.
- decision block 1502 if all the log entries have already been selected, then the routine continues at block 509 , else the routine continues at block 503 .
- the routine extracts the values for the fields of the selected log entry.
- the routine invokes the filter log entry routine, which returns an indication as to whether the selected log entry should be filtered out.
- decision block 505 if the filter log entry routine indicates that the selected log entry should be filtered out, then the routine skips to block 508 , else the routine continues at block 506 .
- the routine invokes the normalize log entry routine to normalize the values of the fields of the selected log entry.
- the routine invokes the generate dimensions routine to update the dimension tables based on the selected log entry and to add an entry into the log entry fact table.
- the routine updates the statistics for the log file. For example, the routine may track the number of log entries that have been filtered out. The routine then loops to block 501 to select the next log entry.
- the routine outputs the log file statistics.
- the routine invokes the identify sessions routine that scans the log entry table to identify the user sessions and updates a session dimension table.
- the routine invokes the generate aggregate statistics routine to generate various statistics and then completes.
- FIG. 6 is a flow diagram of the filter log entry routine in one embodiment.
- the filter log entry routine is passed a log entry and determines whether the log entry should be filtered out.
- the routine determines whether the filter out conditions have been satisfied.
- the routine determines whether the log entry has a field count problem. A field count problem arises when the number of fields in the log entry does not correspond to the number of expected fields for that log entry. The number and types of fields may be defined in a “fields” directive line of the log file.
- decision block 602 the routine determines whether the log entry is outside of a specified time range. The routine compares the time field of the log entry to the time range.
- the time range may be specified so that only those log entries within that time range are processed.
- the routine determines whether the IP address of the log entry should be ignored. For example, a log entry may be ignored if the entry originated from a server whose function is to ping the customer's web server at periodic intervals.
- the routine determines whether the log entry corresponds to a comment (e.g., a “g#remarks” directive).
- the routine determines whether the success code associated with the log entry indicates that log entry should be ignored. For example, if the success code indicates a failure, then the log entry may be ignored.
- the routine determines whether the log entry is requesting a resource whose extension indicates that the log entry should be ignored. For example, the routine may ignore log entries requesting graphic files, such as those in the “.gif” format.
- the routine determines whether the values within the fields of the log entry are corrupt. For example, a value in the date field that indicates a date of February 30th is corrupt.
- the various filtering, conditions may be specified in a configuration file. For example, the time range, IP addresses, and so on may be specified in the configuration file. These configuration files may be specified on a customer-by-customer basis.
- FIG. 7 is a flow diagram illustrating the normalize log entry routine.
- the routine normalizes the values of the fields in the passed log entry.
- the routine converts the time of the log entry into a standard time such as Greenwich Mean Time.
- the routine collects the time based on the variation between the times of the customer web servers. For example, the time of one web server may be five minutes ahead of the time of another web server. This correction may be based on current time information collected from computer systems that generated the events and then correlated to base current time information.
- the routine normalizes, the values of the fields of the log entry. This normalization may include processing search strings to place them in a canonical form.
- a search string of “back pack” may have a canonical form of “backpack.”
- Other normalization of search strings may include stemming of words (e.g., changing “clothes” and “clothing” to “cloth”), synonym matching, and first and last word grouping. The first word grouping for the search strings of “winter clothing” and “winter shoes” results in the string of “winter.”
- FIG. 8 is a flow diagram of the generate dimensions routine in one embodiment.
- This routine identifies a value for each dimension associated with the passed log entry and ensures that the dimension tables contains entries corresponding to those values.
- each entry in a dimension table includes the attribute value (e.g., user identifier) and a hash value. The hash value may be used by the loader when transferring information to the main data warehouse.
- each entry has a local identifier, which may be an index into the local dimension table. The loader maps these local identifiers to their corresponding main identifiers that are used in the main data warehouse.
- the routine invokes a routine that identifies the logical site associated with the log entry and ensures that an entry for the logical site is in the logical site dimension table.
- the routine invokes a routine that identifies the user associated with the log entry and ensures that an entry for the user is in the user dimension table.
- the routine invokes a routine that identifies the URI associated with log entry and ensures that an entry for that URI is in the URI dimension table.
- the routine invokes a routine that identifies the page type based on the parser configuration data and ensures that an entry for that page type is in the page type dimension table.
- the routine invokes a routine that identifies the various events associated with the log entry based on the parser configuration data and ensures that an entry for each event type is in the corresponding event table.
- the routine identifies other dimensions (e.g., referrer URI) as appropriate.
- the routine adds an entry to the log entry table that is linked to each of the identified dimensions using the local identifiers.
- the routine updates the statistics information based on the log entry and then returns.
- FIG. 9 is a flow diagram of the identify logical site routine in one embodiment.
- This routine compares the site information of the passed log entry with the logical site definitions in the parser configuration data.
- the routine selects the next logical site definition from the parser configuration data.
- decision block 902 if all the logical site definitions have already been selected, then the routine continues the block 905 , else the routine continues at block 903 .
- decision block 903 if the URI of the log entry matches the selected logical site definition, then the routine continues at block 904 , else the routine loops to block 901 to select the next logical site definition.
- the routine updates the logical site dimension table to ensure that it contains an entry for the logical site defined by the selected logical site definition. The routine then returns.
- the routine updates the logical site dimension table to ensure that it contains a default logical site definition and then returns. The log entries that do not map to a logical site definition are mapped to a default logical site.
- FIG. 10 is a flow diagram of the identify user routine in one embodiment.
- This routine may use various techniques to identify the user associated with the passed log entry.
- the selection of the technique is configured based on the customer web site. For example, one customer may specify to use a cookie to identify users. In absence of a user identifier in the cookie, the industry norm is to identify users based on their IP addresses.
- This routine illustrates a technique in which a combination of cookies and IP addresses are used to identify a user.
- the routine extracts the user identifier from the cookie associated with the log entry.
- the format of a cookie may be specified on a customer-by-customer basis.
- the routine continues at block 1006 , else the routine continues at block 1003 .
- the extraction may not be successful if, for example, the log entry did not include a cookie.
- the routine extracts the IP address from the log entry.
- decision block 1004 if the IP address is determined to be unique, then routine continues at block 1006 , else the routine continues at block 1005 .
- Certain IP addresses may not be unique. For example, an Internet service provider may use one IP address for many of its users. The Internet service provider performs the mapping of the one IP address to the various users.
- the routine extracts the browser identifier from the log entry. The combination of IP address and browser identifier may uniquely identify a user.
- the routine updates the user dimension table to ensure that it has an entry for this user and then returns.
- FIG. 11 is a flow diagram of the identify page type routine in one embodiment.
- This routine uses the page type definitions of the parser configuration data to identify the page type associated with the log entry.
- the routine selects the next page type definition from the parser configuration data.
- decision block 1101 if all the page type definitions have already been selected, then no matching page type has been found and the routine returns, else the routine continues at block 1103 .
- decision block 1103 if the log entry matches the selected page type definition, then the routine continues at block 1104 , else the routine loops to block 1101 to select the next page type definition.
- the routine updates the page type dimension table to ensure that it contains an entry for the page type represented by the selected page type definition. The routine then returns.
- FIG. 12 is a flow diagram illustrating the identify events routine in one embodiment.
- This routine determines whether the log entry corresponds to any of the events specified in the parser configuration data.
- the routine selects the next type of event from the parser configuration data.
- decision block 1202 if all the event types have already been selected, then the routine returns, else the routine continues at block 1203 .
- the routine selects the next event definition of the selected event type.
- decision block 1204 if all the event definitions of the selected event type have already been selected, then the log entry does not correspond to this type of event and the routine loops to block 1201 to select the next type of event, else the routine continues at block 1205 .
- the routine continues at block 1206 , else the routine loops to block 1203 to select the next event definition of the selected event type.
- the routine updates the dimension table for the selected type of the event to ensure that it contains an entry for the selected event definition.
- the routine then loops to block 1201 to select the next type of event. In this way, the routine matches no more than one event definition for a given event type. For example, if there are two event definitions for the event type “Keyword Search,” then if the first one processed matches, then the second one is ignored.
- FIG. 13 is a flow diagram illustrating the identify sessions routine in one embodiment.
- This routine scans the log entry table of the local data warehouse to identify user sessions.
- a user session may be delimited by a certain period of inactivity (e.g., thirty minutes).
- the criteria for identifying a session may be configurable on a customer-by-customer basis.
- the routine selects the next user from the user dimension table.
- decision block 1302 if all the users have already been selected, then the routine returns, else the routine continues at block 1303 .
- the routine selects the next log entry for the selected user in time order.
- decision block 1304 if all log entries for the selected user have already been selected, then the routine loops to block 1301 to select the next user, else the routine continues at block 1305 .
- decision block 1305 if the selected log entry indicates that a new session is starting (e.g., its time is more than 30 minutes greater than that of the last log entry processed), then the routine continues at block 1306 , else the routine loops to block 1303 to select the next log entry for the selected user.
- the routine updates a session fact table to add an indication of the new session. The routine then loops to block 1303 to select the next log entry for the selected user. The routine may also update the log entries to reference their sessions.
- FIG. 14 is a flow diagram of the generate aggregate statistics routine in one embodiment.
- This routine generate statistics based on analysis of the fact and dimension tables used by the parser.
- the routine selects the next fact table of intent.
- decision block 1402 if all the fact tables have already been selected, then the routine returns, else the routine continues at block 1403 .
- the routine selects the next entry of the selected fact table.
- decision block 1404 if all the entries of the selected fact table have already been selected, then the routine loops to block 1401 to select the next fact table, else the routine continues at block 1405 .
- the routine aggregates various statistics about the selected fact table. The routine then loops to block 1404 to select the next entry of the fact table.
- FIGS. 15 - 17 are flow diagrams illustrating components of the loader in one embodiment.
- FIG. 15 is a flow diagram of the load log data routine implementing the main routine of the loader in one embodiment. This routine controls the moving of the data from the local data warehouse (created and used by the parser) into the main data warehouse.
- the routine invokes the create partitions routine to create partitions for the main data warehouse as appropriate.
- the routine loops loading the dimension tables into the main data warehouse.
- the routine selects the next dimension table.
- decision block 1503 if all the dimension tables have already been selected, then the routine continues at block 1505 , else the routine continues at block 1504 .
- the routine invokes the load dimension table routine for the selected dimension table.
- the routine then loops to block 1502 to select the next dimension table.
- the routine loops adding the entries to the fact tables of the main data warehouse.
- the routine selects the next fact table in order.
- the order in which the fact tables are to be loaded may be specified by configuration information.
- the fact tables may be loaded in order based on their various dependencies. For example, a log entry fact table may be dependent on a user dimension table that is itself a fact table.
- decision block 1506 if all the fact tables have already been loaded, then the routine returns, else the routine continues at block 1507 .
- the routine invokes the load fact table routine for the selected fact table. The routine then loops to block 1505 to select the next fact table.
- FIG. 16 is a flow diagram of the load dimension table routine in one embodiment.
- This routine maps the local identifiers used in the local data warehouse to the main identifiers used in the main data warehouse.
- the routine selects the next entry from the dimension table.
- decision block 1602 if all the entries of the dimension table have already been selected, then the routine returns, else the routine continues at block 1603 .
- the routine retrieves an entry from the dimension table of the main data warehouse corresponding to the selected entry.
- decision block 1604 if the entry is retrieved, then the routine continues at block 1606 , else the dimension table does not contain an entry and the routine continues at block 1605 .
- the routine adds an entry to the dimension table of the main data warehouse corresponding to the selected entry from the dimension table of the local data warehouse.
- the routine creates a mapping of the local identifier (e.g., index into the local dimension table) of the selected entry to the main identifier (e.g., index into the main dimension table) for that selected entry.
- the routine then loops to block 1601 to select the next entry of the dimension table.
- FIG. 17 is a flow diagram of the load fact table routine in one embodiment.
- This routine adds the facts of the local data warehouse to the main data warehouse.
- the routine maps the local identifiers for the dimensions used in the local warehouse to the main identifiers of dimensions used in the main data warehouse.
- the routine selects the next entry in the fact table.
- decision block 1702 if all the entries of the fact table have already been selected, then the routine returns, else the routine continues at block 1703 .
- the routine selects the next dimension for the selected entry.
- decision block 1704 if all the dimensions for the selected entry have already been selected, then the routine continues at block 1706 , else the routine continues at block 1705 .
- the routine retrieves the main identifier for the selected dimension and then loops to block 1703 to select the next dimension.
- the routine stores an entry in the fact table of the main data warehouse. The routine then loops to block 1701 to select the next entry in the fact table.
- FIG. 18 is a flow diagram illustrating the identify user aliases routine in one embodiment.
- This routine tracks the different user identifiers as a user switches from one web site to another.
- the routine maps the user identifiers used by a referrer web site to the user identifiers used by the referred-to web site. In this way, the same user can be tracked even though different web sites use different identifiers for that user.
- This routine may be invoked as part of the parsing of the log files.
- decision block 1801 if the log entry indicates a referrer web site, then the routine continues at block 1802 , else the routine returns.
- the routine identifies the user identifier for the referrer web site.
- the routine creates a mapping between the referrer user identifier and the referred-to user identifier. The routine then returns.
Abstract
Description
- The described technology relates to analyzing of data relating to events generated by a computer program.
- Today's computer networking environments, such as the Internet, offer mechanisms for delivering documents between heterogeneous computer systems. One such network, the World Wide Web network, which comprises a subset of Internet sites, supports a standard protocol for requesting and receiving documents known as web pages. This protocol is known as the Hypertext Transfer Protocol, or “HTTP.” HTTP defines a message passing protocol for sending and receiving packets of information between diverse applications. Details of HTTP can be found in various documents including T. Berners-Lee et al.,Hypertext Transfer Protocol—HTTP 1.0, Request for Comments (RFC) 1945, MIT/LCS, May 1996. Each HTTP message follows a specific layout, which includes among other information, a header which contains information specific to the request or response. Further, each HTTP request message contains a universal resource identifier (a “URI”), which specifies to which network resource the request is to be applied. A URI is either a Uniform Resource Locator (“URL”) or Uniform Resource Name (“URN”), or any other formatted string that identifies a network resource. The URI contained in a request message, in effect, identifies the destination machine for a message. URLs, as an example of URIs, are discussed in detail in T. Berners-Lee, et al., Uniform Resource Locators (URI), RFC 1738, CERN, Xerox PARC, Univ. of Minn., December 1994.
- FIG. 1 illustrates how a browser application enables users to navigate among nodes on the web network by requesting and receiving web pages. For the purposes of this discussion, a web page is any type of document that abides by the HTML format. That is, the document includes an “<HTML>” statement. Thus, a web page is also referred to as an HTML document. The HTML format is a document mark-up language, defined by the Hypertext Markup Language (“HTML”) specification. HTML defines tags for specifying how to interpret the text and images stored in an HTML document. For example, there are HTML tags for defining paragraph formats and for emboldening and underlining text. In addition, the HTML format defines tags for adding images to documents and for formatting and aligning text with respect to images. HTML tags appear between angle brackets, for example, <HTML>. Further details of HTML are discussed in T. Berners-Lee and D. Connolly,Hypertext Markup Language—2.0, RFC 1866, MIT/W3C, November 1995.
- In FIG. 1, a
web browser application 101 is shown executing on aclient computer 102, which communicates with aserver computer 103 by sending and receiving HTTP packets (messages). HTTP messages may also be generated by other types of computer programs, such as spiders and crawlers. The web browser “navigates” to new locations on the network to browse (display) what is available at these locations. In particular, when the web browser “navigates” to a new location, it requests a new document from the new location (e.g., the server computer) by sending an HTTP-request message 104 using any well-known underlying communications wire protocol. The HTTP-request message follows the specific layout discussed above, which includes aheader 105 and aURI field 106, which specifies the network location to which to apply the request. When the server computer specified by URI receives the HTTP-request message, it interprets the message packet and sends a return message packet to the source location that originated the message in the form of an HTTP-response message 107. It also stores a copy of the request and basic information about the requesting computer in a log file. In addition to the standard features of an HTTP message, such as theheader 108, the HTTP-response message contains the requested HTMLdocument 109. When the HTTP-response message reaches the client computer, the web browser application extracts the HTML document from the message, and parses and interprets (executes) the HTML code in the document and displays the document on a display screen of the client computer as specified by the HTML tags. HTTP can also be used to transfer other media types, such as the Extensible Markup Language (“XML”) and graphics interchange format (“GIF”) formats. - The World Wide Web is especially conducive to conducting electronic commerce (“e-commerce”). E-commerce generally refers to commercial transactions that are at least partially conducted using the World Wide Web. For example, numerous web sites are available through which a user using a web browser can purchase items, such as books, groceries, and software. A user of these web sites can browse through an electronic catalog of available items to select the items to be purchased. To purchase the items, a user typically adds the items to an electronic shopping cart and then electronically pays for the items that are in the shopping cart. The purchased items can then be delivered to the user via conventional distribution channels (e.g., an overnight courier) or via electronic delivery when, for example, software is being purchased. Many web sites are also informational in nature, rather than commercial in nature. For example, many standards organizations and governmental organizations have web sites with a primary purpose of distributing information. Also, some web sites (e.g., a search engine) provide information and derive revenue from advertisements that are displayed.
- The success of any web-based business depends in large pail on the number of users who visit the business's web site and that number depends in large part on the usefulness and ease-of-use of the web site. Web sites typically collect extensive information on how its users use the site's web pages. This information may include a complete history of each HTTP request received by and each HTTP response sent by the web site. The web site may store this information in a navigation file, also referred to as a log file or click stream file. By analyzing this navigation information, a web site operator may be able to identify trends in the access of the web pages and modify the web site to make it easier to use and more useful. Because the information is presented as a series of events that are not soiled in a useful way, many software tools are available to assist in this analysis. A web site operator would typically purchase such a tool and install it on one of the computers of the web site. There are several drawbacks with the use of such an approach of analyzing navigation information. First, the analysis often is given a low priority because the programmers are typically busy with the high priority task of maintaining the web site. Second, the tools that are available provide little more than standard reports relating to low-level navigation through a web site. Such reports are not very useful in helping a web site operator to visualize and discover high-level access trends. Recognition of these high-level access trends can help a web site operator to design the web site. Third, web sites are typically resource intensive, that is they use a lot of computing resources and may not have available resources to effectively analyze the navigation information.
- It would also be useful to analyze the execution of computer programs, other than web server programs. In particular, many types of computer programs generate events that are logged by the computer programs themselves or by other programs that receive the events. If a computer program does not generate explicit events, another program may be able to monitor the execution and generate events on behalf of that computer program. Regardless of how event data is collected, it may be important to analyze that data. For example, the developer of an operating system may want to track and analyze how the operating system is used so that the developer can focus resources on problems that are detected, optimize services that are frequently accessed, and so on. The operating system may generate a log file that contains entries for various types of events (e.g., invocation of a certain system call).
- FIG. 1 illustrates how a browser application enables users to navigate among nodes on the web network by requesting and receiving web pages.
- FIG. 2A is a block diagram illustrating components of the data warehouse system in one embodiment.
- FIG. 2B is a block diagram illustrating details of the components of the data warehouse system in one embodiment.
- FIG. 3 is a block diagram illustrating the sub-components of the data processor component in one embodiment.
- FIG. 4 is a block diagram illustrating some of the tables of the local data warehouse and the main data warehouse in one embodiment.
- FIG. 5 is a flow diagram illustrating the parse log data routine that implements the parser in one embodiment.
- FIG. 6 is a flow diagram of the filter log entry routine in one embodiment.
- FIG. 7 is a flow diagram illustrating the normalize log entry routine.
- FIG. 8 is a flow diagram of the generate dimensions routine in one embodiment.
- FIG. 9 is a flow diagram of the identify logical site routine in one embodiment.
- FIG. 10 is a flow diagram of the identify user routine in one embodiment.
- FIG. 11 is a flow diagram of the identify page type routine in one embodiment.
- FIG. 12 is a flow diagram illustrating the identify events routine in one embodiment.
- FIG. 13 is a flow diagram illustrating the identify sessions routine in one embodiment.
- FIG. 14 is a flow diagram of the generate aggregate statistics routine in one embodiment.
- FIG. 15 is a flow diagram of the import log data routine implementing the importer in one embodiment.
- FIG. 16 is a flow diagram of the load dimension table routine and one embodiment.
- FIG. 17 is a flow diagram of the load fact table routine in one embodiment.
- FIG. 18 is a flow diagram illustrating the identify user aliases routine in one embodiment.
- A method and system for providing customers with access to and analysis of event data (e.g., navigation data collected at customer web sites) is provided. The event data may be stored in log files and supplemented with data from other sources, such as product databases and customer invoices. In one embodiment, a data warehouse system collects customer data from the customer web sites and stores the data at a data warehouse server. The customer data may include application event data (e.g., click stream log files), user attribute data of users of the customer web site (e.g., name, age, and gender), product data (e.g., catalog of products offered for sale by the customer), shopping cart data (i.e., identification of the products currently in a user's shopping cart), and so on. The data warehouse server interacts with the customer servers to collect to the customer data on a periodic basis. The data warehouse server may provide instructions to the customer servers identifying the customer data that is to be uploaded to the data warehouse server. These instructions may include the names of the files that contains the customer data and the name of the web servers on which the files reside. These instructions may also indicate the time the day when the customer data is to be uploaded to the data warehouse server. When the data warehouse server receives customer data, it converts the customer data into a format that is more conducive to processing by decision support system applications by which customers can analyze their data. For example, the data warehouse server may analyze low-level navigation events (e.g., each HTTP request that is received by the customer web site) to identify high-level events (e.g., a user session). The data warehouse server then stores the converted data into a data warehouse. The data warehouse server functions as an application service provider that provides various decision support system applications to the customers. For example, the data warehouse server provides decision support system applications to analyze and graphically display the results of the analysis for a customer. The decision support system applications may be accessed through a web browser. In one embodiment, the customer servers are connected to the data warehouse server via the Internet and the data warehouse server provides data warehousing services to multiple customers.
- The data warehouse system may provide a data processor component that converts the log files into a format that is more conducive to processing by the decision support system applications. In one embodiment, the converted data is stored in a data warehouse that includes fact and dimension tables. Each fact table contains entries corresponding to a type of fact derived from the log files. For example, a web page access fact table may contain an entry for each web page access identified in the log files. Each entry may reference attributes of the web page access, such as the identity of the web page and identity of the accessing user. The values for each attribute are stored in a dimension table for that attribute. For example, a user dimension table may include an entry for each user and the entries of the web access fact table may include a user field that contains an index (or some other reference) to the entry of the user dimension table for the accessing user. The user dimension table may contain the names of the users and other user-specific information. Alternatively, the user dimension table may itself also be a fact table that includes references to dimension tables for the attributes of users. The data warehouse may also include fact tables and dimension tables that represent high-level facts and attributes derived from the low-level facts and attributes of the log files. For example, high-level facts and attributes may not be derivable from only the data in a single log entry. For example, the higher level category (e.g., shoes or shirts) of a web page may be identified using a mapping of web page URIs to categories. These categories may be stored in a category dimension table. Also, certain facts, such as the collection of log entries that comprise a single user web access session or visit, may only be derivable by analyzing a series of log entries.
- The data processor component may have a parser component and a loader component. The parser of the data processor parses and analyzes a log file and stores the resulting data in a local data warehouse that contains information for only that log file. The local data warehouse may be similar in structure (e.g., similar fact and dimension is tables) to the main data warehouse used by decision support system applications. The local data warehouse may be adapted to allow efficient processing by the parser. For example, the local data warehouse may be stored in primary storage (e.g., main memory) for speed of access, rather than in secondary storage (e.g., disks). The parser may use parser configuration data that defines, on a customer-by-customer basis, the high-level data to be derived from the log entries. For example, the parser configuration data may specify the mapping of URIs to web page categories. The loader of the data processor transfers the data from the local data warehouse to the main data warehouse. The loader may create separate partitions for the main data warehouse. These separate partitions may hold the customer data for a certain time period (e.g., a month's worth of data). The loader adds entries to the main fact tables (i.e., fact tables of the main data warehouse) for each fact in a local fact table (i.e., fact table of the local data warehouse). The loader also adds new entries to the main dimension tables to represent attribute values of the local dimension tables that are not already in the main dimension tables. The loader also maps the local indices (or other references) of the local dimension tables to the main indices used by the main dimension tables.
- FIG. 2A is a block diagram illustrating components of the data warehouse system in one embodiment. The data warehouse system includes customer components that execute on the customer servers and data warehouse components that execute on the data warehouse server. The
customer servers 210 and thedata warehouse server 260 are interconnected via theInternet 250. Customer components executing on a customer server includes adata collection component 220 and adata viewer 230. The data viewer may reside on a client computer of the customer, rather than a server. The data collection component collects the customer data from thestorage devices 240 of the customer servers. The data viewer provides access for viewing of data generated by the decision support system applications of the data warehouse server. In one embodiment, the data viewer may be a web browsed The data warehouse server includes adata receiver component 270, thedata processor component 280, thedata warehouse 290, and decisionsupport system applications 291. The data receiver component receives customer data sent by the data collection components executing at the various customer web sites. The data processor component processes the customer data and stores it in the data warehouse. The decision support system application provides the customer with tools for analyzing and reviewing the customer data that is stored in the main data warehouse. Analysis performed on and reports generated from are described in U.S. patent application Ser. No. ______ (Attorney Ref. No. 34821-8010US), entitled “Identifying and Reporting on Combinations of Events in Usage Data,” and U.S. patent application Ser. No. ______ (Attorney Ref No. 34821-8013US), entitled “Extracting and Displaying Usage Data for Graphical Structures,” which are being filed concurrently and which are hereby incorporated by reference. In one embodiment, each customer has its own set of dimension and fact tables so that the information of multiple customers are not intermingled. - FIG. 2B is a block diagram illustrating details of the components of the data warehouse system in one embodiment. The
data collection component 220 includes amonitor sub-component 221 and apitcher sub-component 222. The data collection component is described in more detail in U.S. patent application Ser. No. ______ (Attorney Ref. No. 34821-800/US), entitled “Method and System for Monitoring Resource via the Web,” which is being filed concurrently and which is hereby incorporated by reference. The pitcher is responsible for retrieving instructions from the data warehouse server, collecting the customer data in accordance with the retrieved instructions, and uploading the customer data to the data warehouse server. The monitor is responsible for monitoring the operation of the pitcher and detecting when the pitcher may have problems in collecting and uploading the customer data. When the monitor detects that a problem may occur, it notifies the data warehouse server so that corrective action may be taken in advance of the collecting and uploading of the customer data. For example, the pitcher may use certain log on information (e.g., user ID and password) to access a customer web server that contains customer data to be uploaded. The monitor may use that log on information to verify that the log on information will permit access to the customer data. Access may be denied if, for example, a customer administrator inadvertentry deleted from the customer web server the user ID used by the pitcher. When the monitor provides advance notification of a problem, the problem might be corrected before the pitcher attempts to access the customer data. The monitor also periodically checks the pitcher to ensure that the pitcher is executing and, if executing, executing correctly. - The data receiver component of the data warehouse server includes a status receiver sub-component271, a
catcher sub-component 272, anFTP server 273, astatus database 274, and a collecteddata database 275. The status receiver receives status reports from the customer servers and stores the status information in the status database. The catcher receives and processes the customer data that is uploaded from the customer web sites and stores the data in the collected data database. The data processor component includes aparser sub-component 281 and aloader sub-component 282. The parser analyzes the low-level events of the customer data and identifies high-level events and converts the customer data into a format that facilitates processing by the decision support system applications. The loader is responsible for storing the identified high-level events in thedata warehouse 290. In one embodiment, a customer may decide not to have the data collection component executing on its computer systems. In such a case, the customer server may include anFTP client 245 that is responsible for periodically transferring the customer data to theFTP server 273 of the data warehouse server. The data receiver may process this customer data at the data warehouse server in the same way as the pitcher processes the data at the customer servers. The processed data is then stored in the collected data database. - FIG. 3 is a block diagram illustrating the sub-components of the data processor component in one embodiment. The
data processor component 300 includes aparser 310, data storage area 370, and aloader 330. The data processor component inputsparser configuration data 340 and alog file 350 and updates themain data warehouse 360. The parser configuration data may include a mapping of actual web sites to logical sites and a mapping of a combination of Uniform Resource Identifiers (“URls”) and query strings of the log entries to page definitions (e.g., categories) and event definitions. The parser processes the entries of the log file to generate facts and dimensions to eventually be stored in the main data warehouse. The parser identifies events in accordance with the parser configuration data. The parser includes a filterlog entry component 311, a normalizelog entry component 312, a generatedimensions component 313, anidentify sessions component 314, and a generateaggregate statistics component 315. The filter log entry component identifies which log entries should not be included in the main data warehouse. For example, a log entry that has an invalid format should not be included. The normalize log entry component normalizes the data in a log entry. For example, the component may convert all times to Greenwich Mean Time (“GMT”). The generate dimensions component identifies the various dimensions related to a log entry. For example, a dimension may be the Uniform Resource Identifier of the entry or the logical site identifier. The identify sessions component processes the parsed log file data stored in the local data warehouse to identify user sessions. A user session generally refers to the concept of a series of web page accesses that may be related in some way, such as by temporal proximity. The generate aggregate statistics component aggregates data for the log file being processed as each log entry is processed or after the log file is parsed. The data storage area 320 includes alocal data warehouse 321. In one embodiment, the local data warehouse is stored non-persistently (or temporarily) in main memory of the computer system. The local data warehouse may contain fact tables and dimension tables that correspond generally to the tables of themain data warehouse 360. The loader retrieves the information from the local data warehouse and stores the information in the main data warehouse. The loader includes a createpartitions component 331, a loaddimension table component 332, and a loadfact table component 333. The create partitions components creates new partitions for the main data warehouse. A partition may correspond to a collection of information within a certain time range. For example, the main data warehouse may have a partition for each month, which contains all the data for that month. The load dimension table component and the load fact table component are responsible for loading the main data warehouse with the dimensions and facts that are stored in the local data warehouse. - In one embodiment, the log file is a web server log file of a customer. The log file may be in the “Extended Log File Foimat” as described in the document “http://www.w3.org/TR/WD-logfile-960323” provided by the World Wide Web Consortium, which is hereby incorporated by reference. According to that description, the log file contains lines that are either directives or entries. An entry corresponds to a single HTTP transaction (e.g., HTTP request and an HTTP response) and consists of a sequence of fields (e.g., integer, fixed, URI, date, time, and string). The meaning of the fields in an entry is specified by a field directive specified in the log file. For example, a field directive may specify that a log entry contains the fields date, time, client IP address, server IP address, and success code. Each entry in the log file would contain these five fields.
- The parser configuration data defines logical sites, page definitions, and event definitions. A logical site is a collection of one or more IP addresses and ports that should be treated as a single web site. For example, a web site may actually have five web servers with different IP addresses that handle HTTP requests for the same domain. These five IP addresses may be mapped to the same logical site to be treated as a single web site. The page definitions define the format of the URIs of log entries that are certain page types. For example, a URI with a query string of “category=shoes” may indicate a page type of “shoes.” Each event definition defines an event type and a value for that event type. For example, a log entry with a query string that includes “search=shoes” represents an event type of “search” with an event value of “shoes.” Another log entry with a query string of “add=99ABC” may represent an event type of “add” an item to the shopping cart with an event value of item number “99ABC.”
- FIG. 4 is a block diagram illustrating some of the tables of the local data warehouse and the main data warehouse in one embodiment. These data warehouses are databases that include fact tables and dimension tables. A fact table contains an entry for each instance of fact (e.g., web page access). A dimension table contains an entry for each possible attribute value of an attribute (e.g., user). The entries of a fact table contain dimension fields that refer to the entries into the dimension tables for their attribute values. A table may be both a fact table and a dimension table. For example, a user dimension table with an entry for each unique user may also be a fact table that refers to attributes of the users that are stored in other dimension tables. The data warehouses contain a log entry table401, a user table 402, a logical site table 403, a URI table 404, a referrer URI table 405, a page type table 406, event type tables 407, a query string table 408, and a referrer query string table 409. The log entry table is a fact table that contains an entry for each log entry that is not filtered out by the parser. The other tables are dimension tables for the log entry table. The user table contains an entry for each unique user identified by the parser. The logical site table contains an entry for each logical site as defined in the parser configuration data. The URI table contains an entry for each unique URI of an entry in the log entry table. The referred URI table contains an entry for each referrer URI of the log entry table. The page type table contains an entry for each page type identified by the parser as defined in the parser configuration data. The data warehouse contains an event table for each type of event defined in the parser configuration data. Each event table contains an entry for each event value of that event type specified in an entry of the log entry table. The query string table contains an entry for each unique query string identified in an entry of the log entry table. The referrer query string contains an entry for each unique referrer query string identified in an entry of the log entry table.
- Table 1 is an example portion of a log file. The “#fields” directive specifies the meaning of the fields in the log entries. Each field in a log entry is separated by a space and an empty field is represented by a hyphen. The #fields directive in this example indicates that each entry includes the date and time when the transaction was completed (i.e., “date” and “time”), the client IP address (i.e., “c-ip”), and so on. For example, the first log entry has a data and time of “2000-06-01 07:00:04” and a client IP address of “165.21.83.61.”
TABLE 1 #Software: Microsoft Internet Information Server 4.0 #Version: 1.0 #Date: 2000-06-01 07:00:04 #Fields: date time c-ip cs-usemame s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query sc-status sc-win32- status sc-bytes cs-bytes time-taken s-port cs-version cs(User-Agent) cs(Cookie) cs( Referrer) 2000-06-01 07:00:04 165.21.83.161 - W3SVC2 COOK_002 206.191.163.41 GET /directory/28.ASP - 200 0 148428 369 9714 80 HTTP/1.0 Mozilla/3.04+(Win95+1) ASPSESSIONIDQQGGQGPG=JBCCFIPBBHHDANBAFFGLGPH http://allrecipes.com/Default.asp 2000-06-01 07:00:20 4.20.197.70 - W3SVC2 COOK_002 206.191.163.41 GET /Default.asp - 302 0 408 259 30 80 HTTP/1.0 Mozilla/4.0+(compatible:+Keynote-Perspective+4.0)- - 2000-06-01 07:00:20 4.20 197.70 - W3SVC2 COOK_002 206.191.163.41 GET /Default.asp - 200 0 41245 266 200 80 HTTP/1.0 Mozilla/4.0+(compatible:+Keynote-Perspective+4.0)- - 2000-06-01 07:00.27 204.182.65.192 - W3SVC2 COOK_002 206.191.163.41 HEAD /Default.asp - 302 0 254 66 40 80 HTTP/1.0 Ipswitch_WhatsUp/3.0 - - 2000-06-01 07:00:32 24.10.69.137 - W3SVC2 COOK_002 206.191.163.41 GET /directory/541.asp - 200 0 22427 459 421 80 HTTP/1.0 Mozilla/4.7+[en](Win98+U) ASPSESSIONIDQQGGQGPG=BHBCFIPBEJPNOMDPKCGLKNGC; +ARSiteUser=1%2DC2B25364%2D3775%2D11D4%2DBC1%2D0050049BD2E4;+ARSites=ALR=1 http://allrecipes.com/directory/34.asp 2000-06-01 07:00:34 192.102.216.101 - W3SVC2 COOK_002 206.191.163.41 GET /encyc/terms/L/7276.asp - 200 0 20385 471 290 80 HTTP/1.0 Mozilla/4.7+[en]+(X11;+1:+SunOS+5.5.5+sun4u) ASPSESSIONIDQQGGQGPG= PKBCFIPBIKONBPDHKDMMEHCE http://search.allrecipes.com/gsearchresults.asp?site=aIlrecipes&allrecipes=allrecipes& allsites=1&q1=loin 2000-06-01 07:00:34 216.88.216.227 - W3SVC2 COOK_002 206.191.163.41 GET /Default.asp -200 0 41253 258 180 80 HTTP//1.1 Mozila/4.0+(compatible:*MSIE+4.01:+MSN+2.5:MSN+2.5:+Windows+98)- - 2000-06-01 07:00:36 199.203.4.10 - W3SVC2 COOK_002 206.191.163.41 GET /Default.asp - 302 0 408 485 30 80 HTTP/1.0 Mozilla/4.0+(compatible:+MSIE:+5.01:+Windows+98:+TUCOWS) SITESERVER=ID=22f117fb3708b2278f3c 426796a78e2a- 2000-06-01 07:00:37 199.203.4.10 - W3SVC2 COOK_002 206.191.163.41 GET /Default.asp - 200 0 41277 492 421 80 HTTP/1.0 Mozilla/4.0+(compatible:+MSIE.+5.01:+Windows+98:+TUCOWS) SITESERVER=ID=22fii71b3708b2278f3c 426796a78e2a- 2000-06-01 07:00:43 24.10.69.137 - W3SVC2 COOK_002 206.191.163.41 GET /directory/34.asp - 200 0 17835 458 320 80 HTTP/1.0 Mozilla/4.7+[en]+(Win98:+U)ASPSESSIONIDQQGGQGPG=BHBCFIPBEJPNOMDPKCGLKNGC; +ARSiteUser=1%2DC2B25364%2D3775%,2D11D4%2D11D4%2DBAC1%2D0050049BD2E4;+ARSites=ALR=1 http://allrecipes. com/directory/25.asp 2000-06-01 0700:47 199.203.4.10 - W3SVC2 COOK_002 206.191.16341 GET /jumpsite.asp jumpsite=5&Go.x=16&Go.y=14 302 0 341 611 40 80 HTTP/1.0 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+98;+ TUCOWS) SITESERVER=ID=22f117tb3708b2278f3c426796a78e2a:+ASPSESSIONIDQQGGQGPG=FCCCFIPBKJM BDJJHBNCOEDGII http://allrecipes.com)Default.asp 2000-06-01 07:00:47 24.10.69.137 - W3SVC2 COOK_002 206.191.163.41 GET /directory/538.asp - 200 0 27471 459 881 80 HTTP/1.0 Mozilla/4.7+[en]+(Win98:+U) ASPSESSIONIDQQGGQGPG=BHBCFIPBEJPNOMDPKCGLKNGC; +ARSiteUser+1%2DC2B25364%2D3775%2D11D4%2DBAC1%2D0050049BD2E4+ARSites=ALR=1 http://allrecipes.com/directory/34.asp 2000-06-01 07:00:47 207.136.48.117 - W3SVC2 COOK_002 206.191.163.41 GET /directory/511.asp - 200 0 77593 369 12538 80 HTTP/1.0 Mozilla/3.01Gold+(Win95:+) ASPSESSIONIDQQGGQGPG=MFACFIPBDBN PBFBOENJKHJN; +ARSiteUser=1%2DC2B251E5%2D3775%2D11D4%2DBAC1%2D0050049BD2E4+ARSites=ALR=1 http://allrecipes.com/directory/506.asp 2000-06-01 07:00.49 192.102.216.101 - W3SVC2 COOK_002 206.191.163.41 GET /encyc/Al.asp ARRefSite= 15&ARRefCookie=1-C2B253B8-3775-11D4-BAC1-0050049BD2E4 200 0 47193 457 260 80 HTTP/1.0 Mozilla/4.7+ [en]+(X11+1;+SunOS+5.5.1.+sun4u) ASPSESSIONIDQQGGQGPG=PKBCFIPBIKONBPDHKDMMEHCE http://porkrecipe.com/hints/tips.asp - Table 2 is an example portion of parser configuration data. The logical site definitions map a server IP address, port, and root URI to a logical site. For example, the entry “LOGICALSITEURIDEFINITION=209.114.94.26,80,/,1” maps all the accesses to port80 of IP address 209.114.94.26 at URIs with a prefix “/” to
logical site 1. The page type definitions map a logical site identifier, URI pattern, and query string pattern to a page type. For example, the entry “PAGEKEYDEFINITION=news item, news item, 1, {prefix}=homepage include/industrynews_detail. asp, <NewsItemID># {Uri}” indicates that a page type of “news item” is specified forlogical site 1 by a URI pattern of “/homepage_include/industrynews_detail.asp.” The definition also indicates that the event value is “<NewsItemID>#{Uri},” where the URI of the log entry is substituted for “{Uri} and the value of NewsItemID in the query string is substituted for “<NewsItemID>.” The event type definitions map a site identifier, URI pattern, and query string pattern to an event type and value. The definitions also specify the name of the event type and the name of the dimension table for that event type. For example, the entry “EVENTDEFINITION=View News Article, View News Article, 1, {prefix}=/homepage_include/industrynews_detail.asp, <NewsItemId>=*, <Newsltemld>” indicates that View News Article event types are stored in the View News Article dimension table. That event type is indicated by a URI with “Homepage_include/industrynews_detail.asp,” and the event value is the sting that follows “<Newsltemld>=” in the query string.TABLE 2 LOGICALSITEURIDEFINITION= 209.114.94.26. 80./.1 PAGEKEYDEFINITION= news item, news item, 1, {prefix}=/homepage_include/industrynews_detail.asp., <NewsItermId>#{Uri} PAGEKEYDEFINITION= page, page, 1...{Uri} EVENTDEFINITION= Login, 1, {prefix} =/:registration/login.asp,, EVENTDEFINITION= Logout, Logout, 1, {prefix}=/registration/logout.asp,, EVENTDEFINITION= Register Page 1,Register Page EVENTDEFINITION= Register Page 2, Register Page 2, 1, {prefix}=/registration/register2.asp. <UserID=,EVENTDEFINITION= Registration Confirmation. Registration Confirmation, 1, {prefix}=/registration/register3.asp,, EVENTDEFINITION= Abort Registration, Abort Registration, 1, {prefix}=/registration/registrationabort.asp,, EVENTDEFINITION= Member Services, Member Services, 1, {prefix}=/registration/memherservices.asp,, EVENTDEFINITION= Change Password, Change Password, 1, 55 prefix}=registration/changepassword.asp,, EVENTDEFINITION= Profile Edit, Profile Edit, 1, {prefix}=/registration/profile.asp,, EVENTDEFINITION= Change Affiliation, Change Affiliation, 1, {prefix}=/registration/changeaffiliation.asp,<UserID>=*, EVENTDEFINITION= Change Secret Question, Change Secret Question, 1, {prefix}=/registration/changesecretquestion.asp,, EVENTDEFINITION= Forgot Information, Forgot Information, 1, {prefix}=/registration/forgotinfo.asp,, EVENTDEFINITION= Forgot Password, Forgot Password, 1. {prefix}=/registration/forgotpassword.asp,, EVENTDEFINITION= Forgot Signin, Forgot Signin, 1, {prefix}=/registration/forgotsignin.asp,, EVENTDEFINITION= View News Article, View News Article, 1, {prefix}=/homepage_include/industrynews_detail.asp, <NewsItemId>=*.<NewsItemId> - FIGS.5-14 are flow diagrams of components of the parser in one embodiment. FIG. 5 is a flow diagram illustrating the parse log data routine that implements the main routine of parser in one embodiment. The routine processes each entry in the log file based on the parser configuration data. The routine filters out certain log entries, normalizes the attribute values of the log entries, and generates entries in the dimension tables for the attributes of the log entries. After processing all the log entries, the parser identifies user sessions and generates various statistics. In blocks 501-508, the routine loops selecting and processing each log entry. In
block 501, the routine selects the next log entry of the log file starting with the first log entry. The routine may also pre-process the header information of the log file to identify the fields of the log entries. Indecision block 1502, if all the log entries have already been selected, then the routine continues atblock 509, else the routine continues atblock 503. Inblock 503, the routine extracts the values for the fields of the selected log entry. Inblock 504, the routine invokes the filter log entry routine, which returns an indication as to whether the selected log entry should be filtered out. Indecision block 505, if the filter log entry routine indicates that the selected log entry should be filtered out, then the routine skips to block 508, else the routine continues atblock 506. Inblock 506, the routine invokes the normalize log entry routine to normalize the values of the fields of the selected log entry. Inblock 507, the routine invokes the generate dimensions routine to update the dimension tables based on the selected log entry and to add an entry into the log entry fact table. Inblock 508, the routine updates the statistics for the log file. For example, the routine may track the number of log entries that have been filtered out. The routine then loops to block 501 to select the next log entry. Inblock 509, the routine outputs the log file statistics. Inblock 510, the routine invokes the identify sessions routine that scans the log entry table to identify the user sessions and updates a session dimension table. Inblock 511, the routine invokes the generate aggregate statistics routine to generate various statistics and then completes. - FIG. 6 is a flow diagram of the filter log entry routine in one embodiment. The filter log entry routine is passed a log entry and determines whether the log entry should be filtered out. In blocks601-607, the routine determines whether the filter out conditions have been satisfied. In
decision block 601, the routine determines whether the log entry has a field count problem. A field count problem arises when the number of fields in the log entry does not correspond to the number of expected fields for that log entry. The number and types of fields may be defined in a “fields” directive line of the log file. Indecision block 602, the routine determines whether the log entry is outside of a specified time range. The routine compares the time field of the log entry to the time range. The time range may be specified so that only those log entries within that time range are processed. Indecision block 603, the routine determines whether the IP address of the log entry should be ignored. For example, a log entry may be ignored if the entry originated from a server whose function is to ping the customer's web server at periodic intervals. Indecision block 604, the routine determines whether the log entry corresponds to a comment (e.g., a “g#remarks” directive). Indecision block 605, the routine determines whether the success code associated with the log entry indicates that log entry should be ignored. For example, if the success code indicates a failure, then the log entry may be ignored. Indecision block 606, the routine determines whether the log entry is requesting a resource whose extension indicates that the log entry should be ignored. For example, the routine may ignore log entries requesting graphic files, such as those in the “.gif” format. Indecision block 607, the routine determines whether the values within the fields of the log entry are corrupt. For example, a value in the date field that indicates a date of February 30th is corrupt. One skilled in the art would appreciate that the various filtering, conditions may be specified in a configuration file. For example, the time range, IP addresses, and so on may be specified in the configuration file. These configuration files may be specified on a customer-by-customer basis. - FIG. 7 is a flow diagram illustrating the normalize log entry routine. The routine normalizes the values of the fields in the passed log entry. In
block 701, the routine converts the time of the log entry into a standard time such as Greenwich Mean Time. Inblock 702, the routine collects the time based on the variation between the times of the customer web servers. For example, the time of one web server may be five minutes ahead of the time of another web server. This correction may be based on current time information collected from computer systems that generated the events and then correlated to base current time information. Inblock 703, the routine normalizes, the values of the fields of the log entry. This normalization may include processing search strings to place them in a canonical form. For example, a search string of “back pack” may have a canonical form of “backpack.” Other normalization of search strings may include stemming of words (e.g., changing “clothes” and “clothing” to “cloth”), synonym matching, and first and last word grouping. The first word grouping for the search strings of “winter clothing” and “winter shoes” results in the string of “winter.” - FIG. 8 is a flow diagram of the generate dimensions routine in one embodiment. This routine identifies a value for each dimension associated with the passed log entry and ensures that the dimension tables contains entries corresponding to those values. In one embodiment, each entry in a dimension table includes the attribute value (e.g., user identifier) and a hash value. The hash value may be used by the loader when transferring information to the main data warehouse. Also, each entry has a local identifier, which may be an index into the local dimension table. The loader maps these local identifiers to their corresponding main identifiers that are used in the main data warehouse. In
block 801, the routine invokes a routine that identifies the logical site associated with the log entry and ensures that an entry for the logical site is in the logical site dimension table. Inblock 802, the routine invokes a routine that identifies the user associated with the log entry and ensures that an entry for the user is in the user dimension table. Inblock 803, the routine invokes a routine that identifies the URI associated with log entry and ensures that an entry for that URI is in the URI dimension table. Inblock 804, the routine invokes a routine that identifies the page type based on the parser configuration data and ensures that an entry for that page type is in the page type dimension table. Inblock 805, the routine invokes a routine that identifies the various events associated with the log entry based on the parser configuration data and ensures that an entry for each event type is in the corresponding event table. Inblock 806, the routine identifies other dimensions (e.g., referrer URI) as appropriate. Inblock 807, the routine adds an entry to the log entry table that is linked to each of the identified dimensions using the local identifiers. Inblock 808, the routine updates the statistics information based on the log entry and then returns. - FIG. 9 is a flow diagram of the identify logical site routine in one embodiment. This routine compares the site information of the passed log entry with the logical site definitions in the parser configuration data. In
block 901, the routine selects the next logical site definition from the parser configuration data. Indecision block 902, if all the logical site definitions have already been selected, then the routine continues theblock 905, else the routine continues atblock 903. Indecision block 903, if the URI of the log entry matches the selected logical site definition, then the routine continues atblock 904, else the routine loops to block 901 to select the next logical site definition. Inblock 904, the routine updates the logical site dimension table to ensure that it contains an entry for the logical site defined by the selected logical site definition. The routine then returns. Inblock 905, the routine updates the logical site dimension table to ensure that it contains a default logical site definition and then returns. The log entries that do not map to a logical site definition are mapped to a default logical site. - FIG. 10 is a flow diagram of the identify user routine in one embodiment. This routine may use various techniques to identify the user associated with the passed log entry. In one embodiment, the selection of the technique is configured based on the customer web site. For example, one customer may specify to use a cookie to identify users. In absence of a user identifier in the cookie, the industry norm is to identify users based on their IP addresses. This routine illustrates a technique in which a combination of cookies and IP addresses are used to identify a user. In
block 1001, the routine extracts the user identifier from the cookie associated with the log entry. The format of a cookie may be specified on a customer-by-customer basis. Indecision block 1002, if the extraction from the cookie was successful, then the routine continues atblock 1006, else the routine continues atblock 1003. The extraction may not be successful if, for example, the log entry did not include a cookie. Inblock 1003, the routine extracts the IP address from the log entry. Indecision block 1004, if the IP address is determined to be unique, then routine continues atblock 1006, else the routine continues atblock 1005. Certain IP addresses may not be unique. For example, an Internet service provider may use one IP address for many of its users. The Internet service provider performs the mapping of the one IP address to the various users. Inblock 1005, the routine extracts the browser identifier from the log entry. The combination of IP address and browser identifier may uniquely identify a user. Inblock 1006, the routine updates the user dimension table to ensure that it has an entry for this user and then returns. - FIG. 11 is a flow diagram of the identify page type routine in one embodiment. This routine uses the page type definitions of the parser configuration data to identify the page type associated with the log entry. In
block 1101, the routine selects the next page type definition from the parser configuration data. Indecision block 1101, if all the page type definitions have already been selected, then no matching page type has been found and the routine returns, else the routine continues atblock 1103. Indecision block 1103, if the log entry matches the selected page type definition, then the routine continues atblock 1104, else the routine loops to block 1101 to select the next page type definition. Inblock 1104, the routine updates the page type dimension table to ensure that it contains an entry for the page type represented by the selected page type definition. The routine then returns. - FIG. 12 is a flow diagram illustrating the identify events routine in one embodiment. This routine determines whether the log entry corresponds to any of the events specified in the parser configuration data. In
block 1201, the routine selects the next type of event from the parser configuration data. Indecision block 1202, if all the event types have already been selected, then the routine returns, else the routine continues atblock 1203. Inblock 1203, the routine selects the next event definition of the selected event type. Indecision block 1204, if all the event definitions of the selected event type have already been selected, then the log entry does not correspond to this type of event and the routine loops to block 1201 to select the next type of event, else the routine continues atblock 1205. Inblock 1205, if the log entry matches the selected event definition, then the routine continues atblock 1206, else the routine loops to block 1203 to select the next event definition of the selected event type. Inblock 1206, the routine updates the dimension table for the selected type of the event to ensure that it contains an entry for the selected event definition. The routine then loops to block 1201 to select the next type of event. In this way, the routine matches no more than one event definition for a given event type. For example, if there are two event definitions for the event type “Keyword Search,” then if the first one processed matches, then the second one is ignored. - FIG. 13 is a flow diagram illustrating the identify sessions routine in one embodiment. This routine scans the log entry table of the local data warehouse to identify user sessions. In one embodiment, a user session may be delimited by a certain period of inactivity (e.g., thirty minutes). The criteria for identifying a session may be configurable on a customer-by-customer basis. In
block 1301, the routine selects the next user from the user dimension table. Indecision block 1302, if all the users have already been selected, then the routine returns, else the routine continues atblock 1303. Inblock 1303, the routine selects the next log entry for the selected user in time order. Indecision block 1304, if all log entries for the selected user have already been selected, then the routine loops to block 1301 to select the next user, else the routine continues atblock 1305. Indecision block 1305, if the selected log entry indicates that a new session is starting (e.g., its time is more than 30 minutes greater than that of the last log entry processed), then the routine continues atblock 1306, else the routine loops to block 1303 to select the next log entry for the selected user. Inblock 1306, the routine updates a session fact table to add an indication of the new session. The routine then loops to block 1303 to select the next log entry for the selected user. The routine may also update the log entries to reference their sessions. - FIG. 14 is a flow diagram of the generate aggregate statistics routine in one embodiment. This routine generate statistics based on analysis of the fact and dimension tables used by the parser. In
block 1401, the routine selects the next fact table of intent. Indecision block 1402, if all the fact tables have already been selected, then the routine returns, else the routine continues atblock 1403. Inblock 1403, the routine selects the next entry of the selected fact table. Indecision block 1404, if all the entries of the selected fact table have already been selected, then the routine loops to block 1401 to select the next fact table, else the routine continues atblock 1405. Inblock 1405, the routine aggregates various statistics about the selected fact table. The routine then loops to block 1404 to select the next entry of the fact table. - FIGS.15-17 are flow diagrams illustrating components of the loader in one embodiment. FIG. 15 is a flow diagram of the load log data routine implementing the main routine of the loader in one embodiment. This routine controls the moving of the data from the local data warehouse (created and used by the parser) into the main data warehouse. In
block 1501, the routine invokes the create partitions routine to create partitions for the main data warehouse as appropriate. In blocks 1502-1504, the routine loops loading the dimension tables into the main data warehouse. Inblock 1502, the routine selects the next dimension table. Indecision block 1503, if all the dimension tables have already been selected, then the routine continues atblock 1505, else the routine continues atblock 1504. Inblock 1504, the routine invokes the load dimension table routine for the selected dimension table. The routine then loops to block 1502 to select the next dimension table. In blocks 1505-1507, the routine loops adding the entries to the fact tables of the main data warehouse. Inblock 1505, the routine selects the next fact table in order. The order in which the fact tables are to be loaded may be specified by configuration information. The fact tables may be loaded in order based on their various dependencies. For example, a log entry fact table may be dependent on a user dimension table that is itself a fact table. Indecision block 1506, if all the fact tables have already been loaded, then the routine returns, else the routine continues atblock 1507. Inblock 1507, the routine invokes the load fact table routine for the selected fact table. The routine then loops to block 1505 to select the next fact table. - FIG. 16 is a flow diagram of the load dimension table routine in one embodiment. This routine maps the local identifiers used in the local data warehouse to the main identifiers used in the main data warehouse. In
block 1601, the routine selects the next entry from the dimension table. Indecision block 1602, if all the entries of the dimension table have already been selected, then the routine returns, else the routine continues atblock 1603. Inblock 1603, the routine retrieves an entry from the dimension table of the main data warehouse corresponding to the selected entry. Indecision block 1604, if the entry is retrieved, then the routine continues atblock 1606, else the dimension table does not contain an entry and the routine continues atblock 1605. Inblock 1605, the routine adds an entry to the dimension table of the main data warehouse corresponding to the selected entry from the dimension table of the local data warehouse. Inblock 1606, the routine creates a mapping of the local identifier (e.g., index into the local dimension table) of the selected entry to the main identifier (e.g., index into the main dimension table) for that selected entry. The routine then loops to block 1601 to select the next entry of the dimension table. - FIG. 17 is a flow diagram of the load fact table routine in one embodiment. This routine adds the facts of the local data warehouse to the main data warehouse. The routine maps the local identifiers for the dimensions used in the local warehouse to the main identifiers of dimensions used in the main data warehouse. In
block 1701, the routine selects the next entry in the fact table. Indecision block 1702, if all the entries of the fact table have already been selected, then the routine returns, else the routine continues atblock 1703. Inblock 1703, the routine selects the next dimension for the selected entry. Indecision block 1704, if all the dimensions for the selected entry have already been selected, then the routine continues atblock 1706, else the routine continues atblock 1705. Inblock 1705, the routine retrieves the main identifier for the selected dimension and then loops to block 1703 to select the next dimension. Inblock 1706, the routine stores an entry in the fact table of the main data warehouse. The routine then loops to block 1701 to select the next entry in the fact table. - FIG. 18 is a flow diagram illustrating the identify user aliases routine in one embodiment. This routine tracks the different user identifiers as a user switches from one web site to another. In particular, the routine maps the user identifiers used by a referrer web site to the user identifiers used by the referred-to web site. In this way, the same user can be tracked even though different web sites use different identifiers for that user. This routine may be invoked as part of the parsing of the log files. In
decision block 1801, if the log entry indicates a referrer web site, then the routine continues atblock 1802, else the routine returns. Inblock 1802, the routine identifies the user identifier for the referrer web site. Inblock 1803, the routine creates a mapping between the referrer user identifier and the referred-to user identifier. The routine then returns. - From the above description it will be appreciated that although specific embodiments of the technology have been described for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention. For example, the processing of the parser may be performed by the data collection component before sending the data to the data warehouse server. Accordingly, the invention is not limited except by the appended claims.
Claims (49)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/320,171 US20030120670A1 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/613,847 US6785666B1 (en) | 2000-07-11 | 2000-07-11 | Method and system for parsing navigation information |
US10/320,171 US20030120670A1 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/613,847 Division US6785666B1 (en) | 2000-07-11 | 2000-07-11 | Method and system for parsing navigation information |
Publications (1)
Publication Number | Publication Date |
---|---|
US20030120670A1 true US20030120670A1 (en) | 2003-06-26 |
Family
ID=24458912
Family Applications (8)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/613,847 Expired - Lifetime US6785666B1 (en) | 2000-07-11 | 2000-07-11 | Method and system for parsing navigation information |
US10/005,224 Expired - Lifetime US7117193B1 (en) | 2000-07-11 | 2001-12-05 | Parsing navigation information to identify occurrences of events of interest |
US10/007,646 Expired - Lifetime US6917972B1 (en) | 2000-07-11 | 2001-12-05 | Parsing navigation information to identify occurrences corresponding to defined categories |
US10/320,288 Abandoned US20030131081A1 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
US10/320,091 Abandoned US20030126111A1 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
US10/320,092 Expired - Lifetime US6873981B2 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
US10/320,171 Abandoned US20030120670A1 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
US11/495,089 Expired - Lifetime US7464122B1 (en) | 2000-07-11 | 2006-07-27 | Parsing navigation information to identify occurrences of events of interest |
Family Applications Before (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/613,847 Expired - Lifetime US6785666B1 (en) | 2000-07-11 | 2000-07-11 | Method and system for parsing navigation information |
US10/005,224 Expired - Lifetime US7117193B1 (en) | 2000-07-11 | 2001-12-05 | Parsing navigation information to identify occurrences of events of interest |
US10/007,646 Expired - Lifetime US6917972B1 (en) | 2000-07-11 | 2001-12-05 | Parsing navigation information to identify occurrences corresponding to defined categories |
US10/320,288 Abandoned US20030131081A1 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
US10/320,091 Abandoned US20030126111A1 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
US10/320,092 Expired - Lifetime US6873981B2 (en) | 2000-07-11 | 2002-12-16 | Method and system for parsing navigation information |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/495,089 Expired - Lifetime US7464122B1 (en) | 2000-07-11 | 2006-07-27 | Parsing navigation information to identify occurrences of events of interest |
Country Status (1)
Country | Link |
---|---|
US (8) | US6785666B1 (en) |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050267889A1 (en) * | 2004-02-09 | 2005-12-01 | Coremetrics, Inc. | System and method of managing software product-line customizations |
US20060080271A1 (en) * | 2004-10-08 | 2006-04-13 | Jeff Williams | Dynamic reporting tool for a fact warehouse, a fact contained within the fact warehouse, and a method for refreshing the fact warehouse |
EP1716508A1 (en) * | 2004-01-19 | 2006-11-02 | Nigel Hamilton | Method and system for recording search trails across one or more search engines in a communications network |
US20080059310A1 (en) * | 2006-09-05 | 2008-03-06 | Thomas Publishing Company | Marketing method and system using domain knowledge |
US20110264710A1 (en) * | 2010-04-27 | 2011-10-27 | International Business Machines Corporation | Monitoring and reporting productivity in enterprise environment |
US20110307509A1 (en) * | 2010-03-23 | 2011-12-15 | Google Inc. | Conversion Path Performance Measures And Reports |
US8521774B1 (en) | 2010-08-20 | 2013-08-27 | Google Inc. | Dynamically generating pre-aggregated datasets |
US20140214778A1 (en) * | 2006-02-17 | 2014-07-31 | Google Inc. | Entity Normalization Via Name Normalization |
CN105718565A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data warehouse model construction method and construction apparatus |
CN107241220A (en) * | 2017-05-26 | 2017-10-10 | 京信通信系统(中国)有限公司 | The aging of product monitors automatic analysis method and system |
US11055282B1 (en) * | 2020-03-31 | 2021-07-06 | Atlassian Pty Ltd. | Translating graph queries into efficient network protocol requests |
US11093985B2 (en) * | 2018-09-25 | 2021-08-17 | Valideck International | System, devices, and methods for acquiring and verifying online information |
Families Citing this family (153)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7543078B2 (en) * | 1999-12-31 | 2009-06-02 | Subdomain Identity Partners | Individuals' URL identity exchange and communications |
US7266510B1 (en) * | 2000-09-01 | 2007-09-04 | International Business Machines Corporation | Method for graphically representing clickstream data of a shopping session on a network with a parallel coordinate system |
US8255791B2 (en) | 2000-11-29 | 2012-08-28 | Dov Koren | Collaborative, flexible, interactive real-time displays |
JP3276145B1 (en) * | 2000-12-26 | 2002-04-22 | 株式会社ログ・モンスターズ | Internet advertising method and system |
US7447182B2 (en) * | 2001-04-06 | 2008-11-04 | Nortel Networks Limited | Discovering an address of a name server |
US7062551B2 (en) * | 2001-05-24 | 2006-06-13 | International Business Machines Corporation | Method and apparatus to solve compatibility between heterogeneous web server access logs formats |
US6963874B2 (en) * | 2002-01-09 | 2005-11-08 | Digital River, Inc. | Web-site performance analysis system and method utilizing web-site traversal counters and histograms |
US20030079136A1 (en) * | 2001-08-21 | 2003-04-24 | Emmanuel Ericta | Security framework |
US8429688B1 (en) * | 2001-09-19 | 2013-04-23 | At&T Intellectual Property I, L.P. | Broadcast programming guide |
US10255362B2 (en) * | 2001-11-28 | 2019-04-09 | Benjamin Rodefer | Method for performing a search, and computer program product and user interface for same |
US7107338B1 (en) * | 2001-12-05 | 2006-09-12 | Revenue Science, Inc. | Parsing navigation information to identify interactions based on the times of their occurrences |
US20030131106A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Web-page performance toolbar |
US7631035B2 (en) * | 2002-01-09 | 2009-12-08 | Digital River, Inc. | Path-analysis toolbar |
US20030131097A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Interactive path analysis |
US20030128231A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Dynamic path analysis |
US20030131093A1 (en) * | 2002-01-09 | 2003-07-10 | International Business Machines Corporation | System for generating usage data in a distributed information processing environment and method therefor |
US20030163586A1 (en) * | 2002-02-27 | 2003-08-28 | Steve Schnetzler | Server persistence using a URL identifier |
JP2003337644A (en) * | 2002-03-14 | 2003-11-28 | Sony Corp | Electronic device, program, program providing device and recording medium |
US20030204490A1 (en) * | 2002-04-24 | 2003-10-30 | Stephane Kasriel | Web-page collaboration system |
US20030202009A1 (en) * | 2002-04-24 | 2003-10-30 | Stephane Kasriel | Integration toolbar |
US7254632B2 (en) * | 2002-04-26 | 2007-08-07 | P-Cube Ltd. | Apparatus and method for pattern matching in text based protocol |
US7613994B2 (en) | 2002-05-29 | 2009-11-03 | International Business Machines Corporation | Document handling in a web application |
US20030233343A1 (en) * | 2002-06-18 | 2003-12-18 | International Business Machines Corporation | System and method for generating custom business reports for a WEB application |
US8219709B2 (en) * | 2002-07-05 | 2012-07-10 | Carolyn J Hughes | Method for internet name sharing |
EP1533958A4 (en) * | 2002-07-22 | 2007-09-05 | Fujitsu Ltd | Electronic mail server; electronic mail delivery relaying method; and computer program |
US8122153B2 (en) * | 2002-07-31 | 2012-02-21 | Subdomain Identity Partners | Individuals' URL identity exchange and communications |
US7640267B2 (en) | 2002-11-20 | 2009-12-29 | Radar Networks, Inc. | Methods and systems for managing entities in a computing device using semantic objects |
US7584208B2 (en) | 2002-11-20 | 2009-09-01 | Radar Networks, Inc. | Methods and systems for managing offers and requests in a network |
US7035838B2 (en) * | 2002-12-06 | 2006-04-25 | General Electric Company | Methods and systems for organizing information stored within a computer network-based system |
US20040158619A1 (en) * | 2003-02-10 | 2004-08-12 | Claus Pedersen | Method and apparatus for provisioning content |
US20040193612A1 (en) * | 2003-03-31 | 2004-09-30 | Chang William I. | System and method for testing, monitoring, and tracking distributed transactions using a search engine |
US20040254919A1 (en) * | 2003-06-13 | 2004-12-16 | Microsoft Corporation | Log parser |
US8112458B1 (en) | 2003-06-17 | 2012-02-07 | AudienceScience Inc. | User segmentation user interface |
US7966333B1 (en) * | 2003-06-17 | 2011-06-21 | AudienceScience Inc. | User segment population techniques |
US20050027695A1 (en) * | 2003-07-29 | 2005-02-03 | John Peter Thomas | Identifying files suitable for attaching to a text message |
US9928522B2 (en) | 2003-08-01 | 2018-03-27 | Oath (Americas) Inc. | Audience matching network with performance factoring and revenue allocation |
US9118812B2 (en) * | 2003-08-01 | 2015-08-25 | Advertising.Com Llc | Audience server |
US8150732B2 (en) * | 2003-08-01 | 2012-04-03 | Tacoda Llc | Audience targeting system with segment management |
US8464290B2 (en) * | 2003-08-01 | 2013-06-11 | Tacoda, Inc. | Network for matching an audience with deliverable content |
US20050125290A1 (en) * | 2003-08-01 | 2005-06-09 | Gil Beyda | Audience targeting system with profile synchronization |
US9117217B2 (en) | 2003-08-01 | 2015-08-25 | Advertising.Com Llc | Audience targeting with universal profile synchronization |
US7805332B2 (en) | 2003-08-01 | 2010-09-28 | AOL, Inc. | System and method for segmenting and targeting audience members |
US8094804B2 (en) | 2003-09-26 | 2012-01-10 | Avaya Inc. | Method and apparatus for assessing the status of work waiting for service |
US20050114321A1 (en) * | 2003-11-26 | 2005-05-26 | Destefano Jason M. | Method and apparatus for storing and reporting summarized log data |
US20050114505A1 (en) | 2003-11-26 | 2005-05-26 | Destefano Jason M. | Method and apparatus for retrieving and combining summarized log data in a distributed log data processing system |
US7599939B2 (en) * | 2003-11-26 | 2009-10-06 | Loglogic, Inc. | System and method for storing raw log data |
US8234256B2 (en) | 2003-11-26 | 2012-07-31 | Loglogic, Inc. | System and method for parsing, summarizing and reporting log data |
US20050114706A1 (en) * | 2003-11-26 | 2005-05-26 | Destefano Jason Michael | System and method for the collection and transmission of log data over a wide area network |
US7617531B1 (en) * | 2004-02-18 | 2009-11-10 | Citrix Systems, Inc. | Inferencing data types of message components |
US7433876B2 (en) * | 2004-02-23 | 2008-10-07 | Radar Networks, Inc. | Semantic web portal and platform |
US7953859B1 (en) | 2004-03-31 | 2011-05-31 | Avaya Inc. | Data model of participation in multi-channel and multi-party contacts |
US7734032B1 (en) | 2004-03-31 | 2010-06-08 | Avaya Inc. | Contact center and method for tracking and acting on one and done customer contacts |
US8000989B1 (en) | 2004-03-31 | 2011-08-16 | Avaya Inc. | Using true value in routing work items to resources |
US8868670B2 (en) * | 2004-04-27 | 2014-10-21 | Avaya Inc. | Method and apparatus for summarizing one or more text messages using indicative summaries |
US7337370B2 (en) * | 2004-05-28 | 2008-02-26 | International Business Machines Corporation | System and method for capturing significant events at web portlets |
US7609846B2 (en) * | 2004-07-13 | 2009-10-27 | Eastman Kodak Company | Matching of digital images to acquisition devices |
US8738412B2 (en) | 2004-07-13 | 2014-05-27 | Avaya Inc. | Method and apparatus for supporting individualized selection rules for resource allocation |
US20060020510A1 (en) * | 2004-07-20 | 2006-01-26 | Vest Herb D | Method for improved targeting of online advertisements |
US20070271145A1 (en) * | 2004-07-20 | 2007-11-22 | Vest Herb D | Consolidated System for Managing Internet Ads |
US8234141B1 (en) | 2004-09-27 | 2012-07-31 | Avaya Inc. | Dynamic work assignment strategies based on multiple aspects of agent proficiency |
US7949121B1 (en) | 2004-09-27 | 2011-05-24 | Avaya Inc. | Method and apparatus for the simultaneous delivery of multiple contacts to an agent |
US7580922B2 (en) * | 2005-01-04 | 2009-08-25 | International Business Machines Corporation | Methods for relating data in healthcare databases |
JP4720213B2 (en) * | 2005-02-28 | 2011-07-13 | 富士通株式会社 | Analysis support program, apparatus and method |
US7809752B1 (en) | 2005-04-14 | 2010-10-05 | AudienceScience Inc. | Representing user behavior information |
US7676467B1 (en) | 2005-04-14 | 2010-03-09 | AudienceScience Inc. | User segment population techniques |
US8316446B1 (en) | 2005-04-22 | 2012-11-20 | Blue Coat Systems, Inc. | Methods and apparatus for blocking unwanted software downloads |
US7809127B2 (en) | 2005-05-26 | 2010-10-05 | Avaya Inc. | Method for discovering problem agent behaviors |
US8330759B1 (en) | 2005-06-30 | 2012-12-11 | AudienceScience Inc. | Defining one or more used segments based upon extent of completion of a process |
US20070033233A1 (en) * | 2005-08-05 | 2007-02-08 | Hwang Min J | Log management system and method of using the same |
US7779042B1 (en) | 2005-08-08 | 2010-08-17 | Avaya Inc. | Deferred control of surrogate key generation in a distributed processing architecture |
US7822587B1 (en) | 2005-10-03 | 2010-10-26 | Avaya Inc. | Hybrid database architecture for both maintaining and relaxing type 2 data entity behavior |
GB2446723A (en) * | 2005-10-06 | 2008-08-20 | Avaya Tech Llc | Database data extensions |
US7752230B2 (en) * | 2005-10-06 | 2010-07-06 | Avaya Inc. | Data extensibility using external database tables |
US7787609B1 (en) | 2005-10-06 | 2010-08-31 | Avaya Inc. | Prioritized service delivery based on presence and availability of interruptible enterprise resources with skills |
US20070192461A1 (en) * | 2005-11-03 | 2007-08-16 | Robert Reich | System and method for dynamically generating and managing an online context-driven interactive social network |
US20070106746A1 (en) * | 2005-11-04 | 2007-05-10 | Sijian Zhang | Method and System Showing The Most Meaningful Page From A Book Or Content To Customers As The First Page |
US7565366B2 (en) * | 2005-12-14 | 2009-07-21 | Microsoft Corporation | Variable rate sampling for sequence analysis |
US7571151B1 (en) | 2005-12-15 | 2009-08-04 | Gneiss Software, Inc. | Data analysis tool for analyzing data stored in multiple text files |
US8737173B2 (en) | 2006-02-24 | 2014-05-27 | Avaya Inc. | Date and time dimensions for contact center reporting in arbitrary international time zones |
US20070208784A1 (en) * | 2006-03-03 | 2007-09-06 | Beeston Ralph T | Parsing computer system logging information collected by common logging |
US20070239799A1 (en) * | 2006-03-29 | 2007-10-11 | Anirudh Modi | Analyzing log files |
WO2008021832A2 (en) | 2006-08-09 | 2008-02-21 | Radar Networks, Inc. | Harvesting data from page |
US7936867B1 (en) | 2006-08-15 | 2011-05-03 | Avaya Inc. | Multi-service request within a contact center |
US8391463B1 (en) | 2006-09-01 | 2013-03-05 | Avaya Inc. | Method and apparatus for identifying related contacts |
US8938063B1 (en) | 2006-09-07 | 2015-01-20 | Avaya Inc. | Contact center service monitoring and correcting |
US8811597B1 (en) | 2006-09-07 | 2014-08-19 | Avaya Inc. | Contact center performance prediction |
SE0950005L (en) * | 2006-11-10 | 2008-05-11 | Densitech Ab | Providing security in relation to mobile terminals |
US20090055369A1 (en) * | 2007-02-01 | 2009-02-26 | Jonathan Phillips | System, method and apparatus for implementing dynamic community formation processes within an online context-driven interactive social network |
US7917507B2 (en) * | 2007-02-12 | 2011-03-29 | Microsoft Corporation | Web data usage platform |
US8429185B2 (en) | 2007-02-12 | 2013-04-23 | Microsoft Corporation | Using structured data for online research |
US7822755B2 (en) * | 2007-03-06 | 2010-10-26 | Yahoo! Inc. | Methods of processing and segmenting web usage information |
US20080228773A1 (en) * | 2007-03-14 | 2008-09-18 | Hand Held Products, Inc. | Apparatus and method for data input device |
US7912825B2 (en) * | 2007-06-21 | 2011-03-22 | Microsoft Corporation | Configurable plug-in architecture for manipulating XML-formatted information |
US8381193B2 (en) * | 2007-09-06 | 2013-02-19 | International Business Machines Corporation | Apparatus, system, and method for visual log analysis |
US20090076887A1 (en) | 2007-09-16 | 2009-03-19 | Nova Spivack | System And Method Of Collecting Market-Related Data Via A Web-Based Networking Environment |
US8504534B1 (en) | 2007-09-26 | 2013-08-06 | Avaya Inc. | Database structures and administration techniques for generalized localization of database items |
US8306967B2 (en) | 2007-10-02 | 2012-11-06 | Loglogic, Inc. | Searching for associated events in log data |
US8244667B1 (en) * | 2007-10-18 | 2012-08-14 | Google Inc. | Querying multidimensional data with independent fact and dimension pipelines combined at query time |
US7925694B2 (en) | 2007-10-19 | 2011-04-12 | Citrix Systems, Inc. | Systems and methods for managing cookies via HTTP content layer |
US9552356B1 (en) * | 2007-12-21 | 2017-01-24 | Amazon Technologies, Inc. | Merging client-side and server-side logs |
US8856182B2 (en) | 2008-01-25 | 2014-10-07 | Avaya Inc. | Report database dependency tracing through business intelligence metadata |
US8090877B2 (en) | 2008-01-26 | 2012-01-03 | Citrix Systems, Inc. | Systems and methods for fine grain policy driven cookie proxying |
US20090271497A1 (en) * | 2008-04-29 | 2009-10-29 | Lee Roberts | Affiliate Program Redirection System |
US8244224B2 (en) * | 2008-11-20 | 2012-08-14 | Research In Motion Limited | Providing customized information to a user based on identifying a trend |
US8260926B2 (en) | 2008-11-25 | 2012-09-04 | Citrix Systems, Inc. | Systems and methods for GSLB site persistence |
CN101499098B (en) * | 2009-03-04 | 2012-07-11 | 阿里巴巴集团控股有限公司 | Web page assessed value confirming and employing method and system |
US8200617B2 (en) | 2009-04-15 | 2012-06-12 | Evri, Inc. | Automatic mapping of a location identifier pattern of an object to a semantic type using object metadata |
US10628847B2 (en) | 2009-04-15 | 2020-04-21 | Fiver Llc | Search-enhanced semantic advertising |
WO2010120929A2 (en) * | 2009-04-15 | 2010-10-21 | Evri Inc. | Generating user-customized search results and building a semantics-enhanced search engine |
US8862579B2 (en) | 2009-04-15 | 2014-10-14 | Vcvc Iii Llc | Search and search optimization using a pattern of a location identifier |
CN102301652A (en) * | 2009-04-27 | 2011-12-28 | 国际商业机器公司 | Message switching |
JP5338461B2 (en) * | 2009-05-01 | 2013-11-13 | ブラザー工業株式会社 | Management device, information generation program, and information generation method |
US8549019B2 (en) * | 2009-05-26 | 2013-10-01 | Google Inc. | Dynamically generating aggregate tables |
US9305105B2 (en) * | 2009-05-26 | 2016-04-05 | Google Inc. | System and method for aggregating analytics data |
US20100306363A1 (en) * | 2009-05-26 | 2010-12-02 | Erwien Saputra | Determining completion of a web server download session at a database server |
US8751544B2 (en) * | 2009-09-02 | 2014-06-10 | Google Inc. | Method and system for pivoting a multidimensional dataset |
US8412719B1 (en) | 2009-09-02 | 2013-04-02 | Google Inc. | Method and system for segmenting a multidimensional dataset |
US8565386B2 (en) | 2009-09-29 | 2013-10-22 | Avaya Inc. | Automatic configuration of soft phones that are usable in conjunction with special-purpose endpoints |
US8583584B2 (en) * | 2009-10-20 | 2013-11-12 | Google Inc. | Method and system for using web analytics data for detecting anomalies |
US20110119100A1 (en) * | 2009-10-20 | 2011-05-19 | Jan Matthias Ruhl | Method and System for Displaying Anomalies in Time Series Data |
US9516069B2 (en) | 2009-11-17 | 2016-12-06 | Avaya Inc. | Packet headers as a trigger for automatic activation of special-purpose softphone applications |
US8286258B2 (en) * | 2009-12-04 | 2012-10-09 | Institute For Information Industry | Monitor method and monitor apparatus for monitoring data of hardware |
US8108377B2 (en) * | 2009-12-15 | 2012-01-31 | Facebook, Inc. | Predictive resource identification and phased delivery of structured documents |
US8768926B2 (en) * | 2010-01-05 | 2014-07-01 | Yahoo! Inc. | Techniques for categorizing web pages |
US20150213484A1 (en) * | 2010-03-22 | 2015-07-30 | Ashok Amara | System and method for tracking related events |
US8645384B1 (en) * | 2010-05-05 | 2014-02-04 | Google Inc. | Updating taxonomy based on webpage |
US8060414B1 (en) * | 2010-05-25 | 2011-11-15 | Amazon Technologies, Inc. | Encoding and decoding of product properties in a uniform resource locator |
US8712999B2 (en) * | 2010-06-10 | 2014-04-29 | Aol Inc. | Systems and methods for online search recirculation and query categorization |
US8954451B2 (en) * | 2010-06-30 | 2015-02-10 | Hewlett-Packard Development Company, L.P. | Selecting microblog entries based on web pages, via path similarity within hierarchy of categories |
US8799310B2 (en) * | 2010-07-23 | 2014-08-05 | Hewlett-Packard Development Company, L.P. | Method and system for processing a uniform resource locator |
US20120078925A1 (en) * | 2010-09-27 | 2012-03-29 | International Business Machines Corporation | Searching within log files |
US8775606B2 (en) * | 2010-12-02 | 2014-07-08 | Yahoo! Inc. | System and method for counting network users |
US8521655B2 (en) * | 2011-06-06 | 2013-08-27 | Bizequity Llc | Engine, system and method for providing cloud-based business intelligence |
US20130067115A1 (en) * | 2011-09-12 | 2013-03-14 | Isaac Omar Lapanc | Method And System For Mapping Domain Prefixes To Qualified URLs |
US9020463B2 (en) * | 2011-12-29 | 2015-04-28 | The Nielsen Company (Us), Llc | Systems, methods, apparatus, and articles of manufacture to measure mobile device usage |
US9659095B2 (en) * | 2012-03-04 | 2017-05-23 | International Business Machines Corporation | Managing search-engine-optimization content in web pages |
US9507825B2 (en) | 2012-09-28 | 2016-11-29 | Oracle International Corporation | Techniques for partition pruning based on aggregated zone map information |
US8996544B2 (en) | 2012-09-28 | 2015-03-31 | Oracle International Corporation | Pruning disk blocks of a clustered table in a relational database management system |
US9430550B2 (en) | 2012-09-28 | 2016-08-30 | Oracle International Corporation | Clustering a table in a relational database management system |
US10642837B2 (en) | 2013-03-15 | 2020-05-05 | Oracle International Corporation | Relocating derived cache during data rebalance to maintain application performance |
US20140324422A1 (en) * | 2013-04-18 | 2014-10-30 | WTF Technology Partners, Inc. | Synchronous audio distribution to portable computing devices |
US10223453B2 (en) * | 2015-02-18 | 2019-03-05 | Ubunifu, LLC | Dynamic search set creation in a search engine |
WO2016137426A1 (en) * | 2015-02-23 | 2016-09-01 | Hewlett Packard Enterprise Development Lp | Navigation menu based on crowd data |
EP3403187A4 (en) * | 2016-01-14 | 2019-07-31 | Sumo Logic | Single click delta analysis |
CN107547409B (en) * | 2016-06-24 | 2020-12-25 | 华为技术有限公司 | Route searching method and device and routing equipment |
US11282091B2 (en) * | 2016-09-30 | 2022-03-22 | Transitiv, Inc. | Systems, methods, and devices for dynamic page feed management |
US11127044B2 (en) | 2016-12-22 | 2021-09-21 | The Nielsen Company (Us), Llc | Methods and apparatus to expand panelist enrollment |
US10942937B2 (en) | 2017-04-14 | 2021-03-09 | Seagate Technology Llc | Data mining systems |
CA3062634A1 (en) * | 2017-05-10 | 2018-11-15 | Embee Mobile, Inc. | System and method for the capture of mobile behavior, usage, or content exposure |
KR102408476B1 (en) * | 2017-07-10 | 2022-06-14 | 십일번가 주식회사 | Method for predicing purchase probability based on behavior sequence of user and apparatus therefor |
US11086876B2 (en) | 2017-09-29 | 2021-08-10 | Oracle International Corporation | Storing derived summaries on persistent memory of a storage device |
US11232166B2 (en) * | 2019-01-17 | 2022-01-25 | Salesforce.Com, Inc. | Networked page access and addressing based on partial categorization indexing |
CN111651701A (en) * | 2020-06-05 | 2020-09-11 | 厦门理工学院 | Internet of things big data window browsing system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6006225A (en) * | 1998-06-15 | 1999-12-21 | Amazon.Com | Refining search queries by the suggestion of correlated terms from prior searches |
US6157618A (en) * | 1999-01-26 | 2000-12-05 | Microsoft Corporation | Distributed internet user experience monitoring system |
US6633316B1 (en) * | 1999-05-13 | 2003-10-14 | International Business Machines Corporation | Method and apparatus for implementing direct link selection of cached, previously visited links in nested web pages |
Family Cites Families (116)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE59108895D1 (en) * | 1991-12-18 | 1998-01-08 | Siemens Ag | Procedures for keeping the time in computer networks |
US5557686A (en) * | 1993-01-13 | 1996-09-17 | University Of Alabama | Method and apparatus for verification of a computer user's identification, based on keystroke characteristics |
US5619709A (en) * | 1993-09-20 | 1997-04-08 | Hnc, Inc. | System and method of context vector generation and retrieval |
US5950173A (en) * | 1996-10-25 | 1999-09-07 | Ipf, Inc. | System and method for delivering consumer product related information to consumers within retail environments using internet-based information servers and sales agents |
US5715399A (en) * | 1995-03-30 | 1998-02-03 | Amazon.Com, Inc. | Secure method and system for communicating a list of credit card numbers over a non-secure network |
US5742762A (en) * | 1995-05-19 | 1998-04-21 | Telogy Networks, Inc. | Network management gateway |
US6018342A (en) * | 1995-07-03 | 2000-01-25 | Sun Microsystems, Inc. | Automatically generated content-based history mechanism |
US5712979A (en) * | 1995-09-20 | 1998-01-27 | Infonautics Corporation | Method and apparatus for attaching navigational history information to universal resource locator links on a world wide web page |
US5930786A (en) | 1995-10-20 | 1999-07-27 | Ncr Corporation | Method and apparatus for providing shared data to a requesting client |
US5809478A (en) * | 1995-12-08 | 1998-09-15 | Allstate Insurance Company | Method for accessing and evaluating information for processing an application for insurance |
US6016482A (en) * | 1996-01-11 | 2000-01-18 | Merrill Lynch & Co., Inc. | Enhanced collateralized funding processor |
US5821937A (en) * | 1996-02-23 | 1998-10-13 | Netsuite Development, L.P. | Computer method for updating a network design |
US5809235A (en) * | 1996-03-08 | 1998-09-15 | International Business Machines Corporation | Object oriented network event management framework |
US5894554A (en) * | 1996-04-23 | 1999-04-13 | Infospinner, Inc. | System for managing dynamic web page generation requests by intercepting request at web server and routing to page server thereby releasing web server to process other requests |
US5848396A (en) * | 1996-04-26 | 1998-12-08 | Freedom Of Information, Inc. | Method and apparatus for determining behavioral profile of a computer user |
US5864871A (en) | 1996-06-04 | 1999-01-26 | Multex Systems | Information delivery system and method including on-line entitlements |
US5796633A (en) | 1996-07-12 | 1998-08-18 | Electronic Data Systems Corporation | Method and system for performance monitoring in computer networks |
US5787425A (en) | 1996-10-01 | 1998-07-28 | International Business Machines Corporation | Object-oriented data mining framework mechanism |
US5983200A (en) * | 1996-10-09 | 1999-11-09 | Slotznick; Benjamin | Intelligent agent for executing delegated tasks |
US5974572A (en) * | 1996-10-15 | 1999-10-26 | Mercury Interactive Corporation | Software system and methods for generating a load test using a server access log |
US5870559A (en) * | 1996-10-15 | 1999-02-09 | Mercury Interactive | Software system and associated methods for facilitating the analysis and management of web sites |
US6983478B1 (en) * | 2000-02-01 | 2006-01-03 | Bellsouth Intellectual Property Corporation | Method and system for tracking network use |
US6012087A (en) * | 1997-01-14 | 2000-01-04 | Netmind Technologies, Inc. | Unique-change detection of dynamic web pages using history tables of signatures |
JPH10214284A (en) * | 1997-01-30 | 1998-08-11 | Victor Co Of Japan Ltd | On-line shopping system and server for the same |
US6112238A (en) | 1997-02-14 | 2000-08-29 | Webtrends Corporation | System and method for analyzing remote traffic data in a distributed computing environment |
US5899992A (en) * | 1997-02-14 | 1999-05-04 | International Business Machines Corporation | Scalable set oriented classifier |
AUPO525497A0 (en) * | 1997-02-21 | 1997-03-20 | Mills, Dudley John | Network-based classified information systems |
US6026433A (en) * | 1997-03-17 | 2000-02-15 | Silicon Graphics, Inc. | Method of creating and editing a web site in a client-server environment using customizable web site templates |
US6023507A (en) * | 1997-03-17 | 2000-02-08 | Sun Microsystems, Inc. | Automatic remote computer monitoring system |
US5945989A (en) * | 1997-03-25 | 1999-08-31 | Premiere Communications, Inc. | Method and apparatus for adding and altering content on websites |
US6424974B1 (en) * | 1997-03-31 | 2002-07-23 | International Business Machines Corporation | Storing P-code in a database |
US6144944A (en) * | 1997-04-24 | 2000-11-07 | Imgis, Inc. | Computer system for efficiently selecting and providing information |
US6553416B1 (en) | 1997-05-13 | 2003-04-22 | Micron Technology, Inc. | Managing computer system alerts |
US6094672A (en) * | 1997-05-19 | 2000-07-25 | Novell, Inc. | Method and system for time synchronization management |
US6012086A (en) * | 1997-06-24 | 2000-01-04 | Sony Corporation | Internet event timer recording for video and/or audio |
US6029141A (en) * | 1997-06-27 | 2000-02-22 | Amazon.Com, Inc. | Internet-based customer referral system |
US5905985A (en) * | 1997-06-30 | 1999-05-18 | International Business Machines Corporation | Relational database modifications based on multi-dimensional database modifications |
US6205447B1 (en) * | 1997-06-30 | 2001-03-20 | International Business Machines Corporation | Relational database management of multi-dimensional data |
US5987135A (en) | 1997-07-25 | 1999-11-16 | Prc Inc. | System and method for controlling and monitoring remote distributed processing system |
US5999924A (en) * | 1997-07-25 | 1999-12-07 | Amazon.Com, Inc. | Method and apparatus for producing sequenced queries |
US6076092A (en) * | 1997-08-19 | 2000-06-13 | Sun Microsystems, Inc. | System and process for providing improved database interfacing using query objects |
US5960428A (en) * | 1997-08-28 | 1999-09-28 | International Business Machines Corporation | Star/join query optimization |
US6000832A (en) * | 1997-09-24 | 1999-12-14 | Microsoft Corporation | Electronic online commerce card with customer generated transaction proxy number for online transactions |
US6574661B1 (en) | 1997-09-26 | 2003-06-03 | Mci Communications Corporation | Integrated proxy interface for web based telecommunication toll-free network management using a network manager for downloading a call routing tree to client |
US6035332A (en) * | 1997-10-06 | 2000-03-07 | Ncr Corporation | Method for monitoring user interactions with web pages from web server using data and command lists for maintaining information visited and issued by participants |
US6108004A (en) * | 1997-10-21 | 2000-08-22 | International Business Machines Corporation | GUI guide for data mining |
JP3811874B2 (en) * | 1997-10-28 | 2006-08-23 | 富士通株式会社 | Time correction method and mobile phone terminal device used in the method |
US6112181A (en) * | 1997-11-06 | 2000-08-29 | Intertrust Technologies Corporation | Systems and methods for matching, selecting, narrowcasting, and/or classifying based on rights management and/or other information |
US6151601A (en) | 1997-11-12 | 2000-11-21 | Ncr Corporation | Computer architecture and method for collecting, analyzing and/or transforming internet and/or electronic commerce data for storage into a data storage area |
US6128624A (en) * | 1997-11-12 | 2000-10-03 | Ncr Corporation | Collection and integration of internet and electronic commerce data in a database during web browsing |
US6151584A (en) | 1997-11-20 | 2000-11-21 | Ncr Corporation | Computer architecture and method for validating and collecting and metadata and data about the internet and electronic commerce environments (data discoverer) |
US6052785A (en) * | 1997-11-21 | 2000-04-18 | International Business Machines Corporation | Multiple remote data access security mechanism for multitiered internet computer networks |
US6289342B1 (en) * | 1998-01-05 | 2001-09-11 | Nec Research Institute, Inc. | Autonomous citation indexing and literature browsing using citation context |
US6078924A (en) | 1998-01-30 | 2000-06-20 | Aeneid Corporation | Method and apparatus for performing data collection, interpretation and analysis, in an information platform |
US6073135A (en) * | 1998-03-10 | 2000-06-06 | Alta Vista Company | Connectivity server for locating linkage information between Web pages |
US6199063B1 (en) * | 1998-03-27 | 2001-03-06 | Red Brick Systems, Inc. | System and method for rewriting relational database queries |
JP2002510088A (en) * | 1998-03-27 | 2002-04-02 | インフォミックス ソフトウェア, インコーポレイテッド | Processing precomputed views |
US6154738A (en) * | 1998-03-27 | 2000-11-28 | Call; Charles Gainor | Methods and apparatus for disseminating product information via the internet using universal product codes |
US6233618B1 (en) * | 1998-03-31 | 2001-05-15 | Content Advisor, Inc. | Access control of networked data |
GB2336007B (en) | 1998-04-01 | 2003-01-29 | Mitel Corp | Agent-based data mining and warehousing |
US6101486A (en) * | 1998-04-20 | 2000-08-08 | Nortel Networks Corporation | System and method for retrieving customer information at a transaction center |
US6219639B1 (en) * | 1998-04-28 | 2001-04-17 | International Business Machines Corporation | Method and apparatus for recognizing identity of individuals employing synchronized biometrics |
US6169997B1 (en) * | 1998-04-29 | 2001-01-02 | Ncr Corporation | Method and apparatus for forming subject (context) map and presenting Internet data according to the subject map |
US6714931B1 (en) * | 1998-04-29 | 2004-03-30 | Ncr Corporation | Method and apparatus for forming user sessions and presenting internet data according to the user sessions |
US6189004B1 (en) * | 1998-05-06 | 2001-02-13 | E. Piphany, Inc. | Method and apparatus for creating a datamart and for creating a query structure for the datamart |
US6212524B1 (en) * | 1998-05-06 | 2001-04-03 | E.Piphany, Inc. | Method and apparatus for creating and populating a datamart |
US6161103A (en) * | 1998-05-06 | 2000-12-12 | Epiphany, Inc. | Method and apparatus for creating aggregates for use in a datamart |
US6233341B1 (en) * | 1998-05-19 | 2001-05-15 | Visto Corporation | System and method for installing and using a temporary certificate at a remote site |
US6138121A (en) * | 1998-05-29 | 2000-10-24 | Hewlett-Packard Company | Network management event storage and manipulation using relational database technology in a data warehouse |
CA2261460A1 (en) * | 1998-06-04 | 1999-12-04 | Richard B. Mcevoy | Mailing label |
US6278966B1 (en) * | 1998-06-18 | 2001-08-21 | International Business Machines Corporation | Method and system for emulating web site traffic to identify web site usage patterns |
FR2780178B1 (en) | 1998-06-18 | 2001-08-10 | Inst Nat Rech Inf Automat | METHOD FOR TRANSFORMING AND TRANSPORTING DATA BETWEEN AGENT SERVERS PRESENT ON MACHINES AND A CENTRAL AGENT SERVER PRESENT ON ANOTHER MACHINE |
JP4105293B2 (en) | 1998-06-30 | 2008-06-25 | 富士通株式会社 | Network monitoring system, monitoring device and monitored device |
US6192364B1 (en) | 1998-07-24 | 2001-02-20 | Jarg Corporation | Distributed computer database system and method employing intelligent agents |
US6178418B1 (en) * | 1998-07-28 | 2001-01-23 | Noetix Corporation | Distributed data warehouse query and resource management system |
US6317787B1 (en) * | 1998-08-11 | 2001-11-13 | Webtrends Corporation | System and method for analyzing web-server log files |
AU5797599A (en) | 1998-08-31 | 2000-03-21 | Aprisma Management Technologies, Inc. | Method and apparatus for managing data for use by data applications |
US6311205B1 (en) * | 1998-10-19 | 2001-10-30 | International Business Machines Corporation | Persistent user groups on servers managed by central servers |
US6289378B1 (en) * | 1998-10-20 | 2001-09-11 | Triactive Technologies, L.L.C. | Web browser remote computer management system |
US6192357B1 (en) * | 1998-11-03 | 2001-02-20 | Platinum Technology, Inc. | Method and apparatus for optimizing query generation by selectively utilizing attributes or key values |
US6360246B1 (en) * | 1998-11-13 | 2002-03-19 | The Nasdaq Stock Market, Inc. | Report generation architecture for remotely generated data |
US6256676B1 (en) * | 1998-11-18 | 2001-07-03 | Saga Software, Inc. | Agent-adapter architecture for use in enterprise application integration systems |
US6802042B2 (en) * | 1999-06-01 | 2004-10-05 | Yodlee.Com, Inc. | Method and apparatus for providing calculated and solution-oriented personalized summary-reports to a user through a single user-interface |
US6466970B1 (en) * | 1999-01-27 | 2002-10-15 | International Business Machines Corporation | System and method for collecting and analyzing information about content requested in a network (World Wide Web) environment |
US20010012346A1 (en) * | 1999-01-29 | 2001-08-09 | Alex Terry | Interactive billing system utilizing a thin web client interface |
US6246998B1 (en) * | 1999-02-25 | 2001-06-12 | Fujitsu Limited | System and method for home grocery shopping including item categorization for efficient delivery and pick-up |
US6584464B1 (en) * | 1999-03-19 | 2003-06-24 | Ask Jeeves, Inc. | Grammar template query system |
US6519585B1 (en) * | 1999-04-27 | 2003-02-11 | Infospace, Inc. | System and method for facilitating presentation of subject categorizations for use in an on-line search query engine |
US6618719B1 (en) * | 1999-05-19 | 2003-09-09 | Sybase, Inc. | Database system with methodology for reusing cost-based optimization decisions |
US6401077B1 (en) * | 1999-05-28 | 2002-06-04 | Network Commerce, Inc. | Method and system for providing additional behavior through a web page |
US6279033B1 (en) * | 1999-05-28 | 2001-08-21 | Microstrategy, Inc. | System and method for asynchronous control of report generation using a network interface |
US6341290B1 (en) | 1999-05-28 | 2002-01-22 | Electronic Data Systems Corporation | Method and system for automating the communication of business information |
US6467052B1 (en) * | 1999-06-03 | 2002-10-15 | Microsoft Corporation | Method and apparatus for analyzing performance of data processing system |
US6446086B1 (en) * | 1999-06-30 | 2002-09-03 | Computer Sciences Corporation | System and method for logging transaction records in a computer system |
US6665682B1 (en) * | 1999-07-19 | 2003-12-16 | International Business Machines Corporation | Performance of table insertion by using multiple tables or multiple threads |
US6408292B1 (en) * | 1999-08-04 | 2002-06-18 | Hyperroll, Israel, Ltd. | Method of and system for managing multi-dimensional databases using modular-arithmetic based address data mapping processes on integer-encoded business dimensions |
US6286104B1 (en) * | 1999-08-04 | 2001-09-04 | Oracle Corporation | Authentication and authorization in a multi-tier relational database management system |
US6532023B1 (en) * | 1999-08-12 | 2003-03-11 | International Business Machines Corporation | Recording selected applet events of a user interaction sequence |
US6539391B1 (en) * | 1999-08-13 | 2003-03-25 | At&T Corp. | Method and system for squashing a large data set |
US6449739B1 (en) * | 1999-09-01 | 2002-09-10 | Mercury Interactive Corporation | Post-deployment monitoring of server performance |
US7533344B1 (en) * | 1999-09-10 | 2009-05-12 | Ricoh Company, Ltd. | Application unit monitoring and reporting system and method |
US6792458B1 (en) * | 1999-10-04 | 2004-09-14 | Urchin Software Corporation | System and method for monitoring and analyzing internet traffic |
US6105020A (en) * | 1999-10-11 | 2000-08-15 | International Business Machines Corporation | System and method for identifying and constructing star joins for execution by bitmap ANDing |
US6484179B1 (en) * | 1999-10-25 | 2002-11-19 | Oracle Corporation | Storing multidimensional data in a relational database management system |
US6611846B1 (en) * | 1999-10-30 | 2003-08-26 | Medtamic Holdings | Method and system for medical patient data analysis |
US6798413B1 (en) * | 1999-12-03 | 2004-09-28 | Dcs, Inc. | Workflow management system |
US6665658B1 (en) * | 2000-01-13 | 2003-12-16 | International Business Machines Corporation | System and method for automatically gathering dynamic content and resources on the world wide web by stimulating user interaction and managing session information |
US6496857B1 (en) * | 2000-02-08 | 2002-12-17 | Mirror Worlds Technologies, Inc. | Delivering targeted, enhanced advertisements across electronic networks |
US6389510B1 (en) * | 2000-04-25 | 2002-05-14 | Lucent Technologies Inc. | Method and apparatus for caching web-based information |
US6678700B1 (en) * | 2000-04-27 | 2004-01-13 | General Atomics | System of and method for transparent management of data objects in containers across distributed heterogenous resources |
US6728769B1 (en) * | 2000-05-04 | 2004-04-27 | Sun Microsystems, Inc. | Method and apparatus for providing a highly interactive transaction environment in a distributed network |
US20020038388A1 (en) * | 2000-09-13 | 2002-03-28 | Netter Zvi Itzhak | System and method for capture and playback of user interaction with web browser content |
US6836773B2 (en) * | 2000-09-28 | 2004-12-28 | Oracle International Corporation | Enterprise web mining system and method |
US20020082911A1 (en) * | 2000-12-22 | 2002-06-27 | Dunn Charles L. | Online revenue sharing |
US20020143925A1 (en) * | 2000-12-29 | 2002-10-03 | Ncr Corporation | Identifying web-log data representing a single user session |
US7003565B2 (en) * | 2001-04-03 | 2006-02-21 | International Business Machines Corporation | Clickstream data collection technique |
-
2000
- 2000-07-11 US US09/613,847 patent/US6785666B1/en not_active Expired - Lifetime
-
2001
- 2001-12-05 US US10/005,224 patent/US7117193B1/en not_active Expired - Lifetime
- 2001-12-05 US US10/007,646 patent/US6917972B1/en not_active Expired - Lifetime
-
2002
- 2002-12-16 US US10/320,288 patent/US20030131081A1/en not_active Abandoned
- 2002-12-16 US US10/320,091 patent/US20030126111A1/en not_active Abandoned
- 2002-12-16 US US10/320,092 patent/US6873981B2/en not_active Expired - Lifetime
- 2002-12-16 US US10/320,171 patent/US20030120670A1/en not_active Abandoned
-
2006
- 2006-07-27 US US11/495,089 patent/US7464122B1/en not_active Expired - Lifetime
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6006225A (en) * | 1998-06-15 | 1999-12-21 | Amazon.Com | Refining search queries by the suggestion of correlated terms from prior searches |
US6157618A (en) * | 1999-01-26 | 2000-12-05 | Microsoft Corporation | Distributed internet user experience monitoring system |
US6633316B1 (en) * | 1999-05-13 | 2003-10-14 | International Business Machines Corporation | Method and apparatus for implementing direct link selection of cached, previously visited links in nested web pages |
Cited By (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8572100B2 (en) | 2004-01-19 | 2013-10-29 | Nigel Hamilton | Method and system for recording search trails across one or more search engines in a communications network |
EP1716508A1 (en) * | 2004-01-19 | 2006-11-02 | Nigel Hamilton | Method and system for recording search trails across one or more search engines in a communications network |
EP1716508A4 (en) * | 2004-01-19 | 2008-01-23 | Nigel Hamilton | Method and system for recording search trails across one or more search engines in a communications network |
US20090030876A1 (en) * | 2004-01-19 | 2009-01-29 | Nigel Hamilton | Method and system for recording search trails across one or more search engines in a communications network |
US7650344B2 (en) * | 2004-02-09 | 2010-01-19 | Coremetrics, Inc. | System and method of managing software product-line customizations |
US20050267889A1 (en) * | 2004-02-09 | 2005-12-01 | Coremetrics, Inc. | System and method of managing software product-line customizations |
US20060080271A1 (en) * | 2004-10-08 | 2006-04-13 | Jeff Williams | Dynamic reporting tool for a fact warehouse, a fact contained within the fact warehouse, and a method for refreshing the fact warehouse |
US10223406B2 (en) | 2006-02-17 | 2019-03-05 | Google Llc | Entity normalization via name normalization |
US9710549B2 (en) * | 2006-02-17 | 2017-07-18 | Google Inc. | Entity normalization via name normalization |
US20140214778A1 (en) * | 2006-02-17 | 2014-07-31 | Google Inc. | Entity Normalization Via Name Normalization |
US8788321B2 (en) | 2006-09-05 | 2014-07-22 | Thomas Publishing Company | Marketing method and system using domain knowledge |
US20080059310A1 (en) * | 2006-09-05 | 2008-03-06 | Thomas Publishing Company | Marketing method and system using domain knowledge |
US8548851B2 (en) | 2010-03-23 | 2013-10-01 | Google Inc. | Conversion path performance measures and reports |
US11544739B1 (en) | 2010-03-23 | 2023-01-03 | Google Llc | Conversion path performance measures and reports |
US20110307509A1 (en) * | 2010-03-23 | 2011-12-15 | Google Inc. | Conversion Path Performance Measures And Reports |
US9245279B2 (en) * | 2010-03-23 | 2016-01-26 | Google Inc. | Conversion path performance measures and reports |
US10360586B2 (en) | 2010-03-23 | 2019-07-23 | Google Llc | Conversion path performance measures and reports |
US20110264710A1 (en) * | 2010-04-27 | 2011-10-27 | International Business Machines Corporation | Monitoring and reporting productivity in enterprise environment |
US9715669B2 (en) * | 2010-04-27 | 2017-07-25 | International Business Machines Corporation | Monitoring and reporting productivity in enterprise environment |
US10216805B1 (en) | 2010-08-20 | 2019-02-26 | Google Llc | Dynamically generating pre-aggregated datasets |
US9430519B1 (en) | 2010-08-20 | 2016-08-30 | Google Inc. | Dynamically generating pre-aggregated datasets |
US8521774B1 (en) | 2010-08-20 | 2013-08-27 | Google Inc. | Dynamically generating pre-aggregated datasets |
CN105718565A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data warehouse model construction method and construction apparatus |
CN107241220A (en) * | 2017-05-26 | 2017-10-10 | 京信通信系统(中国)有限公司 | The aging of product monitors automatic analysis method and system |
US11093985B2 (en) * | 2018-09-25 | 2021-08-17 | Valideck International | System, devices, and methods for acquiring and verifying online information |
US11055282B1 (en) * | 2020-03-31 | 2021-07-06 | Atlassian Pty Ltd. | Translating graph queries into efficient network protocol requests |
Also Published As
Publication number | Publication date |
---|---|
US20030131081A1 (en) | 2003-07-10 |
US7117193B1 (en) | 2006-10-03 |
US6785666B1 (en) | 2004-08-31 |
US7464122B1 (en) | 2008-12-09 |
US6873981B2 (en) | 2005-03-29 |
US20030144988A1 (en) | 2003-07-31 |
US20030126111A1 (en) | 2003-07-03 |
US6917972B1 (en) | 2005-07-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6873981B2 (en) | Method and system for parsing navigation information | |
US7035925B1 (en) | Parsing navigation information to identify interactions based on the times of their occurrences | |
US7949702B2 (en) | Method and apparatus for synchronizing cookies across multiple client machines | |
US7752394B1 (en) | Method and system for an extensible caching framework | |
Eirinaki et al. | Web mining for web personalization | |
US7698281B2 (en) | Use of extensible markup language in a system and method for influencing a position on a search result list generated by a computer network search engine | |
US6094657A (en) | Apparatus and method for dynamic meta-tagging of compound documents | |
US6286043B1 (en) | User profile management in the presence of dynamic pages using content templates | |
EP1315114B1 (en) | System and method of presenting channelized data | |
US6993529B1 (en) | Importing data using metadata | |
US5987454A (en) | Method and apparatus for selectively augmenting retrieved text, numbers, maps, charts, still pictures and/or graphics, moving pictures and/or graphics and audio information from a network resource | |
US6959319B1 (en) | System and method for automatically personalizing web portals and web services based upon usage history | |
US6564251B2 (en) | Scalable computing system for presenting customized aggregation of information | |
US20030191737A1 (en) | Indexing system and method | |
US20040260680A1 (en) | Personalized indexing and searching for information in a distributed data processing system | |
US7631263B2 (en) | Methods, systems, and computer program products for characterizing links to resources not activated | |
US20100185599A1 (en) | Exposing rich internet application content to search engines | |
US20040249824A1 (en) | Semantics-bases indexing in a distributed data processing system | |
US20040205554A1 (en) | Systems and methods for accessing multiple internal information sources of a business from a composite web document | |
CA2402437C (en) | Methods and systems for monitoring quality assurance | |
De Meo et al. | EC-XAMAS: supporting e-commerce activities by an XML-based adaptive multi-agent system | |
US20080177761A1 (en) | Dynamically optimized storage system for online user activities | |
JP3597104B2 (en) | Electronic catalog automatic collection system | |
AU773065B2 (en) | An indexing system and method | |
Manandhar | An efficient search paradigm implementing a unique identifier |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: REVENUE SCIENCE, INC., WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DIGIMINE, INC.;REEL/FRAME:014185/0138 Effective date: 20030527 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: COMERICA BANK, CALIFORNIA Free format text: SECURITY AGREEMENT;ASSIGNOR:REVENUE SCIENCE, INC.;REEL/FRAME:020393/0843 Effective date: 20010302 |
|
AS | Assignment |
Owner name: AUDIENCESCIENCE, INC. FKA REVENUE SCIENCE, INC. FK Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:COMERICA BANK;REEL/FRAME:024023/0574 Effective date: 20100226 |