CA2739090C - Methods and apparatus for assessing the quality of locate and marking operations with respect to facilities maps - Google Patents

Abstract

Methods, apparatus and systems including a computer comprising at least one hardware processor, at least one tangible storage medium (memory), and at least one input/output (I/O) interface for evaluating a quality of a locate and/or marking operation. First information relating to the locate and/or marking operation is compared to second information obtained from at least one facilities map. One or more indications of a quality assessment of the locate and/or marking operation is automatically generated based on such a comparison, and the one or more indications of the quality assessment are electronically stored on the at least one tangible storage medium, and/or electronically transmitted via the at least one I/O interface, so as to provide an electronic record of the quality assessment.

Description

METHODS AND APPARATUS FOR ASSESSING THE QUALITY OF LOCATE AND
MARKING OPERATIONS WITH RESPECT TO FACILITIES MAPS
Background [0001] Field service operations may be any operation in which companies dispatch technicians and/or other staff to perform certain activities, for example, installations, services and/or repairs. Field service operations may exist in various industries, examples of which include, but are not limited to, network installations, utility installations, security systems, construction, medical equipment, heating, ventilating and air conditioning (HVAC) and the like.

[0002] An example of a field service operation in the construction industry is a so-called "locate and marking operation," also commonly referred to more simply as a "locate operation" (or sometimes merely as "a locate"). In a typical locate operation, a locate technician visits a work site in which there is a plan to disturb the ground (e.g., excavate, dig one or more holes and/or trenches, bore, etc.) so as to determine a presence or an absence of one or more underground facilities (such as various types of utility cables and pipes) in a dig area to be excavated or disturbed at the work site. In some instances, a locate operation may be requested for a "design" project, in which there may be no immediate plan to excavate or otherwise disturb the ground, but nonetheless information about a presence or absence of one or more underground facilities at a work site may be valuable to inform a planning, permitting and/or engineering design phase of a future construction project.

[0003] In many states, an excavator who plans to disturb ground at a work site is required by law to notify any potentially affected underground facility owners prior to undertaking an excavation activity. Advanced notice of excavation activities may be provided by an excavator (or another party) by contacting a "one-call center." One-call centers typically are operated by a consortium of underground facility owners for the purposes of receiving excavation notices and in turn notifying facility owners and/or their agents of a plan to excavate. As part of an advanced notification, excavators typically provide to the one-call center various information relating to the planned activity, including a location (e.g., address) of the work site and a description of the dig area to be excavated or otherwise disturbed at the work site.

[0004] Figure 1 illustrates an example in which a locate operation is initiated as a result of an excavator 110 providing an excavation notice to a one-call center 120. An excavation notice also is commonly referred to as a "locate request," and may be provided by the excavator to the one-call center via an electronic mail message, information entry via a website maintained by the one-call center, or a telephone conversation between the excavator and a human operator at the one-call center. The locate request may include an address or some other location-related information describing the geographic location of a work site at which the excavation is to be performed, as well as a description of the dig area (e.g., a text description), such as its location relative to certain landmarks and/or its approximate dimensions, within which there is a plan to disturb the ground at the work site. One-call centers similarly may receive locate requests for design projects (for which, as discussed above, there may be no immediate plan to excavate or otherwise disturb the ground).

[0005] Using the information provided in a locate request for planned excavation or design projects, the one-call center identifies certain underground facilities that may be present at the indicated work site. For this purpose, many one-call centers typically maintain a collection "polygon maps" which indicate, within a given geographic area over which the one-call center has jurisdiction, generally where underground facilities may be found relative to some geographic reference frame or coordinate system.

[0006] Polygon maps typically are provided to the one-call centers by underground facilities owners within the jurisdiction of the one call center ("members" of the one-call center). A one-call center first provides the facility owner/member with one or more maps (e.g., street or property maps) within the jurisdiction, on which are superimposed some type of grid or coordinate system employed by the one-call center as a geographic frame of reference. Using the maps provided by the one-call center, the respective facilities owners/members draw one or more polygons on each map to indicate an area within which their facilities generally are disposed underground (without indicating the facilities themselves). These polygons themselves do not precisely indicate geographic locations of respective underground facilities; rather, the area enclosed by a given polygon generally provides an over-inclusive indication of where a given facilities owner's underground facilities are disposed. Different facilities owners/members may draw polygons of different sizes around areas including their underground facilities, and in some instances such polygons can cover appreciably large geographic regions (e.g., an entire subdivision of a residential area), which may further obfuscate the actual/precise location of respective underground facilities.

[0007] Based on the polygon maps collected from the facilities owners/members, the one-call center may in some instances create composite polygon maps to show polygons of multiple different members on a single map. Whether using single member or composite polygon maps, the one-call center examines the address or location information provided in the locate request and identifies a significant buffer zone around an identified work site so as to make an over-inclusive identification of facilities owners/members that may have underground facilities present (e.g., to err on the side of caution).
In particular, based on this generally over-inclusive buffer zone around the identified work site (and in some instances significantly over-inclusive buffer zone), the one-call center consults the polygon maps to identify which member polygons intersect with all or a portion of the buffer zone so as to notify these underground facility owners/members and/or their agents of the proposed excavation or design project. Again, it should be appreciated that the buffer zones around an indicated work site utilized by one-call centers for this purpose typically embrace a geographic area that includes but goes well beyond the actual work site, and in many cases the geographic area enclosed by a buffer zone is significantly larger than the actual dig area in which excavation or other similar activities are planned.
Similarly, as noted above, the area enclosed by a given member polygon generally does not provide a precise indication of where one or more underground facilities may in fact be found.

[0008] In some instances, one-call centers may also or alternatively have access to various existing maps of underground facilities in their jurisdiction, referred to as "facilities maps." Facilities maps typically are maintained by facilities owners/members within the jurisdiction and show, for respective different utility types, where underground facilities purportedly may be found relative to some geographic reference frame or coordinate system (e.g., a grid, a street or property map, GPS latitude and longitude coordinates, etc.). Facilities maps generally provide somewhat more detail than polygon maps provided by facilities owners/members; however, in some instances the information contained in facilities maps may not be accurate and/or complete. For at least this reason, whether using polygon maps or facilities maps, as noted above the one-call center utilizes a significant buffer zone around an identified work site so as to make an over-inclusive identification of facilities owners/members that may have underground facilities present.

[0009] Once facilities implicated by the locate request are identified by a one-call center (e.g., via the polygon map/buffer zone process), the one-call center generates a "locate request ticket" (also known as a "locate ticket," or simply a "ticket"). The locate request ticket essentially constitutes an instruction to inspect a work site and typically identifies the work site of the proposed excavation or design and a description of the dig area, typically lists on the ticket all of the underground facilities that may be present at the work site (e.g., by providing a member code for the facility owner whose polygon falls within a given buffer zone), and may also include various other information relevant to the proposed excavation or design (e.g., the name of the excavation company, a name of a property owner or party contracting the excavation company to perform the excavation, etc.). The one-call center sends the ticket to one or more underground facility owners 140 and/or one or more locate service providers 130 (who may be acting as contracted agents of the facility owners) so that they can conduct a locate and marking operation to verify a presence or absence of the underground facilities in the dig area. For example, in some instances, a given underground facility owner 140 may operate its own fleet of locate technicians (e.g., locate technician 145), in which case the one-call center 120 may send the ticket to the underground facility owner 140. In other instances, a given facility owner may contract with a locate service provider to receive locate request tickets and perform a locate and marking operation in response to received tickets on their behalf.

[0010] Upon receiving the locate ticket, a locate service provider or a facility owner (hereafter referred to as a "ticket recipient") may dispatch a locate technician to the work site of planned excavation to determine a presence or absence of one or more underground facilities in the dig area to be excavated or otherwise disturbed. A typical first step for the locate technician includes utilizing an underground facility "locate device,"
which is an instrument or set of instruments (also referred to commonly as a "locate set") for detecting facilities that are concealed in some manner, such as cables and pipes that are located underground. The locate device is employed by the technician to verify the presence or absence of underground facilities indicated in the locate request ticket as potentially present in the dig area (e.g., via the facility owner member codes listed in the ticket). This process is often referred to as a "locate operation."

[0011] In one example of a locate operation, an underground facility locate device is used to detect electromagnetic fields that are generated by an applied signal provided along a length of a target facility to be identified. In this example, a locate device may include both a signal transmitter to provide the applied signal (e.g., which is coupled by the locate technician to a tracer wire disposed along a length of a facility), and a signal receiver which is generally a hand-held apparatus carried by the locate technician as the technician walks around the dig area to search for underground facilities. The transmitter is connected via a connection point to a target object (in this example, underground facility) located in the ground, and generates the applied signal coupled to the underground facility via the connection point (e.g., to a tracer wire along the facility), resulting in the generation of a magnetic field. The magnetic field in turn is detected by the locate receiver, which itself may include one or more detection antenna.
The locate receiver indicates a presence of a facility when it detects electromagnetic fields arising from the applied signal. Conversely, the absence of a signal detected by the locate receiver generally indicates the absence of the target facility.

[0012] In yet another example, a locate device employed for a locate operation may include a single instrument, similar in some respects to a conventional metal detector. In particular, such an instrument may include an oscillator to generate an alternating current that passes through a coil, which in turn produces a first magnetic field. If a piece of electrically conductive metal is in close proximity to the coil (e.g., if an underground facility having a metal component is below/near the coil of the instrument), eddy currents are induced in the metal and the metal produces its own magnetic field, which in turn affects the first magnetic field. The instrument may include a second coil to measure changes to the first magnetic field, thereby facilitating detection of metallic objects.

[0013] In addition to the locate operation, the locate technician also generally performs a "marking operation," in which the technician marks the presence (and in some cases the absence) of a given underground facility in the dig area based on the various signals detected (or not detected) during the locate operation. For this purpose, the locate technician conventionally utilizes a "marking device" to dispense a marking material on, for example, the ground, pavement, or other surface along a detected underground facility.
Marking material may be any material, substance, compound, and/or element, used or which may be used separately or in combination to mark, signify, and/or indicate.

Examples of marking materials may include, but are not limited to, paint, chalk, dye, and/or iron. Marking devices, such as paint marking wands and/or paint marking wheels, provide a convenient method of dispensing marking materials onto surfaces, such as onto the surface of the ground or pavement.

[0014] In some environments, arrows, flags, darts, or other types of physical marks may be used to mark the presence or absence of an underground facility in a dig area, in addition to or as an alternative to a material applied to the ground (such as paint, chalk, dye, tape) along the path of a detected utility. The marks resulting from any of a wide variety of materials and/or objects used to indicate a presence or absence of underground facilities generally are referred to as "locate marks." Often, different color materials and/or physical objects may be used for locate marks, wherein different colors correspond to different utility types. For example, the American Public Works Association (APWA) has established a standardized color-coding system for utility identification for use by public agencies, utilities, contractors and various groups involved in ground excavation (e.g., red = electric power lines and cables; blue = potable water; orange =
telecommunication lines; yellow = gas, oil, steam). In some cases, the technician also may provide one or more marks to indicate that no facility was found in the dig area (sometimes referred to as a "clear").

[0015] As mentioned above, the foregoing activity of identifying and marking a presence or absence of one or more underground facilities generally is referred to for completeness as a "locate and marking operation." However, in light of common parlance adopted in the construction industry, and/or for the sake of brevity, one or both of the respective locate and marking functions may be referred to in some instances simply as a "locate operation" or a "locate" (i.e., without making any specific reference to the marking function). Accordingly, it should be appreciated that any reference in the relevant arts to the task of a locate technician simply as a "locate operation" or a "locate"
does not necessarily exclude the marking portion of the overall process. At the same time, in some contexts a locate operation is identified separately from a marking operation, wherein the former relates more specifically to detection-related activities and the latter relates more specifically to marking-related activities.

[0016] Inaccurate locating and/or marking of underground facilities can result in physical damage to the facilities, property damage, and/or personal injury during the excavation process that, in turn, can expose a facility owner or contractor to significant legal liability. When underground facilities are damaged and/or when property damage or personal injury results from damaging an underground facility during an excavation, the excavator may assert that the facility was not accurately located and/or marked by a locate technician, while the locate contractor who dispatched the technician may in turn assert that the facility was indeed properly located and marked. Proving whether the underground facility was properly located and marked can be difficult after the excavation (or after some damage, e.g., a gas explosion), because in many cases the physical locate marks (e.g., the marking material or other physical marks used to mark the facility on the surface of the dig area) will have been disturbed or destroyed during the excavation process (and/or damage resulting from excavation).
Summary

[0017] As discussed above, in various field service operations, a number of field technicians typically are dispatched to perform field operations at any given time, and over any given time period each technician may be assigned numerous work orders, or "tickets"
specifying aspects of the field operations to be performed. The volume of tickets per technician may be particularly high in the construction industry, especially in connection with locate and marking operations. The inventors have recognized and appreciated that implementing and performing meaningful oversight and quality control activities in a timely fashion for several field technicians each performing several field operations in a given time period may present challenges, and that failure to perform meaningful oversight and quality control activities may adversely affect customer satisfaction.

[0018] Additionally, the inventors have appreciated that the time, effort, and cost that is associated with re-performing work in the field, or with correcting and/or improving poorly performed field calls, may be unacceptable. Consequently, the inventors have realized that a need exists for methods of providing oversight and quality control in field service operations in order to improve customer satisfaction, to identify and reduce the number of poorly performed tickets, and to improve visibility into distributed workforce operations.

[0019] In view of the foregoing, various inventive embodiments disclosed herein relate generally to methods, apparatus and systems for computer-aided determination of quality assessment for locate and marking operations. In some embodiments, a quality assessment decision is solely under the discretion of a human reviewer, albeit facilitated in some respects by computer-aided display of information, and electronic record-keeping and communication functions associated with the quality assessment result(s).
In other embodiments, information related to a locate and marking operation is electronically analyzed such that a quality assessment is not based solely on human discretion, but rather based at least in part on some predetermined criteria and/or metrics that facilitate an automated determination of quality assessment.

[0020] More specifically, in some embodiments, methods, apparatus and systems according to the present disclosure relate to at least partially automating oversight and quality assessment in underground facility locate applications and/or other field service operations. For example, in some embodiments, an automated quality assessment system may receive information related to a locate and marking operation from one or more sources of electronic data (also referred to herein as "field information" or "field data"), analyze the contents of the received electronic data, and automatically assess the quality of the operation based at least in part on the analysis. In other embodiments, automated analysis of at least some of the received electronic data relating to the locate and marking operation facilitates further analysis and/or quality assessment by a human, in which the quality assessment is not based solely on the discretion of the human, but is significantly informed in some manner by automated analysis of data.

[0021] In some exemplary implementations in which a quality of a locate and marking operation is assessed via an at least partially automated process, some or all of the available field information (e.g., which in some instances is derived from data contained in an electronic record of the locate and marking operation) is compared to "reference information" or "reference data" (which in some instances is derived from data contained in a "reference" electronic record). Examples of types of reference information/data used in a quality assessment process according to various embodiments discussed herein may include, but are not limited to: 1) information/data derived from or relating to one or more facilities maps that illustrate the presumed physical locations of underground facilities purportedly present in a geographic area proximate to or surrounding and subsuming the work site; 2) information/data derived from or relating to one or more previous locate and marking operations at or near the work site (referred to herein as "historical tickets" or "historical data"); and/or 3) information/data relating to one or more environmental landmarks present in a geographic area proximate to or surrounding and subsuming the dig area (e.g., the work site and its environs), or within the dig area itself (referred to herein as "landmark information," which may be available, for example, from facilities maps, historical tickets, and/or field data collected at or around the time of the locate and marking operation being assessed).

[0022] In other aspects, the quality assessment of the locate operation may be performed, in whole or in part, by one or more analysis components (e.g., one or more processors executing instructions) separate and/or remote from the locate and/or marking device used in connection with the locate operation. Alternatively, the assessment may be performed, in whole or in part, by one or more analysis components incorporated within or otherwise coupled to a locate device, a marking device, and/or a combined locate and marking device. Depending on the nature of the assessment, it may be performed substantially in real time with respect to the generation of field information/data used in connection with the assessment (e.g., one or more of locate information, marking information and landmark information contained in electronic records of a locate and marking operation and/or an electronic manifest of same), otherwise during a locate and/or marking operation, or after completion of a locate and/or marking operation.

[0023] In some embodiments described herein, a notification may be generated based on the quality assessment performed. The notification may provide one or more indications of the quality of the locate and marking operation as a whole, or of some aspect thereof. For example, the notification may provide an indication of a degree of correspondence or discrepancy between field data contained in the electronic record of the locate and marking operation and reference data contained in the reference electronic record to which it is compared. Likewise, the notification may provide an indication that the locate and marking operation is or is not approved based on the comparison of the field data to the reference data. The notification may be transmitted electronically or otherwise conveyed, for example, to one or more parties associated with one or more underground facilities within the dig area or in a geographic area proximate to or surrounding and subsuming the work site, one or more parties associated with the performance or oversight of the locate and marking operation, and/or one or more parties associated with excavation of the dig area, for example.

[0024] In exemplary embodiments in which the reference information comprises data derived from or relating to one or more facilities maps, field information including geographic information, facility type information, and/or other information relating to an underground facility detected and/or marked during a locate and marking operation may be compared to reference information including geographic and/or other information relating to the corresponding facility as indicated on one or more facilities maps (e.g., all or some of the contents of a current electronic record of a locate and marking operation may be compared to reference information derived from one or more facilities maps). The comparison may generally involve determining whether or not there is agreement between the field information relating to the locate and marking operation and the reference information provided by the one or more facilities maps, which may in turn involve identifying at least one correspondence or discrepancy between the compared information, and in some instances a degree of correspondence.

[0025] In yet other embodiments, a first electronic representation of field information relating to a locate and marking operation (e.g., data in an electronic record, an electronic manifest, etc.), as well as a second electronic representation of reference information (e.g., data in a reference electronic record from any of a variety of sources) to which the first electronic representation is compared, may be visually rendered (e.g., via a computer-generated visual representation in a display field) such that the electronic representations are overlaid to provide a visual aid to an automated assessment process.

[0026] In sum, one embodiment of the present invention is directed to a method, performed in a computer comprising at least one hardware processor, at least one tangible storage medium, and at least one input/output (I/0) interface, for evaluating a quality of a locate and/or marking operation to identify a presence or an absence of at least one underground facility at a work site. The method comprises: A) comparing first information relating to the locate and/or marking operation to second information obtained from at least one facilities map; B) automatically generating, based on A), at least one indication of a quality assessment of the locate and/or marking operation; and C) electronically storing on the at least one tangible storage medium, and/or electronically transmitting via the at least one I/0 interface, the at least one indication of the quality assessment so as to provide an electronic record of the quality assessment.

[0027] Another embodiment is directed to an apparatus for evaluating a quality of a locate and/or marking operation to identify a presence or an absence of at least one underground facility at a work site. The apparatus comprises: at least one input/output (I/O) interface; at least one memory storing processor-executable instructions; and a processor coupled to the memory and the at least one I/0 interface. Upon execution of the processor-executable instructions by the processor, the processor: A) compares first information relating to the locate and/or marking operation to second information obtained from at least one facilities map; B) automatically generates, based on A), at least one indication of a quality assessment of the locate and/or marking operation; and C) controls the at least one memory so as to electronically store, and/or controls the at least one I/0 interface so as to electronically transmit, the at least one indication of the quality assessment so as to provide an electronic record of the quality assessment.

[0028] Another embodiment is directed to at least one computer-readable storage medium encoded with instructions that, when executed by a processor in a computer comprising at least one input/output (I/0) interface, perform a method for evaluating a quality of a locate and/or marking operation to identify a presence or an absence of at least one underground facility within a work site. The method comprises: A) comparing first information relating to the locate and/or marking operation to second information obtained from at least one facilities map; B) automatically generating, based on A), at least one indication of a quality assessment of the locate and marking operation; and C) electronically storing on the at least one computer-readable storage medium, and/or electronically transmitting via the at least one I/0 interface, the at least one indication of the quality assessment so as to provide an electronic record of the quality assessment.

[0029] For purposes of the present disclosure, the term "dig area"
refers to a specified area of a work site within in which there is a plan to disturb the ground (e.g., excavate, dig holes and/or trenches, bore, etc.), and beyond which there is no plan to excavate in the immediate surroundings. Thus, the metes and bounds of a dig area are intended to provide specificity as to where some disturbance to the ground is planned at a given work site. It should be appreciated that a given work site may include multiple dig areas.

[0030] The term "facility" refers to one or more lines, cables, fibers, conduits, transmitters, receivers, or other physical objects or structures capable of or used for carrying, transmitting, receiving, storing, and providing utilities, energy, data, substances, and/or services, and/or any combination thereof. The term "underground facility" means any facility beneath the surface of the ground. Examples of facilities include, but are not limited to, oil, gas, water, sewer, power, telephone, data transmission, cable television (TV), and/or internet services.

[0031] The term "locate device" refers to any apparatus and/or device, used alone or in combination with any other device, for detecting and/or inferring the presence or absence of any facility, including without limitation, any underground facility. In various examples, a locate device often includes both a locate transmitter and a locate receiver (which in some instances may also be referred to collectively as a "locate instrument set,"
or simply "locate set").

[0032] The term "marking device" refers to any apparatus, mechanism, or other device that employs a marking dispenser for causing a marking material and/or marking object to be dispensed, or any apparatus, mechanism, or other device for electronically indicating (e.g., logging in memory) a location, such as a location of an underground facility.
Additionally, the term "marking dispenser" refers to any apparatus, mechanism, or other device for dispensing and/or otherwise using, separately or in combination, a marking material and/or a marking object. An example of a marking dispenser may include, but is not limited to, a pressurized can of marking paint. The term "marking material" means any material, substance, compound, and/or element, used or which may be used separately or in combination to mark, signify, and/or indicate. Examples of marking materials may include, but are not limited to, paint, chalk, dye, and/or iron. The term "marking object"
means any object and/or objects used or which may be used separately or in combination to mark, signify, and/or indicate. Examples of marking objects may include, but are not limited to, a flag, a dart, and arrow, and/or an RFID marking ball. It is contemplated that marking material may include marking objects. It is further contemplated that the terms "marking materials" or "marking objects" may be used interchangeably in accordance with the present disclosure.

[0033] The term "locate mark" means any mark, sign, and/or object employed to indicate the presence or absence of any underground facility. Examples of locate marks may include, but are not limited to, marks made with marking materials, marking objects, global positioning or other information, and/or any other means. Locate marks may be represented in any form including, without limitation, physical, visible, electronic, and/or any combination thereof.
100341 The terms "actuate" or "trigger" (verb form) are used interchangeably to refer to starting or causing any device, program, system, and/or any combination thereof to work, operate, and/or function in response to some type of signal or stimulus.
Examples of actuation signals or stimuli may include, but are not limited to, any local or remote, physical, audible, inaudible, visual, non-visual, electronic, mechanical, electromechanical, biomechanical, biosensing or other signal, instruction, or event. The terms "actuator" or "trigger" (noun form) are used interchangeably to refer to any method or device used to generate one or more signals or stimuli to cause or causing actuation.
Examples of an actuator/trigger may include, but are not limited to, any form or combination of a lever, switch, program, processor, screen, microphone for capturing audible commands, and/or other device or method. An actuator/trigger may also include, but is not limited to, a device, software, or program that responds to any movement and/or condition of a user, such as, but not limited to, eye movement, brain activity, heart rate, other data, and/or the like, and generates one or more signals or stimuli in response thereto. In the case of a marking device or other marking mechanism (e.g., to physically or electronically mark a facility or other feature), actuation may cause marking material to be dispensed, as well as various data relating to the marking operation (e.g., geographic location, time stamps, characteristics of material dispensed, etc.) to be logged in an electronic file stored in memory. In the case of a locate device or other locate mechanism (e.g., to physically locate a facility or other feature), actuation may cause a detected signal strength, signal frequency, depth, or other information relating to the locate operation to be logged in an electronic file stored in memory.
100351 The terms "locate and marking operation," "locate operation," and "locate"
generally are used interchangeably and refer to any activity to detect, infer, and/or mark the presence or absence of an underground facility. In some contexts, the term "locate operation" is used to more specifically refer to detection of one or more underground facilities, and the term "marking operation" is used to more specifically refer to using a marking material and/or one or more marking objects to mark a presence or an absence of one or more underground facilities. The term "locate technician" refers to an individual performing a locate operation. A locate and marking operation often is specified in connection with a dig area, at least a portion of which may be excavated or otherwise disturbed during excavation activities.
100361 The term "user" refers to an individual utilizing a locate device and/or a marking device and may include, but is not limited to, land surveyors, locate technicians, and support personnel.
[0037] The terms "locate request" and "excavation notice" are used interchangeably to refer to any communication to request a locate and marking operation. The term "locate request ticket" (or simply "ticket") refers to any communication or instruction to perform a locate operation. A ticket might specify, for example, the address or description of a dig area to be marked, the day and/or time that the dig area is to be marked, and/or whether the user is to mark the excavation area for certain gas, water, sewer, power, telephone, cable television, and/or some other underground facility. The term "historical ticket"
refers to past tickets that have been completed.
100381 The term "complex event processing (CEP)" refers to a software and/or hardware-implemented (e.g., facilitated by a computer system, distributed computer system, computational analysis coded in software, and/or a combination thereof) technique relating to recognizing one or more events, patterns of events, or the absence of an event or pattern of events, within one or more input streams of information and performing one or more actions and/or computations in response to such recognition, in accordance with specified rules, criteria, algorithms, or logic. CEP generally involves detection of relationships between information contained in input streams (which input streams may include indications of previously recognized events), such as causality, membership, timing, event-driven processes, detection of complex patterns of one or more events, event streams processing, event correlation and abstraction, and/or event hierarchies. CEP may complement and contribute to technologies such as, but not limited to, service oriented architecture (SOA), event driven architecture (EDA), and/or business process management (BPM). CEP allows the information contained in the events flowing through all of the layers of a service business, an enterprise information technology infrastructure and/or management operation to be discovered, analyzed, and understood in terms of its impact on management goals and business processes, and acted upon in real time or as a management process.
[0039] The following U.S. published application are referenced:
[0040] U.S. publication no. 2008-0228294-Al, published September 18, 2008, filed March 13, 2007, and entitled "Marking System and Method With Location and/or Time Tracking;"
[0041] U.S. publication no. 2008-0245299-Al, published October 9, 2008, filed April 4, 2007, and entitled "Marking System and Method;"
[0042] U.S. publication no. 2009-0013928-Al, published January 15, 2009, filed September 24, 2008, and entitled "Marking System and Method;"
[0043] U.S. publication no. 2009-0202101-Al, published August 13, 2009, filed February 12, 2008, and entitled "Electronic Manifest of Underground Facility Locate Marks;"
[0044] U.S. publication no. 2009-0202110-Al, published August 13, 2009, filed September 11, 2008, and entitled "Electronic Manifest of Underground Facility Locate Marks;"
[0045] U.S. publication no. 2009-0201311-A1, published August 13, 2009, filed January 30, 2009, and entitled "Electronic Manifest of Underground Facility Locate Marks;"
[0046] U.S. publication no. 2009-0202111-A1, published August 13, 2009, filed January 30, 2009, and entitled "Electronic Manifest of Underground Facility Locate Marks;"
[0047] U.S. publication no. 2009-0204625-Al, published August 13, 2009, filed February 5, 2009, and entitled "Electronic Manifest of Underground Facility Locate Operation;"
[0048] U.S. publication no. 2009-0204466-Al, published August 13, 2009, filed September 4, 2008, and entitled "Ticket Approval System For and Method of Performing Quality Control In Field Service Applications;"

[0049] U.S. publication no. 2009-0207019-Al, published August 20, 2009, filed April 30, 2009, and entitled "Ticket Approval System For and Method of Performing Quality Control In Field Service Applications;"
[0050] U.S. publication no. 2009-0210284-Al, published August 20, 2009, filed April 30, 2009, and entitled "Ticket Approval System For and Method of Performing Quality Control In Field Service Applications;"
[0051] U.S. publication no. 2009-0210297-A1, published August 20, 2009, filed April 30, 2009, and entitled "Ticket Approval System For and Method of Performing Quality Control In Field Service Applications;"
[0052] U.S. publication no. 2009-0210298-Al, published August 20, 2009, filed April 30, 2009, and entitled "Ticket Approval System For and Method of Performing Quality Control In Field Service Applications;"
[0053] U.S. publication no. 2009-0210285-Al, published August 20, 2009, filed April 30, 2009, and entitled "Ticket Approval System For and Method of Performing Quality Control In Field Service Applications;"
[0054] U.S. publication no. 2009-0204238-Al, published August 13, 2009, filed February 2, 2009, and entitled "Electronically Controlled Marking Apparatus and Methods;"
[0055] U.S. publication no. 2009-0208642-Al, published August 20, 2009, filed February 2, 2009, and entitled "Marking Apparatus and Methods For Creating an Electronic Record of Marking Operations;"
[0056] U.S. publication no. 2009-0210098-Al, published August 20, 2009, filed February 2, 2009, and entitled "Marking Apparatus and Methods For Creating an Electronic Record of Marking Apparatus Operations;"
[0057] U.S. publication no. 2009-0201178-Al, published August 13, 2009, filed February 2, 2009, and entitled "Methods For Evaluating Operation of Marking Apparatus;"
[0058] U.S. publication no. 2009-0202112-Al, published August 13, 2009, filed February 11, 2009, and entitled "Searchable Electronic Records of Underground Facility Locate Marking Operations;" and [0059] U.S. publication no. 2009-0204614-Al, published August 13, 2009, filed February 11, 2009, and entitled "Searchable Electronic Records of Underground Facility Locate Marking Operations."
[0060] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. It should also be appreciated that terminology explicitly employed herein should be accorded a meaning most consistent with the particular concepts disclosed herein.
Brief Description of the DrawinEs [0061] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.
[0062] Figure 1 shows an example in which a locate and marking operation is initiated as a result of an excavator providing an excavation notice to a one-call center.
[0063] Figure 2 illustrates a block diagram of an automated quality assessment system for assessing the quality of a field service operation, in accordance with the present disclosure;
[0064] Figure 3 illustrates a flow diagram of an example of a process for automatically assessing the quality of a field service operation, in accordance with the present disclosure;
[0065] Figure 4 illustrates a functional block diagram of an example of an automated quality assessment application, in accordance with the present disclosure;
[0066] Figure 5 is an example of a facilities map to which information relating to a locate and/or marking operation may be compared;
[0067] Figure 6 illustrates a flow diagram of an example of a method of automatically performing a quality assessment, in accordance with the present disclosure;

[0068] Figure 7 illustrates a flow diagram of an example of a process for automatically generating a quality assessment of a locate and marking operation based, at least in part, on a comparison of information obtained from equipment used during the locate and marking operation and one or more facilities maps, in accordance with the present disclosure;
[0069] Figure 8 illustrates a flow diagram of an example of a process for determining the distance between two sets of geo-location points, in accordance with the present disclosure;
[0070] Figure 9 illustrates a block diagram of a computer, on which some embodiments in accordance with the present disclosure may be implemented; and [0071] Figure 10 is a diagram of an overlay of a locate and/or marking operation on an aggregated facilities map, in accordance with the present disclosure.
Detailed Description [0072] Following below are more detailed descriptions of various concepts related to, and embodiments of, inventive systems, methods and apparatus for analyzing locate and marking operations with respect to facilities maps. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways, as the disclosed concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes.
[00731 Various inventive embodiments disclosed herein relate to methods, apparatus and systems for performing oversight and quality control in field service operations, such as locate and marking operations. In general, approvers and/or managers may review the quality of these locate and marking operations in real time and/or within a certain amount of time (e.g., within one day) of completion of the operation. The review of a locate and marking operation by a human (e.g., an approver or manager) and the determination of a quality assessment for the operation based solely on the discretion of the human is referred to herein as a "manual quality assessment."

[0074] Some embodiments described herein are related to methods, apparatus and systems for at least partially automating oversight and quality assessment in underground facility locate operations and/or other field service operations. For example, in some embodiments, an automated quality assessment system may receive "field information"
(also referred to as "field data") related to a locate and marking operation from one or more sources of electronic data (e.g., electronic records of locate and marking operations generated by various locate equipment, an electronic manifest for same, ticket information, service-related information, etc.), electronically analyze the contents of the field information/data by comparing it to "reference information" (also referred to as "reference data") derived from or related to one or more facilities maps, and automatically assess the quality of the operation based at least in part on the analysis (e.g., according to predetermined criteria on which the comparison is based and metrics for the criteria).
100751 In other embodiments, automated analysis of field information/data facilitates further analysis and/or quality assessment by a human, in which the quality assessment is not based solely on the discretion of the human, but is significantly informed in some manner by automated analysis of data. As contrasted with the above-discussed "manual quality assessment" of a locate and marking operation by a human, this type of assessment (e.g., based on some degree of electronic analysis of data relating to a locate and marking operation) is referred to herein as "automated quality assessment."
100761 In some embodiments, methods, apparatus and systems according to the present invention may automatically output one or more of a variety of indications of the assessed quality of a locate operation. In one aspect, the indication of the assessed quality of a locate operation may be a categorized into one or more of a plurality of quality categories. Any suitable number and type of categories may be used, as the invention is not limited in this respect. For example, in some embodiments, a locate operation may be automatically categorized as either, (a) approved - no further action needed;
(b) satisfactory, but the locate technician needs coaching or training; (c) unsatisfactory - the ticket needs quality control (QC) action; or (d) real-time prompt - an aspect of the assessment may be suitable for prompting the locate technician in real time with respect to, for example, performing an immediate verification and/or corrective action. In other implementations, a score, grade, or other graduated indication (e.g., based on some maximum range or scale) may be provided as an indication of quality assessment in connection with a locate and marking operation.
[0077] Figure 2 is a block diagram of an automated quality assessment system 1800.
Automated quality assessment system 1800 may be, for example, a computer system having at least one hardware processor 1803, a memory 1805 that comprises at least one tangible storage medium (e.g., RAM, ROM, Flash memory, one or more magnetic storage devices, one or more optical storage devices, or any other type of tangible storage medium), and at least one communications interface 1801. Memory 1805 may store computer-readable instructions of an automated quality assessment application 1200, which may be executed by processor 1803. When executed by processor 1803, automated quality assessment application 1200 may obtain information associated with a field service operation (e.g., a locate and marking operation) from data sources 1216 via communication interface 1801, analyze the data to assess the quality of the field service operation and may output (e.g., via communication interface 1801) one or more indications of the quality assessment of the field service operation. In some implementations, one or more indications of the quality assessment may be stored in the memory and/or transmitted via the communication interface to provide an electronic record of the quality assessment.
[0078] Figure 3 is a flow chart of process 1900 that may be performed by quality assessment application 1200 to automatically assess the quality of a field service operation, such as, for example, a locate and marking operation. Process 1900 begins at act 1901, where the automated quality assessment application receives electronic information associated with a field service operation. The process next continues to act 1903, where the automated quality assessment application analyzes at least some of the received information to automatically generate a quality assessment of the field service operation.
The process next continues to act 1905, where the automated quality assessment application outputs an indication of the quality of the field service operation that is based on the assessment generated in the act 1903.
[0079] Referring to Figure 4, a more detailed functional block diagram of automated quality assessment application 1200 and data sources 1216 is presented.
Automated quality assessment application 1200 may be, for example, a rules-based computer software application that includes, for example, an information processing component 1210, quality assessment outcomes 1212 (e.g., one or more indications of the quality assessment), and a feedback component 1214. Automated quality assessment application 1200 may be fed by any number of data sources 1216, which may include various types of electronic information and/or records of data associated with locate and marking operations performed in the field (e.g., both "field information/data" and "reference information/data").
[00801 For example, the automated quality assessment application 1200 of the present disclosure may automatically review a variety of field information, which may include "closed" or completed tickets (i.e., tickets pursuant to which a locate and marking operation has been performed) and their associated manifests (which may or may not include digital images relating to the locate operation), and/or any information relating thereto, in essentially real time and/or within a specified amount of time, such as within one day, from the ticket being closed. In some embodiments discussed in further detail below, closed tickets may be reviewed by automatically interrogating received data associated with a locate and marking operation against various metrics, such as reference information/data derived from or relating to one or more facilities maps.
[0081] In some embodiments, information processing component 1210 of automated quality assessment application 1200 may be, for example, a rules-based software component for analyzing the contents of any information that is available in data sources 1216 and then automatically performing an assessment with respect to the quality of a locate operation that is performed in the field. For each locate and marking operation that is assessed, information processing component 1210 may automatically generate a quality assessment outcome 1212 that corresponds to the results of the automatic quality assessment.
[0082] Any suitable type of outcome may be generated. For example, in some embodiments, the outcome generated may be a categorization of the locate operation into one of a plurality of quality categories (also referred to herein as "scoring"
categories or "grading" categories). For example, based on the automatic quality assessment, a locate operation may be categorized as:
= APPROVED ¨ the locate operation is approved, no further action needed;

= SATISFACTORY ¨ the locate operation is approved, but the locate technician needs coaching or training;
= UNSATISFACTORY ¨ the locate operation is not approved, the ticket needs QC action; or = PROMPT ¨ an aspect of the locate operation assessment may be suitable for transmitting a real-time prompt to the locate technician with respect to, for example, performing a substantially immediate verification and/or corrective action.
[0083] Other examples of possible outcomes generated by automated ticket application 1200 include, but are not limited to, a numerical score (e.g., a score of 0-100%), a grade (e.g., a grade of A-F), or other graduated indicator, based on some range, scale and/or resolution (granularity), that is indicative of the quality of the assessed locate operation.
[0084] Feedback component 1214 of automated quality assessment application 1200 generates the real-time prompts. For example, once the nature of the real-time prompt is determined, feedback component 1214 queries the ticket information in order to ensure that the prompt is directed to the proper originating locate technician.
Additional details of the operation of automated quality assessment application 1200 are described with reference to the method of Figure 15.
[0085] Examples of data sources 1216 that may be processed by information processing component 1210 of automated quality assessment application 100 may include, but are not limited to, one or more tickets 1220, a virtual white lines (VWL) application 1230, a ticket assessment application 1240, locating equipment data 1250, an electronic manifest (EM) application 1260, a collection of facilities maps 1280, an archive of historical tickets 1290, and any other electronic information and/or records 1295. In implementation, the various data sources 1216 may be supplied by multiple entities (not shown) and accessible to automated quality assessment application 1200 via, for example, a networked computing system for supporting locate operations, an example of which is described with reference to Figures 14 and 15.
[0086] In various embodiments of automated quality assessment based on information/data derived from the data sources 1216, it should be appreciated that some of this information/data may be treated as "field information/data" and some of this information/data may be treated as "reference information/data" to which the field information/data is compared during the assessment process. Additionally, it should be appreciated that some of the information/data available from the data sources 1216 may be used to "pre-process" or filter one or both of the field information/data and the reference information/data prior to comparison for some types of assessments.
[0087] Tickets 1220 of data sources 1216 are locate request tickets that may be submitted by excavators and processed by one-call centers. Tickets 1220 may include textual ticket information 1222 that comprises instructions with respect to performing a locate operation, such as, but not limited to, a ticket and/or work order number, date information, geographic location information (e.g., address information), excavation information, excavator information, site information (e.g., a description of the dig area), locate operations instructions information, caller information, remarks information, task information, and any combinations thereof.
[0088] Textual descriptions of dig areas included in tickets may, in some instances, be very imprecise as to exact physical locations at which digging is planned.
Therefore, when a locate request is submitted by an excavator, it may be beneficial for the excavator to supplement the locate request with a visit to the site of the dig area for the purpose of indicating the particular geographic location of the proposed excavation. For example, marks (e.g., white paint) on the ground at the location at which digging is planned may be used to physically indicate a dig area in order to communicate to a locate technician the extent of the boundaries where digging is planned. These marks may be chalk marks or paint that is applied to the surface of the ground, and are generally known as "white lines."
[0089] VWL application 1230 of data sources 1216 is a computer software application that provides an electronic drawing tool that may be used by excavators for electronically marking up, for example, a digital aerial image of the dig area. In this manner, instead of (or in addition to) physically visiting the site of the dig area and marking white lines on the ground at that site, an excavator may electronically draw markings (e.g., white lines) on an aerial image of the site, indicating where digging is planned. These marked up digital images may be saved as, for example, VWL images 1232, which may include accompanied with metadata pertaining to various information in the images. One or more VWL images 1232 in turn may be associated with, for example, tickets 1220 and transmitted to locate companies.
[0090] VWL application 1230 may be implemented, for example, as described in U.S.
Patent Application Serial No. 12/366,853 filed February 6, 2009, entitled "Virtual white lines for delimiting planned excavation sites;" U.S. Patent Application Serial No.
12/475,905 filed June 1, 2009, entitled "Virtual white lines for delimiting planned excavation sites of staged excavation projects;" U.S. Patent Application Serial No.
12/422,364 filed April 13, 2009, entitled "Virtual white lines (VWL) application for indicating a planned excavation or locate path.
[0091] In one example, the dig area indicators in a VWL image may include two-dimensional (2D) drawing shapes, shades, points, symbols, coordinates, data sets, or other indicators to indicate on a digital image the dig area in which excavation is to occur. To generate the electronic image having dig area indicators, an image (e.g., an aerial image) of the work site may be sent to an excavator via a network, the excavator may use a computing device executing the VWL application 1230 to create a VWL image by marking up the image to include one or more dig area indicators precisely delimiting one or more dig areas within the work site and, in response, the marked-up VWL
image may be received from the excavator via the network.
[0092] As noted above, a VWL image 1232 may include metadata corresponding to any markings or content in the image; in particular, geographic information including geographic coordinates (e.g., latitude and longitude values) for any dig area indicators marked on the image may accompany or be included in an image file as metadata, and these geographic coordinates may be employed in some manner as part of a quality assessment process. For example, as discussed further below, in one embodiment geographic information derived from a virtual white lines (VWL) application 1230 (e.g., geographic coordinates associated with one or more dig area indicators contained in a VLW image 1232) may be used by automated quality assessment application 1200 to filter or limit the contents of either field data or reference data prior to analysis/comparison.
[0093] In particular, in one exemplary implementation, geographic coordinates associated with a dig area indicator may be used to select contents that relates only to a geographic area including the geographic coordinates for the dig area indicator, or contents that falls within a predetermined radius of the geographic coordinates for the dig area indicator or a polygon-shaped buffer zone around the geographic coordinates for the dig area indicator. In yet another example, geographic coordinates associated with a dig area indicator may be used to filter out some contents that does not relate to a specifically delimited dig area within a work site as defined by the VWL application (e.g., first geographic information or another portion of information may be selected from the field data, and/or second geographic information or another portion of information may be selected from the reference data, that relates only to a geographic area delimited by the VWL geographic information). Accordingly, it should be appreciated that in some embodiments, the dig area indicator coordinates may identify a plurality of points along a perimeter of the delimited dig area, and these coordinates may be used to select specific geographic information (e.g., filter out geographic information outside of the delimited dig area). In other embodiments, the dig area indicator coordinates may identify a single point, in which case the coordinates may be used to select particular information based at least in part on the coordinates for the single point.
[00941 With respect to locating equipment data 1250, as noted above, a locate technician may use locating equipment, such as a locate instrument set (including a locate receiver device), a marking device, or a combined locate and marking device, so as to perform a locate and marking operation. Locating equipment data 1250 of data sources 1216 may be any information that is collected and/or generated (e.g., one or more electronic records) by any type of locating equipment equipped with components that are capable of collecting electronic information and/or creating electronic records about locate and marking operations that are performed in the field. In some examples, locating equipment data 1250 is constituted by "marking information" or marking device data 1252 that is associated generally with the marking functionality of a locate and marking operation, and/or "locate information" or locate receiver data 1254 that is associated generally with the locating/detection functionality of a locate and marking operation.
Locating equipment data 1250 also may include "landmark information" that may be acquired by suitably configured locating equipment (e.g., a marking device, a locate device, or a combined locate and marking device capable of operating in a "landmark mode"), which information may be acquired either independently or as part of (e.g., during or proximate in time to) a locate and marking operation.

[00951 In one example, marking device data 1252 of locating equipment data 1250 may be electronic information and/or one or more electronic records of data that is provided by electronic marking devices and/or marking systems. Examples of electronic marking devices and/or marking systems that may provide marking device data 1252 may include, but are not limited, to those described in reference to U.S. Patent Application No.
11/696,606, filed April 4, 2007 and published October 9, 2008, entitled "Marking system and method," U.S. Patent Application No. 11/685,602, filed March 13, 2007 and published September 18, 2008, entitled "Marking system and method," U.S. Non-provisional Application Serial No. 12/568,087, filed on September 28, 2009, entitled "Methods and Apparatus for Generating an Electronic Record of Environmental Landmarks Based on Marking Device Actuations," U.S. Non-provisional Application Serial No. 12/539,497, filed on August 11, 2009, entitled "Methods and Apparatus for Generating an Electronic Record of a Marking Operation based on Marking Device Actuations".
[0096] Table 1 shows one example of a sample of marking device data 1252 of locating equipment data 1250 that may be captured as the result of, for example, an actuation of a marking device. In some exemplary implementations, an electronic record of a marking operation may include multiple data entries as shown in the example of Table 1 for respective actuations of a marking device to dispense a marking material (e.g., in some cases there may be one set of data as shown in Table 1 for each actuation). In this manner, each time a marker is dispensed (so as to indicate a presence or absence of a given underground facility), data is collected relating to the geographic location of the dispensed marker (e.g., geo-location data). Additionally, data relating to a characteristic of the dispensed marker (e.g., color and/or brand) is included in the data entries of the electronic record, as well as other data germane to the marking operation.

Table 1: Example marking device data 1252 of locating equipment data 1250 Service provider ID 0482 Locate technician ID 4815 Marking Device ID 7362 Timestamp data 12-Jul-2008; 09:35:15.2 Geo-location data N35 43.57518, W078 49.78314 (deg. and dec.
min.) Marking material data Color=Red, Brand=ABC
Temperature data 73 degrees F
Humidity data 30%
_ Light data 4.3 volts Compass data 213 degrees Inclinometer data -40 Accelerometer data 0.275g Battery strength data 73%
[0097] Table 2 below shows another example of marking device data 1252 of locating equipment data 1250 that may be captured as the result of, for example, one or more actuations of a marking device. Specifically, Table 2 illustrates multiple "actuation data sets" of an electronic record of a marking operation as generated by a marking device, in which each actuation data set includes information associated with multiple actuation event entries logged during a corresponding actuation and dispensing of a locate mark.
Table 2 shows three actuation data sets of an electronic record, corresponding to three actuations of the marking device (e.g., act-1, act-2, and act-3). As may be appreciated from the information shown in Table 2, multiple pieces of geo-location data are logged for each actuation of a marking device (in addition to various other information).

Table 2 Example actuation data set for act-1 Service provider ID 0482 User ID 4815 Device ID 7362 T1 timestamp data 12-Jul-2008; 09:35:15.2 T2 timestamp data 12-Jul-2008; 09:35:16.1 Duration (At) 00:00:00.9 Ti geo-location data 2650.9348,N,08003 .5057,W
1st interval location 2650.9353,N,08003.5055,W
data act-1 2" interval location 2650.9356,N,08003 .5055 ,W
data Nth interval location 2650.9246,N,08003.5240,W
data T2 _geo-location data 2650.9255,N,08003. 5236,W
Product data Color=Red, Brand=ABC, Type/Batch = 224B-1 Locate request data Requestor: XYZ Construction Company, Requested service address: 222 Main St, Orlando, FL
Table 2 (continued) Example actuation data set for act-2 Service provider ID 0482 User ID 4815 Device ID 7362 Ti timestamp data 12-Jul-2008; 09:35:17.5 T2 timestamp data 12-Jul-2008; 09:35:18.7 Duration (At) 00:00:01.2 Ti geo-location data 2650.9256,N,08003 .5234,W
1st interval location 2650.9256,N,08003.5226,W
data 2nd interval location 2650.9256,N,08003 .5217,W
act-2 data =
=
Nth interval location 2650.9260,N,08003 .5199,W
data T2 geo-location data 2650.9266,N,08003 .5196,W
Product data Color=Red, Brand=ABC, Type/Batch = 224B-1 Locate request data Requestor: XYZ Construction Company, Requested service address: 222 Main St, Orlando, FL

Table 2 (continued) Example actuation data set for act-3 Service provider ID 0482 User ID 4815 Device ID 7362 Ti timestamp data 12-Jul-2008; 09:35:18.7 T2 timestamp data 12-Jul-2008; 09:35:19.8 duration (At) 00:00:01.1 Ti geo-location data 2650.9273,N,08003.5193,W
1st interval location 2650.9281,N,08003.5190,W
data act-3 2"d interval location 2650.9288,N,08003.5188,W
data Nth interval location 2650.9321,N,08003.5177,W
data T2 geo-location data 2650.9325,N,08003.5176,W
Product data Color=Red, Brand=ABC, Type/Batch = 224B-1 Locate request data Requestor: XYZ Construction Company, Requested service address: 222 Main St, Orlando, FL
[0098] With regard to the marking material color information that may be included in marking device data 1252 as exemplified in Tables 1 and 2, Table 3 shows an example of the correlation of marking material color to the type of facility to be marked.
Table 3: Correlation of color to facility type Marking Facility Type material color White Proposed excavation Pink Temporary survey markings Red Electric power lines, cables or conduits, and lighting cables Yellow Gas, oil, steam, petroleum, or other hazardous liquid or gaseous materials Orange Communications, cable TV, alarm or signal lines, cables, or conduits Blue Water, irrigation, and slurry lines Purple Reclaimed water, irrigation and slurry lines Green Sewers, storm sewer facilities, or other drain lines Black Mark-out for errant lines [0099] In another example, locate receiver data 1254 of locating equipment data 1250 may be electronic information (e.g., one or more electronic records) of data that is provided by electronic locate receiver devices and/or systems. Examples of a locate receiver device that may provide locate receiver data 1254 are described in U.S. Non-provisional Application Serial No. 12/569,192, filed on September 29, 2009, entitled "Methods, Apparatus, and Systems for Generating Electronic Records of Locate and Marking Operations, and Combined Locate and Marking Apparatus for Same".
1001001 Table 4 below shows an example of a sample of locate receiver data 1254 of locating equipment data 1250 that may be captured, for example, at one or more times during operation/use of an appropriately configured locate receiver. Different models of locate receivers and transmitters are available from a variety of manufacturers and have different features; accordingly, it should be appreciated that the information content and type provided in Table 4 is exemplary of possible information relating to locate receivers on which a quality assessment of a locate operation may be based, and that other types and values for information are possible. With respect to information potentially provided by a given locate receiver as shown in Table 4 below, the "gain" is typically a measure of the degree of sensitivity of a locate receiver antenna that is picking up a signal emanating from along an underground facility (alternatively, "gain" may be viewed as a degree of amplification being applied to a received signal). Gain may be expressed in terms of any scale (e.g., 0-100), as a numeric value or percentage. "Signal strength"
refers to the strength of a received signal at a given gain value; signal strength similarly may be expressed in terms of any scale, as a numeric value or percentage. Generally speaking, higher signal strengths at lower gains typically indicate more reliable information from a locate receiver, but this may not necessarily be the case for all locate operations.
1001011 In some exemplary implementations, an electronic record of a locate operation as obtained from a locate receiver may include multiple data entries as shown in the example of Table 4. Each such entry may not only include information about various operating parameters of the locate receiver (e.g., signal strength, gain), but may additionally include location information (geo-location data) associated with detected facilities, as well as various environmental data. The logging of a given entry by a locate receiver may automatically result from one or more conditions (e.g., signal strength exceeding a particular threshold) and/or respective data entries may be manually logged by a technician using the locate receiver (e.g., via a push button, touch screen, trigger actuation, or other interaction facilitated by a user interface of the locate receiver). In this manner, multiple pieces of data may be collected for an electronic record of a locate operation, including multiple pieces of geo-location data for a given underground facility detected via the locate receiver.
Table 4: Example locate receiver data 1254 of locating equipment data 1250 Service provider ID 0482 Locate technician ID 4815 Locate Device ID 7345 Timestamp data 12-Jul-2008; 09:35:15.2 Geo-location data N35 43.57518, W078 49.78314 (deg. and dec.
min.) Locate mode Mode=PASSIVE
Gain 35 (on a scale of 1-100) Sig. strength 85% (on a scale of 0-100%) Signal frequency 60 Hz Facility depth 3.4 feet , Temperature data 73 degrees F
Humidity data 30%
Light data 4.3 volts Compass data 213 degrees Inclinometer data -40 Accelerometer data 0.275g Battery strength data 85%
[00102] In another example, both marking device data 1252 and locate receiver data 1254 of locating equipment data 1250 may be electronic information (e.g., one or more electronic records) of data that is provided by a combined locate and marking device. An example of such a combined locate and marking device is described in U.S. Non-provisional Application Serial No. 12/569,192, filed on September 29, 2009, entitled "Methods, Apparatus, and Systems for Generating Electronic Records of Locate and Marking Operations, and Combined Locate and Marking Apparatus for Same".
[00103] Table 5 below illustrates one non-limiting example of four actuation data sets that may be collected in an electronic record generated by a combined locate and marking device, in which each data set corresponds, for example, to a separate actuation event to dispense marking material. It should be appreciated, however, that these are merely examples, and that various alternative electronic records may be generated according to the aspects of the invention, for example reflecting different types of information associated with operation of a combination locate and marking device.

100104] Each of the four records of Table 5 includes general information not limited to either the locate receiver functionality or marking functionality of the combination device, such as an identification of the service provider (Service provided ID), an identification of the user (User 1D), an identification of the device (Device ID), and information about the requestor of the locate operation and the requested address (Locate request data). In addition, an entry describing the mode of data collection (e.g., Manual) for the device is also collected, which may indicate that information is logged into the record(s) upon actuation of the combined locate and marking device. Information about the actuation itself, such as the time of actuation (Timestamp data), actuation duration, and geographical location (geo-location data) at the start, during, and/or at and end of the actuation may also be included. The data sets also include information relating to the locate receiver functionality of the combination locate and marking device, including the receiver detection mode (i.e., PEAK in Table 5), the strength of a detected signal, and the frequency of the detected signal. Information relating to a depth measurement (Facility depth) is also included, as is information about the marking material to be dispensed by the combination locate and marking device. Again, it should be appreciated that Table 5 is an illustration of one electronic record including multiple data sets that may be generated in association with operation of a combination locate and marking device, and that other forms of electronic records are also possible.

Table 5 Electronic Record for Combination Locate and Marking Device Service provider ID 0482 User ID 4815 Device ID 7362 Device mode Mode=MANUAL .
Timestamp data 12-Jul-2008; 09:35:15 Actuation duration 0.5 sec .
Start actuation location 2650.9348,N,08003.5057,W
data _ End actuation location 2650.9353,N,08003. 5055,W
Record # 1001 - data Locate mode Mode=PEAK
Signal strength (% of 85%
. maximum) . Signal frequency _ 1 kHz Facility depth 3.4 meters . Marking material data Color=RED, Brand=ABC
Locate request data Requestor¨XYZ Construction Company, Requested service address=222 Main St, Orlando, FL
Table 5 (continued) Electronic Record for Combination Locate and Marking Device Service provider ID 0482 .
User ID 4815 Device ID 7362 .
Device mode Mode=MANUAL
Timestamp data 12-Jul-2008; 09:35:18 _ Actuation duration 0.4 sec Start actuation location 2650.9256,N,08003.5234,W
data End actuation location 2650.9256,N,08003.5226,W
Record # 1002 data Locate mode Mode=PEAK
Signal strength (% of 85%
maximum) Signal frequency 1 kHz Facility depth 3.4 meters Marking material data Color=RED, Brand=ABC
Locate request data Requestor¨XYZ Construction Company, Requested service address=222 Main St, Orlando, FL

-34-Table 5 (continued) Electronic Record for Combination Locate and Marking Device Service provider ID 0482 User ID 4815 Device ID 7362 Device mode Mode=MANUAL
Timestamp data 12-Jul-2008; 09:35:21 Trigger pull duration 0.5 sec Start actuation location 2650.9273,N,08003.5193,W
data Record End actuation location 2650.9281,N,08003.5190,W
# 1003 data Locate mode Mode=PEAK
Signal strength (% of 85%
maximum) Signal frequency 1 kHz Facility depth 3.4 meters Marking material data Color=RED, Brand=ABC
Locate request data Requestor¨XYZ Construction Company, Requested service address=222 Main St, Orlando, FL
Table 5 (continued) Electronic Record for Combination Locate and Marking Device Service provider ID 0482 User ID 4815 Device ID 7362 Device mode Mode=MANUAL
Timestamp data 12-Jul-2008; 09:35:25 Actuation (actuation) 0.5 sec duration Start actuation location 2650.9321,N,08003.5177,W
data Record End actuation location 2650.9325,N,08003.5176,W
# 1004 data Locate mode Mode=PEAK
Signal strength (% of 85%
maximum) Signal frequency 1 kHz Facility depth 3.4 meters Marking material data Color=RED, Brand=ABC
Locate request data Requestor¨XYZ Construction Company, Requested service address=222 Main St, Orlando, FL
[00105] While the collection and logging of locate information and marking information to generate an electronic record is discussed in some aspects, for purposes of

-35-illustration, in terms of actuation data sets (i.e., a set of data that is associated and logged with a corresponding actuation of a locate device, marking device, or combined locate and marking device), it should be appreciated that electronic records as discussed herein are not limited in this respect. More generally, an electronic record of a locate and/or marking operation may be generated in any of a variety of manners, have a variety of file formats and/or data structures, and include any of a variety of locate information and/or marking information (some of which may be germane to one or more actuations of a device, some of which may be common to multiple actuations or the overall locate and/or marking operation in general, and some of which may not be related to specific actuations). For example, in some exemplary implementations electronic records may be a "flat files"
including a succession of time stamped "event entries" of various locate information and/or marking information (logged automatically as a result of one or more particular conditions, e.g., exceeded thresholds for various signals, or manually as a result of user actuation of a device), or a differently formatted file (e.g., an ASCII file, an XML file) having a data structure that segregates or separates in some manner the locate information and/or marking information into multiple different fields.
[00106] It should also be appreciated that one or both of the marking device data 1252 and locate receiver data 1254 of locating equipment data 1250, received from any of the marking devices, locate devices, or combined locate and marking devices referenced above, may include landmark information (in addition to, or alternatively to, locate information and marking information). Landmark information may include any information relating to one or more environmental landmarks of interest (e.g., in and around the work site/dig area and/or generally in the vicinity of the locate and marking operation). Examples of landmark information include, but are not limited to, geo-location data of an environmental landmark, type of environmental landmark, and a time stamp for any acquired information relating to an environmental landmark. In some instances, landmark information may be acquired from locate equipment particularly configured to operate in a landmark mode so as to acquire such information, as well as one or more other modes (e.g., "locate mode" or "marking mode") to accomplish functions relating to detection and/or marking of underground facilities.
1001071 Tables 6A and 6B below show examples of landmark information that may be included in an electronic record forming part of either marking device data 1252 or locate

-36-receiver data 1254 of locating equipment data 1250. Table 6A shows the format and content of an electronic record entry for a utility pole, which includes one geo-location data point, and Table 6B shows the format and content of an electronic record entry for a pedestal, which includes four geo-location data points (i.e., one for each corner of the pedestal). As noted above, it should be appreciated that the format and content shown below in Tables 6A and 6B is provided primarily for purposes of illustration, and that a variety of formats and content may be employed for an electronic record entry for landmark information.
Table 6A: Example record of landmark information acquired for a utility pole Service provider ID 0482 User ID 4815 Device ID 7362 Record Type of EL Type=utility pole # 1 timestamp data 12-Jul-2008; 09:35:17.5 geo-location data 2650.9256,N,08003.5234,W
Locate request data Requestor: XYZ Construction Company, Requested service address: 222 Main St, Orlando, FL
Table 6B: Example record of landmark information acquired for a pedestal Service provider ID 0482 User ID 4815 Device ID 7362 Type of EL Type= pedestal timestamp data 12-Jul-2008; 09:35:17.5 geo-location data 2650.9256,N,08003.5234,W
Type of EL Type= pedestal timestamp data 12-Jul-2008; 09:35:21.2 Record geo-location data 2650.9256,N,08003.5226,W
# 2 Type of EL Type= pedestal timestamp data 12-Jul-2008; 09:35:26.7 geo-location data 2650.9288,N,08003.5188,W
Type of EL Type= pedestal timestamp data 12-Jul-2008; 09:35:33.5 geo-location data 2650.9321,N,08003.5177,W
Locate request data Requestor: XYZ Construction Company, Requested service address: 222 Main St, Orlando, FL

-37-[00108] Electronic Manifest (EM) application 1260 of data sources 1216 is a computer software application that may be used to create an electronic manifest of a locate operation.
As discussed above, an electronic manifest may include a digital (e.g., aerial) image of the work site/dig area and its surroundings, upon which one or more "electronic locate marks"
have been placed for indicating corresponding physical locate marks that have been placed on the ground, pavement or other surface at the site, thereby indicating the geo-locations and types of facilities present. One or more landmarks also may be indicated on the digital image together with the electronic locate marks. Via the EM
application 1260, the digital images may be marked up "manually" by a technician (e.g., using a stylus or other type of user interface in conjunction with the digital image displayed in a display field) to include one or more electronic locate marks and/or one or more identifiers for environmental landmarks. Alternatively, a digital image may be marked up "automatically" by importing data, for example, from one or more pieces of locate equipment (e.g., a locate device, a marking device, or a combined locate and marking device) and overlaying the imported data on the digital image.
[00109] In one example, the starting digital images to be marked up using EM
application 1260 may be VWL images 1232 that are associated with tickets 1220.
In this manner, the resulting EM image contains the original dig area indicator (e.g., from the VWL images) to delimit the dig area for the locate and marking operation, together with any electronic locate marks and/or landmarks added to the image via the EM
application.
The marked up digital images may be saved as, for example, EM images 1262, which may be associated with, for example, tickets 1220 and may be used by locate companies to support proof of work compliance. In some embodiments, EM application 1260 may implemented as described in U.S. Patent Application Serial No. 12/369,232, filed February 11,2009 entitled "Searchable records of underground facility locate marking operations".
[00110] As noted above in connection with VWL images 1232 provided by VWL
application 1230, an EM image 1262 may include metadata corresponding to any markings or content in the image; in particular, geographic coordinates (e.g., latitude and longitude values) for any dig area indicator, electronic locate marks, and/or landmarks marked on the image may accompany or be included in an image file as metadata.

Accordingly, these geographic coordinates, as well as any other information provided by

-38-EM application, may be employed in some manner as part of a quality assessment process (e.g., as field information/data, or in some instances as reference information/data, or in some instances to pre-process or filter one or both of field information/data and reference information/data prior to comparison).
[00111] Facilities maps 1280 of data sources 1216 are any physical, electronic, or other representation of the geographic location, type, number, and/or other attributes of a facility or facilities. Facilities maps 1280 may be supplied by the various facility owners and may indicate the geographic location of the facility lines (e.g., pipes, cables, and the like) owned and/or operated by the facility owner. For example, facilities maps 1280 may be supplied by the owner of the gas facilities, power facilities, telecommunications facilities, water and sewer facilities, and so on. In the process of performing the automatic quality assessment, information processing component 1210 may aggregate the information that is contained in multiple facilities maps 1280 in order to determine all the facilities that are present in and around a certain work site/dig area.
[00112] As indicated above, facilities maps may be provided in any of a variety of different formats. As facilities maps often are provided by facility owners of a given type of facility, typically a set of facilities maps includes a group of maps covering a particular geographic region and directed to showing a particular type of facility disposed/deployed throughout the geographic region. One facilities map of the set of maps is sometimes referred to in the relevant arts as a "plat."
[00113] Perhaps the simplest form of facilities maps is a set of paper maps that cover a particular geographic region. In addition, some facilities maps may be provided in electronic form. An electronic facilities map may in some instances simply be an electronic conversion (i.e., a scan) of a paper facilities map that includes no other information (e.g., electronic information) describing the content of the map, other than what is printed on the paper maps.
[00114] Alternatively, however, more sophisticated facilities maps also are available which include a variety of electronic information, including geographic information and other detailed information, regarding the contents of various features included in the maps.
In particular, facilities maps may be formatted as geographic information system (GIS) map files, in which map features (e.g., facility lines; environmental landmarks relating to

-39-facilities such as pedestal boxes, utility poles, fire hydrants, manhole covers and the like;
one or more architectural elements such as buildings; and/or one or more traffic infrastructure elements such as streets, intersection, curbs, ramps, etc.) are represented as shapes and/or lines, and the file provides metadata describing the geographic locations and types of map features.
[00115] For example, a GIS map file may indicate a facility line using a straight line, which may have a diamond shape at each endpoint of the line to indicate where the line begins and terminates. Such a GIS map file may also include various shapes or symbols indicating different environmental landmarks, architectural elements, and/or traffic infrastructure elements. Examples of information provided by metadata for the map file (i.e., included as part of the electronic file for the map) includes, but is not limited to, information about the geo-location of various points along a given line, the termination points of a given line (e.g., the diamond shapes indicating the start and end of the line), the type of facility line (e.g., facility type and whether the line is a service line or main), geo-location of various shapes and/or symbols for other features represented in the map (environmental landmarks, architectural elements, traffic infrastructure elements), and type information relating to shapes and/or symbols for such other features.
From the foregoing, it should be appreciated that in some instances, given that the geo-locations of two termination or end-points of a given facility line may be provided by the map, the geo-location of any point on the facility line may be determined from these two end-points.
[00116] Facilities maps may include additional information that may be useful to a quality assessment process. For example, various information that may be included in a legend of the facilities map, or otherwise associated with the facilities map (e.g., included in the metadata or otherwise represented on the map), and available for use in a quality assessment process, may include, but is not limited to, a date of the facilities map (e.g., when the map was first generated/created, and/or additional dates corresponding to updates/revisions), a number of revisions to the facilities map (e.g., revision number, which may in some instances be associated with a date), one or more identifiers for a source, creator, owner and/or custodian of the facilities map (e.g., the owner of the facility type represented in the map), various text information (e.g., annotations to update one or more aspects or elements of the map), and any other legend information that may be included or represented in the map.

-40-[00117] Figure 5 shows an example of a visual representation of a portion of an electronic facilities map 500. In this example, facilities map 500 is a telecommunications facilities map that is supplied by a telecommunications company. Facilities map 500 shows telecommunications facilities in relation to certain landmarks, such as streets and roads, using lines and shapes. As discussed above, the electronic facilities map may include metadata indicating what various lines, symbols and/or shapes represent, and indicating the geo-location of these lines, symbols and/or shapes.
[00118] Historical tickets 1290 of data sources 1216 may include any records of locate request tickets performed in the past for the same work site/dig area specified in the present ticket 1220 subject to quality assessment. In the process of performing the automatic quality assessment of closed ticket 1220, information processing component 1210 may aggregate the information that is contained in one or more historical tickets 1290 relating to the same work site/dig area in order to determine the facilities that have been located and/or marked during past locate operations at that site.
[00119] Figure 6 shows a flow diagram of an exemplary process 1300 for performing a quality assessment of an underground facility locate and/or marking operation, as implemented by automated quality assessment application 1200. While the example provided in Figure 6 is a more specific example of the generic process 1900 discussed above in connection with Figure 3, and describes an automated quality assessment based on a completed or closed ticket for which it is presumed that a locate and/or marking operation was actually performed by a technician, it should be appreciated that the concepts generally outlined in the process 1300 may be applied to various types of available information relating to a requested locate operation and marking operation, whether performed separately or in tandem, and irrespective of actual performance of the locate operation and/or the marking operation, so as to assess the quality of the requested operation.
[00120] Process 1300 begins at act 1310, where a completed (i.e., closed) ticket is received and associated information to be used in assessing the quality of the locate and/or marking operation performed in connection with the ticket is collected by automated quality assessment application 1200. The associated ticket information may include, for example, the originating ticket information (e.g., textual ticket information 1222 of a certain ticket 1220), and one or more of the VWL images (e.g., a VWL image 1232 of a

-41-certain ticket 1220), the originating ticket assessment (e.g., a ticket assessment outcome 1242 of a certain ticket 1220), the locating equipment data (e.g., marking device data 1252 and/or locate receiver data 1254 of a certain ticket 1220), the EM images (e.g., a EM
image 1262 of a certain ticket 1220), and any other information (e.g., from other electronic information and/or records 1295).
[00121] The process then continues to act 1312, where the received information is used to automatically assess the quality of the locate and/or marking operation. In the example of Figure 6, a locate operation is categorized as either (a) APPROVED ¨ the operation is approved, no further action needed; (b) SATISFACTORY ¨ the operation is approved, but the locate technician needs coaching or training; (c) UNSATISFACTORY ¨ the operation is not approved, the ticket needs QC action; or (d) PROMPT ¨ an aspect of the operation assessment may be suitable for transmitting a real-time prompt to the locate technician with respect to, for example, performing a substantially immediate verification and/or corrective action. However, the invention is not limited in this respect, as any suitable indication of quality may be provided as a result of an automatic quality assessment, such as, a numerical score (e.g., a score from 0-100%), a letter grade, another type of graduated indictor based on some scale or range, or any other indication of quality.
Additional details and examples of how quality may be automatically assessed at act 1312 and an indication (e.g., a categorization) of quality may be automatically generated at act 1314 are discussed below. It should be appreciated that the invention is not limited to these particular examples, and that such examples are provided primarily for the purposes of illustration.
[00122] In some embodiments, the quality assessment of a locate and/or marking operation performed in act 1312 of Figure 6 may be based entirely or in part on a comparison of information about the performance of the locate and/or marking operation (e.g., "field information," i.e., one or more of locate information, marking information, landmark information and EM information, obtained from one or more of a locate device, a marking device, a combination locate and marking device, and an EM
application) and reference information obtained from one or more facilities maps.
1001231 Various types of field information about the performance of the locate and/or marking operation may be compared to any of a number of different types of reference information that may be obtained from one or more facilities maps. Some examples of

-42-types of field information about the performance of a locate and/or marking operation and types of reference information obtained from one or more facilities maps that may be compared to assess the quality of the locate and/or marking operation are provided below.
However, the invention is not limited to comparing the particular types of information provided in these examples. Furthermore, it should be appreciated that various techniques for comparing field information and reference information obtained from one or more facilities maps may be used, and the invention is not limited to any particular technique.
[00124] In some exemplary embodiments discussed in greater detail below, geographic information in the field data is compared to geographic information in the reference data.
For example, field geo-location data (e.g., one or more sets of latitude and longitude coordinates) relating to the detection and/or marking of a given underground facility during a locate and/or marking operation, and/or field geo-location data relating to one or more environmental landmarks, may be compared to reference geo-location data derived from one or more facilities maps.
[00125] More specifically, in some implementations, latitude and longitude coordinates corresponding to a detected and/or marked facility, and/or latitude and longitude coordinates corresponding to one or more environmental landmarks (field geo-location data), are compared to latitude and longitude coordinates (transformed if necessary to a common reference frame) derived from one or more facilities maps (reference geo-location data). In this manner, a correspondence or discrepancy (or degree of correspondence) may be ascertained between the field geo-location data and the reference geo-location data. As discussed in greater detail below, a first set of field latitude and longitude coordinates, constituting lines or curves representing underground facilities detected and/or marked during the locate and/or marking operation, and/or one or more latitude and longitude coordinates constituting points or polygons representing environmental landmarks, may be compared to a corresponding set of reference latitude and longitude coordinates to determine a degree of matching between the two sets, in a manner akin to pattern matching. This may be useful in determining how closely the locate marks formed by the technician correspond to the presumed physical location(s) of the underground facilities according to one or more facilities maps.
[00126] Although comparisons of field geo-location data and reference geo-location data to facilitate an automated quality assessment process are described in some

-43-exemplary embodiments discussed in greater detail below, it should be appreciated that more generally, in other embodiments, a variety of other information contained in field information/data and reference information/data may be used as a basis for an automated quality assessment. For example, field information pertaining to the number and/or types of facilities detected and/or marked during a locate and marking information may be compared to similar reference information derived from one or more facilities maps, without regard to geographic information (e.g., by noting from the facilities map(s) what types of facilities are present in the map(s) corresponding to a given work site/dig area, how many facilities lines of a particular type are present in the map(s), and/or number of facility lines that are abandoned or no longer active). In another example, field information pertaining to an arrangement or pattern (i.e., relative positions) of multiple lines for a same type of facility and/or multiple different facility types detected and/or marked during a locate and marking information may be compared to similar reference information ("relative position information") derived from one or more facilities maps, irrespective of the presumed physical geographic location(s) of the respective facilities/lines (i.e., the general pattern of lines detected and/or marked in the field may be compared to the general pattern of corresponding lines as illustrated in one or more facilities maps). The foregoing and other examples of assessments based on different types of information derived from facilities maps is discussed in further detail below in connection with various embodiments.
[00127] Figure 7 is a flow chart of an exemplary process 600 for performing the act 1312 of the process 1300 shown in Figure 6, according to one embodiment, and illustrates one technique that may be used to perform an assessment based on a comparison of field information (more specifically marking information) obtained from a marking device with reference information obtained from one or more facilities maps. It should be appreciated that the concepts discussed below in connection with a comparison of marking information from a marking device to reference information derived from facilities maps may be applied similarly and more generally to locate information, landmark information, information derived from an EM image provided by an EM application, and other types of field information relating to a locate and/or marking operation, any of which may be compared to a variety of reference information derived from facilities maps.
In particular, in the example discussed below in connection with Figures 7 and 8 and Tables 7-10, it

-44-should be appreciated that the geographic information forming part of the field data may be obtained similarly in other implementations from locate information, landmark information, an EM image, and other sources of geographic information relating to the locate and/or marking operation.
[00128] The process 600 begins at act 601, a particular facility type that has been marked (and/or detected) during a locate and/or marking operation may be selected for comparison to one or more facilities maps. For example, if it is determined that a locate technician has marked electric lines, gas lines, and sewer lines during the locate and/or marking operation, one of these facility types may be selected initially for comparison to reference data derived from one or more facilities maps. It should be understood that the process 600 may be repeated to perform one or more additional comparisons for one or more of the other facility types; for example, if electric lines are the facility type that is selected at act 601, process 600 may be performed again (e.g., subsequently or in parallel) to select one of the other facility types that has been marked.
[00129] In one aspect of this embodiment, as part of act 601, all of the different facility types marked and represented in the available electronic record of the marking operation are first determined from data in the electronic record indicating the color(s) of marking material used during the locate and/or marking operation. As explained above in connection with Table 3, each marking material color corresponds to a particular facility type. Thus, if actuation data in an electronic record of a marking operation indicates that red, yellow, and green marking material were each used during the marking operation, then it may be determined that electric lines, gas lines, and sewer lines were marked.
[00130] Once it is determined what types of a facilities were marked during the marking operation, a particular type of facility marked may be selected as the subject for assessment by comparison to one or more facilities maps in any of a variety of ways. For example, in some embodiments, a type of facility for which such a comparison has not yet been performed may be selected. In situations where there are multiple facility types for which the comparison has not yet been performed, the facility type may be selected via user input pursuant to a selection query, may be selected based on a risk assessment associated with each facility type, or alternatively may be selected randomly or based on some other factor (e.g., the time at which the facility was marked relative to the other facilities, the number or length of locate marks used to mark the facility).

-45-[00131] Once a particular facility type is selected at act 601, the process continues to act 603, where particular marking information regarding the locate marks for the selected facility type is extracted as necessary from the available electronic record to provide the field data for comparison to reference data from one or more facilities maps.
For example, if the selected facility type is electric lines, then the actuation data sets of an electronic record generated by the marking device (e.g., of the type shown in Table 2 above) indicating that red marking material was dispensed during the locate and/or marking operation may be used to provide the field data.
[00132] More specifically, as a result of act 603, a set of field geo-location data points (e.g., coordinate pairs of GPS latitude and longitude values) are identified that are indicative of geographical locations at which marking material of the color of the selected facility type was dispensed. For example, as shown in Table 2, an actuation data set for a marking device actuation includes a set of geo-location data points at which the marking device was actuated (and dispensed marking material). In the example of Table 2, the geo-location data set for act-1 includes the data points 2650.9348N,08003.5057W, 2650.9353N,08003.5055W, and 2650.9356N,08003.5055W. At act 603, any one or more of these points (along with the other geo-location data points at which the marking device was actuated using the same color) may be selected to be included in the field data. In one example, all geo-location data points from the marking device actuation data sets for the selected facility type (i.e., all geo-location data from all actuation data sets indicating the marker color corresponding to the selected facility type) are selected to be in the set of field geo-location data points that is compared to reference geo-location data from one or more facilities maps. However, the invention is not limited in this respect, as in some embodiments, only a subset of these geo-location data points may be selected.
[00133] For example, in some embodiments, geo-location data points at which marking material was dispensed as part of a dotting pattern may be disregarded and not included in the field data, while geo-location data points dispensed as part of a lines pattern may be included in the field data. As known in the relevant art, there are multiple types of marking patterns that may be used in a marking operation. For example, a "dotting"
pattern may be utilized to quickly mark the location of the target facility.
Often the dotting pattern is used in conjunction with the initial locate operation to detect the target facility. A "lines" pattern is typically the end product of the marking operation. This

-46-pattern extends the dotting pattern in order to create locate marks in the form of dashed lines that indicate the presence or absence of an underground facility.
[00134] Because the locate marks for the dotting pattern are initial marks that may be "revised" by the marks from the lines pattern, in some embodiments it may be desirable to exclude the geo-location points of marks from the dotting pattern from the field data.
Geo-location points of marks from a dotting pattern may be identified and distinguished from geo-location points of marks from a lines pattern in a variety of ways.
An example of one such way is described below, though a variety other techniques could be used.
[00135] In some embodiments, geo-location points from a marking device actuation data set corresponding to a dotting pattern may be distinguished from geo-location points corresponding to a lines pattern based on information in the actuation data set for the marking device actuation that identifies the duration of the actuation (see Table 2). If the duration of the marking device actuation is less than a threshold amount of time, the actuation may be considered to correspond to a dotting pattern, and the geo-location points from this actuation data set may be excluded from the field data. If, on the other hand, the duration is equal to or greater than the threshold, the actuation may be considered to correspond to a lines pattern, and the geo-location points from this actuation data set may be included in the set of field geo-location data points to be compared to reference information from a facilities map.
[00136] After act 603, process 600 continues to act 605, where one or more facilities maps corresponding to the selected type of facility are accessed and the geo-locations of the facility line(s) to which the locate marks are to be compared are obtained from the map(s). In some embodiments, a database may be maintained that stores, in electronic form, facilities maps (or "plats") provided by facility owners. In various aspects, the database may index the maps by facility owner (or facility type) and/or geographical region, to facilitate access to and selection of the facilities map of the selected facility type for the proper geographic region. As such, as a result of act 605, a set of "reference geo-location data points" indicating the presumed physical location of a facility line of the type of the selected facility may be obtained from the selected facilities map(s), for comparison to the set of field geo-location data points obtained in act 603.

-47-[00137] More specifically, once an appropriate facilities map is identified, accessed and selected, the facility line or lines indicated in the facilities map, for which reference geo-location data points are used as a basis for comparison, may be selected in a variety of ways, as the invention is not limited to selecting these facility line or lines in any particular way. For example, in some embodiments, the ticket for the locate and/or marking operation may include a geo-location for the work site (or a street address for the work site that may be geo-coded) at which the locate and/or marking operation was performed. A
facilities map for the selected facility type that includes this geo-location may be selected, and the facility line of the selected facility type that is closest to this geo-location may be the facility line that is selected from this facilities map so as to provide the set of reference geo-location data points.
[00138] In other exemplary implementations, the point that is the centroid of the set of field geo-location data points obtained in act 603 may be determined, and a facilities map for the selected facility type that includes this geo-location may be selected. The facility line of the selected facility type that is closest to this geo-location may be the facility line that is selected from this facilities map so as to provide a basis for comparison. In yet other implementations, when a VWL image provided by an excavator is available, the geo-locations for the boundary of the dig area in which work is planned (e.g., latitude and longitude coordinates for the dig area indicator in the VWL image that denotes/delimits the dig area) may be determined from the VWL image. The facilities map for the selected facility type that includes these "VWL geo-locations" may be selected, and the facility lines of the selected facility type that are within the area delimited by the VWL geo-locations may be selected for comparison.
[00139] Once the facility line or lines for comparison have been selected from the appropriate plat or plats, a set of reference geo-location data points for the facility line(s) may be obtained in any of a variety of ways. For example, in some embodiments, as discussed above, an electronic facilities map may include metadata that indicates the geo-locations of at least the termination (start and end) points of the selected facility line(s).
From this termination point geo-location data, a set of geo-location data points of the presumed physical location of the facility lines(s) may be derived. In various examples, this set of reference geo-location data points may have virtually any number of points between the termination points, depending in part on the granularity/resolution desired for

-48-the set of reference geo-location data points. In some cases, a given plat or plat provides "interim" geo-location data points for a given facility line, between the two termination points, and these interim geo-location data points (as provided by the metadata accompanying a plat), may be used directly for the set of reference geo-location data points. In other instances, a set of reference geo-location data points may be constructed merely from the termination points as indicated in the plat metadata, and any number of interim points may be included in this set (again, in some cases based on a desired granularity/resolution).
[00140] After act 605, the process next continues to act 607, where the set of field geo-location data points obtained at act 603 and the set of reference geo-location data points obtained from the facilities map at act 605 are converted to a common reference frame (e.g., a common coordinate system).
[00141] As known in the relevant art, a geographic or "global" coordinate system (i.e., a coordinate system in which geographic locations on Earth are identified by a latitude and a longitude value, e.g., (LAT,LON)) may be used to identify geographic locations of locate marks and a facility line. In a "geocentric" global coordinate system (i.e., a coordinate system in which the Earth is modeled as a sphere), latitude is defined as the angle from a point on the surface of a sphere to the equatorial plane of the sphere, whereas longitude is defined as the angle east or west of a reference meridian between two geographical poles of the sphere to another meridian that passes through an arbitrary point on the surface of the sphere. Thus, in a geocentric coordinate system, the center of the Earth serves as a reference point that is the origin of the coordinate system.
However, in actuality the Earth is not perfectly spherical, as it is compressed towards the center at the poles. Consequently, using a geocentric coordinate system can result in inaccuracies.
[00142] In view of the foregoing, the Earth is typically modeled as an ellipsoid for purposes of establishing a global coordinate system. The shape of the ellipsoid that is used to model the Earth and the way that the ellipsoid is fitted to the geoid of the Earth is called a "geodetic datum." In a "geodetic" global coordinate system, the latitude of a point on the surface of the ellipsoid is defined as the angle from the equatorial plane to a line normal to the reference ellipsoid passing through the point, whereas the longitude of a point is defined as the angle between a reference plane perpendicular to the equatorial

-49-plane and a plane perpendicular to the equatorial plane that passes through the point. Thus, geodetic latitude and longitude of a particular point depends on the geodetic datum used.
[00143] A number of different geodetic global coordinate systems exist that use different geodetic datums, examples of which include WGS84, NAD83, NAD27, OSGB36, and ED50. As such, a geographic point on the surface of Earth may have a different latitude and longitude values in different coordinate systems. For example, a stop sign at the corner Maple St. and Main St. may have a latitude and longitude of (LATI, LONI) in the WGS84 coordinate system, but may have a latitude and longitude of (LAT2, LON2) in the NAD83 coordinate system (where LATI LAT2 and/or LONI WM).
Thus, when comparing one geographic point to another geographic point to determine the distance between them, it is desirable to have both geographic points in the same global coordinate system.
[00144] Additionally, when determining a geographic location based on information derived from a map (e.g., a facilities map), the coordinate system provided by the map may not be a global coordinate system, but rather may be a "projected"
coordinate system.
As appreciated in the relevant art, representing the curved surface of the Earth on a flat surface or plane is known as a "map projection." Representing a curved surface in two dimensions causes distortion in shape, area, distance, and/or direction.
Different map projections cause different types of distortions. For example, a projection could maintain the area of a feature but alter its shape. A map projection defines a relation between spherical coordinates on the globe (i.e., longitude and latitude in a global coordinate system) and flat planar x,y coordinates (i.e., a horizontal and vertical distance from a point of origin) in a projected coordinate system. A facilities map may provide geographic location information in one of several possible projected coordinate systems.
[00145] Thus, to compare a first geographic location derived from a facilities map and expressed in x,y coordinates to a second geographic location expressed as LAT,LON
coordinates in some geodetic global coordinate system (e.g., as obtained from a location tracking system), it is desirable to have respective coordinate pairs for the first location and the second location either in the same geodetic global coordinate system or projected coordinate system (projected from the same geodetic geographical coordinate system). As such, at act 607 of the process 600, if the set of reference geo-location data points from the selected facilities map and the set of field geo-location data points obtained from the

-50-marking information are in different coordinate systems, one or both of these sets of geo-location data points may be converted (e.g., transformed) so that they are in a common frame of reference (e.g., the same global coordinate system or the same projected coordinate system projected from the same global coordinate system).
[00146] For example, in some embodiments, the geo-location points obtained from the marking information may be geo-locations in the WGS84 coordinate system (i.e., the coordinate system typically used by GPS equipment), whereas the geo-location points obtained from the facilities map may be in the NAD83 coordinate system. Thus, at act 607, the coordinates from the marking information and the coordinates from the facilities map may be put in a common frame of reference. For example, the coordinates from the facilities map may be converted to the WGS84 coordinate system, the coordinates from the marking information may be converted to the NAD83 coordinate system, or the coordinates from the facilities map and the coordinates from the marking information may both be converted to a third common coordinate system.
[00147] Process 600 then continues to act 609, where the set of field geo-location data points are compared to the set of reference geo-location data points in the common reference frame to assess the quality of the locate and/or marking operation.
A variety of different techniques may be used to compare the two sets of geo-location data points, and examples of several possible techniques are provided below. However, the invention is not limited to using the particular technique described in connection with Figure 8, as various embodiments may use different techniques.
[00148] In some embodiments, the shape of the facility line as indicated on the facilities map may be compared to the shape of the line as indicated by the locate marks dispensed during the locate and/or marking operation, irrespective of their relative geo-locations, such that quality is assessed based on how close these shapes are to each other.
[00149] In other embodiments, the set of field geo-location data points may be compared to the set of reference geo-location data points to determine geographic distance between them. Figure 8 shows an illustrative process 700 for determining distance between points in two sets, X and Y, of geo-location data points. Each of these sets may include a plurality of geo-location data points (e.g., latitude and longitude values or x,y coordinate pairs), and the geo-location data points in set X typically are in the same

-51-reference frame (e.g., coordinate system) as the geo-location data points in set Y. In embodiments in which the process 700 is used to compare a set of field geo-location data points to a set of reference geo-location data points obtained from one or more facilities map(s), set X may include the field points and set Y may include the reference points.
Process 700 defines the distance between the two sets X and Y as a vector of distances do...dn, where each distance d, indicates the distance between a point xi in set X and the point ye in set Y that is closest to xi.
[00150] Referring to Figure 8, the illustrative process 700 begins at act 10 by initializing a variable n to zero. The process continues to act 20, where a point xn in the set X is selected, where xn is the Ilth point in the set X. The process next continues to act 30, where the point in set Y that is closest to the point xr, is identified and is set as the variable Ye. That is, among all the points in the set Y, the selected point ye is the one closest to point xn. The process then continues to act 40, where the distance between xn and ye is recorded and stored in the variable dn. The process next continues to act 50, where it is determined whether there are any more points in the set X to process. When it is determined that the set X contains one or more points yet to be processed, the process 700 continues to act 60, where the value of n is incremented by one. The process then returns act 20, where the next point in the set X is selected. If, at act 50, it is determined that there are no more points in set X to process, the process 700 ends.
[00151] It should be appreciated that each of the sets X and Y may include any number of geo-location data points, as the present disclosure is not limited in this respect. For example, in some embodiments, one or both of the sets may have only one geo-location data point specifying a single point on Earth. In other embodiments, one or both sets may have multiple geo-location data points specifying multiple points on Earth.
[00152] Additionally, the process 700 may be applied to determine a measure of distance between any two sets of points in any space in which a measure of distance can be defined between two points. Thus, the application of the process 700 is not limited to geo-location data points expressed in an absolute frame of reference that ties the geo-location data to specific points on Earth. For example, in some embodiments, the geo-location data points in set X and set Y may not be expressed in latitude and longitude, but rather may be expressed as locations (e.g., distance and direction) relative to some other reference point (e.g., an arbitrary reference point, a reference point defined by one or more

-52-facilities maps, a reference point defined by some environmental landmark, or some other reference point).
[001531 The process 700 is also not limited to any particular technique for determining the distance between two points, as any of numerous techniques may be used.
For example, in an embodiment where the geo-location data points are expressed in latitudinal and longitudinal coordinates, a distance between two points may be calculated according to the great-circle distance in spherical geometry, using Vincenty's inverse method for computing geographical distance between two points, or using some other method. In some embodiments in which the coordinates for the two points are each two-dimensional Cartesian coordinates in a common grid system, the straight line distance between these two points may be determined using the following formula: d = sqrt((x2- x1)2 +
(y2- yi)2).
Referring back to Figure 7, after the comparison in act 609, the process continues to act 611, where a quality assessment of the locate and/or marking operation that is based, at least in part, on the comparison in act 609 is generated. The quality assessment may be generated in any of a variety of ways, including, for example, in the manner discussed above in connection with act 1312 of Figure 6. Several examples of techniques for generating a quality assessment are described below. However, the invention is not limited to any particular technique for generating a quality assessment.
[001541 In embodiments in which the process illustrated in Figure 8 is used to perform the comparison in act 609, in some exemplary implementations the quality assessment may be based on the percentage of locate marks that are within a threshold distance of any point on the corresponding facility line (as indicated on the facilities map(s)) used as a basis for comparison. That is, as discussed above, the process of Figure 8 generates a vector of distances do...dn, where each distance d indicates the distance between one of the points at which a locate mark was dispensed (x0...x) and the closest point ye on the corresponding facility line. Thus, in some embodiments, the quality assessment may be based on the percentage of these distances that are within some predetermined range or threshold.
1001551 Table 7 below shows one possible technique for generating a quality assessment of a locate and/or marking operation in this way using a scoring table.
Techniques for generating a scoring table and computing a score using a scoring table are

-53-described in greater detail in U.S. Non-provisional Patent Application Serial No.
12/493,109, filed June 26, 2009, entitled "Methods and Apparatus for Quality Assessment of a Field Service Operation". As shown in Table 7, the criterion on which the quality of locate and/or marking operation is being assessed is listed in the leftmost column. For this criterion, the table includes one or more expected or reference values or ranges for the criterion, also referred to as "metrics," against which information about the locate and/or marking operation is measured/compared. The metrics are divided into several "scoring categories," namely, value(s)/condition(s) that, if met, result in a particular score.
[001561 For purposes of the analysis illustrated in Table 7, field information/data is referred to as ACTUAL DATA, and reference information/data is referred to as EXPECT
DATA. A quality assessment for the indicated criterion is based on a comparison of the ACTUAL DATA to the EXPECT DATA (e.g., so as to determine in what scoring category the ACTUAL DATA falls as a result of the comparison). For purposes of the discussion that follows, although examples based on numeric scores are provided, the term "score" as used herein is intended to more generally denote any of a variety of graduated indicators for a quality assessment (which in turn may be based on a variety of ranges, scales and resolutions/granularity for the indicators).
Table 7 Expected value or range (metrics) Criterion Preferred Marginal Unacceptable Less than 95%
EXP: Percentage of 95% or of points are points within threshold more of 50% or more of within 1 foot, distance of any facility points are but 50% or more points are line point (as indicated within 1 outside 2 feet.
of points are on facilities maps) foot.
within 2 feet.
ACT: Percentage of 98% of points within threshold points are distance of any facility within 1 line point (as indicated foot.
on facilities maps) 1001571 In the example of Table 7, the criterion on which the quality of the locate operation is being assessed is the percentage of points at which locate marks were dispensed that are within some threshold distance of the closest corresponding point on the facility line, as indicated by the facilities map(s). Additionally, in this example, there are

-54-three scoring categories: Preferred; Marginal; and Unacceptable. For each scoring category, there is a metric used to evaluate the contents of the distance vector [do...dn]
resulting from the comparison of the field data and the reference data to determine in which scoring category the results of the comparison fall. In the example of Table 7, an evaluation of the distance vector shows that 98% of the points at which locate marks were dispensed were within one foot of the corresponding closest point on the facility line being marked (where the location of the facility line is determined from the facilities map(s)).
As such, the locate and/or marking operation falls into the preferred category.
[00158] With reference again to act 1905 of Figure 3 (or, similarly, act 1314 of Figure 6), in some embodiments a score, grade, or categorization may be assigned as an output to categorize the quality assessment process based on into which scoring category the assessment falls. For example, in some embodiments, each scoring category may be associated with a number of points (e.g., 2 points for Preferred, 1 point for Marginal, and 0 points for Unacceptable), and the quality assessment may be awarded the number of points associated with the scoring category into which it falls. Thus, for example, in the example of Table 7, 2 points may be awarded, because the operation falls in the "Preferred" scoring category.
[00159] In some embodiments, the number of points awarded may be converted to a percent score that is based on the number of points awarded and a maximum possible number of points. Thus, for example, in the example of Table 7, the locate and/or marking operation received two points out of a maximum possible two points. As such, the locate and/or marking operation may be assigned a score of 2/2 or 100%. If the assessment results were to fall in the "Marginal" category and receive only one point, then it may be assigned a score of 1/2 or 50%. Similarly, if the assessment results were to fall in the unacceptable category and receive zero points, then it may be assigned a score of 0/2 or 0%.
[00160] In some embodiments, a range of percent scores may be converted to letter scores to provide an indication of quality. For example, a percent score of 100-90% may be converted to a letter score of A, 89-80% may be converted to a letter score of B, 79-70% may be converted to a letter score of C, 69-60% may be converted to a letter score of D, and <60% may be converted to a letter score of F. In yet another example, a range of percent scores may be converted to a simple PASS/FAIL score. For example, a percent

-55-score of 100-60% may be converted to a score of PASS and a percent score of <60% may be converted to a score of FAIL.
[00161] In some embodiments, the quality assessment illustrated in Table 7 may be used in the process of Figure 6 to categorize the locate and/or marking operation as either "Approved" "Coach" or "QC Referral. For example, Table 7 may be used at act 1312 to assess the quality of the locate and/or marking operation. Based on this assessment, the quality of the operation may be categorized at act 1314. For example, if the operation falls in the "Preferred" scoring category in Table 7 it may be categorized as "Approved" at act 1314; if the operation falls in the "Marginal" scoring category, it may be categorized as "Coach;" and if the operation falls in the "Unacceptable" scoring category it may be categorized as "QC Referral."
[00162] In the example of Table 7, three scoring categories are used, such that the locate and/or marking operation is classified as either Preferred, Marginal, and Unacceptable. However, the number of scoring categories is merely illustrative, as any number of scoring categories could be used, and various mutually exclusive metrics may be assigned to these scoring categories. For example, in some embodiments, five scoring may be used (e.g., Excellent, Good, Average, Poor, Unacceptable), while in other embodiments more than five scoring categories may be used.
[00163] In addition, it should be appreciated that the percentage values and distance threshold values used in the metrics in Table 7 (and in Tables 8-10 described below) are merely illustrative and that a variety of different percentage values and distance threshold values may be used. In some embodiments, the distance threshold values may be based on legal requirements pertaining to locate and/or marking operations. For example, some governments (e.g., state governments) may dictate that a locate mark placed on the ground is within a certain "tolerance zone" around the underground facility (e.g., 12 inches, 18 inches, 24 inches, 30 inches, 36 inches, etc.). Thus, in some embodiments, one or more of the metrics used in a scoring table may be based on a tolerance zone dictated by government regulations.
[00164] In the example provided by Table 7, a single criterion is provided for all of the facility lines marked. However, in some embodiments, a separate criterion may be used for each facility line marked. For example, as shown in Table 8 below, if during a locate

-56-and/or marking operation, a gas line, a power line, and a water line were marked, then a separate criterion may be provided for each of these facility lines. This enables the accuracy of each facility line that was marked during the locate and/or marking operation to be assessed independent of the other facility lines.
Table 8 Expected value or range (metrics) Criterion Preferred Marginal Unacceptable EXP: Type=Gas; Less than 95%
95% or Percentage of points of points are more of 50% or more of within threshold within 1 points are foot, distance of any facility but 50% or more points are within 1 outside 2 feet.
line point (as indicated of points are foot.
on facilities maps) within 2 feet.
ACT: Type=Gas;
Percentage of points 98% of within threshold points are distance of any facility within 1 line point (as indicated foot.
on facilities maps) EXP: Type=Electric; Less than 95%
95% or Percentage of points of points are more of 50% or more of within threshold within 1 points are are p foot, distance of any facility but 50% or more within 1 outside 2 feet.
line point (as indicated foot. of points are on facilities maps) within 2 feet.
ACT: Type=Electric;
10% of points Percentage of points are outside of 1 within threshold foot, but 75% of distance of any facility points are within line point (as indicated 2 feet.
on facilities maps) EXP: Type=Water; Less than 95%
95% or Percentage of points of points are more of 50% or more of within threshold within 1 points are foot, distance of any facility but 50% or more points are within 1 outside 2 feet.
line point (as indicated foot. of points are on facilities maps) within 2 feet.
ACT: Type=Water;
Percentage of points 80% of points within threshold are outside 1 distance of any facility foot.
line point (as indicated on facilities maps) [00165] As discussed above, in some embodiments, each scoring category may be associated with a number of points (e.g., 2 points for Preferred, 1 point for Marginal, and 0

-57-points for Unacceptable), and the quality assessment may be awarded the number of points associated with the scoring category into which it falls. Thus, for example, in the example of Table 8,2 points may be awarded for marking of the gas line(s), 1 point may be awarded for the marking of the electric line(s), and 0 points may be awarded for the marking of the water line(s). Thus, the operation may receive a score of three points out of a maximum possible total of six points, for a score of 3/6 or 50%.
[00166] As discussed above, the inventors have appreciated that, while a facilities map may provide some information about the geo-location of one or more facility line(s), this geo-location may not be perfectly accurate and, with some facilities maps, may be quite inaccurate. Thus, in some embodiments, the quality assessment (e.g., the score) that is generated by the process of Figure 7 may be weighted based on the perceived reliability of a facilities map. The perceived reliability of a facilities map may be determined in a variety of ways, and the invention is not limited to any particular way of determining the perceived reliability of a facilities map. For example, in some embodiments, factors that may be considered as indicators of reliability may be the age of the map, the revision number of the map, the facility company that generated the map, whether the map indicates that one or more facility lines are present in a joint trench (i.e., a trench in which multiple lines for different facilities are placed), and whether the map indicates that one or more facility lines are present in a conduit run (i.e., a pipe through which multiple facility lines are run).
[00167] Table 9 shows an example of a scoring table in which each criterion has been assigned a weight based on the perceived reliability of the locate and/or marking operation.
In this example, the map for the water facility is perceived to be inaccurate (e.g., because it is old, because the facilities maps from this particular water company are known, based on past experience, to be inaccurate, or based on some other factor(s)), while the map for the gas facility and the map for the electric facility are believed to be highly accurate (e.g., because these maps are relatively new, because the gas and/or electric companies that provided the maps are known, based on past experience, to provide accurate maps, because the maps indicate that facility lines are present in a joint trench or in a conduit run, or based on some other factor(s)). Thus, the Gas criterion and the Electric criterion are each assigned a weight of 2, while the Water criterion is assigned a weight of 0.5.

-58-Table 9 Expected value or range Criterion Preferred Marginal Unacceptable Weight Weighted (score=2) (score=1) (scorel) Factor Score EXP:
Type=Gas;
Less than 95%
Percentage of o points within f points are 95% or more within 1 foot, 50% or more of threshold of points are but 50% or points are distance of any within 1 foot. more of points outside 2 feet.
facility line are within 2 point (as feet.
indicated on facilities maps) ACT:
Type=Gas;
Percentage of points within threshold 98% of points are within 5 x2 4 distance of any feet.
facility line point (as indicated on facilities maps) EXP:
Type=Electric;
Percentage of points within threshold distance of any facility line point (as indicated on facilities maps) ACT:
Type=Electric;
Percentage of 10% of points points within are outside of 1 threshold foot, but 75% of x2 2 distance of any points are facility line within 2 feet.
point (as indicated on facilities maps) EXP:
Type=Water;
Percentage of points within threshold distance of any facility line

-59-Expected value or range Criterion Preferred Marginal Unacceptable Weight Weighted (score=2) (score=1) (score=0) Factor Score point (as indicated on facilities maps) ACT:
Type=Water;
Percentage of points within 80% of points threshold are outside 1 x0.5 0 distance of any foot.
facility line point (as indicated on facilities maps) [00168] In the example of Table 9, the locate and/or marking operation receives six points (i.e., 4+2+0) out of a total possible 9 points. Thus, the locate and/or marking operation receives a score of 6/9 points or 66.7%.
[00169] In the examples provided by Table 7, Table 8, and Table 9, for generating a quality assessment of the locate and/or marking operation, the sole criterion for assessing quality is based on a comparison of the location of locate marks dispensed during a locate operation and the location of a facility line as indicated by one or more facilities map(s).
However, the invention is not limited in this respect, as in some embodiments, this criterion may be one of a number of criteria that is used at act 611 (or act 1312 of Figure 6) to generate a quality assessment. There are variety of techniques by which this criterion may be used in combination with other criteria to generate a quality assessment, one example of which is provided below. However, the invention is not limited to using the particular technique described below or any other particular technique.
[00170] In some embodiments, a scoring table, similar to Table 7 may be used to assess the quality of a locate and/or marking operation based on a plurality of different criteria.
An example of such a scoring table is shown below in Table 10. Table 10 is similar to Table 7, except that instead of a single criterion in the left-most column, there are multiple criteria. In addition, in Table 7, each criterion may be assigned a weight factor, such that some criteria (e.g., criteria that are deemed more important) may optionally be given greater weight than others in the quality assessment. As with Table 7, for each criterion in Table 10 actual data (field data) obtained from the locate and/or marking operation being

-60-evaluated may be compared with expected data (reference data) values or ranges for that criterion, and a number of points may be awarded based on the scoring category into which the locate and/or marking operation falls for that criterion and a weight factor assigned to that scoring category. For example, if the weight factor for a particular criterion is 5 and the locate and/or marking operation falls into the "Preferred" category for that criterion, then 10 points (i.e. 2x5) would be awarded for that criterion based on the example given above in connection with Table 7.
Table 10 Expected value or range Criterion Preferred Marginal Unacceptable Weight Weighted (score=2) (score=1) (score=0) Factor Score EXP:
Percentage of points Less than 95%
within of points are threshold 95% or more of within 1 foot, 50% or more of distance of points are within but 50% or points are any facility 1 foot. more of points outside 2 feet.
line point are within 2 (as indicated feet.
on facilities maps) ACT:
Percentage of points within threshold 98% of points distance ofxl 2 are within 1 foot.
any facility line point (as indicated on facilities maps) EXP: Dig N35 43.57518, N35 43.57518, N35 43.57518, W078 49.78314 area geo- W078 49.78314 W078 49.78314 ¨
>0.2 to <0.5 location <0.2 miles >0.5 miles miles N35 43.57518, ACT: Geo- W078 49.78314 P/F
location data 0.04 mi EXP: Locate Before 05FEB09 On 05FEB09 After 05FEB09 ¨
Date ACT:
04FEB09;
Timestamp x2 4 09:35:15.2 data

-61-Expected value or range Criterion Preferred Marginal Unacceptable Weight Weighted (score=2) (score=1) (score=0) Factor Score EXP:
40 mins 10-20 40 mins >20 Elapsed 40 mins ¨ ¨
mins mins time ACT:
Timestamp 54 mins x2 2 data EXP:
RED color data RED color data Type=Electr n/a ¨ ¨
present absent ic power ACT: Color data RED present x5 10 EXP: Geo-N35 43.57518' N35 43 .57518, location N35 43.57518' W078 49.78314 W078 49.78314 W078 49.78314 ¨
>0.1 to < 0.2 Oto 0.1mi >0.2 mi ACT: Geo- N35 43.57518, location data W078 49.78314 x2 4 0.04 mi EXP: Gain 0-45 >45-70 >70-100 .
ACT: Gain 35 xl 2 .
EXP: Sig.
100-85% <85-65% <65% ¨ ¨
strength ACT: Signal 83% xl 1 data _ EXP:
YELLOW color YELLOW color Type=Gas, n/a ¨ ¨
data absent data present oil ACT: Color YELLOW x5 10 data absent EXP:
ORANGE color ORANGE color Type=Com, n/a ¨ ¨
data absent data present CATV
ACT: Color ORANGE x5 10 data absent EXP: BLUE color data n/a BLUE color Type=Water present data absent ¨
ACT: Color BLUE present x5 10 data _ EXP: Geo-N35 43.57518 N35 43.57518, location W078 49.7831,4 NW3057 84 34.957.75813814 W078 49.78314 ¨ ¨
>0.1 to < 0.2 Oto 0.1mi >0.2 mi nil ACT: Geo- N35 43.57518, location data W078 49.78314 x2 2 0.14 mi EXP: Gain 0-45 >45-70 >70-100

-62-Expected value or range Criterion Preferred Marginal Unacceptable Weight Weighted (score=2) (score=1) (score=0) Factor Score ACT: Gain 35 xl 2 EXP: Sig.
100-85% <85-65% <65%
strength ACT: Signal 87%
xl 2 data EXP: GREEN color n/a GREEN color Type=Sewer data absent data present ACT: Color GREEN absent x5 10 data EXP:
PURPLE color PURPLE color Type=Irrigat data absent n/a data present ion ACT: Color PURPLE absent x5 10 data EXP: Locate technician Lookup table n/a Not found ID
ACT:
Locate technician ID
EXP:
Marking Lookup table n/a Not found Device ID
ACT:
Marking 7362 P/F
Device ID
EXP: Locate Lookup table n/a Not found Device TE, ACT:
Locate 7345 P/F
Device ID
EXP: Temp 50-80 20-<50, >80-<20, >110 ( F) 110 ACT: Temp 73F
x2 4 data =
EXP:
0-40% >40-90% >90-100%
Humidity ACT:
Humidity 52% x2 2 data EXP: Light 4.0 to 5.0 volts 2.0 to <4.0 volts <2.0 volts ACT: Light 4.3 volts x2 4 data EXP: <-30 to -60 <-60 to -90 Inclinometer -30 to 30 degrees degrees or >30 degrees or >60 ¨
to 60 degrees to 90 degrees ACT: -17 xl 2

-63-Expected value or range Criterion Preferred Marginal Unacceptable Weight Weighted (score=2) (score=1) (score=0) Factor Score Inclinometer data EXP:
Acceleromet 0.2g to 1.0g >1.0g to 1.5g >1.5g er data ACT: 0.375g xl 2 EXP:
Battery 100-85% <85-50% <50%
strength data ACT:
Battery 93% x5 10 strength data Total points earned out of a possible 112 = 105 Percent Score = 93.8%
[00171] Once the number of points awarded for each criterion has been determined, a total number of points may be computed by summing together the points awarded for each of the criteria together. The maximum number of points possible for the locate and/or marking operation may be determined by first determining the sum of all weight factors and then multiplying this sum by the point value of the "Preferred" result. A
percentage score may be determined by dividing the number of points awarded by the maximum number of points possible and multiplying the result by 100.
[00172] For example and referring to Table 10, the sum of the weight factors is 56 and the point value of the "Preferred" result is 2. Therefore, in this example the maximum number of points possible for the locate operation is 56 x 2, which is 112.
The sum of the points earned for the current locate operation, in the example of Table 10, is 105. Thus, the percent score for the current locate operation, which in this example is 105/112 x 100 = 93.8%.
[001731 As discussed above, a range of percent scores may be converted to letter scores, so that a letter score letter score indicative of the quality of the locate and/or marking operation may be assigned. For example, a percent score of 100-90% may be converted to a letter score of A, 89-80% may be converted to a letter score of B, 79-70%
may be converted to a letter score of C, 69-60% may be converted to a letter score of D, and <60%
may be converted to a letter score of F. In yet another example, a range of percent scores may be converted to a simple PASS/FAIL score. For example, a percent score of

-64-60% may be converted to a score of PASS and a percent score of <60% may be converted to a score of FAIL.
[00174] In other embodiments, the numerical quality assessment score may be used to automatically categorize a locate operation as either APPROVED, COACH, or QC
Referral. In one example, using the numeric scoring system of 0 to 100%, a score of 60%
or below may automatically render an assessment of QC Referral, in which case, after act 1314 of Figure 6, the process continues to act 1322. A score of >60% to 80%
may automatically render an assessment of COACH, in which case the process continues to act 1318, and a score of >80% to 100% may automatically render an assessment of APPROVED, and the process may continue to act 1316.
[00175] The example of Table 10 shows a scoring table have a number of various different criteria. The number of criteria and the particular criteria used are merely illustrative, as any number or type of criteria may be used.
[00176] Thus, the constructs provided by Tables 7, 8, 9, and 10 above illustrate various concepts germane to assessing the quality of locate and/or marking operations based at least on reference information derived from one or more facilities maps, which reference information may be used alone or in combination with other information that may provide for a variety of criteria by which such operations may be assessed. As noted above, while various examples were provided above based on a comparison of geographic information contained in field data with geographic information contained in reference data, it should be appreciated that the invention is not limited to comparisons of only geographic information. For example, field information pertaining to the number and/or types of facilities detected and/or marked during a locate and marking information may be compared to similar reference information derived from one or more facilities maps, without regard to geographic information (e.g., by noting from the facilities map(s) what types of facilities are present in the map(s) corresponding to a given work site/dig area, and/or how many facilities lines of a particular type are present in the map(s)). In another example, field information pertaining to an arrangement or pattern (i.e., relative positions) of multiple lines for a same type of facility and/or multiple different facility types detected and/or marked during a locate and marking information may be compared to similar reference information derived from one or more facilities maps, irrespective of the presumed physical geographic location(s) of the respective facilities/lines (i.e., the general

-65-pattern of lines detected and/or marked in the field may be compared to the general pattern of corresponding lines as illustrated in one or more facilities maps). The various concepts discussed herein may be applied similarly to the foregoing various types of information to facility an automated quality assessment process based on one or more facilities maps.
[00177] In the illustrative process of Figure 7, a separate facilities map is selected for each facility type (or facility company). However, in some embodiments, rather than using a separate facilities map for each facility type or facility company, an aggregated facilities map may be generated by combining data from multiple facilities maps, and the aggregated facilities map may be selected at act 601 of Figure 7 and compared to data obtained from the locate and/or marking operation. For example, if gas lines, water lines, and power lines are to be marked during a locate and/or marking operation in a particular location, an aggregated facilities map may be generated by accessing the facilities map from the gas company for the location, the facilities map from the water company for the location, and the facilities map from the electric company from the location, extracting information about the location of map features (e.g., facility lines, streets, and/or other map features) from each of these facilities maps, converting the locations to a common frame of reference (e.g., using the techniques discussed above in connection with Figure 7), and combining the extracted features into a single aggregated map. Thus, rather than performing the process of Figure 7 three times with three separate facilities maps (i.e., once using the gas facilities map, once using the water facilities map, and once using the electric facilities map), the aggregated facilities map may be used each time.
1001781 In some embodiments, such an aggregated facilities map may be displayed (e.g., via a display device 816 of computer 800) to a human user (e.g., the locate technician performing the marking operation, a supervisor of the locate technician, and/or some other user) to provide the user with a visual depiction of the presumed physical location of the facilities present at the work site. In addition, in some embodiments, an electronic representation of the physical locate marks dispensed on the ground may be generated and rendered visually. In other words, information obtained from the performance of the locate and/or marking operation about the locations at which locate marks were dispensed may be obtained in order to "electronically recreate" the locate and/or marking operation and render it visually (e.g., which may be displayed on a display device).

-66-[00179] In some embodiments, the visual rendering of the locate and/or marking operation may be overlaid on the aggregated facilities map to provide a user with a visual picture of where locate marks were dispensed relative to the presumed physical locations of facility lines.
[00180] By way of example, Figure 10 illustrates a perspective view of an overlay 1000 of recreated locate operation on a corresponding aggregated facilities map 1005, which may be formed by the aggregation of one or more facilities maps file.
[00181] Aggregated facilities map 1005 is an example of an aggregation of facilities map against which a recreated locate operation 1100 may be compared. In this example, overlay 1000 also shows lines pattern 1116 of the recreated locate operation correlated to telecommunications line 1116 of aggregated facilities map 1005, lines pattern 1112 correlated to sewer line 1012, lines pattern 1114 correlated to telecommunications line 1014, and lines pattern 1110 correlated to power line 1010.
Overlay 1000 of Figure 10 is an example of a comparison that has substantially no discrepancies between the locations at which marking material was dispensed and the presumed physical locations of facility lines. However, those skilled in the art will recognize that discrepancies may occur. In such situations, the overlay of the locate marks for a particular facility line may be displaced from the presumed physical location of the facility line.
[00182] More specifically, various techniques may be employed to assist the human user in making the visual comparisons of field information and reference information. For example, the information used to render line patterns may be suitably filtered, interpolated, smoothed or otherwise processed, to enhance the appearance of the line patterns.
Additionally, features corresponding to field data and features corresponding to reference data may be differentiated in a display field in any of a variety of manners (e.g., different line types, symbols or patterns; different colors or shades of related colors;
different vertical planes of display, etc.) to allow for visual perception of both the field data and the reference data.
[00183] To this end, in one embodiment, each of the field data and the reference data, if present in a computer-aided visual rendering, as well as any constituent information forming part of the field data and the reference data, may be displayed as separate "layers"

-67-of the visual rendering, such that a viewer of the visual rendering may turn on and turn off displayed data based on a categorization of the displayed data. For example, all field data may be categorized generally under one layer designation (e.g., "Field"), and independently enabled or disabled for display (e.g., hidden) accordingly.
Similarly, all reference data may be categorized generally under another layer designation (e.g., "Reference") and independently enabled or disabled for display accordingly.
Respective layers may be enabled or disabled for display in any of a variety of manners;
for example, in one implementation, a "layer directory" or "layer legend" pane may be included in the display field (or as a separate window selectable from the display field of the visual rendering), showing all available layers, and allowing a viewer to select each available layer to be either displayed or hidden, thus facilitating comparative viewing of layers.
[00184] Furthermore, any of the above-mentioned general categories for layers may have sub-categories for sub-layers, such that each sub-layer may also be selectively enabled or disabled for viewing by a viewer. For example, under the general layer designation of "Field," different facility types that may have been marked (and indicated in the field data by color, for example) may be categorized under different sub-layer designations (e.g., "Field ¨ Electric;" "Field ¨ Gas;" etc.); in this manner, a viewer may be able to hide the electric field data while viewing the gas field data, or vice versa, in addition to having the option to view or hide all field data. Sub-layer designations similarly may be employed for the reference data (e.g., "Reference ¨
water/sewer;"
"Reference ¨ CATV"). Virtually any characteristic of the information available for display may serve to categorize the information for purposes of displaying layers or sub-layers.
[00185] Conclusion [00186] In sum, information relating to a locate and/or marking operation may be compared to a variety of information derived from one or more facilities maps for purposes of assessing a quality of the locate and/or marking operation. The types of field information being compared to reference information derived from one or more facilities maps may include geographic information, facility type information, and/or other information relating to the facilities identified and/or marked during the locate and/or marking operation. For example, the comparison may generally involve determining whether there is agreement between the locate and/or marking operation and information

-68-derived from one or more facilities maps, which may in turn involve identifying at least one correspondence or discrepancy between the compared data, and in some instances a degree of correspondence.
1001871 While various inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein, and each of such variations and/or modifications is deemed to be within the scope of invention, which is solely defined by the claims. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the inventive teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein.
It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims, inventive embodiments may be practiced otherwise than as specifically described. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein.
1001881 The above-described embodiments can be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.
1001891 Figure 9 shows an illustrative computer 800 that is an example of a type of computer that may serve as quality assessment system 1800 and may execute quality assessment application 1200. Computer 800 comprises a memory 810 that stores a plurality of facilities maps 830a.. .830n, one or more processing units 812 (also referred to herein simply as "processors"), one or more communication interfaces 814, one or more

-69-display units 816, and one or more user input devices 818. The memory 810 may comprise any computer-readable media, and may store computer instructions (also referred to herein as "processor-executable instructions") for implementing the various functionalities described herein. The processing unit(s) 812 may be used to execute the instructions. For example, memory 810 may store computer-executable instructions of automated quality assessment application 1200 which, when executed by processor(s) 812, performs the processes illustrated in Figure 6, 7, and/or 8 to compare data obtained from a locate and/or marking operation with data obtained from a facilities map(s) 830, and generate a quality assessment based on this comparison.
[00190] The communication interface(s) 814 may be coupled to a wired or wireless network, bus, or other communication means and may therefore allow the computer 800 to transmit communications to and/or receive communications from other devices.
The display unit(s) 816 may be provided, for example, to allow a user 240 to view various information in connection with execution of the instructions. The user input device(s) 818 may be provided, for example, to allow the user to make manual adjustments, make selections, enter data or various other information, and/or interact in any of a variety of manners with the processor during execution of the instructions.
1001911 In some embodiments, computer 800 may be situated on the locating equipment. For example, computer 800 may be affixed to a marking device, a locate receiver, and/or a combined marking device and locate receiver. In some embodiments, the techniques for performing a quality assessment may be performed in real-time during a locate and/marking operation. Thus, for example, in some embodiments, a quality score for the locate and/marking operation may be computed in real-time during the performance of the locate and/or marking operation. Moreover, in some embodiments, such a real time score may be displayed (e.g., via display device(s) 816) to the field service technician during performance of the locate and/or marking operation.
[00192] In some embodiments in which a quality score is computed in real-time during the performance of a locate and/or marking operation, the functionality of the locating equipment may be altered based on the score at a given point in time. For example, if during the locate and/or marking operation, the quality score drops below a certain threshold, the marking device may be disabled.

-70-[00193] The various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine.
[00194] In this respect, various inventive concepts may be embodied as a computer readable storage medium (or multiple computer readable storage media) (e.g., a computer memory, one or more floppy discs, compact discs, optical discs, magnetic tapes, flash memories, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention discussed above. The computer readable medium or media can be transportable, such that the program or programs stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the present invention as discussed above.
[00195] The terms "program" or "software" are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor to implement various aspects of embodiments as discussed above. Additionally, it should be appreciated that according to one aspect, one or more computer programs that when executed perform methods of the present invention need not reside on a single computer or processor, but may be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the present invention.
[00196] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc.
that perform particular tasks or implement particular abstract data types.
Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.

-71-[00197] Also, data structures may be stored in computer-readable media in any suitable form. For simplicity of illustration, data structures may be shown to have fields that are related through location in the data structure. Such relationships may likewise be achieved by assigning storage for the fields with locations in a computer-readable medium that convey relationship between the fields. However, any suitable mechanism may be used to establish a relationship between information in fields of a data structure, including through the use of pointers, tags or other mechanisms that establish relationship between data elements.
[00198] Also, various inventive concepts may be embodied as one or more methods, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.
[00199] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents, and/or ordinary meanings of the defined terms.
[00200] The indefinite articles "a" and "an," as used herein in the specification and in the claims, should be understood to mean "at least one", where appropriate.
[00201] The phrase "and/or," as used herein in the specification and in the claims, should be understood to mean "either or both" of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases.
Multiple elements listed with "and/or" should be construed in the same fashion, i.e., "one or more" of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the "and/or" clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to "A and/or B", when used in conjunction with open-ended language such as "comprising" can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.
[00202] As used herein in the specification and in the claims, "or" should be understood to have the same meaning as "and/or" as defined above. For example, when separating

-72-items in a list, "or" or "and/or" shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as "only one of" or "exactly one of;" or, when used in the claims, "consisting of," will refer to the inclusion of exactly one element of a number or list of elements. In general, the term "or" as used herein shall only be interpreted as indicating exclusive alternatives (i.e.
"one or the other but not both") when preceded by terms of exclusivity, such as "either,"
"one of," "only one of," or "exactly one of" "Consisting essentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.
[00203] As used herein in the specification and in the claims, the phrase "at least one,"
in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements.
This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase "at least one" refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B," or, equivalently "at least one of A and/or B") can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.
[00204] In the claims, as well as in the specification above, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "involving,"
"holding,"
"composed of;" and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases "consisting of' and "consisting essentially of' shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 2111.03.

Claims (33)

CLAIMS:

1. A method for evaluating a quality of a locate and/or marking operation performed by a locate technician in response to a locate ticket and in advance of planned excavation activities at a dig area identified by the locate ticket, the method executed by a computer comprising at least one hardware processor, at least one tangible storage medium, and at least one input/output (I/0) interface, the method comprising:
A) receiving first information relating to the locate and/or marking operation, the locate and/or marking operation comprising identifying, in advance of the planned excavation activities, a presence or an absence of at least one underground facility within the dig area identified by the locate ticket, the first information comprising a first set of geo-location data points indicating geographic locations at which at least one facility line of the at least one underground facility was detected and/or marked during the locate and/or marking operation;
B) comparing the first information to second information obtained from at least one facilities map, the second information comprising a second set of geo-location data points indicating a presumed actual location of at least one facility line represented in the at least one facilities map, the comparing comprising:
B1) comparing the first set of geo-location data points with the second set of geo-location data points; and B2) determining a distance between each point in the first set of geo-location data points and a nearest point in the second set of geo-location data points to generate a vector of distances;
C) automatically generating, based on B), at least one indication of a quality assessment of the locate and/or marking operation; and D) electronically storing on the at least one tangible storage medium, and/or electronically transmitting via the at least one I/O interface, the at least one indication of the quality assessment so as to provide an electronic record of the quality assessment.

2. The method of claim 1, wherein the computer comprises at least one display device, and wherein the method further comprises:
displaying the at least one indication of the quality assessment on the at least one display device.

3. The method of claim 1, wherein the locate and/or marking operation is performed by at least one technician, and wherein D) comprises:
transmitting at least one feedback message to the at least one technician prior to completion of the location and/or marking operation, the feedback message being generated based at least in part on the at least one indication of the quality assessment generated in C).

4. The method of claim 1, wherein the second information obtained from the at least one facilities map includes type information indicating at least one type of facility represented in the at least one facilities map.

5. The method of claim 4, wherein the type information includes a number of different types of facilities represented in the at least one facilities map.

6. The method of claim 1, wherein the second information obtained from the at least one facilities map includes a number of facility lines of a same type as the at least one underground facility intended to be detected and/or marked during the locate and/or marking operation.

7. The method of claim 1, wherein the second information obtained from the at least one facilities map includes relative position information relating to a relative position, pattern, or arrangement of multiple facility lines represented in the at least one facilities map.

8. The method of claim 1, wherein the second information obtained from the at least one facilities map includes information relating to one or more environmental landmarks, one or more architectural elements, or one or more traffic infrastructure elements represented in the at least one facilities map.

9. The method of claim 1, wherein the second information obtained from the at least one facilities map relates at least in part to at least one of:
a date of the at least one facilities map;
a number of revisions to the at least one facilities map;
a source, creator, owner and/or custodian of the at least one facilities map;
text information included the at least one facilities map; and legend information included in the at least one facilities map.

10. The method of claim 1, wherein the second information obtained from the at least one facilities map relates at least in part to at least one conduit run or at least one joint trench represented in the at least one facilities map.

11. The method of claim 1, wherein the first information is generated by locating equipment used by the field-service technician to perform the locate and/or marking operation, wherein the locating equipment comprises at least one of a marking device, a locate device, and a combined locate and marking device, and wherein A) comprises:
receiving the first information from the locating equipment.

12. The method of claim 11, wherein the at least one of the marking device, the locate device, and the combined locate and marking device includes the computer including the at least one hardware processor executing the method.

13. The method of claim 11, wherein A), B), C) and D) are performed during or immediately following the locate and/or marking operation, and wherein the method further comprises:
E) altering at least one operating characteristic of the locating equipment based on the at least one indication of the quality assessment.

14. The method of claim 13, wherein the at least one indication of the quality assessment comprises a numeric score indicative of the quality of the locate and/or marking operation, and wherein E) further comprises:
El) altering the at least one operating characteristic of the locating equipment if the numeric score is below a predetermined threshold.

15. The method of claim 13, wherein E) comprises disabling the locating equipment if the at least one indication of the quality assessment is unsatisfactory.

16. The method of claim 11, wherein the first information includes at least one of:
at least one location at which the locating equipment was used to perform the locate and/or marking operation;
facility-type information identifying at least one underground facility for which the presence or the absence was detected and/or marked during the locate and/or marking operation;
facility location information identifying a location of the at least one underground facility for which the field-service technician detected and/or marked the presence or the absence during the locate and/or marking operation;
marking material type information identifying at least one characteristic of a marking material used to mark the presence or the absence of the at least one underground facility;
signal information identifying a signal strength measured by the locate device at the location of the at least one underground facility for which the field-service technician detected the presence or the absence during the locate and/or marking operation;
position information identifying an angle and/or acceleration of the locating equipment during the locate and/or marking operation; and environmental information describing at least one environmental condition present during the locate and/or marking operation.

17. The method of claim 1, wherein the first information comprises an electronic manifest of the locate and/or marking operation, the electronic manifest comprising:
at least one image of the work site; and at least one electronic marking on the image indicating the presence or the absence of the at least one underground facility.

18. The method of claim 17, wherein the electronic manifest further comprises first geographic information indicating the location of the at least one electronic marking.

19. The method of claim 1, wherein the method further comprises:
B0) prior to B), selecting for comparison at least some of the first information or at least some of the second information based at least in part on a dig area indicator that indicates the dig area of the work site on a digital image.

20. The method of claim 19, wherein B0) comprises:
receiving, via the at least one I/0 interface, geographic coordinates for the dig area indicator; and selecting the at least some of the first information, or the at least some of the second information, that relates only to a geographic area including the geographic coordinates for the dig area indicator.

21. The method of claim 20, wherein the geographic area is within a predetermined radius of the geographic coordinates for the dig area indicator.

22. The method of claim 19, wherein the dig area indicator delimits the dig area on the digital image, and wherein B0) comprises:
receiving, via the at least one I/O interface, dig area indicator geographic information including geographic coordinates for the dig area indicator; and selecting the at least some of the first information, or the at least some of the second information, that relates only to a geographic area delimited by the dig area indicator geographic information.

23. The method of claim 1, wherein the method further comprises:
B0) prior to B), selecting the at least one facilities map based at least in part on a dig area indicator that indicates the dig area of the work site on a digital image.

24. The method of claim 23, wherein B0) comprises:
receiving, via the at least one I/O interface, dig area indicator geographic information including geographic coordinates for the dig area indicator; and selecting the at least one facilities map that includes a geographic area including the geographic coordinates for the dig area indicator.

25. The method of claim 1, wherein:
the at least one facilities map includes a plurality of facilities maps;
the second information comprises a plurality of pieces of information derived from different ones of the plurality of facilities maps; and prior to B), the method comprises aggregating the plurality of pieces of information derived from the different ones of the plurality of facilities maps to obtain the second information.

26. The method of claim 25, wherein a confidence value is associated with each of the plurality of pieces of information, and wherein the method comprises aggregating the plurality of pieces of information by selecting at least one piece of information from the plurality of pieces of information based on respective confidence values associated with the plurality of pieces of information.

27. The method of claim 1, wherein the first set of geo-location data points is expressed in a first reference frame and the second set of geo-location data points is expressed in a second reference frame different from the first reference frame, and wherein the processor transforms at least one of the first set of geo-location data points and the second set of geo-location data points to a common reference frame.

28. The method of claim 1, wherein C) comprises:
C1) generating the at least one indication of the quality assessment based, at least in part, on the vector of distances generated in B2).

29. The method of claim 28, wherein Cl comprises:
generating the at least one indication of the quality assessment based, at least in part, on a percentage of distance values in the vector of distances that are below a first threshold.

30. The method of claim 1, wherein C) further comprises:
generating the at least one indication of the quality assessment as a score or grade having one of a plurality of possible values.

31. The method of claim 30, wherein B) comprises:
providing at least one quality assessment criterion relating to the second information obtained from the at least one facilities map;
providing at least two scoring categories for the at least one quality assessment criterion, each scoring category associated with a scoring value or grade;
for each scoring category providing an expected data value or range of expected data values;
determining, for the at least one quality assessment criterion, into which of the at least two scoring categories the locate and/or marking operation falls by comparing the first information to the expected data value or range of expected data values for at least one of the at least two scoring categories; and assigning to the locate and/or marking operation the scoring value or grade associated with the scoring category into which the locate and/or marking operation falls.

32. An apparatus for evaluating a quality of a locate and/or marking operation performed by a locate technician in response to a locate ticket and in advance of planned excavation activities at a dig area identified by the locate ticket, the apparatus comprising:
at least one input/output (I/0) interface;
at least one memory storing processor-executable instructions; and a processor coupled to the memory and the at least one I/O interface, wherein upon execution of the processor-executable instructions by the processor, the processor performs the method according to any one of claims 1 to 31.

33. A computer-readable storage medium encoded with instructions that, when executed by a processor in the computer comprising the at least one input/output (I/0) interface, perform the method according to any one of claims 1 to 31.