US8788523B2 - Systems, methods and software for processing phrases and clauses in legal documents - Google Patents

Systems, methods and software for processing phrases and clauses in legal documents Download PDF

Info

Publication number
US8788523B2
US8788523B2 US12/553,752 US55375209A US8788523B2 US 8788523 B2 US8788523 B2 US 8788523B2 US 55375209 A US55375209 A US 55375209A US 8788523 B2 US8788523 B2 US 8788523B2
Authority
US
United States
Prior art keywords
clauses
clause
phrases
documents
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/553,752
Other versions
US20110055206A1 (en
Inventor
Kingsley Martin
Trace Liggett
Gary Berosik
Thomas Zielund
Dietmar Dorr
Jochen L. Leidner
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thomson Reuters Enterprise Centre GmbH
Original Assignee
Thomson Reuters Global Resources ULC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Reuters Global Resources ULC filed Critical Thomson Reuters Global Resources ULC
Priority to US12/553,752 priority Critical patent/US8788523B2/en
Assigned to WEST SERVICES, INC. reassignment WEST SERVICES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ZIELUND, THOMAS, BEROSIK, GARY, DORR, DIETMAR, LEIDNER, JOCHEN
Assigned to THOMSON REUTERS GLOBAL RESOURCES reassignment THOMSON REUTERS GLOBAL RESOURCES ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: WEST SERVICES, INC.
Publication of US20110055206A1 publication Critical patent/US20110055206A1/en
Priority to US14/337,105 priority patent/US9811518B2/en
Application granted granted Critical
Publication of US8788523B2 publication Critical patent/US8788523B2/en
Assigned to THOMSON REUTERS GLOBAL RESOURCES UNLIMITED COMPANY reassignment THOMSON REUTERS GLOBAL RESOURCES UNLIMITED COMPANY CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: THOMSON REUTERS GLOBAL RESOURCES
Assigned to THOMSON REUTERS ENTERPRISE CENTRE GMBH reassignment THOMSON REUTERS ENTERPRISE CENTRE GMBH ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: THOMSON REUTERS GLOBAL RESOURCES UNLIMITED COMPANY
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/279Recognition of textual entities
    • G06F40/289Phrasal analysis, e.g. finite state techniques or chunking
    • G06F17/30
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00Digital computers in general; Data processing equipment in general
    • G06F15/16Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1865Transactional file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/93Document management systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • G06F40/174Form filling; Merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/205Parsing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/10Office automation; Time management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/18Legal services; Handling legal documents

Definitions

  • Various embodiments of the present invention concern systems, methods, and software for detecting and/or correlating phrases in documents, such as legal contracts.
  • a common legal task for lawyers and other legal professionals entails drafting, analyzing, and/or negotiating transactional documents, such as legal contracts.
  • caselaw documents which include citations to other legal cases, that can inform lawyers about the validity, reliability, and origins of the legal text and reasoning within a given judicial opinion
  • contracts typically lack such indicators. Nevertheless, lawyers and other legal professionals often seek information about the precedential value and the reliability of contract (deal) documents, particularly clauses, definitions, and key phrases within these documents.
  • the present inventors identified a need for better tools to help lawyers and other legal professionals work with and analyze phrases and clauses within transactional documents.
  • One exemplary system includes a phrase discovery engine or module, which automatically processes one or more documents, such as legal contracts, and identifies and groups phrases and clauses (super phrases or uberphrases) within the documents.
  • the phrase discovery engine can be used to assist in referencing industry standard clauses or phrases, to obtain alternative clause and/or phrase languages, or to obtain negotiation leverage.
  • FIG. 1 is a diagram of an exemplary desktop platform corresponding to one or more embodiments of the present invention.
  • FIG. 2 is a diagram of an exemplary web service platform corresponding to one or more embodiments of the present invention.
  • FIG. 3 is a flow chart of an exemplary method corresponding to one or more embodiments of the present invention.
  • FIG. 4 is a flow chart of an exemplary method corresponding to one or more embodiments of the present invention.
  • FIG. 5 is a conceptual illustration of an action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
  • FIG. 6 is a conceptual illustration of another action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
  • FIG. 7 is a conceptual illustration of another action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
  • FIG. 8 is a conceptual illustration of another action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
  • FIG. 9 is an illustration of a document profile corresponding to one or more embodiments of the present invention.
  • FIG. 10 is a facsimile of a graphical user interface portion of an exemplary multi-functional document draft and analysis system corresponding to one or more embodiments of the present invention.
  • Various embodiments of the present invention are useful in helping lawyers and other legal professionals address risk relating to drafting and/or entering into legal contracts.
  • risk there are several types of risk regarding legal contracts.
  • One type of risk is the risk of illegality or unenforceability which relates to violating legal principles.
  • An agreement creates a private law between the parties; however, this private law is subservient to the laws (both statutory and common law) of the jurisdiction that govern the agreement, often dictated by where the agreement is executed.
  • Such examples include statutory requirements. (e.g. trust principle that distribution of principal may be permitted only for “health, support, education and maintenance”) and common law principles. (e.g. non-compete clauses).
  • Another risk concerns failure to comply with external (regulatory) or internal (contractual) requirements.
  • An additional risk is failing to secure the best terms (negotiation risk).
  • An agreement is a bargain between parties in which each party seeks to secure the best terms for themselves, which can be measured by other similar documents executed between parties having similar bargaining power.
  • Business norms vary by bargaining power and change over time.
  • Related to the previous risk is the risk of deviating from business norms and market standards (uncertainty risk).
  • An agreement between parties is a quantifiable expression of risk that each party is willing to bear; uncertainty or unpredictability increases the risk. In order to correctly assess the level of risk, the parties must understand the value of the assets affected and their future state (asset-specific risk).
  • the present invention is designed to function within a variety of usage scenarios.
  • One scenario allows the user to select a group of merger agreements, statutes, etc. to compare against each other or against a specific merger agreement, statute, etc.
  • Another scenario entails a user selecting a paragraph from a merger agreements, statutes, etc. to compare against a group of merger agreements, statutes, etc.
  • Yet another scenario entails a user drafting a merger agreement, and automatically receiving phrase or clause suggestions during the drafting process, with the phrase or clause suggestions automatically determined in the background based on a group of merger agreements, statutes, etc.
  • a group could be a user-defined set, a set defined by and/or for a particular law firm or corporation, or a set defined by the provider of the phrase processing engine or other third-party.
  • FIG. 1 shows that one such platform takes the form of web data service such as Westlaw.com ( FIG. 1 ), desktop and/or firm side software ( FIG. 2 ), multiple monitor UI, and/or a combination.
  • web data service such as Westlaw.com ( FIG. 1 )
  • desktop and/or firm side software FIG. 2
  • multiple monitor UI and/or a combination.
  • One embodiment includes a transactional negotiation leverage and research tool which benchmarks deal documents against market standards or other reference sets, which distinguishes market standard language from deal-specific or negotiated language, which links to judicial interpretation of contractual language, and/or validates codified (statutory and regulatory) language.
  • Another embodiment involves a transactional drafting and review tool which finds alternative clauses and phrases and provides a taxonomic framework for editorial links and procedural guidance.
  • Yet another embodiment of the present invention exemplifies prototypical documents which could include master (non-negotiated) documents for a first draft and reference documents (e.g. buyer weighted sales agreement).
  • Other embodiments include alert functions to identify and report documents that have been affected by a change in the law, providing automated document and clause type classification with clause, definition and deal term libraries, and document vetting to identify valuable, reusable deal documents through processing-efficient phrase matching.
  • FIG. 3 shows a flow chart 300 of an exemplary phrase discovery and processing method.
  • Flow chart 300 includes process blocks 310 - 380 , which in addition to being representative of exemplary processing activity are also representative of an exemplary functional partitioning or architecture. The flow chart is therefore applicable to software, firmware, and hardware implementations.
  • the exemplary method begins with annotating one or more documents, such a set of documents from a legal corpus.
  • FIG. 4 shows an exemplary annotation method 310 via process blocks 311 - 316 .
  • Block 311 entails sentence splitting.
  • Block 312 entails tokenizing the document.
  • Block 313 entails stemming and lemmatization of the document(s).
  • Block 314 entails tagging parts of speech within the documents.
  • Block 315 a named entity extractor, replaces named entities (numbers, dates, orgs, etc.) with a placeholder. This could also be done for pronouns with option of some co-reference analysis.
  • Block 316 performs a chunking function, which entails identifying various non-recursive phrases within the document, such as noun phrases, verb phrases, adjective phrases, adverb phrases, etc. Chunker may not work well with the placeholders but could use a dummy name to get around this aspect.
  • this parser is shallow in that it typically segments the text in a manner that uses only a single level of depth, meaning that there is no recursive nature to the resulting parsed elements. Benefits include matching or relating things that are grammatically related in addition to words that are related, eliminating a lot of statistically determined phrases because they aren't meaningful in the language of choice (speeds up the process by not having to include many of the candidates).
  • phrase “sound mind and body” would have a bracket indicating a phrase instead of going through all different word combination scenarios to identify the phrase. This would also reduce dramatically or potentially even replace phrase identification.
  • Chunking adds annotations on the document the same way a tokenizer does.
  • Phrase identification could look for sequences of tokens ignoring chunks, sequences of chunks ignoring tokens, or sequence of tokens respecting certain aspects of chunk boundaries to improve several performance measures (precision, accuracy, speed, memory).
  • Some of the document annotators might be useful only in the English language and maybe a few others. However, using only a tokenizer can be theoretically applied to any language.
  • the outputs of all these formatting techniques are annotated/tagged documents. After the documents are annotated or tagged, exemplary execution continues at block 320 of FIG. 3 .
  • Block 320 entails identifying co-occurring phrases within the set of documents.
  • a phrase is defined as a sequence of two or more semi-contiguous tokens that are found in two or more documents (identical matching). Semi-contiguous takes into account that several stop words may occur between the tokens of a phrase within the document. Once these stop words are removed, the tokens in an identified phrase would become continuous.
  • Some embodiments provide an upper limit of the number of tokens in a row (example, 5 tokens in a row) or a statistical cut-off in defining or identifying phrases.
  • One exemplary embodiment is based on an n-gram model to identify phrases.
  • an additional exemplary embodiment of the present invention could use a variation of n-grams without a maximum (e.g. 5 tokens in a row maximum) and use a confidence-based threshold (n-grams with pruning the search).
  • the pruned n-grams approach would cease identifying a phrase when there is no longer enough support for the token sequence. For example if beer and diaper are not a phrase, this approach can determine that beer, diaper, president are not going to be a phrase either.
  • Another exemplary embodiment utilizes a sequence alignment technique to identify phrases occurring in two or more documents. Sequence alignment looks at all tokens sequences between two documents simultaneously and decides where the phrases reside. Examples of sequence alignment algorithms include Needleman-Wunsch algorithm, BLAST, or a multiple alignment technique that can be used for more than two documents simultaneously. Once the phrases are identified, they are also associated with a number indicating the number of documents they occur in, the type of document they occur, for example, will, trust, contract, contract type, end-user license agreement, etc. execution continues at block 330 .
  • Block 330 entails linking or clustering the identified phrases into equivalence classes or families.
  • phrase linkage occurs when two or more phrases are similar enough to represent the same phrase.
  • Several techniques can be utilized for linking including simple, complete and average linkage.
  • an exemplary embodiment uses external corpus/resources such as Wikipedia, Westlaw® Headnotes with KeyNumbers, Black's Law dictionary (corpus of conceptual entities) to determine the linkage between two or more phrases.
  • the exemplary embodiment uses an algorithm that looks at the words in the phrase and associates the phrase with a concept. It then associates the phrase with a surrogate identifier, trying to match that with other surrogate identifiers that it has found in Wikipedia, for example. If two of the phrases that have not otherwise been linked are found to have a common concept according to the external resource, then an assumption is made that the two phrases are sufficiently similar to be linked or clusters into the same equivalence class. When phrase linkage is complete, execution proceeds to block 340 .
  • Block 340 entails grouping or clustering the phrase clusters or equivalence classes based on phrase co-occurrence into larger phrase groupings.
  • the exemplary embodiment deems phrase co-occurrence to occur when sets of equivalent classes show up together frequently enough, that is within a threshold number of documents together.
  • Exemplary thresholds include support thresholds, confidence thresholds, strength, and interestingness even when you don't know what you are searching for.
  • An additional exemplary embodiment includes the method of frequent item-set mining (FIM). It is important to note that proximity between phrases does not matter nor does order for this step. However, phrase proximity plays a role in defining and utilizing super-phrases.
  • Block 350 entails defining a set of clauses (super phrases or uber phrases) based on phrase proximity and uniqueness.
  • a super-phrase is a collection of text areas that are associated with a phrase set. More particularly, a superphrase is a region of text bounded by two phrases in close proximity.
  • One exemplary embodiment uses association rules mining (ARM) to find associations within a set of equivalence classes.
  • the result is inference rules—rules to arrive at inferences from a set of clauses, and make statements about those clauses. For example, the inference rules say 90% of the contracts clauses also had these clauses as well.
  • This embodiment also provides recommendation rules based on inference rules to be used in risk-avoidance recommendations and drafting recommendations given the proximity and uniqueness constraints. These recommendation rules could provide for benchmarking or best practices standards to compare to user's work product or even a checklist for certain documents. If high thresholds of certain sets are met, a comparison of those sets to what user has drafted in order to provide recommendations.
  • Other exemplary embodiments for determining and generating super-phrases/uberphrases include tf-idf, indexing, clustering.
  • the parameters for system could be user defined or a machine learning technique could be used to automatically determine the optimal parameters.
  • Another feature allows for suffix arrays as a data structure choice when given large sequences of text to quickly access substrings.
  • the suffix array of a string can be used as an index to quickly locate every occurrence of a substring within the string. Finding every occurrence of the substring is equivalent to finding every suffix that begins with the substring. Thanks to the lexicographical ordering, these suffixes are grouped together in the suffix array, and can be found efficiently with a binary search.
  • an index of phrases, phrase clusters, groups of phrase clusters, and superphrases is available for use.
  • each of the phrases, groups, and superphrases is associated not only with a document identifier and document positional information for the associated text, but also metadata regarding the origin, authors, law firms, dates, jurisdictions, type of document (will, real estate agreement, mergers and acquisition agreement, confidentiality agreement, license agreement, etc.)
  • tags one or more of the phrases or clauses from the document corpus as good or bad (valid or invalid). This tagging may be done automatically based on a classifier (for example, an support vector machine) trained with known good or bad phrases or clauses, or manually by legal experts.)
  • Block 360 entails receiving user text input.
  • the input text is presented from an active edit window of a word processing application such as the Word or WordPerfect application.
  • the input is a set of documents, such as agreements, from a law firm.
  • the input takes the form of a single agreement or clause or paragraph of an agreement.
  • the input may be provided by a browser application. For example, a user may encounter an end-user license agreement (EULA) and an application program interface may automatically or with specific user direction identify the EULA as input. Exemplary execution continues at block 370 .
  • EULA end-user license agreement
  • Block 370 entails processing the input text against the index structure created using process blocks 310 - 350 .
  • this entails essentially processing the input text using blocks 310 - 350 to identify phrases, phrase groups, and superphrases and then identifying a ranked list of most similar phrases, phrase groups, and superphrases from the index structure.
  • corresponding documents, phrases, and associated meta data for the most similar phrases will be available.
  • Block 380 entails outputting results of the processing in block 370 .
  • the results can take a variety for forms, such as listing of similar clauses or phrases and associated metadata; a listing of links for a set of documents having phrases or clauses most similar to the put one; and/or an annotated version of the input text indicating portions in conformity with known standard phrases as well as clauses that have deviated from such norms.
  • the norms may be a specific set of documents or clauses that have for example preapproved for use by law department for a given corporation or by a given law firm.
  • FIGS. 5-9 show a more conceptual version of an exemplary phrase discovery methodology.
  • FIG. 5 illustrates building a unique word list from any corpus of documents. In creating this word list, any standard search indexing techniques such as stemming, synonyms, etc. may be used.
  • FIG. 6 represents an act of detecting common phrases.
  • Phrase detection uses common word sequences appearing in two or more documents with defined proximity to generate candidate phrase lists.
  • Phrase detection rules combine parsers for filtering, comparing and scoring candidate phrases. The rules may be adjusted for different document domains and include minimum number and/or percentage of matching words and maximum number or percentage of unmatched or stop words. In addition, some embodiments take into consideration word alternatives in phrases. Within a phrase some words or shorter phrases may appear and are identified as common word alternatives. Given the basic rules that the user has inputted, common phrases are discovered. Once these common phrases are recognized, a phrase list is created and the common phrases are saved within the list. Then phrase matching compares phrase lists across multiple documents to identify common phrases.
  • FIG. 7 shows an act of grouping or clustering common phrases. Since phrases may be overlapping, phrase group rules identify individual and phrase groups. Some embodiments allow for non-overlapping clusters, whereas others allow for overlapping clusters.
  • the fourth step includes grouping common clauses or super phrases. Rules are applied to further improve precision and recall and to group into clauses. For example, in 10,000 documents two phrases were always near each other, but in the 100 documents where only one of the phrases appeared, it had a low score and manual review showed they were not good matches. Therefore, those phrases are not considered to be like phrases.
  • FIG. 9 shows that after grouping, the exemplary method identifies the names of clauses.
  • a profile of the document set has been created.
  • the profiles are saved and in future processing, the user can utilize just this profile as a “document”, and/or utilize a set of preclassified profiles to train a classifier that categorizes a given document or clause.
  • a profile can be used to “name” clauses manually for a document type, and deconstruct other documents based off of that set.
  • Scoring defaults can be set for document types and are always manually and programmatically adjustable. Some sample scoring metrics are listed:
  • blocks 160 have been executed for a corpus of documents, for example, from a law firm or from across multiple law firms, such in a collection of agreement for an online legal research service, one can repeat the methodology for a given document and identify related phrases and clauses and documents from the phrases using similarity metrics. Also, one may simply run a query against the phrase families, phrase family groupings, and/or phrase groupings to identity similar phrases as well as their corresponding documents and associated meta data, such as jurisdiction, location of author, date of addition to the corpus, etc.
  • various embodiments of systems incorporating the present invention provide a graphical user interface for defining or controlling user preferences regarding operation of a phrase processing engine.
  • FIG. 10 shows an exemplary user preference interface for controlling phrase functions in some embodiments of the present invention.
  • Some of these results include highlighting of negotiated/deal specific text versus boilerplate versus other (e.g. statutory), indicating degrees of variance, providing summary of risks for document overall and specific clauses, suggesting alternative clauses/additional clauses based on degrees of similarity or dissimilarity, and suggesting relevant caselaw/statutes and other materials.
  • these results provide dynamic document analysis and form creation.

Abstract

Exemplary systems for processing a corpus of documents, such as legal contracts or agreements, are disclosed. The systems include a phrase discovery engine which derives statistics and phrase equivalence classes, groups of phrase equivalence classes, and uberphrases (clauses) bounded by phrases. These can be used to determine origins of phrases or clauses within given legal contract or to suggest alternative phrases and clauses.

Description

RELATED APPLICATION
This application is a continuation of U.S. application Ser. No. 12/354,617, filed Jan. 15, 2009, which claims priority to U.S. Provisional Patent Application 61/011,147, which was filed on Jan. 15, 2008 and which is incorporated herein by reference.
COPYRIGHT NOTICE AND PERMISSION
A portion of this patent document contains material subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyrights whatsoever. The following notice applies to this document: Copyright© 2007, Thomson Reuters Global Resources.
TECHNICAL FIELD
Various embodiments of the present invention concern systems, methods, and software for detecting and/or correlating phrases in documents, such as legal contracts.
BACKGROUND
A common legal task for lawyers and other legal professionals entails drafting, analyzing, and/or negotiating transactional documents, such as legal contracts. Unlike caselaw documents, which include citations to other legal cases, that can inform lawyers about the validity, reliability, and origins of the legal text and reasoning within a given judicial opinion, contracts typically lack such indicators. Nevertheless, lawyers and other legal professionals often seek information about the precedential value and the reliability of contract (deal) documents, particularly clauses, definitions, and key phrases within these documents.
Accordingly, the present inventors identified a need for better tools to help lawyers and other legal professionals work with and analyze phrases and clauses within transactional documents.
SUMMARY
To address this and/or other needs, the present inventors devised, among other things, systems, methods, and software that support authoring and/or review of documents, such as legal contracts. One exemplary system includes a phrase discovery engine or module, which automatically processes one or more documents, such as legal contracts, and identifies and groups phrases and clauses (super phrases or uberphrases) within the documents. The phrase discovery engine can be used to assist in referencing industry standard clauses or phrases, to obtain alternative clause and/or phrase languages, or to obtain negotiation leverage.
BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 is a diagram of an exemplary desktop platform corresponding to one or more embodiments of the present invention.
FIG. 2 is a diagram of an exemplary web service platform corresponding to one or more embodiments of the present invention.
FIG. 3 is a flow chart of an exemplary method corresponding to one or more embodiments of the present invention.
FIG. 4 is a flow chart of an exemplary method corresponding to one or more embodiments of the present invention.
FIG. 5 is a conceptual illustration of an action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
FIG. 6 is a conceptual illustration of another action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
FIG. 7 is a conceptual illustration of another action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
FIG. 8 is a conceptual illustration of another action within a phrase discovery engine corresponding to one or more embodiments of the present invention.
FIG. 9 is an illustration of a document profile corresponding to one or more embodiments of the present invention.
FIG. 10 is a facsimile of a graphical user interface portion of an exemplary multi-functional document draft and analysis system corresponding to one or more embodiments of the present invention.
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENT(S)
This description, which incorporates the Figures and the appended claims, describes one or more specific embodiments of an invention. These embodiments, offered not to limit but only to exemplify and teach the invention, are shown and described in sufficient detail to enable those skilled in the art to implement or practice the invention. Thus, where appropriate to avoid obscuring the invention, the description may omit certain information known to those of skill in the art.
Legal Risk and Usage Scenarios
Various embodiments of the present invention are useful in helping lawyers and other legal professionals address risk relating to drafting and/or entering into legal contracts.
There are several types of risk regarding legal contracts. One type of risk is the risk of illegality or unenforceability which relates to violating legal principles. An agreement creates a private law between the parties; however, this private law is subservient to the laws (both statutory and common law) of the jurisdiction that govern the agreement, often dictated by where the agreement is executed. Such examples include statutory requirements. (e.g. trust principle that distribution of principal may be permitted only for “health, support, education and maintenance”) and common law principles. (e.g. non-compete clauses).
Another risk, known as procedural risk, concerns failure to comply with external (regulatory) or internal (contractual) requirements. An additional risk is failing to secure the best terms (negotiation risk). An agreement is a bargain between parties in which each party seeks to secure the best terms for themselves, which can be measured by other similar documents executed between parties having similar bargaining power. Business norms vary by bargaining power and change over time. Related to the previous risk is the risk of deviating from business norms and market standards (uncertainty risk). An agreement between parties is a quantifiable expression of risk that each party is willing to bear; uncertainty or unpredictability increases the risk. In order to correctly assess the level of risk, the parties must understand the value of the assets affected and their future state (asset-specific risk).
To help legal professionals manage and understand such contractual risk, the present invention is designed to function within a variety of usage scenarios.
One scenario allows the user to select a group of merger agreements, statutes, etc. to compare against each other or against a specific merger agreement, statute, etc. Another scenario entails a user selecting a paragraph from a merger agreements, statutes, etc. to compare against a group of merger agreements, statutes, etc. Yet another scenario entails a user drafting a merger agreement, and automatically receiving phrase or clause suggestions during the drafting process, with the phrase or clause suggestions automatically determined in the background based on a group of merger agreements, statutes, etc. Note: A group could be a user-defined set, a set defined by and/or for a particular law firm or corporation, or a set defined by the provider of the phrase processing engine or other third-party.
Exemplary System Platforms Incorporating Phrase Discovery Engine
The present invention can be implemented on a number of exemplary platforms. FIG. 1 shows that one such platform takes the form of web data service such as Westlaw.com (FIG. 1), desktop and/or firm side software (FIG. 2), multiple monitor UI, and/or a combination.
One embodiment includes a transactional negotiation leverage and research tool which benchmarks deal documents against market standards or other reference sets, which distinguishes market standard language from deal-specific or negotiated language, which links to judicial interpretation of contractual language, and/or validates codified (statutory and regulatory) language. Another embodiment involves a transactional drafting and review tool which finds alternative clauses and phrases and provides a taxonomic framework for editorial links and procedural guidance. Yet another embodiment of the present invention exemplifies prototypical documents which could include master (non-negotiated) documents for a first draft and reference documents (e.g. buyer weighted sales agreement). Other embodiments include alert functions to identify and report documents that have been affected by a change in the law, providing automated document and clause type classification with clause, definition and deal term libraries, and document vetting to identify valuable, reusable deal documents through processing-efficient phrase matching.
Exemplary Phrase Discovery Method and Software Architecture
FIG. 3 shows a flow chart 300 of an exemplary phrase discovery and processing method. Flow chart 300 includes process blocks 310-380, which in addition to being representative of exemplary processing activity are also representative of an exemplary functional partitioning or architecture. The flow chart is therefore applicable to software, firmware, and hardware implementations.
At block 310, the exemplary method begins with annotating one or more documents, such a set of documents from a legal corpus. FIG. 4 shows an exemplary annotation method 310 via process blocks 311-316.
Block 311 entails sentence splitting. Block 312 entails tokenizing the document. Block 313 entails stemming and lemmatization of the document(s). Block 314 entails tagging parts of speech within the documents. Block 315, a named entity extractor, replaces named entities (numbers, dates, orgs, etc.) with a placeholder. This could also be done for pronouns with option of some co-reference analysis.
Block 316 performs a chunking function, which entails identifying various non-recursive phrases within the document, such as noun phrases, verb phrases, adjective phrases, adverb phrases, etc. Chunker may not work well with the placeholders but could use a dummy name to get around this aspect. In the exemplary embodiment, this parser is shallow in that it typically segments the text in a manner that uses only a single level of depth, meaning that there is no recursive nature to the resulting parsed elements. Benefits include matching or relating things that are grammatically related in addition to words that are related, eliminating a lot of statistically determined phrases because they aren't meaningful in the language of choice (speeds up the process by not having to include many of the candidates). For example, the phrase “sound mind and body” would have a bracket indicating a phrase instead of going through all different word combination scenarios to identify the phrase. This would also reduce dramatically or potentially even replace phrase identification. Chunking adds annotations on the document the same way a tokenizer does. Phrase identification could look for sequences of tokens ignoring chunks, sequences of chunks ignoring tokens, or sequence of tokens respecting certain aspects of chunk boundaries to improve several performance measures (precision, accuracy, speed, memory). Some of the document annotators might be useful only in the English language and maybe a few others. However, using only a tokenizer can be theoretically applied to any language. The outputs of all these formatting techniques are annotated/tagged documents. After the documents are annotated or tagged, exemplary execution continues at block 320 of FIG. 3.
Block 320 entails identifying co-occurring phrases within the set of documents. In the exemplary embodiment, a phrase is defined as a sequence of two or more semi-contiguous tokens that are found in two or more documents (identical matching). Semi-contiguous takes into account that several stop words may occur between the tokens of a phrase within the document. Once these stop words are removed, the tokens in an identified phrase would become continuous. Some embodiments provide an upper limit of the number of tokens in a row (example, 5 tokens in a row) or a statistical cut-off in defining or identifying phrases. One exemplary embodiment is based on an n-gram model to identify phrases. Furthermore, an additional exemplary embodiment of the present invention could use a variation of n-grams without a maximum (e.g. 5 tokens in a row maximum) and use a confidence-based threshold (n-grams with pruning the search). The pruned n-grams approach would cease identifying a phrase when there is no longer enough support for the token sequence. For example if beer and diaper are not a phrase, this approach can determine that beer, diaper, president are not going to be a phrase either.
Another exemplary embodiment utilizes a sequence alignment technique to identify phrases occurring in two or more documents. Sequence alignment looks at all tokens sequences between two documents simultaneously and decides where the phrases reside. Examples of sequence alignment algorithms include Needleman-Wunsch algorithm, BLAST, or a multiple alignment technique that can be used for more than two documents simultaneously. Once the phrases are identified, they are also associated with a number indicating the number of documents they occur in, the type of document they occur, for example, will, trust, contract, contract type, end-user license agreement, etc. execution continues at block 330.
Block 330 entails linking or clustering the identified phrases into equivalence classes or families. In the exemplary embodiment, phrase linkage occurs when two or more phrases are similar enough to represent the same phrase. Several techniques can be utilized for linking including simple, complete and average linkage. Additionally, an exemplary embodiment uses external corpus/resources such as Wikipedia, Westlaw® Headnotes with KeyNumbers, Black's Law dictionary (corpus of conceptual entities) to determine the linkage between two or more phrases.
More particularly, the exemplary embodiment uses an algorithm that looks at the words in the phrase and associates the phrase with a concept. It then associates the phrase with a surrogate identifier, trying to match that with other surrogate identifiers that it has found in Wikipedia, for example. If two of the phrases that have not otherwise been linked are found to have a common concept according to the external resource, then an assumption is made that the two phrases are sufficiently similar to be linked or clusters into the same equivalence class. When phrase linkage is complete, execution proceeds to block 340.
Block 340 entails grouping or clustering the phrase clusters or equivalence classes based on phrase co-occurrence into larger phrase groupings. The exemplary embodiment deems phrase co-occurrence to occur when sets of equivalent classes show up together frequently enough, that is within a threshold number of documents together. Exemplary thresholds include support thresholds, confidence thresholds, strength, and interestingness even when you don't know what you are searching for. An additional exemplary embodiment includes the method of frequent item-set mining (FIM). It is important to note that proximity between phrases does not matter nor does order for this step. However, phrase proximity plays a role in defining and utilizing super-phrases.
Block 350 entails defining a set of clauses (super phrases or uber phrases) based on phrase proximity and uniqueness. In the exemplary embodiment, a super-phrase (uberphrase) is a collection of text areas that are associated with a phrase set. More particularly, a superphrase is a region of text bounded by two phrases in close proximity.
One exemplary embodiment uses association rules mining (ARM) to find associations within a set of equivalence classes. The result is inference rules—rules to arrive at inferences from a set of clauses, and make statements about those clauses. For example, the inference rules say 90% of the contracts clauses also had these clauses as well. This embodiment also provides recommendation rules based on inference rules to be used in risk-avoidance recommendations and drafting recommendations given the proximity and uniqueness constraints. These recommendation rules could provide for benchmarking or best practices standards to compare to user's work product or even a checklist for certain documents. If high thresholds of certain sets are met, a comparison of those sets to what user has drafted in order to provide recommendations. Other exemplary embodiments for determining and generating super-phrases/uberphrases include tf-idf, indexing, clustering.
The parameters for system could be user defined or a machine learning technique could be used to automatically determine the optimal parameters. Another feature allows for suffix arrays as a data structure choice when given large sequences of text to quickly access substrings. The suffix array of a string can be used as an index to quickly locate every occurrence of a substring within the string. Finding every occurrence of the substring is equivalent to finding every suffix that begins with the substring. Thanks to the lexicographical ordering, these suffixes are grouped together in the suffix array, and can be found efficiently with a binary search.
When block 350 finishes, an index of phrases, phrase clusters, groups of phrase clusters, and superphrases (or clauses) is available for use. In this index or data structure, each of the phrases, groups, and superphrases is associated not only with a document identifier and document positional information for the associated text, but also metadata regarding the origin, authors, law firms, dates, jurisdictions, type of document (will, real estate agreement, mergers and acquisition agreement, confidentiality agreement, license agreement, etc.) (Some embodiments tag one or more of the phrases or clauses from the document corpus as good or bad (valid or invalid). This tagging may be done automatically based on a classifier (for example, an support vector machine) trained with known good or bad phrases or clauses, or manually by legal experts.)
Block 360 entails receiving user text input. In the exemplary embodiment, the input text is presented from an active edit window of a word processing application such as the Word or WordPerfect application. However, in some embodiments, the input is a set of documents, such as agreements, from a law firm. In other embodiments, the input takes the form of a single agreement or clause or paragraph of an agreement. In some embodiments, the input may be provided by a browser application. For example, a user may encounter an end-user license agreement (EULA) and an application program interface may automatically or with specific user direction identify the EULA as input. Exemplary execution continues at block 370.
Block 370 entails processing the input text against the index structure created using process blocks 310-350. In the exemplary embodiment, this entails essentially processing the input text using blocks 310-350 to identify phrases, phrase groups, and superphrases and then identifying a ranked list of most similar phrases, phrase groups, and superphrases from the index structure. At this point, corresponding documents, phrases, and associated meta data for the most similar phrases (based on one or more computed similarity metrics) will be available.
Block 380 entails outputting results of the processing in block 370. In the exemplary embodiment, the results can take a variety for forms, such as listing of similar clauses or phrases and associated metadata; a listing of links for a set of documents having phrases or clauses most similar to the put one; and/or an annotated version of the input text indicating portions in conformity with known standard phrases as well as clauses that have deviated from such norms. In some embodiments, the norms may be a specific set of documents or clauses that have for example preapproved for use by law department for a given corporation or by a given law firm.
FIGS. 5-9 show a more conceptual version of an exemplary phrase discovery methodology. FIG. 5 illustrates building a unique word list from any corpus of documents. In creating this word list, any standard search indexing techniques such as stemming, synonyms, etc. may be used.
FIG. 6 represents an act of detecting common phrases. Phrase detection uses common word sequences appearing in two or more documents with defined proximity to generate candidate phrase lists. Phrase detection rules combine parsers for filtering, comparing and scoring candidate phrases. The rules may be adjusted for different document domains and include minimum number and/or percentage of matching words and maximum number or percentage of unmatched or stop words. In addition, some embodiments take into consideration word alternatives in phrases. Within a phrase some words or shorter phrases may appear and are identified as common word alternatives. Given the basic rules that the user has inputted, common phrases are discovered. Once these common phrases are recognized, a phrase list is created and the common phrases are saved within the list. Then phrase matching compares phrase lists across multiple documents to identify common phrases.
FIG. 7 shows an act of grouping or clustering common phrases. Since phrases may be overlapping, phrase group rules identify individual and phrase groups. Some embodiments allow for non-overlapping clusters, whereas others allow for overlapping clusters.
Once the common phrases are found and grouped, the fourth step (FIG. 8) includes grouping common clauses or super phrases. Rules are applied to further improve precision and recall and to group into clauses. For example, in 10,000 documents two phrases were always near each other, but in the 100 documents where only one of the phrases appeared, it had a low score and manual review showed they were not good matches. Therefore, those phrases are not considered to be like phrases.
FIG. 9 shows that after grouping, the exemplary method identifies the names of clauses. At this point, a solid collection of clauses in, for example—merger agreements—can be used as an index file or profile for future merger agreement comparison. This set of profiles simply improves performance and reduces processing time; however the option to choose a set and compare documents is still available.
Now a profile of the document set has been created. The profiles are saved and in future processing, the user can utilize just this profile as a “document”, and/or utilize a set of preclassified profiles to train a classifier that categorizes a given document or clause. In addition, a profile can be used to “name” clauses manually for a document type, and deconstruct other documents based off of that set.
The following factors may be taken into account when defining phrase, clause or super phrase:
    • # of matching words
    • # of unmatched words between words
    • Order of matching words
    • Distinguishing words
    • Length of phrase
      • And comparative length to matches
    • # of documents containing phrase
      • E.g. 10,000 merger agreements processed, only 2 files contain a certain phrase, it is likely not a phrase.
    • Proximity of phrases
    • Frequency of phrase groups in super phrase
Scoring defaults can be set for document types and are always manually and programmatically adjustable. Some sample scoring metrics are listed:
    • Length of phrase
    • # of words matched
    • # of distinguishing words
    • Order of words
    • # of documents matched
    • # of words across documents
    • Proximity to other typically associated phrases (e.g. a sample set of 10,000 merger agreements, 9,000 have phrases A & B within 2 paragraphs of each other. While processing a new merger agreement, we discover phrases similar to A&B within 2 paragraphs of each other, so these are almost certainly variations of A&B).
      • Order of words can also be tabulated in a similarly
    • Percent of total text in paragraph that matches phrases.
      Those of skill will appreciate that the exemplary embodiment in a sense provides three types of parsers: Phrase Detection (Identification and matching based on word commonality, proximity and uniqueness); Phrase Grouping (phrase groups based on phrase commonality); and Uberphrases (phrase sets based on phrase group commonality and potentially proximity)
Once blocks 160 have been executed for a corpus of documents, for example, from a law firm or from across multiple law firms, such in a collection of agreement for an online legal research service, one can repeat the methodology for a given document and identify related phrases and clauses and documents from the phrases using similarity metrics. Also, one may simply run a query against the phrase families, phrase family groupings, and/or phrase groupings to identity similar phrases as well as their corresponding documents and associated meta data, such as jurisdiction, location of author, date of addition to the corpus, etc. Indeed, if the corpus is assembled based on user submission, user data such as attorney name, state, jurisdiction, licensure, experience, and so forth can become part of the metadata associated with each document and thus each phrase or clause it contains, enabling one to discern likely origin or pedigree of phrases or clauses as well as popularity or rarity and other metrics.
Exemplary Interface for Controlling Phrase Discovery
Given the variety of use scenarios, various embodiments of systems incorporating the present invention provide a graphical user interface for defining or controlling user preferences regarding operation of a phrase processing engine.
FIG. 10 shows an exemplary user preference interface for controlling phrase functions in some embodiments of the present invention. Some of these results include highlighting of negotiated/deal specific text versus boilerplate versus other (e.g. statutory), indicating degrees of variance, providing summary of risks for document overall and specific clauses, suggesting alternative clauses/additional clauses based on degrees of similarity or dissimilarity, and suggesting relevant caselaw/statutes and other materials. In addition, these results provide dynamic document analysis and form creation.
CONCLUSION
The embodiments described above are intended only to illustrate and teach one or more ways of practicing or implementing the present invention, not to restrict its breadth or scope. The actual scope of the invention, which embraces all ways of practicing or implementing the teachings of the invention, is defined only by the following claims and their equivalents.

Claims (25)

What is claimed is:
1. A computer-implemented method of classifying text in documents comprising:
electronically receiving for storing in a memory a set of electronic documents each comprising electronic text;
parsing by a processor in electrical communication with the memory the electronic text from each of the set of electronic documents in a plurality of phrases or clauses;
associating each of the set of electronic documents with a transactional document type;
associating at least a set of the plurality of phrases or clauses with a clause type classification, the clause type classification being associated with one or more transactional document type; and
generating by the processor a reference set of clauses for each clause type classification and organizing the reference set of clauses into a clause index structure, whereby the reference set of clauses is subsequently used in processing user inputs, the processing user inputs comprising:
receiving at a server a set of user input electronic text input by a user via a remote user device over a network;
parsing by the server the set of user input electronic text to yield a set of user input phrases or clauses;
comparing by the server the set of user input phrases or clauses to the reference set of clauses, the comparing the set of user input phrases or clauses including computing a set of similarity metrics;
identifying a set of one or more match clauses from the reference set of clauses, the set of match clauses determined to be similar to one or more of the set of user input phrases or clauses based at least in part on the computed set of similarity metrics; and
generating for output to the remote user device a signal representing the set of match clauses.
2. The method of claim 1, further comprising:
identifying for association with a clause type classification or a transactional document type one or more of the set of user input phrases or clauses based on the comparison.
3. The method of claim 1, wherein computing a set of similarity metrics comprises:
computing a first similarity metric based on a determined similarity between a first one of the set of user input phrases or clauses and a first one of the reference set of clauses; and
computing a second similarity metric based on a determined similarity between the first one of the set of user input phrases or clauses and a second one of the reference set of clauses.
4. The method of claim 3, further comprising ranking the first and second ones of the reference set of clauses based on the first and second similarity metrics.
5. The method of claim 4, further comprising outputting a ranked listing of the first and second ones of the reference set of clauses.
6. The method of claim 1, wherein the transactional document type corresponds with at least one of a set of legal agreement types.
7. The method of claim 1, further comprising flagging one or more of the set of user input phrases or clauses as deviating from or conforming with one or more phrases or clauses from the reference set of clauses.
8. A system comprising a processor adapted to access and process machine readable instructions stored in a memory and access data stored in a database, the system in electronic communication with a network and further comprising:
an input adapted to receive a set of electronic documents each comprising electronic text;
a parser executed by the processor to parse the electronic text from each of the set of electronic documents and yield a plurality of phrases or clauses;
a classifier executed by the processor to associate each of the set of electronic documents with a transactional document type
and associate at least a set of the plurality of phrases or clauses with a clause type classification, the clause type classification being associated with one or more transactional document type;
a clause index structure stored in a database accessible by the processor and comprising a reference set of clauses for each clause type classification
the input further adapted to receive a set of user input electronic text from a remote user device via a network in electrical communication with the system;
the parser further executed by the processor to parse the set of user input electronic text to yield a set of user input phrases or clauses;
the processor configured to access from the database the clause index structure and to access from memory and execute a set of code adapted to:
compare the set of user input phrases or clauses to the reference set of clauses: compute a set of similarity metrics: and identify a set of one or more match clauses from the reference set of clauses based at least in part on the computed set of similarity metrics; and
a signal generated by the processor and representing the set of match clauses; and
an output in electrical communication with the network and adapted to output the generated signal via the network for intended receipt at the remote user device.
9. The system of claim 8, wherein the transactional document type corresponds with at least one of a set of legal agreement types.
10. The system of claim 8, wherein the processor adapted to access and process machine readable instructions stored in the memory and access data stored in the database comprises one or more processors operatively coupled to one or more memory devices, wherein each of the memory devices includes one or more machine executable instructions sets for execution by the processor.
11. The system of claim 10, wherein one or more of the processor resides on a web server operatively coupled to a legal research database and one or more of the processors resides on a client access device communicatively coupled to the server via a wide or local area network.
12. A transactional document drafting system implemented by a computer comprising a processor adapted to process a set of instructions stored in a memory, the system comprising:
an input adapted to receive from a user an input representing an identified transactional document type of interest to the user;
a first set of instructions executed by the processor to present via a user display a clause index structure representing a library of clauses, the clause index structure and library of clauses being associated with the identified transactional document type, the library of clauses comprised of clauses previously identified from a reference set of documents, and wherein the library of clauses is organized based on a clause type classification and the clause index structure presents the library of clauses based at least in part on the clause type classification;
upon receiving a signal representing a user selection of a clause from the presented library of clauses, a second set of instructions executed by the processor to present for display the selected clause from the library of clauses;
a user interface adapted to insert the selected clause into an electronic document active in a word processing application operating on a user computing device;
the input further adapted to receive the reference set of documents;
a parser executed by the processor to parse electronic text from each of the reference set of documents to identify a set of phrases or clauses;
the processor adapted to process a set of instructions stored in the memory to compare identified phrases or clauses from a first of the reference set of documents with phrases or clauses identified from one or more of the other reference set of documents and to determine a set of default clauses corresponding, respectively, to a set of clause classification types based on the comparison.
13. The system of claim 12 being operated at least in part by a web-based data service connected to a user operated device over an Internet connection.
14. The system of claim 12 being operated locally at a user operated device.
15. The system of claim 12, wherein the second set of instructions comprises instructions executed by the processor to present for display an alternative clause related to the selected clause and instructions allowing a user to select for incorporation in an electronic document either one of the selected clause and alternative clause.
16. The system of claim 12, wherein the identified transactional document type is one from the group consisting of: will, trust, contract, contract type, license agreement, sales agreement, confidentiality agreement, merger agreement, acquisition agreement.
17. The system of claim 12, wherein each of the clauses comprising the library of clauses comprise at least one phrase identified in the reference set of documents.
18. The system of claim 12, wherein the processor is further adapted to process a set of instructions stored in the memory to determine a set of alternative clauses corresponding, respectively, to the set of clause classification types based on the comparison, the alternative clauses being different than the default clauses.
19. A computer-implemented method for drafting transactional documents, the method comprising:
receiving from a user an input representing an identified transactional document type of interest to the user;
executing by a processor a first set of instructions stored in a memory accessible by the processor to present to a user a clause index structure representing a library of clauses, the clause index structure representing a library of clauses being associated with the identified transactional document type, the library of clauses comprised of clauses previously identified from a reference set of documents, and wherein the library of clauses is organized based on a clause type classification and the clause index structure presents the library of clauses based at least in part on the clause type classification;
receiving a signal representing a user selection of a clause from the presented library of clauses;
executing a second set of instructions to present the selected clause from the library of clauses;
incorporating the selected clause into an electronic document active in a word processing application; and
receiving the reference set of documents;
parsing electronic text from each of the reference set of documents to identify a set of phrases or clauses;
comparing identified phrases or clauses from a first of the reference set of documents with phrases or clauses identified from one or more of the other reference set of documents; and
determining a set of default clauses corresponding, respectively, to a set of clause classification types based on the comparison.
20. The method of claim 19 being operated at least in part by a web-based data service connected to a user operated device over an Internet connection.
21. The method of claim 19 being operated locally at a user operated device.
22. The method of claim 19, wherein the second set of instructions comprises instructions executed to present an alternative clause related to the selected clause and instructions allowing a user to select for incorporation in an electronic document either one of the selected clause and alternative clause.
23. The method of claim 19, wherein the identified transactional document type is one from the group consisting of: will, trust, contract, contract type, license agreement, sales agreement, confidentiality agreement, merger agreement, acquisition agreement.
24. The method of claim 19, wherein each of the clauses comprising the library of clauses comprise at least one phrase identified in the reference set of documents.
25. The method of claim 19, further comprising determining a set of alternative clauses corresponding, respectively, to the set of clause classification types based on the comparison, the alternative clauses being different than the default clauses.
US12/553,752 2008-01-15 2009-09-03 Systems, methods and software for processing phrases and clauses in legal documents Active 2030-08-31 US8788523B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US12/553,752 US8788523B2 (en) 2008-01-15 2009-09-03 Systems, methods and software for processing phrases and clauses in legal documents
US14/337,105 US9811518B2 (en) 2008-01-15 2014-07-21 Systems, methods, and software for questionbased sentiment analysis and summarization

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US1114708P 2008-01-15 2008-01-15
US35461709A 2009-01-15 2009-01-15
US12/553,752 US8788523B2 (en) 2008-01-15 2009-09-03 Systems, methods and software for processing phrases and clauses in legal documents

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US35461709A Continuation 2008-01-15 2009-01-15

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/337,105 Continuation US9811518B2 (en) 2008-01-15 2014-07-21 Systems, methods, and software for questionbased sentiment analysis and summarization

Publications (2)

Publication Number Publication Date
US20110055206A1 US20110055206A1 (en) 2011-03-03
US8788523B2 true US8788523B2 (en) 2014-07-22

Family

ID=43626368

Family Applications (2)

Application Number Title Priority Date Filing Date
US12/553,752 Active 2030-08-31 US8788523B2 (en) 2008-01-15 2009-09-03 Systems, methods and software for processing phrases and clauses in legal documents
US14/337,105 Active US9811518B2 (en) 2008-01-15 2014-07-21 Systems, methods, and software for questionbased sentiment analysis and summarization

Family Applications After (1)

Application Number Title Priority Date Filing Date
US14/337,105 Active US9811518B2 (en) 2008-01-15 2014-07-21 Systems, methods, and software for questionbased sentiment analysis and summarization

Country Status (1)

Country Link
US (2) US8788523B2 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150112877A1 (en) * 2008-01-15 2015-04-23 Frank Schilder Systems, methods, and software for questionbased sentiment analysis and summarization
US20160217208A1 (en) * 2013-05-09 2016-07-28 Lexisnexis, A Division Of Reed Elsevier Inc. Systems and methods for generating issue networks
US9454518B1 (en) * 2014-07-01 2016-09-27 Designation Station, LLC Automated processing of transcripts with transcript designations
US10013407B2 (en) 2014-07-01 2018-07-03 Designation Station, LLC Automated processing of transcripts, transcript designations, and/or video clip load files
US10140263B2 (en) 2014-06-06 2018-11-27 Maud GAGNÉ-LANGEVIN System and method for generating task-embedded documents
US10216715B2 (en) 2015-08-03 2019-02-26 Blackboiler Llc Method and system for suggesting revisions to an electronic document
US10515149B2 (en) 2018-03-30 2019-12-24 BlackBoiler, LLC Method and system for suggesting revisions to an electronic document
WO2020118448A1 (en) * 2018-12-13 2020-06-18 Element Ai Inc. System and method for managing rights to assets
EP3680850A1 (en) * 2019-01-11 2020-07-15 Sirionlabs Method and system for determining risk score for a contract document
US10776891B2 (en) 2017-09-29 2020-09-15 The Mitre Corporation Policy disruption early warning system
US11392769B2 (en) 2020-07-15 2022-07-19 Fmr Llc Systems and methods for expert driven document identification
US11436286B1 (en) 2019-04-04 2022-09-06 Otsuka America Pharmaceutical, Inc. System and method for using deconstructed document sections to generate report data structures
US11663410B2 (en) 2021-02-17 2023-05-30 Kyndryl, Inc. Online terms of use interpretation and summarization
US11681864B2 (en) 2021-01-04 2023-06-20 Blackboiler, Inc. Editing parameters
US20230298368A1 (en) * 2022-02-28 2023-09-21 Docusign, Inc. Agreement document model modifications in a document management system
US11954135B2 (en) 2022-09-13 2024-04-09 Briefcatch, LLC Methods and apparatus for intelligent editing of legal documents using ranked tokens

Families Citing this family (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9137238B1 (en) * 2010-08-06 2015-09-15 RightQuestions, LLC Pass-sequences
EP2551810A1 (en) * 2011-07-26 2013-01-30 Tata Consultancy Services Limited Method and system for STP linking the variances in corporation action events received from multiple agents
US9305082B2 (en) * 2011-09-30 2016-04-05 Thomson Reuters Global Resources Systems, methods, and interfaces for analyzing conceptually-related portions of text
US9171069B2 (en) * 2012-07-31 2015-10-27 Freedom Solutions Group, Llc Method and apparatus for analyzing a document
US11232137B2 (en) * 2012-12-18 2022-01-25 RELX Inc. Methods for evaluating term support in patent-related documents
WO2014100202A1 (en) 2012-12-18 2014-06-26 Lexisnexis, A Division Of Reed Elsevier Inc. Systems and methods for patent-related document analysis and searching
US10191893B2 (en) 2013-07-22 2019-01-29 Open Text Holdings, Inc. Information extraction and annotation systems and methods for documents
US8856642B1 (en) 2013-07-22 2014-10-07 Recommind, Inc. Information extraction and annotation systems and methods for documents
US20160171576A1 (en) * 2013-08-22 2016-06-16 Auction.Com, Llc Functionally interrelated user interfaces for conducting transactions
EP3072063A4 (en) * 2013-11-18 2017-11-01 Recommind, Inc. Information extraction and annotation for documents
US9251139B2 (en) * 2014-04-08 2016-02-02 TitleFlow LLC Natural language processing for extracting conveyance graphs
US10347240B2 (en) * 2015-02-26 2019-07-09 Nantmobile, Llc Kernel-based verbal phrase splitting devices and methods
EP3355261A4 (en) * 2015-09-25 2019-04-03 NEC Patent Service,Ltd. Information-processing device, information-processing method, and program-recording medium
US10511653B2 (en) * 2015-10-12 2019-12-17 Roman KISIN Discussion-based document collaboration
US10628522B2 (en) * 2016-06-27 2020-04-21 International Business Machines Corporation Creating rules and dictionaries in a cyclical pattern matching process
US10762142B2 (en) 2018-03-16 2020-09-01 Open Text Holdings, Inc. User-defined automated document feature extraction and optimization
US11048762B2 (en) 2018-03-16 2021-06-29 Open Text Holdings, Inc. User-defined automated document feature modeling, extraction and optimization
AU2018271315A1 (en) * 2018-05-14 2019-11-28 Chen-McCaig, Zack MR Document processing and classification systems
US10467344B1 (en) 2018-08-02 2019-11-05 Sas Institute Inc. Human language analyzer for detecting clauses, clause types, and clause relationships
US10452734B1 (en) 2018-09-21 2019-10-22 SSB Legal Technologies, LLC Data visualization platform for use in a network environment
US10915710B2 (en) 2018-09-27 2021-02-09 International Business Machines Corporation Clause analysis based on collection coherence in legal domain
US10311140B1 (en) * 2018-10-25 2019-06-04 BlackBoiler, LLC Systems, methods, and computer program products for a clause library
US11610277B2 (en) 2019-01-25 2023-03-21 Open Text Holdings, Inc. Seamless electronic discovery system with an enterprise data portal
US11132386B2 (en) * 2019-02-15 2021-09-28 International Business Machines Corporation Fast linking of anonymized datasets
US10565445B1 (en) 2019-04-11 2020-02-18 Capital One Services, Llc Systems and methods for comparing legal clauses
US20210090190A1 (en) * 2019-06-05 2021-03-25 Orula Inc. Data management system for use with agreements and data detailing concepts, designs, and ideas
GB201911760D0 (en) * 2019-08-16 2019-10-02 Eigen Tech Ltd Training and applying structured data extraction models
US11886814B2 (en) 2020-01-24 2024-01-30 Thomson Reuters Enterprise Centre Gmbh Systems and methods for deviation detection, information extraction and obligation deviation detection
US11688027B2 (en) * 2020-03-20 2023-06-27 Codexo Generating actionable information from documents
US11625421B1 (en) * 2020-04-20 2023-04-11 GoLaw LLC Systems and methods for generating semantic normalized search results for legal content
US20220012830A1 (en) * 2020-07-10 2022-01-13 MounTavor, Inc. Method and system for automatic analysis of legal documents using sequence alignemnt
US20220335075A1 (en) * 2021-04-14 2022-10-20 International Business Machines Corporation Finding expressions in texts
US20220358608A1 (en) * 2021-05-07 2022-11-10 Jeff Schwartz System and method for processing discovery documents in a legal industry
US11361151B1 (en) 2021-10-18 2022-06-14 BriefCatch LLC Methods and systems for intelligent editing of legal documents
US20230350954A1 (en) * 2022-05-02 2023-11-02 SparkCognition, Inc. Systems and methods of filtering topics using parts of speech tagging

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6014680A (en) * 1995-08-31 2000-01-11 Hitachi, Ltd. Method and apparatus for generating structured document
US20030023527A1 (en) * 2001-07-27 2003-01-30 Wilce Scot D. Systems and methods for facilitating agreement generation and negotiation via an agreement modeling system
US6869018B2 (en) * 2000-07-31 2005-03-22 Reallegal, Llc Transcript management software and methods therefor
US6996520B2 (en) * 2002-11-22 2006-02-07 Transclick, Inc. Language translation system and method using specialized dictionaries
US20060106837A1 (en) * 2002-11-26 2006-05-18 Eun-Jeong Choi Parsing system and method of multi-document based on elements
US20060121459A1 (en) * 2002-02-20 2006-06-08 Incyte Corporation Receptors and membrane-associated proteins
US20060149720A1 (en) * 2004-12-30 2006-07-06 Dehlinger Peter J System and method for retrieving information from citation-rich documents
US20070282824A1 (en) * 2006-05-31 2007-12-06 Ellingsworth Martin E Method and system for classifying documents
US20080077570A1 (en) * 2004-10-25 2008-03-27 Infovell, Inc. Full Text Query and Search Systems and Method of Use
WO2008052239A1 (en) * 2006-11-03 2008-05-08 Appen Pty Limited Email document parsing method and apparatus
US20080133444A1 (en) * 2006-12-05 2008-06-05 Microsoft Corporation Web-based collocation error proofing
US7437408B2 (en) * 2000-02-14 2008-10-14 Lockheed Martin Corporation Information aggregation, processing and distribution system
US20080270345A1 (en) * 2007-04-27 2008-10-30 Jeff Barlow System and method for using indexes to parse static XML documents
US20090083208A1 (en) * 2006-03-15 2009-03-26 Raghavan Vijay V System, method, and computer program product for data mining and automatically generating hypotheses from data repositories
US7519607B2 (en) * 2002-08-14 2009-04-14 Anderson Iv Robert Computer-based system and method for generating, classifying, searching, and analyzing standardized text templates and deviations from standardized text templates
US20090204596A1 (en) * 2008-02-08 2009-08-13 Xerox Corporation Semantic compatibility checking for automatic correction and discovery of named entities
US20090294529A1 (en) * 1996-07-15 2009-12-03 Messina Kevin M Parsing an identification document in accordance with a jurisdictional format
US20130311490A1 (en) * 2009-01-02 2013-11-21 Apple Inc. Efficient Data Structures for Parsing and Analyzing a Document

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5794236A (en) * 1996-05-29 1998-08-11 Lexis-Nexis Computer-based system for classifying documents into a hierarchy and linking the classifications to the hierarchy
JP3166689B2 (en) * 1997-11-18 2001-05-14 富士ゼロックス株式会社 Thesaurus retrieval and synthesis system
US6430581B1 (en) * 1998-04-10 2002-08-06 Pitney Bowes Inc. Automated court document docketing filing system
US6553385B2 (en) * 1998-09-01 2003-04-22 International Business Machines Corporation Architecture of a framework for information extraction from natural language documents
JP4732593B2 (en) * 1999-05-05 2011-07-27 ウエスト パブリッシング カンパニー Document classification system, document classification method, and document classification software
US6591261B1 (en) * 1999-06-21 2003-07-08 Zerx, Llc Network search engine and navigation tool and method of determining search results in accordance with search criteria and/or associated sites
CA2381460A1 (en) * 1999-08-06 2001-02-15 James S. Wiltshire, Jr. System and method for classifying legal concepts using legal topic scheme
US6772149B1 (en) * 1999-09-23 2004-08-03 Lexis-Nexis Group System and method for identifying facts and legal discussion in court case law documents
US7028259B1 (en) * 2000-02-01 2006-04-11 Jacobson Robert L Interactive legal citation checker
US6429666B1 (en) * 2000-04-17 2002-08-06 Sentronics Corporation Capacitive circuit array for fingerprint sensing
KR100548983B1 (en) * 2000-11-02 2006-02-02 (주)마크텍 Computer system and method for verifying the authenticity of digital documents
US6662178B2 (en) * 2001-03-21 2003-12-09 Knowledge Management Objects, Llc Apparatus for and method of searching and organizing intellectual property information utilizing an IP thesaurus
IL142421A0 (en) * 2001-04-03 2002-03-10 Linguistic Agents Ltd Linguistic agent system
AUPR701701A0 (en) * 2001-08-14 2001-09-06 Mcdonald, Nathan Document analysis system and method
GB2380109A (en) * 2001-09-21 2003-03-26 Hewlett Packard Co Apparatus for interpreting electronic legal documents
US7333966B2 (en) * 2001-12-21 2008-02-19 Thomson Global Resources Systems, methods, and software for hyperlinking names
US20040193596A1 (en) * 2003-02-21 2004-09-30 Rudy Defelice Multiparameter indexing and searching for documents
US7610313B2 (en) * 2003-07-25 2009-10-27 Attenex Corporation System and method for performing efficient document scoring and clustering
US20050203924A1 (en) * 2004-03-13 2005-09-15 Rosenberg Gerald B. System and methods for analytic research and literate reporting of authoritative document collections
US20050210048A1 (en) * 2004-03-18 2005-09-22 Zenodata Corporation Automated posting systems and methods
US8289541B2 (en) * 2006-09-12 2012-10-16 Morgan Stanley Document handling
US8788523B2 (en) * 2008-01-15 2014-07-22 Thomson Reuters Global Resources Systems, methods and software for processing phrases and clauses in legal documents

Patent Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6014680A (en) * 1995-08-31 2000-01-11 Hitachi, Ltd. Method and apparatus for generating structured document
US20090294529A1 (en) * 1996-07-15 2009-12-03 Messina Kevin M Parsing an identification document in accordance with a jurisdictional format
US20120024948A1 (en) * 1996-07-15 2012-02-02 Intelli-Check, Inc. Parsing an identification document in accordance with a jurisdictional format
US7437408B2 (en) * 2000-02-14 2008-10-14 Lockheed Martin Corporation Information aggregation, processing and distribution system
US6869018B2 (en) * 2000-07-31 2005-03-22 Reallegal, Llc Transcript management software and methods therefor
US20030023527A1 (en) * 2001-07-27 2003-01-30 Wilce Scot D. Systems and methods for facilitating agreement generation and negotiation via an agreement modeling system
US20060121459A1 (en) * 2002-02-20 2006-06-08 Incyte Corporation Receptors and membrane-associated proteins
US7519607B2 (en) * 2002-08-14 2009-04-14 Anderson Iv Robert Computer-based system and method for generating, classifying, searching, and analyzing standardized text templates and deviations from standardized text templates
US8285750B2 (en) * 2002-08-14 2012-10-09 Document Analytic Technologies, Llc Computer-based system and method for generating, classifying, searching, and analyzing standardized text templates and deviations from standardized text templates
US6996520B2 (en) * 2002-11-22 2006-02-07 Transclick, Inc. Language translation system and method using specialized dictionaries
US20060106837A1 (en) * 2002-11-26 2006-05-18 Eun-Jeong Choi Parsing system and method of multi-document based on elements
US20080077570A1 (en) * 2004-10-25 2008-03-27 Infovell, Inc. Full Text Query and Search Systems and Method of Use
US20060149720A1 (en) * 2004-12-30 2006-07-06 Dehlinger Peter J System and method for retrieving information from citation-rich documents
US20090083208A1 (en) * 2006-03-15 2009-03-26 Raghavan Vijay V System, method, and computer program product for data mining and automatically generating hypotheses from data repositories
US20070282824A1 (en) * 2006-05-31 2007-12-06 Ellingsworth Martin E Method and system for classifying documents
WO2008052239A1 (en) * 2006-11-03 2008-05-08 Appen Pty Limited Email document parsing method and apparatus
US20080133444A1 (en) * 2006-12-05 2008-06-05 Microsoft Corporation Web-based collocation error proofing
US20080270345A1 (en) * 2007-04-27 2008-10-30 Jeff Barlow System and method for using indexes to parse static XML documents
US20090204596A1 (en) * 2008-02-08 2009-08-13 Xerox Corporation Semantic compatibility checking for automatic correction and discovery of named entities
US20130311490A1 (en) * 2009-01-02 2013-11-21 Apple Inc. Efficient Data Structures for Parsing and Analyzing a Document

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150112877A1 (en) * 2008-01-15 2015-04-23 Frank Schilder Systems, methods, and software for questionbased sentiment analysis and summarization
US9811518B2 (en) * 2008-01-15 2017-11-07 Thomson Reuters Global Resources Systems, methods, and software for questionbased sentiment analysis and summarization
US20160217208A1 (en) * 2013-05-09 2016-07-28 Lexisnexis, A Division Of Reed Elsevier Inc. Systems and methods for generating issue networks
US9940389B2 (en) * 2013-05-09 2018-04-10 Lexisnexis, A Division Of Reed Elsevier, Inc. Systems and methods for generating issue networks
US20180189386A1 (en) * 2013-05-09 2018-07-05 Lexisnexis, A Division Of Reed Elsevier, Inc. Systems and methods for generating issue networks
US10474710B2 (en) * 2013-05-09 2019-11-12 RELX, Inc. Systems and methods for generating issue networks
US10140263B2 (en) 2014-06-06 2018-11-27 Maud GAGNÉ-LANGEVIN System and method for generating task-embedded documents
US9454518B1 (en) * 2014-07-01 2016-09-27 Designation Station, LLC Automated processing of transcripts with transcript designations
US10013407B2 (en) 2014-07-01 2018-07-03 Designation Station, LLC Automated processing of transcripts, transcript designations, and/or video clip load files
US10902187B2 (en) 2014-07-01 2021-01-26 Designation Station, LLC Automated processing of transcripts, transcript designations, and/or video clip load files
US10229098B2 (en) 2014-07-01 2019-03-12 Designation Station, LLC Automated processing of transcripts, transcript designations, and/or video clip load files
US10216715B2 (en) 2015-08-03 2019-02-26 Blackboiler Llc Method and system for suggesting revisions to an electronic document
US11630942B2 (en) 2015-08-03 2023-04-18 Blackboiler, Inc. Method and system for suggesting revisions to an electronic document
US11093697B2 (en) 2015-08-03 2021-08-17 Blackboiler, Inc. Method and system for suggesting revisions to an electronic document
US10489500B2 (en) 2015-08-03 2019-11-26 Blackboiler Llc Method and system for suggesting revisions to an electronic document
US10970475B2 (en) 2015-08-03 2021-04-06 Blackboiler, Inc. Method and system for suggesting revisions to an electronic document
US10824797B2 (en) 2015-08-03 2020-11-03 Blackboiler, Inc. Method and system for suggesting revisions to an electronic document
US10776891B2 (en) 2017-09-29 2020-09-15 The Mitre Corporation Policy disruption early warning system
US10713436B2 (en) 2018-03-30 2020-07-14 BlackBoiler, LLC Method and system for suggesting revisions to an electronic document
US11244110B2 (en) 2018-03-30 2022-02-08 Blackboiler, Inc. Method and system for suggesting revisions to an electronic document
US10515149B2 (en) 2018-03-30 2019-12-24 BlackBoiler, LLC Method and system for suggesting revisions to an electronic document
US11709995B2 (en) 2018-03-30 2023-07-25 Blackboiler, Inc. Method and system for suggesting revisions to an electronic document
WO2020118448A1 (en) * 2018-12-13 2020-06-18 Element Ai Inc. System and method for managing rights to assets
EP3680850A1 (en) * 2019-01-11 2020-07-15 Sirionlabs Method and system for determining risk score for a contract document
US11436286B1 (en) 2019-04-04 2022-09-06 Otsuka America Pharmaceutical, Inc. System and method for using deconstructed document sections to generate report data structures
US11392769B2 (en) 2020-07-15 2022-07-19 Fmr Llc Systems and methods for expert driven document identification
US11681864B2 (en) 2021-01-04 2023-06-20 Blackboiler, Inc. Editing parameters
US11663410B2 (en) 2021-02-17 2023-05-30 Kyndryl, Inc. Online terms of use interpretation and summarization
US20230298368A1 (en) * 2022-02-28 2023-09-21 Docusign, Inc. Agreement document model modifications in a document management system
US11954135B2 (en) 2022-09-13 2024-04-09 Briefcatch, LLC Methods and apparatus for intelligent editing of legal documents using ranked tokens

Also Published As

Publication number Publication date
US20110055206A1 (en) 2011-03-03
US9811518B2 (en) 2017-11-07
US20150112877A1 (en) 2015-04-23

Similar Documents

Publication Publication Date Title
US8788523B2 (en) Systems, methods and software for processing phrases and clauses in legal documents
Verspoor et al. A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools
US20200050638A1 (en) Systems and methods for analyzing the validity or infringment of patent claims
Bagheri et al. Care more about customers: Unsupervised domain-independent aspect detection for sentiment analysis of customer reviews
JP6095621B2 (en) Mechanism, method, computer program, and apparatus for identifying and displaying relationships between answer candidates
Caldarola et al. An approach to ontology integration for ontology reuse
US9542496B2 (en) Effective ingesting data used for answering questions in a question and answer (QA) system
Bhatia et al. Mining privacy goals from privacy policies using hybridized task recomposition
US20140180934A1 (en) Systems and Methods for Using Non-Textual Information In Analyzing Patent Matters
US20160357854A1 (en) Scenario generating apparatus and computer program therefor
Pertile et al. Comparing and combining C ontent‐and C itation‐based approaches for plagiarism detection
Li et al. Bridging semantic gaps between natural languages and APIs with word embedding
US20120035914A1 (en) System and method for handling multiple languages in text
US10606903B2 (en) Multi-dimensional query based extraction of polarity-aware content
CN112231494B (en) Information extraction method and device, electronic equipment and storage medium
Tang et al. Overview of the NTCIR-9 Crosslink Task: Cross-lingual Link Discovery.
US20160253294A1 (en) Inserting links that aid action completion
Hollenstein et al. Inconsistency detection in semantic annotation
Nassif et al. Automatically categorizing software technologies
Solihin et al. Advancement of information extraction use in legal documents
Mirrezaei et al. The triplex approach for recognizing semantic relations from noun phrases, appositions, and adjectives
Janpitak et al. Information security requirement extraction from regulatory documents using GATE/ANNIC
WO2009091917A1 (en) Processing of phrases and clauses in documents
Mariani et al. Reuse and plagiarism in Speech and Natural Language Processing publications
Sundareswara et al. A large-scale exploration of terms of service documents on the web

Legal Events

Date Code Title Description
AS Assignment

Owner name: THOMSON REUTERS GLOBAL RESOURCES, SWITZERLAND

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WEST SERVICES, INC.;REEL/FRAME:023618/0377

Effective date: 20090320

Owner name: WEST SERVICES, INC., MICHIGAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BEROSIK, GARY;ZIELUND, THOMAS;DORR, DIETMAR;AND OTHERS;SIGNING DATES FROM 20090312 TO 20090319;REEL/FRAME:023618/0141

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: THOMSON REUTERS GLOBAL RESOURCES UNLIMITED COMPANY

Free format text: CHANGE OF NAME;ASSIGNOR:THOMSON REUTERS GLOBAL RESOURCES;REEL/FRAME:044264/0597

Effective date: 20161121

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551)

Year of fee payment: 4

AS Assignment

Owner name: THOMSON REUTERS ENTERPRISE CENTRE GMBH, SWITZERLAND

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:THOMSON REUTERS GLOBAL RESOURCES UNLIMITED COMPANY;REEL/FRAME:052056/0198

Effective date: 20200227

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8