The use of quantitative prediction continues to shake up numerous professional services industries by automating or semi-automating tasks previously performed by experts. Professor Daniel Katz (Michigan State University) has offered up an analysis of how quantitative prediction is already changing the legal services industry (Quantitative Legal Prediction – or – How I Learned to Stop Worrying and Start Preparing for the Data Driven Future of the Legal Services Industry, 62 Emory Law Journal ___ (2013) (working draft)). Although Katz’s analysis focuses on legal services, the trends discussed can be applied to other professional service industries, including tax planning and accounting services. Quantitative prediction promises to automate or semi-automate many core questions asked by professionals and their clients: Do I have a case? What is our likely exposure? How much is this going to cost? Are these documents relevant? What will happen if we leave this particular provision out of this contract? How can we best staff this particular legal matter?
Professor Katz explains the technological developments that have lead to the current growth in the use of quantitative prediction and argues that the recent economic downturn, and its associated cost control pressures have further increased the speed at which quantitative prediction solutions have been adopted. He describes these changes as a move toward a business model he terms the “data driven legal practice.” He highlights many areas within the legal industry which have already been moved toward this new model, including an analytics platform offered by a Wolters Kluwer company, TyMetrix, that helps predict an acceptable rate to pay for a given legal service based on industry-wide data. He also highlights examples in the areas of attorney performance management (Lawyer Metrics) and the automation of document review as part of the e-discovery process. However, predicting the outcome of a given case (e.g., whether a case will settle and for how much) remains the challenge offering the most practical value to the industry. Although predictions of case outcomes has been applied to patent disputes (LexMachina), Katz implies similar solutions will be offered for more pedestrian cases as the commercial timing and market appetite become more favorable.
Assuming Professor Katz’s analysis of changes to the professional legal services industry is correct, how might these changes impact companies that provide information to those same professionals? His analysis offers a few possibilities:
- New workflow tools. Whenever the workflow of a professional changes, tools that fit the new workflow will be in demand. Some tools may directly apply quantitative prediction, while others simply integrate probabilistic results into existing workflows. For example, probabilistic results on related material integrated into research workflow (e.g., legal briefs that cite X usually cite Y as well).
- New metadata. The metadata currently captured by information providers from court documents or other legal data is primarily evaluated according to its usefulness in a search context (i.e., will this metadata improve a user’s keyword search or help a user filter their results?) However, the metadata that is most useful for quantitative prediction may differ. For example, the judge or the attorneys involved in a case may be considered of limited use in a search context but may prove to be highly important when used in quantitative prediction. To accommodate the needs of quantitative prediction, providers of data for use in quantitative prediction may need to reevaluate their priorities in the collection of metadata.
- New data. Similar to the reevaluation of metadata, a reevaluation of which sources of data are valuable may also be needed. For example, if court documents are collected to provide explanations of a court’s reasoning, it makes sense to editorially ignore lower court decisions or those issued without an explanation of the court’s reasoning. However, if that same information was being used to feed a predictive model, such exclusions would only make the data incomplete. Similarly, data on court filings, settlements, damage awards, search queries, and other data that may have limited value for substantive research may be increasingly valuable in the context of quantitative prediction.
- Training data/services. Many quantitative predictions rely on algorithms that use a supervised or semi-supervised learning process. In other words, the algorithm is trained and its success is evaluated relative to a subset of data that has been pre-identified or pre-classified by expert reviewers. Further, the size and scope of this training data is often an important component in determining the success of an algorithm’s predictive ability. Traditional providers of legal information would seem to be especially experienced at such document collection and classification tasks.
- Comparison/evaluation services. Professor Katz’s analysis perhaps not surprisingly predicts that legal professionals and legal educators are generally not equipped with the skills that will be necessary to engage in the new model of data driven law practice. This deficiency suggests there may be a demand for trusted third-parties that can compare solutions and evaluate the various strengths and weaknesses of different approaches.
How do you think an increased use of quantitative prediction will change the legal industry and the information and products it consumes?