Abstracts: Bayesian Analysis in Law: Papers presented at Conference: The Many Faces of Contemporary Philosophy and Theory of Law

Abstracts have been posted of papers presented at the Conference: The Many Faces of Contemporary Philosophy and Theory of Law, held 23-24 March 2013, at Jagellonian University, Cracow, Poland. The conference included a special working group on Bayesian analysis in law, abstracts of papers of which begin on page 6 of the abstracts volume and are excerpted below:

Dr Jeroen Keppens: Bayesian Perspectives on the Value of Evidence. Abstract:

Given the interdisciplinary audience, I would like to introduce the Bayesian approach to evidential reasoning in Law. Then I plan to move on the Bayesian modeling techniques and the various concerns and difficulties that arise from it.

Paweł Banaś and Krzysztof Kasparek: Some remarks about controversies concerning applying Bayes theorem to criminal policy-making. Abstract:

The following paper aims at summarizing a discussion concerning the exploitation of Baysesian analysis within criminal policy-making, namely problems with the so called postprison civil commitment of sex offenders as sexually violent predators (SVPs) employed currently in some of the US states. During this process it is determined whether a former convict will be “classified” as SVP. Typically, actuarial instruments are used in order to help decide on this issue. Recently, Richard Wollert has pointed out that exploitation of Bayesian theorem may prove useful in this type of cases when addressing at least some of the questions that may arise. However, his ideas were met with much criticism within risk-assessment community. In this paper we want to present main arguments of both sides of the debate and point to some of the possible problems with Bayesian analysis as used in forensic psychology.

Piotr Bystranowski: Czy da się nauczyć prawników statystyki? Sieci bayesowskie a unikanie błędów probabilistycznych w rozumowaniach prawniczych. Abstract:

Od lat siedemdziesiątych i czasów przełomowych eksperymentów Kahnemana i Tversky’ego powszechnym stało się przekonanie, iż ludzkie osądy w warunkach niepewności często dają rezultaty systematycznie i rażąco niezgodne z regułami matematycznego rachunku prawdopodobieństwa, w tym zwłaszcza z tzw. wzorem Bayesa. Od błędów tego rodzaju nie jest wolna sala sądowa. Przeciwnie – wyniki szeregu procesów karnych pokazują, że wymiar sprawiedliwości jest podatny na wiele błędów w rozumowaniach probabilistycznych (z tzw. złudzeniem prokuratora na czele). Ich skutkiem bywa, na przykład, przypisanie zbyt dużej pewności materiałowi dowodowemu, który z formalnego punktu widzenia zdaje się być dalece nierozstrzygający. Pociąga to za sobą pytanie, w jaki sposób rozwiązać ową ewidentną niezgodność mię-dzy intuicyjnymi rozumowaniami w warunkach niepewności a formalnymi metodami probabilistycznymi. [...] Tych mankamentów zdaje się unikać proponowana przez Normana Fentona i Martina Neila wizualizacja przy pomocy sieci bayesowskich. W ten sposób można modelować nawet najbardziej skomplikowany materiał dowodowy w sposób przejrzysty dla laika. Rola stron procesu ograniczałaby się tu do sprecyzowania prawdopodobieństw a priori i zależności między poszczególnymi zmiennymi, zaś zadanie skonstruowania architektury sieci pozostawiano by ekspertom. O prawomocności obliczeń dokonywanych „pod spodem” można by przekonać strony na prostych przykładach, z wykorzystaniem np. drzewek zdarzeniowych. Zatem zastosowanie sieci bayesowskich w procesie miałoby być, zdaniem Fentona i Neila, najprostszym sposobem uniknięcia błędów probabilistycznych bez konieczności podejmowania beznadziejnego zadania, jakim jest nauczenie prawników statystyki.

Bartosz Janik: Some remarks concerning Bayesian rationality in Law. Abstract:

This paper aims at providing some remarks concerning Bayesian decision theory (BDT) and rationality in the legal perspective. As a first point I would like to provide a philosophical account of rationality and I will try to, while focusing on most appropriate meaning of it, to judge it from a legal point of view. It will be clear that the general notion of legal rationality is very complicated and we must set some particular goals to achieve a more global perspective. In my paper, I will focus on legal reasoning and will try to adopt Rescher’s distinction of cognitive/practical/evaluative rationality for the purpose of this analysis. The main point of this part will be the evaluation, to what extent risk aversion is connected with rationality. The thesis will be formulated in the following manner: the mechanisms of risk avoidance could serve as local rationality–triggers (as to oppose skepticism in Rescher’s terminology and deal with imperfection of our cognitive resources). The second point will be the attempt to show the connection between Bayesian decision theory (which focuses on error minimizing and thus, risk avoidance) and rationality. I will introduce basic formalism of BDT and show how, on that basis, we could formulate the local rationality for legal decision making. Again, the central notion will be the risk and I will present formal mechanism of risk avoidance in BDT. The notion of rationality, as a risk optimizer, will be proposed for this local environment. The last point of the analysis will be the answer to the question to what extent we are free from legal–theoretic assumptions in formulations of rationality. It turns out that the choice of an underlying theory of law will always determine our global notion of rationality but in the local perspective we could formulate general tools and concepts.

Izabela Skoczeń: Why should a lawyer calculate the probability of implicature formation? Abstract:

This paper aims at providing examples of possible applications of methods for calculating the probability of implicature formation (with the use of the bayesian method) in legal situations. The basis for the present considerations will be the notion of scalar implicatures, based on the gricean approach to Pragmatics. Scalars are based on conventional meanings attributed to words with the use of lexical scales (Horn). Placing a word in a definite position in a scale enables the speakers to attribute it a definite meaning, that does not have to be consistent with the lexical meaning that would be understood with the use of classical logic. [...] As experiments have proven, in contexts with data deficit the probability of definite implicature formation is rather not intuitive. A quite striking example is the following situation: if while describing three objects, the speakers has information concerning the features of only two of them, the hearer seems more prone to infer, that the third item disposes of the same feature while hearing an utterance with the numeral “two”, rather than “some”. This surprising result seems most vital for lawyers, as it conveys a hidden pattern of linguistic manipulation. The conventional implicature that should be cancelled due to pragmatic reasons is so strong, that it still influences the meaning. Imagine, that we have three suspects A,B,C and we know that A and B were at the crime scene that day. We don’t know, whether C was at the crime scene. If the probability of omitting scalar implicature cancellation is higher when using expressions like some, rather than numerals, C’s defendant should rather say “Some of the suspects were at the crime scene.” rather than “Two were at the crime scene.”. The later formulation, according to Goodman and Stuhlm¨uller calculations, would boost the probability of the court inferring the implicature that C was also at the crime scene that day. This observation opens an entire new range of possibilities of manipulating implicature formation in contexts, where the hearer is aware of the speaker’s data being insufficient. It is often the in judicial environments, when the provided evidence is too scarce.

For full text of the papers, please contact the authors.

HT Bartosz Janik

Tauberer: New GovTrack Bill Prognosis Methodology Page, with Charts

Dr. Joshua Tauberer has created a new bill prognosis methodology page for GovTrack, his U.S. federal open legislative data service.

The page includes (on right screen) three tabs of charts demonstrating output and functioning of the prognosis methodology. The third tab shows charts of precision vs. recall results, and is intended expressly “for machine learning researchers.”

HT @JoshData

Gittelson et al.: Modeling the forensic two-trace problem with Bayesian networks

Simone Gittelson of l’Institut de Police Scientifique, Ecole des Sciences Criminelles, Université de Lausanne, and colleagues, have published Modeling the forensic two-trace problem with Bayesian networks, forthcoming in Artificial Intelligence and Law.

Here is the abstract:

The forensic two-trace problem is a perplexing inference problem introduced by Evett (J Forensic Sci Soc 27:375–381, 1987). Different possible ways of wording the competing pair of propositions (i.e., one proposition advanced by the prosecution and one proposition advanced by the defence) led to different quantifications of the value of the evidence (Meester and Sjerps in Biometrics 59:727–732, 2003). Here, we re-examine this scenario with the aim of clarifying the interrelationships that exist between the different solutions, and in this way, produce a global vision of the problem. We propose to investigate the different expressions for evaluating the value of the evidence by using a graphical approach, i.e. Bayesian networks, to model the rationale behind each of the proposed solutions and the assumptions made on the unknown parameters in this problem.

Tauberer: Changes to GovTrack Bill Prognosis

Dr. Joshua Tauberer of GovTrack has posted Bill prognosis gets a few improvements, at the GovTrack Blog.

He writes:

…I’m adding three new factors to GovTrack’s analysis: whether the bill was introduced in the first 90 days of the Congress, whether it was introduced in the first year, and whether it was introduced in the last 90 days of the Congress. You can now see that last one in the factors for S. 3637, for example.

The post describes his decision to incorporate a component of the model described in: Yano et al.: Textual Predictors of Bill Survival in Congressional Committees.

For more details please see the complete post.

HT @JoshData

Katz on Quantitative Legal Prediction

Professor Dr. Daniel Martin Katz of the Michigan State University College of Law and the ReInvent Law Laboratory has published Quantitative Legal Prediction – or – How I Learned to Stop Worrying and Start Preparing for the Data Driven Future of the Legal Services Industry, forthcoming in Emory Law Journal.

Here is the abstract:

Do I Have a Case? What is Our Likely Exposure? How much is this Going to Cost? What will happen if we leave this particular provision out of this contract? How can we best staff this particular legal matter? These are core questions asked by sophisticated clients such as general counsels as well as consumers at the retail level. Whether generated by a mental model or a sophisticated algorithm, prediction is a core component of the guidance that lawyers offer. Indeed, it is by generating informed answers to these types of questions that many lawyers earn their respective wage.

Every single day lawyers and law firms are providing predictions to their clients regarding their prospects in litigation and the cost associated with its pursuit (defense). How are these predictions being generated? Precisely what data or model is being leveraged? Could a subset of these predictions be improved by access to outcome data in a large number of ‘similar’ cases. Simply put, the answer is yes. Quantitative legal prediction already plays a significant role in certain practice areas and this role is likely increase as greater access to appropriate legal data becomes available.

This article is dedicated to highlighting the coming age of Quantitative Legal Prediction with hopes that practicing lawyers, law students and law schools will take heed and prepare to survive (thrive) in this new ordering. Simply put, most lawyers, law schools and law students are going to have to do more to prepare for the data driven future of this industry. In other words, welcome to Law’s Information Revolution and yeah – there is going to be math on the exam.

Click here for slides from the presentation version of the paper, Quantitative Legal Prediction.

Yano et al.: Textual Predictors of Bill Survival in Congressional Committees

Tae Yano and Professor Dr. Noah A. Smith, both of Carnegie Mellon University Language Technologies Institute, and Professor Dr. John D. Wilkerson of the University of Washington Deaprtment of Political Science, presented a paper entitled Textual Predictors of Bill Survival in Congressional Committees, at New Directions in Analyzing Text as Data 2012, a conference held 5-6 October 2012 at the Harvard University Institute for Quantitative Social Science.

Here is the abstract:

A U.S. Congressional bill is a textual artifact that must pass through a series of hurdles to become a law. In this paper, we focus on one of the most precarious and least understood stages in a bill’s life: its consideration, behind closed doors, by a Congressional committee. We construct predictive models of whether a bill will survive committee, starting with a strong, novel baseline that uses features of the bill’s sponsor and the committee it is referred to. We augment the model with information from the contents of bills, comparing different hypotheses about how a committee decides a bill’s fate. These models give significant reductions in prediction error and highlight the importance of bill substance in explanations of policy-making and agenda-setting.

A notable technology related to this research is the new probability-of-bill-passage feature on Dr. Joshua Tauberer’s GovTrack service.

Click here for Dr. Tauberer’s comment on the Yano et al. paper.

An interesting discussion among academics and developers arose on Twitter in response to a tweet about this paper.

Call for Proposals: ReInventLaw Dubai 2012: An ‘Un’conference on Law, Technology, Innovation, and Entrepreneurship

A call for presentation proposals — with submission deadline of 15 October 2012 — has been issued for ReInventLaw Dubai 2012: “an ‘un’conference devoted to law, technology, innovation, and entrepreneurship” — to be held 10 December 2012 at Media City in Dubai.

The organizers particularly welcome presentations about innovations in legal services or legal education. Presentations can take the form of 6 Minute Ignite Style Presentations or 12 Minute “TED Style” Presentations.

Registration is free.

The event Website describes the event as follows:

ReInvent Law Dubai is an “un”conference devoted to law, technology, innovation, and entrepreneurship.

Anyone interested in the future of law or technology or entrepreneurship will want to participate. Come hear about the innovative ideas generated by the highly-engaging atmosphere of the event!

The event is being sponsored by The ReInventLaw Laboratory at Michigan State University College of Law, and is modeled on the LawTechCamp London 2012 event held last summer.

For more information, please see the ReInventLaw Dubai 2012 Website.

HT @computational.

Cheng on Reconceptualizing the Burden of Proof as a Probability Ratio

Professor Edward K. Cheng of Vanderbilt University School of Law has posted Reconceptualizing the Burden of Proof, forthcoming in Yale Law Journal.

Here is the abstract:

The burden of proof is conventionally described as an absolute probability threshold – for example, the preponderance standard is commonly equated to anything greater than 0.5. In this Essay, I argue that this characterization of the burden of proof is wrong. Rather than focus on an absolute threshold, the Essay reconceptualizes the preponderance standard as a probability ratio, and I show how doing so eliminates many of the classical problems associated with probabilistic theories of evidence. Using probability ratios eliminates the so-called Conjunction Paradox, and developing the ratio tests under a Bayesian perspective further explains the Blue Bus problem and other puzzles surrounding statistical evidence. By harmonizing probabilistic theories of proof with recent critiques advocating for abductive models (inference to the best explanation), the Essay hopes the bridge a gap in current evidence scholarship.

HT Professor Dr. Dan Kahan

Oldfather et al. on Automated Content Analysis, Court Opinions, and Legal Scholarly Methodology

Professor Chad M. Oldfather of Marquette University School of Law, Professor Dr. Joseph P. Bockhorst of the University of Wisconsin Madison Department of Electrical Engineering and Computer Science, and Brian P. Dimmer, Esq., have published Triangulating Judicial Responsiveness: Automated Content Analysis, Judicial Opinions, and the Methodology of Legal Scholarship, Florida Law Review, 64, 1189-1242 (2012).

Here is the abstract:

The increasing availability of digital versions of court documents, coupled with increases in the power and sophistication of computational methods of textual analysis, promises to enable both the creation of new avenues of scholarly inquiry and the refinement of old ones. This Article advances that project in three respects. First, it examines the potential for automated content analysis to mitigate one of the methodological problems that afflicts both content analysis and traditional legal scholarship — their acceptance on faith of the proposition that judicial opinions accurately report information about the cases they resolve and courts’ decisional processes. Because automated methods can quickly process large amounts of text, they allow for assessment of the correspondence between opinions and other documents in the case, thereby providing a window into how closely opinions track the information provided by the litigants. Second, it explores one such novel measure — the responsiveness of opinions to briefs — in terms of its connection to both adjudicative theory and existing scholarship on the behavior of courts and judges. Finally, it reports our efforts to test the viability of automated methods for assessing responsiveness on a sample of briefs and opinions from the United States Court of Appeals for the First Circuit. Though we are focused primarily on validating our methodology, rather than on the results it generates, our initial investigation confirms that even basic approaches to automated content analysis provide useful information about responsiveness, and generates intriguing results that suggest avenues for further study.

Murphy and Jacobs: Using Effect Size Measures to Reform Determination of Adverse Impact in Equal Employment

Professor Dr. Kevin R. Murphy and Professor Dr. Rick R. Jacobs, both of the Penn State University Department of Psychology, have published Using effect size measures to reform the determination of adverse impact in equal employment litigation, Psychology, Public Policy, and Law, 18, 477-499 (2012).

Here is the abstract:

A critical issue in large-scale employment discrimination litigation is whether the challenged practices have a systematic adverse impact on protected groups of applicants or employees. The methods that are currently used to detect adverse impact are seriously flawed, but these flaws could be addressed by putting more emphasis on commonly used research statistics that measure effect size, such as standardized difference ( d) or the percentage of variance (PV) explained. We recommend incorporating these indices as a baseline for detecting the presence of adverse impact, and that when PV or d is small, observed group differences should not be taken as evidence of adverse impact, regardless of the results of significance tests. Such an approach eliminates spurious charges of unfair discrimination, spares employers and the courts from dealing with claims that are frivolous and based on minutia, and helps enforcement agencies to concentrate their efforts on employment practices that are truly discriminatory. More generally, it will improve the use and interpretation of statistical evidence in litigation.