Call for Papers: DESI V: Workshop on Standards for Using Predictive Coding and Other Machine Learning Algorithms

A call for papers — with submission deadline of 1 May 2013 — has been issued for DESI V: Workshop on Standards for Using Predictive Coding and Other Machine Learning Algorithms, to be held 14 June 2013 in Rome, Italy, following ICAIL 2013: International Conference on Artificial Intelligence and Law.

Papers addressing the following questions are invited for DESI V:

1) How transparent can and should the process be in sharing seed sets or training sets of documents with opposing parties, including the sharing of privileged documents?

2) What differences if any exist between seed sets developed through random sampling versus other forms of judgmental sampling (including picking seed documents using keywords)?

3) How are non-relevant documents used to optimize machine learning algorithms and should they be subject to similar disclosure?

4) Are there ways in which predictive coding and machine learning methods can be tuned to find highly relevant (“hot”) documents in large collections?

5) To what extent is metadata important in tuning predictive coding software to find similarity in documents?

6) In light of past research at the TREC Legal Track and elsewhere, are there absolute targets for metrics in recall and precision that could serve as standards in every case, or are achieving certain metrics dependent on the relevant data set and legal context?

7) What kinds of best practice standards are needed to help improve mutual understanding of what was actually done, and to improve overall “search quality”?

8) How should predictive coding techniques be audited in connection with an entity submitting itself to an ISO 9001 quality measurement process?

9) To what extent can and should machine learning approaches be used in other phases of the litigation process, to assist in aspects of the process such as identification, preservation, and collection?

10)What are the applications of predictive coding and other forms of machine learning in related “compliance” areas, including regulatory, enforcement, and investigations?

The workshop discussion will be grounded in the results of the recently completed TREC Legal Track, especially where supervised learning methods have shown promising results in terms of being able to more cost-effectively demonstrate rates of recall and precision that approximate the best that could be obtained through other methods, including exhaustive manual review.

For more details, please see the complete call.

HT Jurix.

Call for Papers: ICAIL 2013: International Conference on Artificial Intelligence and Law

A call for papers — with paper submission deadline of 18 January 2013 — has been issued for ICAIL 2013: 14th International Conference on Artificial Intelligence and Law, to be held 10-14 June 2013 in Rome, Italy.

The Twitter account for the conference is @ICAIL2013 . The Twitter hashtag for the conference is #ICAIL2013. The conference organizers invite those interested to follow the Twitter account and hashtag and to comment and contribute with the latest news.

The conference features two tracks: one for “regular papers” and one for “innovative applications papers.”

Here is the complete list of deadlines:

• Mentoring program request deadline: November 9, 2012
• Mentoring program paper deadline: November 16, 2012
• Submission of workshop and tutorial proposals: December 7, 2012
• Submission of abstracts (optional): January 11, 2013
• Submission of papers deadline: January 18, 2013
• Notification of acceptance: March 20, 2013
• Final revised and formatted papers due: April 19, 2013
• Conference: June 10 – June 14, 2013

Papers are invited on the following topics:

• Formal and computational models of legal reasoning
• Knowledge acquisition techniques for the legal domain, including natural language processing and data mining
• Computational models of argumentation and decision making
• Legal knowledge representation including legal ontologies and common sense knowledge
• Automatic legal text classification and summarization
• Automated information extraction from legal databases and texts
• Machine learning and data mining applied to legal databases
• Conceptual or model-based legal information retrieval
• E-discovery and e-disclosure
• E-government and e-justice
• Computational models of evidential reasoning
• Modeling norms for multi-agent systems
• Modeling negotiation and contract formation
• Computational models of case-based legal reasoning
• Online dispute resolution
• Intelligent legal tutoring systems
• Intelligent support systems for the legal domain
• Interdisciplinary applications of legal informatics methods and systems

For more information, please see the call for papers.

HT Anne Gardner

[NOTE: Updated 23 November 2012 to add the Twitter account and hashtag. HT Enrico Francesconi]

Gros et al. on Ethics and Authority Sharing for Autonomous Armed Robots

Florian Gros and Catherine Tessier of ONERA, and Thierry Pichevin of CREC, Ecoles de Saint-Cyr Coetquidan, presented a paper entitled Ethics and Authority Sharing for Autonomous Armed Robots (scroll down), at RDA2 2012: the First Workshop on Rights and Duties of Autonomous Agents, held 28 August 2012 in Montpellier, France.

Click here for slides of the presentation.

Here is the abstract:

The goal of this paper is to review several ethical questions that are relevant to the use of autonomous armed robots and to authority sharing between such robots and the human operator. First, we discern the commonly confused meanings of morality and ethics. We continue by proposing leads to answer some of the most common ethical questions raised by literature, namely the autonomy, responsibility and moral status of autonomous robots, as well as their ability to reason ethically. We then present the possible advantages that authority sharing with the operator could provide with respect to these questions.

The principal ethical rules addressed in the paper are the Laws of War and related legal rules.

Click here for the complete proceedings of RDA2 2012: the First Workshop on Rights and Duties of Autonomous Agents.

Click here for abstracts and slides of presentations at RDA2 2012: the First Workshop on Rights and Duties of Autonomous Agents.

Chandler on Machine Learning Judicial Behavior Using a Mathematica to Weka Interface

Professor Seth J. Chandler of the University of Houston Law Center will present a paper entitled Machine Learning Judicial Behavior Using a Mathematica to Weka Interface, at IMS 2012: The International Mathematica Symposium, to be held 11-13 June 2012 in London, England, UK.

Here is the abstract:

Weka is a comprehensive and powerful Java library subject to a GNU General Public License that implements a large number of modern machine learning classification and other methods. These classification methods include Bayesian techniques, nearest neighbor voting, support vector machines, neural nets, and decision trees, meta-methods such as “dagging” as well as simple methods such as “OneR” and “ZeroR.” While Weka can be used both from a command line and using a variety of respectable GUI interfaces provided by its designers, the ability to further manipulate its output or conduct structured experiments can be challenging. This presentation will show how one can use a Mathematica foreign language interface (J/Link) to conduct structured experiments using Weka’s capabilities and to extract information produced by Weka algorithms as the basis for further analysis and visualization using Mathematica. The domain in which the matter will be presented should be topical: an effort to predict the behavior of United States Supreme Court Justices. Using the Mathematica to Weka interface both for construction of machine learning algorithms and as an engine for deriving real-time results, the presentation (a) predicts the results of important pending Supreme Court cases, (b) creates a “Fantasy Supreme Court” that predicts the results of imagined cases to imagined panels of justices, and (c) creates a kind of “time machine” that shows how actual cases might have come out – and might have changed legal and cultural history – had they been decided by different panels of justices.

For the full text of the paper or the slides, please contact the author.

Thanks to Professor Chandler for allowing me to post the abstract.

DGT-TM-2011, Parallel Corpus of All EU Legislation in Translation, Expanded to Include Data from 2004-2010

The DGT Multilingual Translation Memory of the Acquis Communautaire: DGT-TM — a parallel corpus of all European Union legislation, called the Acquis Communautaire, translated into all 22 languages of the EU nations — has been expanded to include EU legislation from 2004-2010, according to an April 2012 announcement on the DGT-TM Website. The updated corpus is called DGT-TM-2011.

The new content comes from the EU Official Journal Series L, 2004-2010.

According to the announcement, DGT-TM-2011 is the largest parallel corpus in the world, and is intended to be used for the following purposes:

• training automatic systems for statistical machine translation (SMT);
• producing monolingual or multilingual lexical and semantic resources such as dictionaries and ontologies;
• training and testing multilingual information extraction software;
• checking translation consistency automatically;
• testing and benchmarking alignment software (for sentences, words, etc.).

The DGT-TM-2011 should be a valuable resource for legal informatics and legal linguistics research and development.

For more information, please see:

• The official announcement;
• A new paper about DGT-TM-2011: Steinberger, R., et al. (2012). DGT-TM: A freely Available Translation Memory in 22 Languages, to be presented at LREC 2012;
• A new post about the DGT-TM-2011: Super-European language translation corpus, at Corpus Linguistics.

HT @moximer.

Mochales and Moens on Argumentation Mining in ECHR Texts

Rachel Mochales Palau and Professor Dr. Marie-Francine Moens, both of Katholieke Universiteit Leuven Afdeling Informatica, have published Argumentation Mining, forthcoming in Artificial Intelligence and Law. Here is the abstract:

Argumentation mining aims to automatically detect, classify and structure argumentation in text. Therefore, argumentation mining is an important part of a complete argumentation analyisis, i.e. understanding the content of serial arguments, their linguistic structure, the relationship between the preceding and following arguments, recognizing the underlying conceptual beliefs, and understanding within the comprehensive coherence of the specific topic. We present different methods to aid argumentation mining, starting with plain argumentation detection and moving forward to a more structural analysis of the detected argumentation. Different state-of-the-art techniques on machine learning and context free grammars are applied to solve the challenges of argumentation mining. We also highlight fundamental questions found during our research and analyse different issues for future research on argumentation mining.

The techniques discussed in the paper are illustrated in part through their application to a corpus of texts issued by the European Court of Human Rights (ECHR).

Call for Papers: SIRE 2011: SIGIR 2011 Information Retrieval for E-Discovery Workshop

A call for papers — with submission deadlines of 13 May 2011 for research papers and 3 June 2011 for position papers — has been posted for SIRE 2011: SIGIR 2011 Information Retrieval for E-Discovery Workshop, to be held 28 July 2011 in Beijing, China.

The workshop will be collocated with SIGIR 2011: The 34th Annual ACM SIGIR Conference.

Papers are invited on the following topics:

• Distributed search of large heterogeneous enterprise information systems, including corporate intranets, archival and backup repositories, cloud-based storage, etc.
• High-recall search of large collections, including those with high densities of relevant documents
• Supervised learning of classifiers for responsiveness, privilege and other factors of interest (sometimes referred to in e-discovery as predictive coding)
• IR techniques that leverage the characteristics of specific types of business records (email, instant messages, voice mail, file systems, etc.)
• Clustering, link analysis, and other methods for discovering structure in large collections, including detection of duplicate documents
• Process design for human-in-the-loop review and exploitation of large data sets, including measurement of inter-reviewer consistency, active learning, etc.
• Evaluation design, including sampling strategies, estimation of confidence intervals, and reusability of large test collections

For more information, please see the call for papers.

HT Jack G. Conrad.

NELIC 2011: New and Emerging Legal Infrastructures Conference

[Update 20 April 2011: Click here for videos of the entire NELIC conference. HT @LSNTAP.]

NELIC 2011: The New and Emerging Legal Infrastructures Conference, will be held 15 April 2011, at the University of California, Berkeley School of Law, Boalt Hall, in Berkeley, California, USA.

According to the announcement, invited speakers will address the following topics:

• “Quantitative Legal Prediction“: such as applying “machine learning” and “natural language processing” to develop “statistical model[s]” of “judicial decision-making”;
• “Legal Financing and Securitization“
• “The Future of Legal Automation“
• “Legal Interfaces and User Experiences“: including implications for access “to the legal system.”

As of today, the speakers include Joshua Walker of Lex Machina; and Daniel Martin Katz of the University of Michigan Center for the Study of Complex Systems.

For registration or more information, please see the conference announcement.

Deadline Extended to 17 January: Call for Papers for ICAIL 2011

[NOTE: The call for papers submission deadline has been extended to 17 January 2011, according to @JackGConrad.]

A call for papers has been issued for ICAIL 2011: The 13th International Conference on Artificial Intelligence and Law, to be held 6-10 June 2011 at the University of Pittsburgh School of Law in Pittsburgh, Pennsylvania, USA.

The conference is organized by IAAIL: The International Association for Artificial Intelligence and Law.

A mentoring program is being offered for authors wishing to submit papers to the conference.

Here are the submission deadlines:

• “Mentoring program request deadline: November 8, 2010
• Mentoring program paper deadline: November 15, 2010
• Submission of workshop and tutorial proposals: December 6, 2010
• Submission of abstracts (optional): January 3, 2011″
• Submission of papers extended deadline: January 17, 2011

Papers are invited on the following topics:

• “Formal and computational models of legal reasoning
• Knowledge acquisition techniques for the legal domain, including natural language processing and data mining
• Computational models of argumentation and decision making
• Legal knowledge representation including legal ontologies and common sense knowledge
• Computational models of evidential reasoning
• Modeling norms for multi-agent systems
• Modeling negotiation and contract formation
• Computational models of case-based legal reasoning
• Conceptual or model-based legal information retrieval
• Automated information extraction from legal databases and texts
• Intelligent legal tutoring systems
• Intelligent support systems for the legal domain
• E-discovery and e-disclosure
• Automatic legal text classification and summarization
• Machine learning and data mining applied to legal databases”

For more information, please see the call for papers.

HT Jack G. Conrad.

December 6 Deadline: ICAIL 2011 Workshop & Tutorial Proposals

[NOTE: 6 December 2010 is the deadline for submitting workshop and tutorial proposals.]

A call for papers has been issued for ICAIL 2011: The 13th International Conference on Artificial Intelligence and Law, to be held 6-10 June 2011 at the University of Pittsburgh School of Law in Pittsburgh, Pennsylvania, USA.

The conference is organized by IAAIL: The International Association for Artificial Intelligence and Law.

A mentoring program is being offered for authors wishing to submit papers to the conference.

Here are the remaining submission deadlines:

• “Submission of workshop and tutorial proposals: December 6, 2010
• Submission of abstracts (optional): January 3, 2011
• Submission of papers deadline: January 10, 2011″

Papers are invited on the following topics:

• “Formal and computational models of legal reasoning
• Knowledge acquisition techniques for the legal domain, including natural language processing and data mining
• Computational models of argumentation and decision making
• Legal knowledge representation including legal ontologies and common sense knowledge
• Computational models of evidential reasoning
• Modeling norms for multi-agent systems
• Modeling negotiation and contract formation
• Computational models of case-based legal reasoning
• Conceptual or model-based legal information retrieval
• Automated information extraction from legal databases and texts
• Intelligent legal tutoring systems
• Intelligent support systems for the legal domain
• E-discovery and e-disclosure
• Automatic legal text classification and summarization
• Machine learning and data mining applied to legal databases”

For more information, please see the call for papers.

HT Jack G. Conrad.