Passing a USA National Bar Exam: a First Corpus for Experimentation: Article Data

  • Adam Zachary Wyner (Creator)
  • Biralatei James Fawei (Creator)
  • Jeff Z Pan (Creator)
  • Pete Edwards (Data Manager)



The dataset is described in the paper "Passing a USA National Bar Exam: a First Corpus for Experimentation".

The National Conference of Bar Examiners in the United States prepares and administers the Multistate Bar Exam (MBE) every year to thousands of aspiring lawyers throughout the country. The exam questions (in the most recent exam) cover the legal spectrum: Constitutional Law, Contracts, Criminal Law and Procedure, Evidence, Real Property, Torts, and Civil Procedure. Thus, the MBE is a broad and deep exploration of the examinee's knowledge of the law as it applies across the US.

The corpus was gathered from National Conference of Bar Examiners (NCBE) materials and prepared for a textual entailment exercise using the Excitement Open Platform. The original dataset contains one hundred questions, each with four possible answers out of which the candidate must pick the correct one; the NCBE provided an answer key to the materials. Given some modifications discussed in the paper, the original dataset was developed into pairs of theories and hypotheses, where each question was paired with one possible answer, yielding four hundred theory-hypothesis pairs. The correct (Gold Standard) answer is indicated as "entailment" and the wrong answer as "nonentailment".

The data are in XML format.

Below is an original example question with answers a.-d., where b. is correct:

After being fired from his job, Mel drank almost a quart of vodka and decided to ride the bus home. While on the bus, he saw a briefcase he mistakenly thought was his own, and began struggling with the passenger carrying the briefcase. Mel knocked the passenger to the floor, took the briefcase, and fled. Mel was arrested and charged with robbery.

Mel should be

[a.] acquitted, because he used no threats and was intoxicated.
[b.] acquitted, because his mistake negated the required specific intent.
[c.] convicted, because his intoxication was voluntary.
[d.] convicted, because mistake is no defense to robbery.
Date made available31 Mar 2016
PublisherUniversity of Aberdeen
Temporal coverage2016 - 2016
Date of data productionMar 2016
Geographical coverageUnites States of America
  • Passing a USA National Bar Exam: a First Corpus for Experimentation

    Wyner, A. Z., Fawei, B. J. & Pan, J. Z., 23 May 2016, LREC 2016, Tenth International Conference on Language Resources and Evaluation. Calzolari, N., Choukri, K., Declerck, T., Goggi, S., Grobelnik, M., Maegaard, B., Mariani, J., Mazo, H., Moreno, A., Odijk, J. & Piperidis, S. (eds.). LREC, 7 p.

    Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

    Open Access
  • Passing a USA national bar exam - A first experiment

    Fawei, B., Wyner, A. & Pan, J. Z., 2015, Legal Knowledge and Information Systems, JURIX 2015: 28th Annual Conference. IOS Press, Vol. 279. p. 179-180 2 p. (Frontiers in Artificial Intelligence and Applications; vol. 279).

    Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

    2 Citations (Scopus)

Cite this