How Soon Will AI Replace Judges?

As a plaintiff’s employment law attorney, the thought of artificial intelligence replacing judges is both intriguing and alarming. The legal world is traditionally slow to adopt technological advancements, yet the potential for AI to revolutionize our field is undeniable. I came across a recent concurring opinion by the Hon. Judge Kevin C. Newsom of the United States Court of Appeals for the Fifth Circuit in Snell v. United Specialty Ins. Co., No. 22-12581, 2024 WL 2717700 (11th Cir. May 28, 2024). In the case, United Casualty Insurance Company refused to defend James Snell, a landscaper, in a lawsuit claiming he had negligently installed a ground-level trampoline in a client’s backyard. Snell sued, arguing that United had breached its insurance contract and seeking a declaration that United had a duty to defend and indemnify him. The entire case came down to the singular issue of whether installing the trampoline was considered “landscaping.” Why would an insurance coverage case pique the interest of an employment discrimination lawyer?

Judge Newsom suggested something quite radical: that AI-powered large language models like OpenAI’s ChatGPT, Google’s Gemini, and Anthropic’s Claude might assist judges with the interpretive analysis of legal texts. The judge’s proposal was to consider whether these AI models could help determine the ordinary meaning of words in legal documents. He stated, “I write separately (and I’ll confess this is a little unusual) simply to pull back the curtain on the process by which I thought through one of the issues in this case—and using my own experience here as backdrop, to make a modest proposal regarding courts’ interpretations of the words and phrases used in legal instruments.” This got me thinking about the broader implications of AI in the judiciary.

The Ordinary Meaning Conundrum

Again, the dispute revolved around whether installing an in-ground trampoline constituted “landscaping” under an insurance policy. This seemingly straightforward issue turned into a linguistic labyrinth, as the court grappled with dictionary definitions and the common understanding of the term. The Judge Newsom spent hours trying to pinpoint the ordinary meaning of “landscaping,” feeling frustrated and stuck.

Imagine if AI could step in here. The judge half-jokingly asked a clerk to query ChatGPT about the ordinary meaning of “landscaping.” The response was surprisingly sensible, aligning closely with the judge’s instincts. He reflected on this by saying, “Is it absurd to think that ChatGPT might be able to shed some light on what the term ‘landscaping’ means? Initially, I answered my own question in the affirmative: Yes, Kevin, that is positively absurd. But the longer and more deeply I considered it, the less absurd it seemed.” Could AI really offer valuable insights into such nuanced questions? This lead Judge Newsom to consider the potential strengths and weaknesses of using AI in judicial interpretation.

Strengths of AI in Legal Interpretation

Judge Newsom identified several potential strengths of using AI in legal interpretation:

1. Training on Ordinary Language: AI models are trained on vast amounts of data from the internet, encompassing how people use language in everyday contexts. This extensive training allows AI to provide statistical predictions about word usage, potentially offering insights into the ordinary meaning of legal terms.
2. Understanding Context: Modern AI models are adept at understanding the context in which words are used. They can distinguish between different meanings of a word based on context, which is crucial in legal interpretation where precision is paramount.
3. Accessibility: AI tools are increasingly accessible to judges, lawyers, and the public. This democratization of technology means that legal professionals can use AI as an inexpensive research tool, complementing traditional methods.
4. Transparency: Using AI for legal research can enhance transparency. When judges rely on AI outputs, they can disclose the specific queries and results, providing a clear trail of how interpretations were reached.
5. Empirical Advantages: Compared to other empirical methods like surveys or corpus linguistics, AI offers a practical and scalable approach to understanding language usage. Surveys can be impractical and time-consuming, while corpus methods may involve too much discretion in data selection.

Potential Drawbacks of AI in Legal Interpretation

Judge Newsom also acknowledged several potential drawbacks of AI in legal interpretation:

1. Hallucinations: This is the killer and biggest risk. AI models can sometimes generate incorrect or fictional information, known as “hallucinations.” This risk necessitates careful oversight and verification of AI outputs to ensure reliability. This issue was notably highlighted when attorneys were sanctioned for submitting briefs with fake cases generated by ChatGPT. These “hallucinations” underscore the need for vigilance when using AI in legal contexts. As these models grow, this risk will diminish. Moreover, as President Ronald Regan said, “Trust but verify.”
2. Bias and Representation: AI training data might not fully capture offline speech, potentially underrepresenting certain communities. This could lead to biases in AI interpretations, which must be addressed to ensure fair and inclusive legal outcomes.
3. Manipulation Risks: There’s a risk that lawyers or judges might try to manipulate AI outputs by crafting specific queries or shopping around among different AI models. Ensuring transparency and using multiple models can help mitigate this risk.
4. Dystopian Fears: The idea of AI replacing judges entirely might conjure dystopian fears. However, Judge Newsom clarified that AI should be used as a supplementary tool, not a replacement for human judgment. He elaborated, “Those, like me, who believe that ‘ordinary meaning’ is the foundational rule for the evaluation of legal texts should consider—consider—whether and how AI-powered large language models like OpenAI’s ChatGPT, Google’s Gemini, and Anthropic’s Claude might—might—inform the interpretive analysis.” The goal is to enhance the interpretive process, not to hand it over to machines.

Practical Considerations and Future Directions

To maximize the utility of AI in legal interpretation, Judge Newsom suggested several practical considerations:

1. Querying AI Models: Crafting the right queries (prompts) is crucial. Users should experiment with different prompts and report the range of results to ensure robust conclusions. Lawyers and judges should as follow questions, including requiring the AI model to source its information.
2. Output Clarity: AI models should provide not just answers but also confidence levels or probabilistic predictions. This helps users understand the certainty of AI outputs and make informed decisions.
3. Temporal Context: Legal interpretations often depend on the historical context of terms. AI models need to account for this by allowing queries to be limited to specific timeframes, ensuring that interpretations reflect the intended meaning at the time of adoption.

Embracing the Future

The integration of AI into the judiciary requires caution and thoughtful consideration. The potential for AI to aid in the interpretation of legal texts is promising but must be carefully managed to ensure it complements rather than replaces human judgment. As legal professionals, we must remain open to technological advancements while safeguarding the principles of fairness and justice. Technological advancements are coming. This is not new. Just during my career, we have advanced from doing all legal research in books to libraries on CD ROM discs to terms and connectors online to natural language searches. AI is coming.

The idea of AI replacing judges might seem far-fetched today, but its role in assisting with legal interpretation is a real and exciting possibility. In navigating this brave new world, it’s crucial to remain open-minded and critical, ensuring that technology serves to enhance justice rather than undermine it. As an employee’s rights attorney, I see the value in leveraging AI to enhance our understanding of legal texts, ensuring more accurate and fair outcomes for our clients. While the journey to fully integrating AI into the judiciary will be complex, it’s a path worth exploring. After all, who wouldn’t want a little help from our digital friends in navigating the intricate world of law?

Just my two cents.

Disclaimer

The views and opinions expressed in this blog are those of the author and do not necessarily reflect the official policy or position of any agency, organization, employer, or company. This blog is intended for informational and educational purposes only and should not be construed as legal advice. Readers should consult with a qualified attorney for advice regarding their individual legal issues. The author is not responsible for any errors or omissions, or for any outcomes related to the use of this information. Use of and access to this blog does not create an attorney-client relationship between the reader and the author. This disclaimer was written by AI.