Safe AI: How to Interact with AI and Agents Without Breaching the Law

Olga Sevriuk rounds up some recent cases that highlight the legal risks of sharing sensitive client data with public AI chatbots and agents and the lessons that can be learned.

The evolving landscape of AI safety

Nowadays the AI adoption often outpaces the development of regulatory and governance frameworks. As a result, AI increasingly influences the work of a wide range of professionals, including regulated practitioners such as advocates, immigration advisers and law firms.

From drafting documents and analysing data to assisting with legal research and client communication, AI tools have become embedded in everyday legal workflows. While some law firms invest in the development of proprietary AI systems, others integrate publicly available tools such as ChatGPT, Gemini, Claude and similar large language models.

This article examines the risks, such as legal privilege waiver and confidentiality breach, that arise when lawyers rely on third-party AI platforms and highlights key precautionary steps that practitioners should communicate to their clients.

Who should be concerned?

This discussion is relevant for:

• Consultants and independent advisers using AI tools in client engagements
• Lawyers and law firms incorporating AI into their legal practice and legal research
• In-house legal professionals who use AI in their daily work or are considering integrating AI into legal operations

Confidentiality and the illusion of safety: case law review

Inputting confidential or personal data into commercial or publicly available AI systems can lead to significant legal consequences, ranging from breaches of confidentiality to the waiver of legal privilege, depending on the context.

Recent case law from both the United States and the United Kingdom demonstrates that such risks are no longer merely theoretical.

UK case law

In a recent decision of the Upper Tribunal (Immigration and Asylum Chamber), the court stated:

“Uploading confidential documents into an open-source AI tool, such as ChatGPT, is to place this information on the internet in the public domain, and thus to breach client confidentiality and waive legal privilege, and any such conduct might itself warrant referral to the SRA and should, in any event, be referred to the Information Commissioner’s Office”. (R v Secretary of State for the Home Department (AI hallucinations; supervision; Hamid)

The background of the case is illustrative.

• A UK-based accredited immigration adviser (who was also a solicitor) submitted a document to the court citing a case that did not exist and referring to case law that was not relevant to the issue of sufficiency of protection in the context of the appeal. The tribunal required the adviser to clarify whether AI had been used in drafting the document, warning that “the citation of cases which do not exist sends that judge on a fool’s errand”.
• Initially, the adviser denied using AI. However, he later self-reported the matter to both the Immigration Advice Authority (IAA) and the Solicitors Regulation Authority (SRA), ultimately confirming that AI had been used.
• It was subsequently discovered that the adviser had relied on AI for a broader range of tasks beyond legal research and case law analysis. The adviser confirmed that AI had occasionally been used for administrative tasks, including summarising decision letters. He also admitted that he had entered client emails explaining Home Office decisions into ChatGPT in order to improve their wording and had uploaded Home Office decision letters to the platform to generate summaries for clients.

This decision represents one of the first UK cases discussing negligent conduct by a regulated legal professional in connection with the use of AI tools. Although the case arose in a specific immigration law context, it sends a broader message to the legal community: uploading confidential documents into publicly available AI tools such as ChatGPT may breach client confidentiality and result in the waiver of legal privilege.

What does this decision teach legal professionals?

First, it points out the obligation to self-report potential misconduct to the relevant regulators, including the Immigration Advice Authority (IAA) and the Solicitors Regulation Authority (SRA), and to bring the matter to the attention of the Information Commissioner’s Office.

Second, the court distinguished between the use of general, non-specialist AI tools and enterprise systems designed with stronger data protection safeguards, referring to Microsoft Copilot as an example of a system that may operate within a more controlled environment and may not place information in the public domain.

Finally, the tribunal stressed that when AI is used for legal research or drafting, legal professionals remain under a professional obligation to verify the accuracy of any AI-generated material and to ensure that staff understand the risks associated with non-specialist AI tools.

US case law

Another case raising concerns regarding legal privilege and the use of AI agents recently took place in a U.S. federal court. The central question the court had to answer was:

“When a user communicates with a publicly available AI platform in connection with a pending criminal investigation, are the user’s communications protected by attorney-client privilege or the work product doctrine?” (United States v Heppner, 25 Cr. 503 (S.D.N.Y. Feb. 17, 2026).

Spoiler: the answer is negative.

In this case, which concerned communications between a defendant (the user) and Claude (the AI platform), the court held that the communications contained in the AI-generated documents were not confidential and were not protected by either attorney-client privilege or the work product doctrine.

One of the core reasons for this ruling was the written privacy policy to which users of Claude consent: “The policy states that Anthropic collects data on both users’ “inputs” and Claude’s “outputs”, may use such data to train Claude, and reserves the right to disclose such data to a range of third parties, including governmental and regulatory authorities”.

The background:

• Bradley Heppner, the former CEO of a financial services company, was accused of criminal conduct, including securities fraud, wire fraud, conspiracy and making false statements. During the investigation, numerous documents and electronic devices were seized from the defendant’s home. Among the seized materials were 31 documents containing communications between the defendant and the generative AI platform Claude. These communications took place after the defendant had received a subpoena.
• The AI-generated documents consisted of reports outlining a potential defence strategy, including arguments the defendant might raise regarding the anticipated factual and legal charges.
• The defendant attempted to apply attorney-client privilege and the work product doctrine in relation to these documents.
• He argued that the information entered into the AI system had been learned from counsel, that the documents were created for the purpose of discussing defence strategy with counsel and obtaining legal advice, and that the contents of the documents were later planned to be shared with his counsel.

The court rejected the application of legal privilege because the AI documents – communications between the defendant and the AI system – lacked essential elements of attorney-client privilege.

1. Claude, as an AI system, is not an attorney, does not owe fiduciary duties and is not subject to professional regulation or discipline. As the court noted, a relationship of trust comparable to that between a lawyer and client cannot arise in such a context.
2. A further reason for denying privilege was the absence of confidentiality. Upon analysing Claude’s written privacy policy, the court found that users consent to Anthropic collecting both users’ “inputs” and Claude’s “outputs”, using such data to train the model and potentially disclosing it to third parties, including governmental and regulatory authorities.
3. A third and particularly important aspect of the case was that the defendant communicated with the AI agent not at the direction of counsel, but on his own initiative. Instead, he approached Claude as if it were de facto legal counsel and sought legal advice through the platform (a use that is clearly excluded under Claude’s disclaimer policy). The defendant’s communications with the AI agent were therefore undertaken voluntarily and were not prompted or authorised by his lawyers.

The work product doctrine was also found to be inapplicable. The court concluded that the AI-generated documents had not been prepared by counsel or at the direction of counsel, but rather by the defendant on his own initiative.

What does this decision teach legal professionals?

The case demonstrates that negligent behaviour by clients when interacting with publicly available AI systems may undermine their own legal defence. Claims of legal privilege may therefore be challenged when a client relies on public AI platforms, particularly where the interaction destroys the element of confidentiality required for privilege to arise.

At the same time, an important question remains: would the outcome have been different if the AI system had been used directly by a lawyer or at the clear direction of counsel? Given that commercial AI platforms often reserve the right to collect, process, and disclose user data to third parties, the risk that privileged information may lose its protected status cannot be excluded. At the same time, it is important to note that Judge Rakoff considered an alternative scenario and the potential applicability of attorney-client privilege if the use of the AI system had been directed by legal counsel.

“Had counsel directed [the defendant] to use Claude, Claude might arguably be said to have functioned in a manner akin to a highly trained professional who may act as a lawyer’s agent within the protection of the attorney-client privilege.”

The recent case law confirms the plausibility of this alternative scenario, though under the work-product doctrine rather than attorney-client privilege. In Warner v. Gilbarco, Inc., decided in February 2026 by the U.S. District Court for the Eastern District of Michigan, the defendant’s request to compel disclosure of AI-related documents (inputs entered into ChatGPT by the plaintiff) was denied. The court held that the plaintiff’s use of ChatGPT did not waive work-product protection, in part because the plaintiff was a pro se litigant and therefore had the right to assert work-product protection over such materials prepared in anticipation of litigation.

This highlights that the context in which such tools are used – and, in particular, whether their use is directed and supervised by legal counsel – matters.

Conclusion

These cases are particularly illustrative and given that they occurred within a short time interval: the UK court delivered its decision in November 2025, and the U.S. court followed in February 2026 – it raises concerns regarding confidentiality and legal privilege in the context of communications with AI systems and serve as a warning for professionals across many industries.

The key insights drawn from the reviewed case law point to a shared judicial concern: entering sensitive or privileged information into publicly available AI platforms such as ChatGPT or Claude may be treated as a disclosure to a third party, effectively placing such information into the public domain. Likewise, the Artificial Intelligence (AI) Guidance for Judicial Office Holders, updated in October 2025, reaffirms this position within the UK judicial community. The guidance states: “Any information that you input into a public AI chatbot should be seen as being published to all the world”.

Even if, in practice, the information does not ultimately reach the public, the act itself raises another important issue – negligence in the handling of confidential information. It is a well-established principle of law that parties seeking protection under confidentiality must take reasonable steps to preserve it.

In other words, if a lawyer and a client were to discuss sensitive legal matters on a crowded train where others could overhear their conversation, it would be difficult to argue that adequate precautions had been taken to preserve confidentiality. The same concern arises when confidential information is entered into publicly available AI systems without appropriate safeguards.

It is important to understand the legal distinction between privacy, confidentiality and legal privilege. Many AI systems provide certain privacy protections under applicable data protection laws or through their privacy policies. However, the existence of privacy safeguards does not necessarily mean that communications with such systems are confidential.

Confidentiality requires that information is shared within a relationship where the recipient is under a duty not to disclose it. Publicly available AI platforms typically do not assume such duties. Instead, their terms often allow providers to process, retain or disclose user data to third parties.

Legal privilege sets an even higher threshold. It protects communications between a client and a lawyer made for the purpose of obtaining or providing legal advice. When information is voluntarily disclosed to a third-party AI platform that is not bound by professional duties of confidentiality, the essential elements required for privilege may no longer be satisfied.

Thus, this article concludes that entering information into publicly available AI systems that cannot guarantee the confidentiality of the data they process creates substantial risks for both confidentiality and, consequently, legal privilege.

Key lessons

Lesson 1: Do not share sensitive data: Sensitive data (confidential, privileged, or commercially sensitive) must not be shared with open-access or public AI tools unless it is already in the public domain. If you seek support in your legal research, you may disable chat history in public AI chatbots; however, you should still avoid entering any sensitive data.

Lesson 2: AI governance must become a formal policy: Just as organisations previously developed privacy and cookie policies in response to the growth of digital services, a similar evolution is now required for AI governance. AI policies should clearly define what categories of data may or may not be entered into AI systems, establish procedures for handling confidential or commercially sensitive information, outline accountability for negligent data handling, and introduce oversight mechanisms for AI-assisted decision-making.

Lesson 3: Ensure accuracy: Any information provided by an AI tool must always be verified before it is used or relied upon.

Lesson 4: Inform clients: Clients should be informed, and agreements should specify that documents generated by a lawyer must not be uploaded to any AI system without the lawyer’s prior knowledge and consent.