The legal industry is increasingly thinking about using AI and Large Language Models (LLMs) like GPT for document review, legal research, and even writing legal briefs. Yet, in our discussions, legal professionals regularly express concern about LLM security.  

• Do we risk a waiver of attorney-client or work-product privileges by sending our data to OpenAI? 
• What if that data includes confidential client information?
• Can LLMs learn from and share the information I send?
• How do large language model providers like Microsoft assure data security and confidentiality?
• What is the law governing these questions and how will it be applied?

Download a PDF of the slides here. 

During the webinar, we had additional insightful questions raised. Please take a look at our answers to those questions below and read our article, “Are LLMs Like GPT Secure? Or Do I Risk Waiving Attorney-Client or Work-Product Privileges?”, that inspired this webinar:

1. Does anonymizing input moot a/c concerns?

That is always an option but would be difficult to accomplish. For reasons stated during the program, we don’t think this is either necessary or practical with a commercial license.

2. ChatGPT states that “user feedback may be considered in shaping the direction of future updates,” so is it that isolated?

The open beta license does say this. Commercial licenses typically prohibit using prompts for training.

3. Is the fact that it can’t learn true for all GPT, including BARD and other LLMs?

To our knowledge, yes. It is a function of the LLM architecture.

4. It is helpful to know that it does not “learn” from current user prompts. But will the developers of GPT-4 incorporate user prompts entered into GPT-4 as training material for GPT-5?

There may never be a GPT 5 (surprising as that may seem). But, the answer is no for commercial licenses.

5. I was under the impression that Chat GPT was learning at an exponential rate because of the millions of users. Is that not correct?

No, it does not learn from your prompts, and particularly not prompts submitted under a commercial license, rather than the beta.

6. I read in the ‘Atlantic’ in an article about the founder of ChatGPT that he did not understand how ChatGPT learned how to code. Is it possible it can learn in some ways?

Anything is possible with these LLMs but they don’t learn from user prompts, at least not those submitted via a commercial license…

7. If the whiteboard is wiped, how does ChatGPT do successive prompts and save them so you can return to the session?

For ChatGPT, that information can be saved for reuse. For commercial licenses accessing GPT through the API, you have to resend the information each time.

8. When the next version of GPT is created, won’t that incorporate millions of user interactions from the prior version?

It may include training from the open ChatGPT beta. Not from commercial license users.

9. Does Chat GPT keep the info you put into the whiteboard after you are done with the session?

Yes for ChatGPT. No for access through commercial licenses.

10. On the user feedback point, does the feedback include the conversation/gpt response or just what you put in that feedback box?

To our knowledge just the information you provide. This may vary depending on the nature of the license.

11. Is it true that some vendors are building models or applications that place a different interface over a foundational model such as Chat GPT? What guarantee to users have that these overlay interfaces won’t be recording the prompts or data that are supplied by the user? And are there any circumstances in which the user wouldn’t know that they were interacting with or being exposed to an interface that collects data in this manner?

ChatGPT is not a foundational module. It is an application that you access through a web browser. To our knowledge commercial applications like we showed only access LLMs through an API. We suggest that commercial users review the license agreements and deal with reputable vendors.

12. Is confidentiality maintained in BARD, Claude, Falcon, etc. as it is in ChatGPT?

In general yes, but each is governed by its own license.

13. Unfortunately there has been a great deal of chatter about not putting private info in the context window when prompting Chat GPT or its equivalents. Would that make expectation of privacy not be customary? Would that mean that it is not reasonable to expect privacy until we get a court decision to say it’s OK?

Not in our view. We didn’t wait for the courts to approve email or cell phones. And confusion or misinformation does not negate a reasonable, objective expectation of privacy.

14. I asked GPT-4 whether it was trained on prompts from GPT-3. This was its response: “As of my last update in September 2021, OpenAI has not publicly disclosed specific details about whether GPT-4 was trained using user prompts made in GPT-3.” Doesn’t that indicate that there is not a reasonable expectation of privacy in prompts submitted to Open AI?

Not in our view. The ChatGPT 3.5 beta expressly stated that OpenAI would use prompt information for training the algorithm. However, OpenAI and Microsoft commercial licenses specifically exclude that right.

15. What is the cost for GPT enterprise as opposed to prior versions?

We don’t know. I would check with OpenAI.

16. Thoughts on how data retention policies might need to be modified to address prompts, etc.

That is not our area of expertise. But with a commercial license, prompts are maintained by your organization or software provider.

17. Restating earlier question: Using the hypothetical at hand re settlement letter, are Chat GPT concerns alleviated by anonymising names and amounts? Reminds me of using translation tools.

We don’t think that is either practical or necessary. However, we wouldn’t recommend using the free ChatGPT beta for settlement letters. Better to use the Enterprise version with the proper data restrictions.