Are We Entering the Era of Prompt Engineers for Legal?
By John Tredennick
Many think this will be the next career opportunity for legal technologists. I am not so sure? Why is that? Because GPT itself is a master at creating effective prompts. Watch this. And I was hardly trying.
Here was our TREC topic:
“Slot Machines — All documents concerning the definition, legality, and licensing of “slot machines” in Florida. What are the key issues around slot machines in Florida?”
We found 200 likely relevant documents about slot machines and had them analyzed by GPT 4.0 Turbo (128k). Here was GPT’s analysis of the topic:
“The topic of slot machines in Florida encompasses a range of issues related to their definition, legality, and licensing within the state. The key issues include:
- Legal and Regulatory Framework: There is a significant focus on the need to clarify existing statutes under Chapter 849 of Florida law to better define and regulate slot machines versus legal “amusement games” (Document 253501, Document 253502, Document 253558, Document 253571, Document 253675, Document 253701).
- Court Rulings and Immediate Installation: A Broward County court’s decision allowed for the immediate installation of slot machines at Hollywood Greyhound, highlighting the role of the judiciary in the ongoing legal battles and decisions impacting the slot machine timeline and process (Document 237379).
- Pari-Mutuel Facilities and Greyhound Racing: Concerns have been raised about the impact of slot machine legislation on the greyhound racing industry, with calls for maintaining live racing schedules and mandated purse payments (Document 243208, Document 243224, Document 243558, Document 243779, Document 244261, Document 244341, Document 244353, Document 244365, Document 244366, Document 244367, Document 244423, Document 244425, Document 244426, Document 244436, Document 244471).
- Senior Arcades and “Gray Machines”: The classification of certain video machines in senior arcades as legal games of skill or illegal slot machines is debated, with recent enforcement actions against senior arcades (Document 254165, Document 254170, Document 254225, Document 255060, Document 255270, Document 256203, Document 256546, Document 256572).
- Economic Impact and Competition: The economic impact of slot machines on the state and competition with other forms of gambling, such as Indian casinos, cruise ships, and the lottery, is discussed, with suggestions to use slot machine revenues to support the pari-mutuel betting industry (Document 243558).
- Public Sentiment and Political Stance: Public sentiment against the expansion of gambling and slot machines in Florida is evident, with citizens urging Governor Bush to oppose such measures (Document 202334, Document 202633).
- Legislative Action: The need for a special legislative session to address slot machine legislation and the potential for a bill to be submitted regarding the use of bill acceptors in slot machines is mentioned (Document 242947, Document 257243, Document 257343, Document 257421, Document 257428).
- Voter-Approved Amendment: A voter-approved constitutional amendment for slot machines in Broward County indicates that public opinion via referendums plays a role in the legality of slot machines (Document 242947).
- Penny Arcades/Slots: The growing popularity of “penny arcades” or “gray machines” in Florida is discussed, with concerns about their lack of regulation and potential impact on problem gambling and the elderly (Document 244261, Document 244341, Document 244353, Document 244363, Document 244365, Document 244366, Document 244367).
- Revenue Allocation: There is a request for slot machine revenues to be used to enhance purses for greyhound racing, suggesting a link between slot machine profits and support for the racing industry (Document 243779).
These issues highlight the complexity of the slot machine debate in Florida, encompassing legal definitions, regulatory challenges, economic considerations, industry impacts, public opinion, and political actions.”
At that point we had a good handle on the topic area. So we simply asked GPT to create a prompt to review and classify additional documents while we headed out to dinner.
We need to review additional documents for relevance to our topic and the information you have provided. Create a prompt we can use to review additional likely relevant documents so we can tell whether they are relevant or not. The prompt should instruct GPT to summarize the document and tell us why it is or is not relevant. GPT should also rank the documents on a scale of 100 to 1 with 100 being the most relevant. Please act as a senior prompt engineer and create the best possible prompt for this assignment.”
Here was GPT’s response:
“As a senior prompt engineer, I would suggest the following prompt for reviewing additional documents for relevance to the topic of slot machines in Florida:
Prompt for Document Review:
You are tasked with reviewing documents to determine their relevance to the topic of slot machines in Florida, specifically concerning their definition, legality, and licensing. For each document you review, please provide a brief summary of its content and then explain why it is or is not relevant to the topic. Consider the following factors in your relevance assessment:
- Does the document discuss the legal status of slot machines in Florida?
- Does it address legislative actions or court rulings related to slot machines?
- Does it mention key stakeholders such as government officials, the pari-mutuel industry, or Native American tribes?
- Does it provide information on public opinion or voter initiatives regarding slot machines?
- Does it cover regulatory issues, economic impacts, or social concerns associated with slot machines?
After summarizing and assessing the document, please rank its relevance on a scale from 100 to 1, with 100 being the most relevant to the topic. Use the following scale as a guide:
- 100-91: Directly discusses slot machines in Florida, including legal definitions, legislative actions, court rulings, or regulatory frameworks.
- 90-81: Addresses the economic or social impact of slot machines in Florida, or provides detailed information on stakeholder positions or actions.
- 80-71: Mentions slot machines in the context of gambling in Florida but may not focus on the specific issues of legality or licensing.
- 70-61: Contains tangential information related to gambling or gaming in Florida that could indirectly affect slot machines.
- 60-51: Briefly references slot machines or related gambling activities in Florida without substantial detail or focus.
- 50-1: Lacks significant content related to slot machines in Florida or discusses unrelated topics.
Please provide your summary, relevance explanation, and relevance ranking for each document reviewed.
This prompt is designed to guide the reviewer through a structured analysis of each document, ensuring that the summary and relevance assessment are aligned with the key aspects of the topic. The ranking scale provides clear criteria for determining the level of relevance, facilitating consistent evaluations across multiple documents.”
I am not suggesting that prompt engineers won’t be required but they will likely be working to polish prompts rather than create them from scratch. At the least you have to be continuously amazed at what these Large Language Models can do.
About the Authors
John Tredennick (JT@Merlin.Tech) is the CEO and founder of Merlin Search Technologies, a software company leveraging generative AI and cloud technologies to make investigation and discovery workflow faster, easier, and less expensive. Prior to founding Merlin, Tredennick had a distinguished career as a trial lawyer and litigation partner at a national law firm.
With his expertise in legal technology, he founded Catalyst in 2000, an international ediscovery technology company that was acquired in 2019 by a large public company. Tredennick regularly speaks and writes on legal technology and AI topics, and has authored eight books and dozens of articles. He has also served as Chair of the ABA’s Law Practice Management Section.
About Merlin Search Technologies
Merlin is a pioneering cloud technology company leveraging generative AI and cloud technologies to re-engineer legal investigation and discovery workflows. Our next generation platform integrates GenAI and machine learning to make the process faster, easier, and less expensive. We’ve also introduced Cloud Utility Pricing, an innovative software hosting model that charges by the hour instead of by the month, saving clients substantial savings on discovery costs when they turn off their sites.
With over twenty years of experience, our team has built and hosted discovery platforms for many of the largest corporations and law firms in the world. Learn more at merlin.tech.
John Tredennick, CEO and founder of Merlin Search Technologies
JT@Merlin.Tech
Transforming Discovery with GenAI
Take a look at our research and Generative AI platform integration work on our GenAI Page.
Subscribe
Get the latest news and insights delivered straight to your inbox!
