by Andrew Dang 3L Student at Stanford Law School
Businesses faced a crucial question when Chat-GPT was launched on November 30, 2020: How much should they depend on Generative AI to run their business? The consensus in the legal industry is that AI will not replace lawyers. However, tech-savvy attorneys are likely to outpace those who do not use it. LLMs, such as GPT-4, automate routine tasks in the legal industry like drafting contracts and summarizing documents. The legal industry is not going to be overtaken by this wave of automation, which is similar to the disruption Uber brought about in the transportation sector. Leading legal organizations are already investing heavily in AI, with LexisNexis integrating AI and Thomson Reuters acquiring Casetext for $650 million, illustrating a survival-of-the-fittest scenario.
Adopting AI does not guarantee success. It requires constant adaptation. The legal battleground illustrates this well: if both sides use GPT-4, the advantage may neutralize, leading to a new phase of AI competition–customization. Legal generative AI must be reliable and controllable in order to become an asset, rather than a liability. Open-source LLMs excel in this area, as they offer transparency and adaptability compared to closed-source models. Open-source AI is the best choice to help the legal industry navigate this new terrain. It offers transparency and adaptability, unlike closed-source models.
Domain-Specific Models: The Case for Their Use
LLMs are able to combine vast amounts of information, which is especially useful when using Retrieval Augmentation. LLMs can be limited by the training data they have and are prone to “hallucination,” however RAG acts as a conduit between models and external databases. RAG relies on technologies such as vector databases that store and manage data to allow for efficient retrieval and embedded models which help categorize and understand text. Although RAG systems are dominated by proprietary closed-source models, organizations should also consider open-source alternatives.
The fine-tuned models perform better than generic models for domain-specific tasks such as separating legal facts from everyday fact. Wikipedia is not a good source of data to train a model that can understand the technical terms used in law. Legal queries may appear simple, but they are nuanced. A surface-level understanding is not sufficient. If asked what the licensor should do when a licensee makes improvements to their data, embedding model might confuse “Licensor has to incorporate the enhancements in their data,” with “Licensor can use all the rights that are not granted to the licensee.”
To overcome the above issues, legal organizations can fine tune open-source embedded models using domain-specific data. As an example, embeddings models like Word2Vec and GloVe that are further trained using legal datasets better capture the legal language nuances. Legal BERT outperformed ChatGPT in the LexGlue Benchmark. This model was pre-trained using the CaseHold dataset. BAAI’s bge-largeen-v1.5, an open-source program from BAAI, ranks third in MTBE benchmark. Massive Text Embedding Benchmark evaluates eight tasks: “Bitext Mining, Classification Clustering Pair Classification Reranking Retrieval Semantic Textual Similarity Summarization”. We showed several examples of large embedding and language models that performed better when trained with domain-specific tasks. We can use the same principles to assume that models at the top of embedding benchmarks perform better when they are trained on domain-specific problems.
Big Tech & Big Data
Big Tech may allow for fine tuning of their embedding model. However, due to their lack of transparency, Big Tech should not be trusted with proprietary data, as it could compromise data privacy. Open-source models, on the other hand offer transparency and data protection for legal organizations. In this data economy, user data is the currency. Legal organizations must protect confidential data. Legislative initiatives like the European Artificial Intelligence Act impose transparency obligations on AI vendors, like Big Tech. The AI Act, which takes inspiration from GDPR’s transparency commitment, is reflected throughout the legislation. The AI Act is designed to protect the “fundamental rights and freedoms” of people, including their right to privacy whenever their personal data are processed.
Stanford’s Transparency Index still states that Big Tech foundation models do not meet the EU AI Act transparency requirements. Big Tech’s lack transparency in model architecture, evaluation data, and training data pose a risk to businesses that rely on black-boxed models. This undermines trust between stakeholders and clients who are concerned about the use of their data and whether AI systems can lead to discriminatory outcomes or harm. Google’s docket records reveal privacy violations. Google is still violating data protection laws. This was evident in Google LLC vs. YouTube where both companies were fined 170 dollars for violating the Children’s Online Privacy Protection Act. Google’s $5 Billion lawsuit against Brown v Google LLC for tracking users incognito session is another example.
OpenAI is also lacking in transparency. OpenAI models are difficult to understand, making it hard to identify biases and flaws. OpenAI’s abrupt firing and rehiring of Sam Altman also raises concerns. OpenAI’s ChatGPT has prompted tech companies to engage in an AI arms-race, focusing on profit before safety. OpenAI, despite its influence in the technology industry, is silent on Altman’s dismissal. The only information that the public knows is that Altman was “not consistently candid” in his communication with the board.
Stakeholders and clients are concerned about the lack of transparency in Big Tech. Businesses rely on AI-powered systems as data is becoming the currency of knowledge economies. With the growth of black-box AI, third-party liability is an issue. Businesses that rely heavily on Big Tech models run the risk of losing their clients and stakeholders, who have legitimate concerns about how data is being used and whether AI could result in discriminatory or harmful outcomes.
Data Privacy Laws
OpenAI was the subject of two class actions in 2023. A second class action was brought against several states that are adopting the EU privacy framework. They have passed legislation to regulate AI’s “heightened risks of harm” as well as affirming individuals’ rights regarding personal information.
As the legal landscape of AI and data privacy changes, so does how AI technologies will be developed and used. Legal organizations need to adapt to the evolving legal landscape and ensure that their AI systems respect data subjects’ rights and creative contributions of individuals. Open-source models help companies align their AI systems with state laws that are emerging, particularly in respect of intellectual property rights and data privacy.
AI Alignment & the Adversarial nature of the Law
AI alignment is a significant part of AI. AI alignment is the development of AI systems that align their goals and behaviors with human values. AI alignment is based on the harm reduction principle. In the legal system, the adversarial aspect is ingrained. The adversarial nature of litigation requires both sides to make efforts to improve their chances of winning.
Legal organizations who remain complacent in the face of this convergence will lose their adversarial battle. However, proactive legal organizations are likely to win. The implementation of an AI system by a firm does not guarantee success. Legal organizations need to constantly train and update their AI systems. The landscape of AI compliance and regulations is still uncertain. The U.S. may choose a product-liability approach to AI regulations, focusing AI on products entering commerce.
Congress could align itself with the EU and extend compliance requirements to AI systems. In the face of regulatory uncertainty, it becomes more important to develop a domain-specific open-source model. These models provide better control, tailored compliance with legal standards and flexibility to adapt to changing regulations.
Winning AI with “GPU Poor Budget
Organisations are debating how to integrate generative AI in their infrastructure. Most enterprises rely upon third-party providers to integrate AI. Vendors take care of the GPU costs, putting developers at a disadvantage. Integration of AI via an API is much easier than hosting a model locally. A standard GPU-accelerated AWS instance can cost more than $2,000 per month to run a large language models like LLaMa 2 and GPT-3.5. GPT-4 will cost more than $4,000 per instance per month. The “GPU-Poor”, or organizations lacking the necessary hardware, cannot use the models that come with the software.
Open-source frameworks are a cost-saving option for legal organizations. LoRA and Q-LoRA, for example, are both “GPU-poor” training tools. LoRA reduces performance and hardware issues by only training low-rank matrices, instead of the entire model. Q-LoRA compresses the model and trains LoRA adapters using four-bit quantization. This further reduces memory usage. Flash-Attention 2, a framework by Tri Dao, is a tool for “GPU-Poor” users. FlashAttention-2 offers accelerated performance of 2x through parallelism. Flash-Attention 2 allows developers to replicate GPT3-175B for 90% less cost. Estimated cost of training LLMS such as GPT-3 is $4,600,000. FlashAttention-2, however, replicates this training for a fraction of that cost. It costs about $458.136. Organizations can use open-source resources such as the OpenAccess-AI-Collective’s Axolotl, which includes implementations of techniques like LoRA, Q-LoRA, and FlashAttention2 in their model training codes.
In the case of LLMs, there is a new paradigm whereby smaller models outperform models twice as large. Microsoft has announced Phi 1,5, a model with 1.5 billion parameters. Phi 1.5, despite its small size, outperformed Facebook/Meta’s Llama-2 model with seven billion parameters.
Mistral AI’s first model, Mistral 7B, reinforces the notion that “bigger doesn’t necessarily mean better.” Mistral 7B outperformed all models in the 7B class and even Facebook/Meta’s Llama213 B model, on benchmarks that measure diverse AI capabilities. Zephyr 7b Beta, a finely tuned version of Mistral outperforms ChatGPT-3.5, Llama-2 70b, and other benchmarks.
Organizations should strategically deploy their AI systems. The use of open-source frameworks can save organisations money on hardware and tokens. Open-source frameworks give legal organizations the control to make changes when necessary, allowing them to experiment and implement new methodologies instead of waiting for providers to update their models. Open-source continues to reduce the barriers to scaling and deploying AI models.
Open-Source LLMs: The Case for Open-Source
No industry can ignore the impact of AI on labor, and the legal field is no exception. Open-source large language model can reduce the risks of artificial intelligence for organizations.
The language complexity of the law is different from that of everyday language. Flagship embedding model excels in everyday tasks. Fine-tuned embedding models are better suited to complex tasks that are required by law. The privacy concerns are greater than the benefits of fine-tuning embedded models that are closed-source. Data, such as client and trade secrets of companies, must be kept safe from Big Tech. Big Tech’s history of litigation and lack of transparency prove that Big Tech is not trustworthy with private data.
Legal organizations should also prepare for any regulatory changes that could affect the use of AI. Privacy frameworks such as the EU AI Act emphasize confidentiality, accountability and transparency when it comes to AI deployments. This aligns with open-source principles. The adversarial nature in legal practice presents unique challenges to AI alignment. Legal systems are based on rigid and sometimes opposing positions. Incorporating AI requires a balanced approach that takes into account technological innovation, ethical concerns, and regulatory compliance. Legal organizations who invest in specialized expertise, data curation and open-source AI will be better prepared to face the challenges that lie ahead.
—————————————————————————————————
[1] MTEB Leaderboard – a Hugging Face Space by mteb, https://huggingface.co/spaces/mteb/leaderboard (last visited Nov 28, 2023).
[2] Rishi Bommasani et al., The Foundation Model Transparency Index, (2023), http://arxiv.org/abs/2310.12941 (last visited Nov 27, 2023).
[3] Albert Q. Jiang et al., Mistral 7B, (2023), http://arxiv.org/abs/2310.06825 (last visited Nov 28, 2023).
[4] HuggingFaceH4/zephyr-7b-beta * Hugging Face, (2023), https://huggingface.co/HuggingFaceH4/zephyr-7b-beta (last visited Nov 28, 2023).