by Dr. Megan Ma, Associate Director, CodeX, and Campbell Hutcheson, CodeX Collaborator
Early this year, we began to explore the potential of supervisory AI agents as “intelligent alarms” for generative AI integrations in legal workflows. We consider that supervisory AI agents provide multi-prong risk mitigation by behaving both as a guardrail and continuous monitoring for legal professionals that use generative AI in their work processes. In the short paper, we had outlined a sample case study of how AI agents may be trained to help better triage and flag incidents of possible noncompliance and inconsistent behavior with explicit, formalized regulation.
We intended to further develop this line of research by building a prototype that envisions how a local small model can be trained on the professional rules of conduct and act as an “AI Professional Code of Conduct Officer,” flagging issues that appear to breach client confidentiality, attorney-client privilege, or client consent, then stopping these data inputs from being sent to enterprise generative AI tools, and redirecting them for further review to the human lawyer.
The demo video recorded is an early prototype of a platform that leverages the Practical Guidance from the State Bar of California and allows a client to select their preferences with regards to (1) data usage and transfer; and (2) generative AI use. Given these parameters, the small model will monitor and identify moments in which an attorney’s behavior may be inconsistent with the conditions specified by the client. The goal is to ensure transparency of communication between attorneys and clients, as well as enable an added verification mechanism that responsible conduct is upheld.
Demo Video
Future Work
We consider that there may be multiple ways in which supervisory AI agents may be deployed. One area of exploration is the development of supervisory AI-as-browser-plugins to mitigate risks when using generative AI that leverages web browsers. This would allow all uses of generative AI outside of enterprise platforms to have added guardrails and continuous monitoring capabilities afforded by the AI Professional Code of Conduct Officer. Furthermore, this would encourage a broader scope of accountability and governance for legal professionals facing uncertainty of use. We will continue to explore additional model rules of professional conduct and aim to release the prototype in open source as a next step in the short-term.