Power Admin AI
← All posts

Trust & Compliance

Your AI's Client Messages Are Discoverable. Most Firms Can't Produce Them.

Firms spend their entire AI evaluation worrying about what the AI might say wrong. The bigger exposure is not being able to prove what it said at all. Here is what a defensible AI audit trail actually contains, and the six questions that expose a vendor who cannot give you one.

By Harry Hedaya10 min read

Every law firm that evaluates AI asks the same first question: what if it tells a client something wrong?

It is a fair question. It is also the smaller half of the risk.

Here is the version almost nobody asks. A former client files a bar grievance in March 2027. She claims your firm never told her about a settlement offer, never returned her calls, and left her in the dark for four months. Some of that period is covered by conversations your AI handled over SMS and email.

Now you have thirty days to produce a complete, timestamped record of every communication between your firm and that client. Not a summary. Not a screenshot. A record that shows who said what, when, through which channel, whether a human approved it, and what data the AI looked at before it answered.

Can your firm produce that today? Most cannot. And the firms that cannot are usually the ones who were most careful about the first question and never thought about the second.

The record problem existed before AI

This is not an AI problem that AI created. It is an old problem that AI makes visible.

Ask any managing partner where the complete communication history for a given client lives. The honest answer is that it lives in six places. Case notes in Litify or Filevine. Emails in somebody's Outlook. Texts in a personal cell phone or a Go High Level thread. Call logs in a phone system that retains recordings for ninety days if anyone bothered to turn recording on. Documents in a shared drive. Intake notes in Lead Docket that never got migrated after the case converted.

Reconstructing four months of client contact from that pile takes a paralegal two days and produces something with holes in it. Firms have been living with this for twenty years because the holes rarely got tested.

AI changes the exposure in two ways.

First, volume. A firm running an AI Super Agent across voice, SMS, and email is generating five to ten times the client touchpoints it generated before, because the AI answers at 9 PM on Sunday and a human did not. More contact means more record.

Second, attribution. When a paralegal sends a bad status update, the analysis is straightforward. A person made a judgment call. When an AI sends a bad status update, the first question from opposing counsel, from the bar, or from your carrier is: who supervised it, and prove it.

ABA Model Rule 5.3 has always required lawyers to supervise nonlawyer assistance. Formal Opinion 512, issued in 2024, made clear that using generative AI does not move that obligation somewhere else. Rule 1.1 comment 8 puts technology competence on you. Rule 1.4 puts client communication on you. None of those rules care whether the message came from a paralegal or a model.

Supervision you cannot document is supervision that did not happen, at least as far as a grievance committee is concerned.

What a defensible audit trail actually contains

Most vendors will tell you they "have logging." Logging is not an audit trail. Logging is a developer feature that helps engineers debug their own software. An audit trail is a legal artifact that survives being handed to somebody hostile.

Here is the difference, concretely. A real audit trail for a single AI-handled client interaction should contain all of the following:

The full message, verbatim, both directions. Inbound and outbound, exactly as the client saw it. Not a paraphrase, not a truncated preview, not a classification label like "status inquiry, resolved."

Channel and timestamp with timezone. SMS at 9:14 PM Eastern is a different fact than email at 9:14 AM Pacific.

The identity resolution. Which client record this message was matched to, and how confident the match was. When a firm has two clients named Maria Rodriguez, the record of which one the AI thought it was talking to matters enormously.

The data the AI read before responding. Case status, last activity date, document checklist, whatever fields were pulled from the CMS/CRM at the moment of the response. This is the single most important element and the one almost every vendor omits. If the AI told a client "your medical records are still outstanding," you need to be able to show that the CMS/CRM said exactly that at 9:14 PM on that date. Otherwise you are defending a hallucination claim with no evidence.

The approval path. Was this draft-mode with a named human approval, auto-sent under a specific rule, or escalated? If a human approved it, which human, and at what time. If it auto-sent, which rule authorized it.

The escalation record. Every message that got routed to a person, who it went to, and whether they responded.

Any write-back the AI performed. If the AI updated a phone number, created a callback task, or moved a case stage, that change belongs in the same trail as the conversation that triggered it.

Immutability and retention. The record cannot be editable after the fact, and it has to outlive your vendor relationship.

That last one is where most firms get hurt, so it deserves its own section.

The exit problem nobody negotiates

Here is a scenario that plays out more often than it should.

A firm runs an AI vendor for eighteen months. The relationship sours, or the vendor gets acquired, or the pricing changes and the firm switches. The firm cancels. Thirty days later the vendor deprecates the account and the conversation history goes with it.

Two years after that, a matter from that period surfaces. The firm has no record of a year and a half of client communication because it lived in a system the firm no longer has access to and never exported.

Your record retention obligation does not end when your vendor contract does. Most states require client file retention for five to seven years after a matter closes, and some categories run longer. No AI vendor's standard contract is written around that timeline.

The fix is not complicated, but it has to be decided before you sign, not after you leave.

The correct architecture is that the AI writes its complete communication record back into a system you own. For most firms that means the CMS/CRM, because Litify, Filevine, Clio, MyCase, and Salesforce all support activity records against a matter, and because your file retention policy already covers whatever lives there. The AI vendor's platform holds the working copy. Your CMS/CRM holds the record of truth.

If the vendor's answer to "where does the record live" is "in our dashboard," you are renting your own file.

Six questions that expose a vendor fast

You do not need to be technical to run this evaluation. Ask these in order and listen for hedging.

1. Show me a full export of one week of client conversations, in a format I can open without your software.

The answer should be a CSV, JSON, or PDF you can hold. If they can only show you a dashboard, the record does not really exist as a record. Watch for "we can build that for you" as a response to a feature that should already exist.

2. Does the export include the case data the AI read before each response?

This is the question that separates serious platforms from wrappers. Most tools log the conversation and nothing about the state of your CMS/CRM at the moment of the answer. Without it, you cannot distinguish "the AI made something up" from "the AI accurately reported what your system said, and your system was stale." Those are wildly different problems with wildly different liability.

3. For any given sent message, can you tell me whether a human approved it and who?

If the platform runs a draft-mode workflow, approval identity should be a field, not a guess. If it does not run draft mode at all, ask why not, and then ask question four with more energy.

4. If we cancel tomorrow, what happens to eighteen months of history, and when?

You want a written answer with a number of days in it, plus a documented export path. "We'd work with you on that" is not an answer.

5. Does the record write back into our CMS/CRM automatically, or only on request?

Automatic write-back is the only version that survives contact with reality. A manual export process is a process that runs for two months and then stops.

6. What is logged when the AI decides not to answer?

Escalations and refusals are the most important events in the system and the ones most likely to be silently dropped. If a client texted "I want to fire you and I'm calling the bar" at 11 PM and the AI escalated it, that escalation is the exact record you will want later. It should be as durable as the messages that got sent.

What this looks like when it is built right

At Power Admin AI, the audit trail is not a reporting feature bolted on at the end. It is the reason the AI Fusion architecture connects to your CMS/CRM through a real API instead of scraping a browser session.

Every inbound message, on every channel, gets matched to a case record and written back as an activity against that matter. Every outbound message is written back the same way, along with the fields the Super Agent read to compose it and the rule or human approval that authorized sending it. Every escalation is a record. Every write the AI performs to a contact record or case stage is logged against the conversation that caused it.

The practical effect is boring, which is the point. When a partner asks "what did we tell this client in April," the answer is one query against the system the firm already trusts, and it comes back complete. Not "here is what I could find," which is the answer that turns a nuisance grievance into a real problem.

It also changes how the trust-building phases work. In draft mode, the approval record accumulates as evidence that a human reviewed AI output before it reached clients. That record is what lets a firm confidently move a category to auto-send later, because there is a documented accuracy history behind the decision rather than a vendor's assurance. The audit trail is not just defense. It is the mechanism that lets you safely expand autonomy.

The one-hour test

You do not have to wait for a vendor evaluation to find out where you stand. Run this today with whatever systems you already have.

Pick a client from six months ago. Give a paralegal one hour and ask for a complete communication record: every call, text, email, and document exchange, in chronological order, with timestamps.

If it takes longer than an hour, or comes back with gaps, that is your actual exposure and it exists whether or not you ever buy AI. Adding AI to that environment without fixing the record layer multiplies the volume of communication you cannot account for.

Fixing the record layer first, and then adding AI that writes into it, gets you the opposite: more client contact and better documentation at the same time. That is the version worth buying.

The firms that get this right are not the ones with the most cautious AI. They are the ones who can prove, on any given day, exactly what their firm said to any given client and who signed off on it.


Find out what your record layer actually looks like

Book a 20-minute working session and we will map where your client communication history currently lives, where the gaps are, and what a write-back architecture would look like against your existing CMS/CRM. No deck, no pitch, just the map.

Book a working session or see how the AI Super Agent handles voice, SMS, and email.

Want to see this in action?

Try Voice AI free, or book a 20-minute call and we'll walk through your firm's numbers.