Legal AI tools used by law firms have a propensity to hallucinate—can anything be done?

Subscribe to enjoy similar stories.A few days ago, the law firm Sullivan & Cromwell apologized to the chief judge of Manhattan’s US Bankruptcy Court for a court filing with AI-hallucinated citations. Andrew Dietderich, co-head of the firm’s global restructuring group, wrote the judge that the firm’s “comprehensive policies and training requirements governing the use of AI tools” had not been followed. A secondary review process also failed.

A database of similar incidents that had around 90 entries a year ago now has 1,333. Many are from pro se litigants and small-firm practitioners. Now add one of America’s leading firms.

It’s unlikely this is just about law firms. Their errors are in public court filings. But all professional service firms, even ones like Boston Consulting Group, Goldman Sachs, and the Big Four accountants, have a similar business model—they have partners who oversee teams of associates and check their work.

The more associates per partner, the more the partners earn. That’s what’s known in professional services as leverage, and it’s so important that law firms are actually ranked by profits per partner. But this model means they’re all going to struggle to capture the economic gains artificial intelligence seems to promise, because partners’ ability to supervise and verify AI-enabled work will become the rate-limiting step to the firms’ growth.The model works because even though associates lack partners’ expertise, they mostly know when they don’t know something and can flag the points in the work they produce that need more senior eyes.

Call it an SOS Post-It note. Partners don’t have to check every citation and comma. They can focus on the places that genuinely require their expertise, so one