A two-partner firm buys a tool in March because a peer recommended it. By June, one partner uses it daily, the other has never logged in, and nobody can say whether it saved any time. The subscription renews anyway.

That is the common shape of a disappointing AI purchase. The product usually works. What was missing was a decision, made before the card came out, about what the thing was for.

Four questions settle it. They take an afternoon.

What decision will this tool not make?

Start here, not with features. Write one sentence naming the judgment that stays with a person no matter what the tool outputs.

For a drafting tool: it does not decide what advice a client receives. For a research tool: it does not decide what is true, only what to go verify. For a meeting summarizer: it does not decide what goes in the client file.

That sentence is worth more than a feature comparison. It tells your team where the tool ends, which is the part people get wrong when the output reads well. In my experience, the failures are rarely the tool being bad. They are someone treating a confident draft as a finished decision.

What does it need to know, and who feeds it?

Every useful AI tool runs on inputs. A tool that drafts client emails needs your tone, your standard disclosures, and your service definitions. A tool that summarizes calls needs to know what your firm considers material.

Somebody has to supply that, and keep supplying it. If nobody owns the inputs, the tool produces generic output and the team quietly stops using it. Name the person before you buy. If no one has the hours, that is your answer for this quarter.

Which of your data can it see?

For a firm holding client financial information, this is the question with teeth. Read what the vendor says about training on your data, where it is stored, and how it is deleted. Ask whether client-identifying detail needs to touch the tool at all, since often the work can be done on anonymized text.

If your firm is regulated, loop in your compliance officer before the trial, not after. Vendor diligence and recordkeeping obligations do not pause for a pilot. Your counsel, not a blog post, is the authority on what your obligations require.

How will you know in ninety days whether it worked?

Pick one measure before you start, and keep it modest. Hours back on a specific task. Turnaround time on a recurring deliverable. Number of drafts that reached a client without rework.

Write the current number down. Look again in ninety days. If it has not moved, cancel without embarrassment. A tool that did not fit is a cheap lesson if you find out in a quarter and an expensive habit if you find out in three years.

What this looks like in practice

A short document. Four headings, a few sentences under each, and a date to revisit. That is the whole exercise.

It sounds slow next to the pace of announcements in this space. It is faster than the alternative, which is a stack of half-used subscriptions and a team that has learned to distrust anything new the firm buys. We build the same discipline into our own work, which is what Considered Build describes, and the underlying competencies are laid out on the AI fluency page.

The firms getting real value here are not the ones with the most tools. They are the ones who can say, in a sentence, what each tool is responsible for and what it is not.

If you want an outside read on where your firm's systems stand before you add anything to them, The Second Look is free and takes about four minutes.