I build AI features for a living, so I am supposed to be bullish on models everywhere, all the time. I am not. Some problems are just not language problems, and calling an LLM is an expensive, fuzzy way to guess.

Ask what job the user is actually trying to do

A classic trap: the PM says “we want a copilot for onboarding.” You wire up streaming, the demo looks amazing in the room, and then real users need to click three times to do what a form would have solved in one. The model is not evil here; the design skipped the part where you respect human attention.

When I get skeptical, I run through this in my head:

• Is the answer in our database? If yes, show it. A table, a chart, a “here is your last invoice” card beats a wall of text.
• Is the user choosing among known options? A select or radio list with search is still UX from 2010, and it is fast.
• Do we need creativity or compliance? Creative drafts: model shines. Rigid policy checks: I want code and tests, not vibes.

Cost is not just money

Token bills matter, but so does time. I have watched a checkout flow that added “AI help” in front of a button users already knew how to use. Median time-to-complete went up, not down. The metric that improved was demo applause, which does not pay servers.

So when does it make sense?

I reach for a model when the input is messy language and the output needs soft judgment: summarization, triage, turning rough notes into a structured draft, suggesting next steps. I also like LLMs for turning structured data into human-readable explanation—after I already fetched the data with boring old SQL.

Saying no to AI in a feature is not a lack of vision. It is the same instinct that keeps you from Redis before you have measured the bottleneck. The hot take is: ship the product, then add intelligence where the pain is real, not where the press release demands it.