Notable Quotes
"An AI agent can be accurate on the narrow task defined in the test dataset and still fail the user completely."
"When people rewrite everything that the AI gives them, you know that you have not earned the user's trust."
"It finished the job, but nobody trusted the output."
"If you keep showing someone a flash sale, then people are trained to assume that another one is always around the corner."
"You cannot A/B test a probabilistic system the way you would test a button color."
Takeaways
A losing experiment is paid-for information. Groupon's flash-sale flop revealed the lever was wrong, not the goal — scarcity beat price-based urgency.
Acceptance rate is the trust metric. The share of output users keep without editing is the strongest available proxy for trust.
DART measures behavior, not opinions. Four signals read off logs and transcripts: decay, acceptance, relevance, and task completion.
