CASE STUDY . 02 . DESIGNING FOR AI . CONCEPT STUDY
Same bot. New brain. The design problem flipped.
In 2022, the hard part was teaching the bot what to say. In 2026, the hard part is teaching it what not to.
ROLE
Concept study, building on my AI design work at Reece
COMPANY
Reece USA
YEAR
2022 reimagined for the LLM era
FOCUS
Conversational design. AI guardrails. Trust.
WHAT 2022 TAUGHT ME
Three AI features, all before the LLM era. Predictable, but limited.
At Reece we designed three AI powered experiences.
Reece Chat let customers search products in natural words inside the mobile app, with guided choices upfront, branch hours, find a part, search a product, so customers saw what the bot could do before typing something it couldn't handle.
Reece Vision let customers photograph a part and find the same part, similar options, or a replacement part; it solved the question a contractor can't type: "What is this, and what replaces it?"
And an employee support chatbot on ServiceNow served 1,500 employees across HR and IT.
All three taught the same lesson.
The hardest design problem was never the happy path. It was the moment the system reached its limit: the search that didn't match, the photo the model wasn't sure about, the question outside the bot's intents. Designing those moments well was the difference between AI people trusted and AI people avoided.
WHAT FLIPPED WITH LLM'S
Then: the bot returned results. Now: it can hold a conversation about the job.
My 2022 customer chat could find "centerset bathroom faucet." It could not answer the question underneath: "I'm replacing a faucet on a small vanity with three holes, four inches apart, what fits?" An LLM can. It can ask the clarifying question, narrow the options, and explain the tradeoffs like a good counter person would.
The design job flipped from matching words to catalog items, to deciding what the assistant should handle, what it must never guess, and when a counter person takes over.
THE REDESIGN: THREE ZONES OF TRUST
Where AI advises freely. Where it must use live data. Where a human takes over.
Zone One - Open Conversation
-
Understanding the customer's job
-
Asking clarifying questions
-
Comparing options
-
Explaining what's installable
The LLM works like a knowledgeable counter person.
Zone Two - Exact data only
-
Prices
-
Stock at the branch
-
Specs
-
Compatibility
The assistant must never guess these from memory. It should pull live answers from the catalog and inventory systems and show them exactly.
Zone Three - Handoff
-
Big orders
-
Returns
-
Anything where the customer is frustrated or the stakes are high.
A warm handoff to the branch with the full conversation attached. In 2022, this was the "View all results" link.
MEASURING QUALITY
A bot that answers everything is easy. A bot you can trust is the work.
Four signals I would design in from day one: how often the assistant resolves a question without a human (containment), how often customers correct or rephrase (a sign the answer missed), the quality of handoffs, measured by whether the human picks up with context intact, and whether conversations end in a confident purchase or an abandoned one.