Most AI support products are sold as one promise. In practice there are three jobs, and each fails in a different way. Pick the layer by the failure you can live with, and pick the metric before you switch anything on.
| Layer | What the AI does | Typical failure | Metric that catches it |
|---|---|---|---|
| Drafting | Writes a reply that a person reviews and sends | Reviewers send drafts without reading them | Share of drafts sent unedited, next to reopen rate on drafted threads |
| Deflection | Answers before a person and steers the customer to self-serve | The customer gives up and the thread looks solved | Repeat contact on the same topic within 72 hours |
| Autonomous resolution | Replies and closes the thread with no person involved | A wrong answer or a commitment the customer’s team holds you to | Share of AI-closed threads reopened or escalated within 7 days |
The 72-hour and 7-day windows are our suggested starting points, not industry standards. Set them to how long your customers usually take to come back.
Layer 1: drafting
Drafting is the safest layer because a person still sends every message. Its failure is quiet. When drafts are right most of the time, reviewers stop reading them, and the one wrong draft goes out under their name.
Watch two numbers every week: the share of AI drafts sent with no edit, and the reopen rate on drafted threads against hand-written ones. Our starting rule: “If unedited sends stay above four in five for two weeks and reopens on drafted threads are higher than on hand-written threads, stop drafting that intent and fix the snippet or help article it came from.” Four in five is our threshold, not a benchmark.
Let drafts draw only on approved material, such as snippets and help articles, not on the model’s general knowledge of your product. A draft that cites no source is one a reviewer should rewrite.
Layer 2: deflection
Deflection means the AI answers and the customer does not come back to a person. The deflection rate looks good until you read the repeat-contact data.
The vendor definitions explain why. As of 2026-11-11, Zendesk’s help article, which describes the platform in place before May 18, 2026, says that for messaging AI agents “By default, the conversation ends 2 hours after the first message in the conversation”, and that this can be increased up to 72 hours. For email AI agents, “The conversation ends 72 hours after the first email in the conversation.” A customer who read an answer, went quiet and never wrote again can count as resolved under rules like these. Set your own definition from how your customers actually return.
Count repeat contact instead. Our rule: “If more than one in five deflected threads on a topic comes back with the same question within 72 hours, move that topic back to drafting until the answer is fixed.” Worked example with hypothetical numbers: a topic deflects 60 threads in a month, and 14 come back with the same question within 72 hours. That is about one in four, above the line, so the topic goes back to drafting.
Layer 3: autonomous resolution
This layer replies and closes the thread. Its main failure is rarely a wrong fact. It is a commitment: a refund nobody approved, a date for a fix, a statement about an outage, or a promise to delete data. Decide before go-live which topics the AI may never answer.
Start with this rule and change it only with evidence: “If the topic involves money, data deletion, security, an outage or a contract term, route it to a person, whatever the model’s confidence.”
Measure the failure directly. Track the share of AI-closed threads that reopened or escalated within 7 days, and keep a short log of every time a person had to correct something the AI said. Worked example with hypothetical numbers: a team closes 200 threads with the AI in a month, and 26 reopen within 7 days. That is about one in eight. Pull the same figure for human-closed threads from the same month before you judge it. If the AI figure is clearly higher, move the intents behind most of those 26 threads out of autonomy and review the rest in two weeks.
Slack Connect details to settle before go-live
Most AI guidance assumes an inbox. In a shared channel, four things need an answer in writing.
Who installs and approves the AI. Slack’s Slack Connect help article says admins can restrict who can invite external people, and that invitations may need admin approval. The same article does not address apps, so it does not say who can install or approve an app in a shared channel. Ask your Slack admin which workspace the AI runs in and who signs off on it, and record the answer.
Who sees the AI’s replies. The same article says people “can only access the channels they’ve been invited to join.” Our inference, not something Slack documents for AI replies: the customer’s invited colleagues can see what the bot posts in the shared channel. Test it before launch. Send a test deflection into a channel with a test external account and check what that account sees.
What the handoff carries. Confirm that the person who takes a handoff sees the whole thread, including the AI’s earlier replies. If your tool does not show that, write the handoff note yourself. When several customer colleagues post in one channel, reply to the person who asked the last question and name them in the first line.
Disclosure and data. Tell the customer early in the thread that they are talking to an AI. Sample wording for the first AI reply: “I’m an AI assistant for [Company]. If this needs a person, I’ll pass the thread to someone on the team, and they’ll reply here.” Do not promise a response time the handoff cannot meet. Ask the vendor in writing where conversation data is stored, how long it is kept and whether it is used for model training. Rules on disclosing AI interactions differ by country and sector, so take them to counsel. This page is not legal advice.
Languages need the same care. Sample 20 threads per language each week and have a reviewer who reads that language check them. If no such reviewer exists, the rule is “keep that language in drafting or human-only mode.”
What it costs
Vendor pricing shows what each vendor counts as a “resolution”, and that matters more than the headline price.
Zendesk’s allowance is per agent per month: as of 2026-11-11, the help article gives “15 automated resolutions/agent/month” on Enterprise, “10 automated resolutions/agent/month” on Professional and Growth, and “5 automated resolutions/agent/month” on Team. It states that “Light agents are not included in the default allocation calculation.” So a team of six Professional agents has 10 per agent, which is 60 if the allowance is counted across the team. Confirm with Zendesk how overage is counted, because the article gives no dollar figure per resolution.
Intercom lists Fin at “$0.99 per outcome”. For Fin used with an existing helpdesk, the page says “Minimum monthly commitment applies (e.g. 50 outcomes)” and “No seats required”. Worked example: if the commitment is 50 outcomes, 20 outcomes in a month still cost 50 times $0.99, which is $49.50, or about $2.48 per outcome you actually used. Compare cost per confirmed fix after the reopen window closes, not the list price. If 40 percent of outcomes are real fixes, each real fix costs $0.99 divided by 0.4, about $2.48.
Seats are a separate line from the AI cost. Slack’s pricing page lists Pro at “$7.25 USD per user / month, when paying annually” and $8.75 per user per month when paying monthly, Business+ at “$15 USD per user / month, when paying annually” and $18 per user per month when paying monthly, and Enterprise+ at “$45USD” per user per month when paying annually, with no monthly price. Those are Slack workspace prices, not support-AI prices. The page does not list Agentforce in any plan, so confirm with Slack before you budget for it.
Build one monthly table before you sign. Hypothetical example: 120 AI outcomes at $0.99 is $118.80. At 40 percent real fixes, that is 48 fixes. Add review time: 40 drafts a day at 45 seconds each is 30 minutes a day, about 10 hours a month over 20 working days. Multiply those hours by your own loaded hourly cost and put that line next to the vendor fee. At this volume, the review hours cost more than the $118.80 outcome line once your loaded rate passes about $11.88 an hour, so the review line usually decides whether drafting pays.
Go-live checklist for the first 30 days
Record a 30-day baseline before go-live: human first response time, reopen rate, and CSAT on closed threads, split by topic. Without it, the day-30 review has nothing to compare against.
Replay before you commit. Take 150 recent threads, run them through the candidate’s trial or your own draft setup, and score each answer as right, incomplete or wrong. Compare wrong-answer counts per intent, not headline accuracy. Any wrong answer on a commitment keeps that intent in drafting.
Choose the first intents by three tests. They are asked at least 20 times a month (our floor), they have one answer that does not change per customer, and they need no account data, money or exception handling. Examples include how to connect an integration, where to find an API key in the docs, and which region an export lands in. Keep billing changes, refunds, bug reports and anything that asks “is this a bug” with people for the whole first 30 days.
Write the escalation path before go-live. Hand off when the customer asks for a person, when the AI’s last two replies did not answer the question, or when the topic is on the do-not-automate list. A specific handoff message helps: “I’m passing this to someone on our team who can check your account. They’ll reply in this thread.”
Split CSAT by path. Tag each thread by the path it took: AI only, AI then human, or human only. A thread where a human took over after an AI reply is tagged AI then human, even when a person sent the last reply. Report CSAT per tag, so a good AI score cannot hide a bad handoff.
Set targets before launch and measure AI and human first replies separately, using the definitions in the Slack Connect first response time guide. Starting targets: an AI first reply within 2 minutes on in-scope intents, and a human reply on every escalated thread within one business hour of the handoff. Use the first response time calculator for handoffs that arrive outside business hours.
Review the 90th percentile of escalated threads every week. If it rises above four business hours for two weeks, check the causes in this order before you change scope. First, routing: were the handoffs assigned at all? Second, the wait between handoff and assignment in the hours when it rises. Third, staffing in those hours. Fourth, which intents produce the slow escalations. Shrink the automated scope only if the slow escalations come from intents that should not have been automated. Otherwise fix routing, staffing or handoff speed.
On day 30, each intent gets one of three outcomes: keep it autonomous, move it back to drafting, or retire it. Write the decision and the number behind it next to the intent in your tracker.