By KYN AI Advisory Team — AI implementation specialists, Singapore
Reply classification sorts each prospect reply into one label so the right next step runs. Four labels do most of the work: interested, not now, wrong person and unsubscribe. Add out of office, question and complaint, and a reply fits one of seven. The value is in the definitions. A label with a vague definition produces vague results, however good the model is.
Why classify replies at all?
Because the next step differs by reply, and a person reading 80 replies a day makes slips. An unsubscribe missed on Tuesday becomes a complaint on Friday. An interested reply read late loses a week.
Classification makes three things routine: suppression, follow-up dates and hand-off to a rep. It does not write the commercial answer. It decides which queue the reply lands in and how fast.
How should each label be defined?
Write each label as a rule a stranger could apply. Include one clear example and one edge case.
Interested. The sender asks for a call, a quote, a demo or more detail about a specific offer.
- Example: "Can you send pricing for the Jakarta site?"
- Edge: "Maybe, send details" counts as interested only if a next step is named. Otherwise treat it as a question. See handling "send me more information" replies.
Not now. The sender is open later and gives or implies a timeframe.
- Example: "Come back after the audit in March."
- Edge: "Not interested" with no reason is a decline, not "not now". Log it as closed and stop the sequence.
Wrong person. The sender is not the right contact and may name someone else.
- Example: "Please speak to our finance manager."
- Edge: if a new name and address are given, create the contact as a draft for review. Do not email the new person automatically.
Unsubscribe. The sender asks to stop receiving messages, in any words.
- Example: "Remove me", "stop", "do not contact".
- Edge: "Please stop" in a longer angry message is still an unsubscribe and also a possible complaint. Both apply.
Out of office. An automatic message with a return date.
Question. The sender asks something that is not a buying signal.
Complaint or legal. The sender disputes how you got their details, mentions a regulator, a lawyer or a breach.
Which labels need rules before the model?
Two should never depend on a model alone: unsubscribe and complaint. Run a code check first for stop words and for regulator and legal terms. If the code check fires, the model's label is ignored for routing, and a person sees the reply.
This is also a defence against hostile text. A reply can carry instructions aimed at the agent, and a model may follow them. Treat the reply as data and keep control logic in code. More on this in prompt injection in business AI agents.
What does each label trigger?
| Label | Action | Owner | |---|---|---| | Interested | Alert the rep, stop the sequence, create a task | Named rep | | Not now | Set a follow-up date, pause the sequence | Agent, with date logged | | Wrong person | Stop for this contact, draft the new contact | Reviewer | | Unsubscribe | Add to the suppression list for all channels | Agent, logged | | Out of office | Pause until the return date | Agent | | Question | Draft an answer for review | Reviewer | | Complaint or legal | Halt the account and escalate | Named senior person |
Suppression should apply across email and messaging, not only the channel the reply came in on. The Singapore Spam Control Act 2007 requires an unsubscribe facility on unsolicited commercial email, and other countries set their own rules, so check the recipient's country.
How do you handle replies that fit two labels?
Allow one primary label and a list of flags. "Not now, but please remove me from the list" has the primary label unsubscribe, because the stop action is the safer one. Rank labels by the cost of getting them wrong.
- Complaint or legal
- Unsubscribe
- Interested
- Wrong person
- Question
- Not now
- Out of office
When two labels apply, the higher one wins for routing, and the lower one is kept as a flag in the log.
How do you test a classifier?
Collect 100 real replies from your own history. Two people label them independently. Resolve their disagreements, because those disagreements show where your definitions are weak. Then run the classifier.
- Report the match rate overall and per label.
- Report the two misses that cost most: unsubscribe missed, and interested missed.
- Re-run after any change to the prompt, the model or the label list.
Keep the 100 as a fixed test set and add new hard cases each month. For the wider measurement set, see how to measure an AI reply desk and the pipeline view in how an AI reply desk works.
What does a worked set of examples look like?
These replies are illustrative. Each shows the label and the reason, so a reviewer can see how the definitions apply.
| Reply text | Primary label | Flags | Reason | |---|---|---|---| | "Send me the Jakarta pricing." | Interested | Price request | Names an offer and asks for detail | | "Not this quarter. Try us in July." | Not now | Date: July | Gives a timeframe | | "I handle HR, not finance. Talk to Wei Ling." | Wrong person | New contact | Names someone else | | "Take me off your list." | Unsubscribe | None | Stop request in plain words | | "I am away until 3 November." | Out of office | Return date | Automatic, dated | | "How is this different from our ERP?" | Question | None | No buying signal yet | | "Where did you get my details? I am reporting this." | Complaint or legal | Data source | Regulator mention |
Add 10 examples per label to your own set, taken from real replies. Definitions improve fast when you argue over real text.
What mistakes do classifiers make?
- Politeness read as interest. "Sounds interesting, thanks" is often a soft decline. Treat it as a question unless a next step is named.
- Sarcasm and irritation. A short angry reply is usually a stop request. Route to a person.
- Forwarded replies. A colleague forwards your email with a note on top. The note is the reply, not the forwarded text.
- Reply to an earlier message. The sender answers step 1 after receiving step 3. Label from the content, then check the thread.
- Empty or image-only replies. A signature image or a thumbs-up emoji has no label. Send these to a person.
Where KYN fits
KYN runs reply classification for an insurance brokerage as part of its outreach system. Sorting one reply costs about S$0.01 in that live system, depending on the model and the text length. The classifier runs under its own restricted account, writes each label to the log with the actor, and cannot send. Messages wait for a named person. See sales systems.
The first 14 days are a review of your stack and interviews with your team. A full build ships in under 90 days. Everything runs in a cloud account in your name.
Frequently asked questions
How many labels should a reply classifier have?
Six to eight. Fewer labels are easier to test and route. If you need more, add flags rather than new primary labels.
Should the classifier see the whole thread or only the last reply?
Give it the last reply as the text to label and the thread as context. Labelling from the thread alone invites errors where the earlier message colours the reading of the new one.
What if the reply is in another language?
Include replies in each language you sell in when you build the 100-reply test set. Report accuracy by language. A classifier that is accurate in English may not be in Bahasa Indonesia or Chinese, so test each.
Checked 30 October 2026.
Curious whether Marketing agents fits your business? Talk to KYN on WhatsApp — no forms, just a conversation.
