Contact reason labeling audit checklist

What auditing labeling is for

Labeling contact reasons is the raw material for almost everything else: if conversations are not classified by reason, you cannot know which campaign brought price inquiries, which customer segment asks the most about billing, or how much of the volume is an inquiry that could be answered on its own. The problem is that labeling degrades without anyone touching it. Someone creates a new label because the existing one did not show up in their list. Another person writes "price inquiry" and another writes "price?". A category created for an old campaign stays alive and empty. Six months later, nobody knows what half the labels mean and the reports can no longer be read.

Think of a beauty salon with three people at the front desk. One labels inquiries as "appointment", another as "appointments", another as "book appointment", and a fourth also uses "appointment" for the reminders the business sends. Three months later, when they want to see how many appointments were requested via WhatsApp versus how many were confirmed, the number does not add up anywhere. It is not a data problem: it is a problem of labels that never had a written definition.

This audit is done once a quarter, on a sample of closed conversations from the period, not on all of them. It does not aim for labeling to be perfect: it aims for it to be usable. There are three questions: does everything that came in have a label? (coverage), is the same thing always labeled the same way? (consistency), are there labels that are not used and take up space? (dead categories).

Step 0: prepare the sample before looking at anything

The audit is ruined if you do it on the conversations you remember. You have to build the sample with a fixed rule and stick to it. Three decisions and you are ready to start.

1. 1Define the period: the full previous closed quarter, not "the last few months". A range with a start and end date, and do not change it halfway through.
2. 2Define the sample size. If the period has few conversations (fewer than 100), audit all of them. If it has hundreds or thousands, take a systematic sample: for example, every tenth closed conversation from the list, sorted by date. Systematic and not "the first 50", because the first 50 are usually from a single week and a single shift.
3. 3Split the sample into two buckets: conversations that came in through WhatsApp and conversations that came in through another channel (Instagram, Messenger). Reasons are not distributed equally and mixing them hides the problem of one channel.
4. 4Before opening the first conversation, export the complete list of existing labels with the number of contacts associated with each one. That list is the backbone of the audit and you will need it in all three stages.

Stage 1: coverage (is anything left untagged?)

Coverage measures what proportion of the sample has a reason tag assigned. It is not enough for the conversation to be tagged: it has to be tagged with a reason, not with the name of the agent who handled it or with the status ("closed", "pending"). Those are other dimensions and they contaminate the reason analysis.

How it is calculated: you count the conversations in the sample that have a reason tag and divide it by the total sample. That percentage is your coverage. Then you do the same calculation by channel and by shift, because coverage almost never fails evenly: it tends to drop on the night shift, on weekends and on the channel the team uses least.

• Overall coverage: conversations with a reason tag over total sample.
• Coverage by channel: WhatsApp, Instagram, Messenger. A channel with low coverage is usually the one the team has not incorporated into the flow.
• Coverage by shift or by agent: it helps to see whether the problem is with the process or with a person. If it is with a person, it is solved with a conversation; if it is with a shift, it is solved with a rule.
• Coverage by time slot: if conversations that come in outside business hours are left untagged, the reason report will be biased toward business hours.

Food service example: a restaurant with its own delivery discovers that overall coverage is high, but that conversations that come in after 23:00 are almost all left untagged. The reason report says that most inquiries are for reservations, when in reality the nighttime inquiries —which are about order delays and address changes— never entered the count. The bias is not in the data: it is in the shift that does not tag.

The signal that coverage is good is not 100%. A 100% usually means that someone tagged anything just to avoid leaving the conversation empty, and that breaks the consistency of the next stage. A reasonable floor to sustain the analysis is that a clear majority of the sample has a reason; if a large portion is left unclassified, the reason report does not represent the business.

Stage 2: consistency (is the same thing called the same?)

Here is the heavy work and this is where the problems that are not visible on a dashboard appear. Consistency is measured in two ways: between tags (that there are no two that mean the same thing) and within a tag (that the assignment criterion does not depend on who handled it).

For consistency between labels, go through the full list of labels and group those that refer to the same reason. You will find three typical patterns: the spelling variant ("price inquiry" and "inquiry about price"), the gender or number variant ("appointment" and "appointments"), and the level-of-detail variant ("inquiry" and "inquiry about stock for a specific product"). The first two are duplicates that must be merged. The third is not always: it can be a legitimate hierarchy, but in that case the general label should not be used when the specific one exists.

For consistency within a label, take the conversations in the sample that carry the same label and read them together. If you find three conversations with the same label that are actually three different reasons, the assignment criteria is not defined. If you find two conversations about the same reason with different labels, there is a duplicate or there is ambiguous criteria.

• Exact duplicates: two labels with the same text except for capitalization, accents or spaces. They are merged without discussion.
• Semantic duplicates: two labels that a new agent would not know how to distinguish. They are merged and only one is kept.
• Implicit hierarchies: a general label and a specific one that overlap. The rule for which one wins is defined and documented.
• Labels from another axis: status, channel, agent or priority mixed with reasons. They are separated; the reason is not the status.
• Per-person criteria: each agent has their own version of the same reason. It is resolved with a written one-line definition per label.

Stage 3: dead categories (which labels are not used?)

A dead category is a label that exists in the list but has almost no associated conversations, or that had none in the audited period. It is not the same as a legitimate low-frequency label: an inquiry about a product that sells little can be rare and valid. The problem is the label left over from an old campaign, from a test that nobody finished deleting, or from a category the business no longer handles.

How they are detected: with the list of labels and their contact count, sorted from lowest to highest. Those at zero or near zero are candidates. But before deleting you must distinguish three cases, because deleting the wrong label removes data you were actually using.

• Truly dead: zero conversations in the period and nobody on the team remembers having used it. It is archived or deleted.
• Seasonally dead: zero in the audited period but with use in the same quarter of the previous year (a year-end campaign, a special date). It is not deleted: it is marked as seasonal and removed from the active list so it does not clutter the report.
• Alive but misused: it has associated conversations, but when reading them the reason does not match the label name. It is not dead: it is a label with a broken definition. The definition is rewritten or it is renamed.

One caution with deletion: if the tool allows archiving instead of eliminating, archive. Deleting a label that has history can leave old conversations without retroactive classification and ruin any comparison against previous periods. Archiving removes it from the daily-use list without touching the past.

Audit table: what to review, what good looks like, and what to do if it fails

How to measure whether tagging improved between audits

An audit without a comparison against the previous one is a snapshot. To know whether the work paid off, save four numbers when you close each audit and compare them with those from the previous quarter. Nothing more than that is needed.

Concentration deserves a clarification because it is the metric that is most often misinterpreted. If 90% of your conversations fall under "general inquiry", you have high coverage and useless tagging: you cannot separate price from stock or from billing. The goal is not for every tag to have the same number of conversations (that would be forcing reality), but for the most used tags to be specific enough for the report to answer a business question.

What to do when something fails

When the audit flags a problem, the order of the repair matters. Fixing coverage before consistency generates more mistagged conversations. Fixing consistency before dead categories leaves duplicate tags that will be used again. The order that works is: first definitions, then merges, then coverage, and finally cleanup of what is dead.

• If coverage is low: don't add new labels. The problem isn't that options are missing, it's that labeling isn't part of the flow. Make it a mandatory step before closing the conversation and check whether the problem is concentrated in one shift.
• If there are duplicates: first merge the exact ones and the obvious semantic ones. One single label per reason. Before merging, check how many conversations each one has, so you don't lose the history of the one with the most usage.
• If the criterion is ambiguous: write a one-line definition for each label. If you can't write it in one line, the label is poorly thought out and needs to be split or merged.
• If a label is truly dead: archive it, don't delete it, so you don't break the historical record. If it's seasonal, mark it and remove it from the active list.
• If a label is alive but misused: don't delete it. Rename it or rewrite its definition; the problem is the name, not the label.
• If the team creates new labels without control: define who can create them and by what criteria. A new label for every spontaneous inquiry is the fastest way back to the mess this audit just cleaned up.

Store example: a hardware store selling via WhatsApp had three labels that looked like the same reason ("stock", "availability", and "do you have?"). After reading twenty conversations with each one together, it became clear that "stock" was used for catalog product inquiries, "availability" for products that had to be ordered from the supplier, and "do you have?" was the catch-all where everything that didn't fit landed. They weren't three reasons: they were two real reasons and one junk label. The first two were merged with a written rule and the third was archived.

How this connects to AI suggestions

If you use AI suggestions to reply, labeling has a second function besides reporting: it gives context to the suggestion. A conversation labeled as a billing inquiry isn't answered the same way as a stock one, and the difference is in the label. That's why messy labeling doesn't just ruin the report: it also makes the suggestion worse, because the system has no way of knowing what the conversation is about. Wando suggests replies and the team approves them, and it learns from the business's real replies; the label remains the human criterion that organizes the whole.

The audit doesn't change because there's AI in the middle. One thing changes: if the system suggests labels, check in the sample how many times the label suggestion was accepted without correction and how many times it was changed. A high percentage of corrections on one specific label is the earliest sign that this label has a definition problem, before it shows up in the reasons report. In a clinic, for example, if the "appointment" suggestion is systematically corrected to "reschedule appointment", the problem isn't the AI: it's that a label for the second case was missing.