Text annotation means adding labels to written language so a model can learn to understand it. The three most common kinds are classification (one or more categories for a whole text), sentiment and intent (what the writer feels or wants) and entity tagging (marking names, dates, amounts and other items inside the text).
Text is where most AI projects start, because so much business data is text: emails, chats, reviews, tickets and documents. It looks simple, but language is full of overlaps, sarcasm and mixed messages, so clear rules matter as much here as anywhere.
Classification
Classification puts each text into one or more categories from a fixed list. It powers ticket routing, spam filters, content moderation and document sorting.
| Variant | How it works | Example |
|---|---|---|
| Single-label | Exactly one category per text | Each support email gets one team: Billing, Cards, Loans or Other |
| Multi-label | Any number of categories can apply | A news article is both Politics and Economy |
| Hierarchical | Categories in levels, from broad to specific | Cards → Lost or stolen → Abroad |
| Binary | Yes or no | Is this message spam? |
Choose single-label only if the categories truly exclude each other. Otherwise annotators are forced to pick one of two right answers and agreement drops. If you need single-label for routing, add a decision rule that says which category wins.
Always include an "Other" or "None" option and check what lands in it. A growing "Other" pile usually means a missing category.
Keep hierarchies shallow. Two or three levels are manageable; deeper trees cause confusion between neighbouring branches.
Sentiment and Intent
Sentiment captures how the writer feels. Intent captures what they want to happen. They are often labeled together, because "angry and wants a refund" calls for a different reply than "angry and just venting".
Choosing a sentiment scale:
| Scale | Labels | Good for |
|---|---|---|
| 3-point | Negative, Neutral, Positive | Most business uses; easiest to agree on |
| 5-point | Very negative to very positive | Reviews, where strength matters |
| With "Mixed" | Adds a label for both at once | Reviews and feedback, which often praise and complain together |
| Aspect-based | Sentiment per topic in the text | "Food: positive, service: negative" |
More points mean finer detail but lower agreement. Start with three plus "Mixed" unless the model needs more.
The hard cases and rules that help:
- Sarcasm: "Great, another fee. Just what I needed." Label the meaning, not the words: negative.
- Mixed feelings: "Slow delivery, but the product is excellent." Use Mixed, or aspect-based labels.
- Polite complaints: "I'd appreciate it if this could be fixed soon." Courteous, but the intent is a complaint.
- Facts without feelings: "My card expires next month." Neutral, even if the topic sounds negative.
Intent labels should be actions your system can take: Change booking, Cancel, Ask for refund, Report a problem. If two intents lead to the same action, merge them.
Entities and Spans
Entity tagging marks specific pieces of a text, called spans and gives each a type. It teaches models to pull facts out of free text.
Example:
[Maria Ingram] PERSON paid [$450] AMOUNT to [Banca Transilvania] ORGANIZATION on [03.03.2026] DATE.
Common entity types are people, organisations, places, dates, amounts, products and IDs; projects usually add their own, such as policy number or drug name.
Span boundaries cause most disagreements. Decide in the guidelines:
- Do titles belong to the name? "Dr Maria Ingram" or "Maria Ingram"?
- Are currency symbols part of the amount? "$450" or "450"?
- What about punctuation and spaces at the edges?
- How are nested entities handled? "University of Chicago" is an organisation that contains a place.
- Are pronouns ("she", "the company") tagged?
Write one rule for each, with examples. Many span disagreements disappear once boundaries are fixed in writing.
Relations go one step further and link entities: Maria Ingram → paid → $450. They are slower to annotate and need their own rules.
A deeper guide is in Named Entity Recognition Annotation: A Practical Guide.
Making Text Annotation Consistent
- Use real examples from your own data in the guidelines, including messy ones with typos, slang and mixed languages.
- Write decision rules for overlaps between categories and between sentiment labels.
- Fix span boundaries in writing before anyone starts.
- Use native or near-native speakers for tone, slang and sarcasm.
- Label a share of texts twice to measure agreement per label; see Inter-Annotator Agreement Explained Simply.
- Remove personal data you don't need before labeling; text often contains names, emails and account numbers. See How to Anonymize Data Before Sending It for Annotation.
Text Annotation by Careful Readers
AnnoTrue annotators are tested on reading, writing and following rules before they see any client data. See the quality before you commit, with a free pilot.