A model learns whatever its training data teaches it, mistakes included. Wrong, inconsistent or one-sided labels put a ceiling on performance that no amount of extra computing power removes. And if the test data is wrong too, you can't even measure the problem.
That is not just a saying. Several well-known studies have measured it and their results point the same way: a smaller amount of carefully checked data often beats a larger amount of careless data.
What the Research Shows
Three studies are especially useful for anyone deciding how much to invest in data quality.
1. Even famous datasets contain many wrong labels. In 2021, researchers from MIT and elsewhere checked the test sets of 10 widely used benchmark datasets (Northcutt, Athalye and Mueller). They estimated an average of at least 3.3% label errors across them and about 6% in the ImageNet validation set. Worse, the errors changed which model looked best: on corrected data, a smaller model could beat a larger one that seemed better on the original labels. In other words, bad test labels led to choosing the wrong model.
2. A small, carefully chosen set can be enough. In 2023, the LIMA study (Zhou and others) fine-tuned a large language model on only 1.000 carefully selected prompts and answers. In human evaluations, its answers were judged equivalent to or better than GPT-4's in 43% of cases. The authors' conclusion: careful selection of a small set mattered more than volume.
3. Good human feedback can beat model size. In OpenAI's InstructGPT study (Ouyang and others, 2022), people preferred answers from a model trained on human feedback with 1.3 billion parameters over those of the 175-billion-parameter GPT-3. The difference came from the quality of the human training signal, not from size.
The lesson for a business is simple: before buying more data or a bigger model, make sure the data you have is right.
What "Quality" Means in Training Data
Quality is more than correct labels. A dataset can have every label right and still train a poor model. Five properties matter:
| Property | The question it answers | How it is checked |
|---|---|---|
| Accuracy | Are the labels correct? | Gold items, reviewer samples, client spot checks |
| Consistency | Are similar items labeled the same way, by everyone, every day? | Agreement between annotators on overlapping items |
| Coverage | Does the data include the cases the model will meet, including rare ones? | Compare the data's mix with real traffic; list missing cases |
| Balance | Are groups and categories represented fairly? | Counts per category, language, region or group |
| Freshness | Does the data still reflect today's products, rules and language? | Dates of the data; review after major changes |
Accuracy gets the most attention, but consistency is often the bigger problem. If half the team calls an item "complaint" and the other half "question", the model learns a coin toss. Consistency mostly comes from clear guidelines, which is why we put so much weight on them in How to Write Annotation Guidelines.
How Bad Data Hurts a Model
Poor data rarely causes a dramatic failure. It causes quiet, expensive problems that show up after launch.
Wrong labels teach wrong rules. If refund requests are often labeled "billing question", the model learns to send them to the wrong team and keeps doing it.
Inconsistent labels lower the ceiling. When annotators disagree, the model can't do better than the confusion it was shown. Training longer won't fix it.
Missing cases become blind spots. A document model trained only on clean scans fails on phone photos. A chatbot trained on polite questions struggles with angry ones.
Unbalanced data becomes unfair output. If one group, language or region is barely present, the model works worse for it. In hiring, lending or health, that is also a legal risk under rules such as the EU AI Act. See The EU AI Act and Training Data.
Wrong test data hides all of the above. As the 2021 study showed, errors in the test set can make a worse model look better. Teams then ship the wrong model with confidence.
Fixing it later costs more. Relabeling after training means paying for the labels twice, plus the training and the time lost. Checking quality while labeling is far cheaper.
How to Get Quality Data
Quality is built into the process, not inspected at the end. The steps that matter most:
- Write clear guidelines with examples for every label and rules for unclear cases.
- Sample data that looks like real use, including rare and messy items.
- Test annotators before real work and use experts where the content needs them.
- Run a pilot and fix the guidelines before scaling.
- Check continuously: hidden gold items, overlap between annotators, reviewer sampling and batch acceptance.
- Keep a clean test set, labeled with extra care and reviewed twice, so you measure the model on the truth.
- Use model errors to decide what to label next.
The full process is in The Data Labeling Process: A Step-by-Step Guide and the checks in Quality Control in Data Labeling: Methods That Work.
Quality You Can Check
AnnoTrue checks every project at five levels and sends a quality report with each batch. If a batch misses the agreed accuracy, we fix it free of charge within 5 working days. See it for yourself with a free pilot.
Sources
- Northcutt, Athalye and Mueller, Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks, NeurIPS 2021
- Zhou and others, LIMA: Less Is More for Alignment, 2023
- Ouyang and others, Training language models to follow instructions with human feedback, 2022