Data Security in Outsourced Labeling: Questions to Ask a Vendor

By Stefan, founderPrivacy, compliance and security4 min read

Before you share any data with a labeling vendor, get clear, written answers to four groups of questions: what the contract says, where the data is stored and who can access it, who the annotators are and where they work and what happens in an incident and at the end of the project. A good vendor answers quickly and specifically. Vague answers are a warning sign.

Use the questions below as a checklist in your vendor conversations and compare the answers side by side.

Question A good answer sounds like
Will you sign an NDA before we share anything? "Yes, before any data or guidelines are shared."
Will you sign a data processing agreement that meets GDPR Article 28? "Yes; here is our standard DPA and we can review yours." See GDPR and AI Training Data.
Who exactly is the legal entity and in which country? A registered company name, address and registration number
Do you use subcontractors or other platforms to find annotators? A clear yes or no; if yes, who they are and under what terms
Who owns the labels and any derived data? "You do, fully."
Will you use our data for anything else, such as training your own models? "No, never."
Do you carry liability insurance? A clear answer with the type of cover

Ask for documents, not just answers: the DPA, a security overview and a list of sub-processors. A vendor that can't share them before signing probably doesn't have them.

Storage, Access and Tools

Question Why it matters
Where will our data be stored, physically and by which provider? Location decides which laws apply and whether transfer rules kick in
Can you work inside our own annotation tool? Data then never leaves your systems
Is data encrypted in transit and at rest? Basic protection if something is intercepted or stolen
Can annotators download, copy, print or screenshot data? Can that be blocked or logged? People copying data out is one of the easiest ways for it to leak
Is access limited per project and per person? Only the people on your project should see your data
Is there two-factor login and an audit log of who accessed what? Lets you investigate if something goes wrong
How are backups handled and are they deleted with the data? Forgotten backups are a common gap
How often do you test your security? The GDPR expects security to be tested and assessed regularly

The GDPR's security article (Article 32) names measures such as pseudonymisation and encryption, the ability to restore data after an incident and regular testing of security measures. A vendor should be able to explain how it meets each.

Questions About the People

In labeling, people see your data item by item. Ask about them as carefully as about the technology.

  • How are annotators vetted? Identity checks, skills tests, a human interview?
  • Are they employees or contractors and are they bound by confidentiality? Ask to see the clause.
  • Which countries do they work from? If your data contains personal data from the EU, access from outside the EU is a transfer under the GDPR. Can the vendor restrict your project to eligible countries?
  • How many people will see our data? Fewer is safer; ask for a small, stable team.
  • Do they work on personal or managed devices? And what rules apply to their workspace?
  • What training do they get on data protection?
  • What happens when someone leaves the project? Access should be removed the same day.
  • Is any work done by AI tools? Some vendors use AI to pre-label or check data. If your data goes into a third-party AI service, that is another place it travels.

At AnnoTrue, every annotator passes a skills test and a video call, signs an independent contractor agreement with confidentiality terms and works without downloading data. Projects can be restricted to eligible countries and client data is always labeled by people, never by AI.

Incidents, Deletion and Red Flags

Incidents

  • How quickly will you tell us about a breach? Under the GDPR, a processor must notify the controller "without undue delay" and the controller then has, where feasible, 72 hours to notify the authority (Article 33). The contract should set a concrete deadline.
  • Do you have a written incident response plan? Ask who leads it and how it is tested.
  • Have you had a data breach before? An honest answer, with what changed afterwards, is a good sign.

End of project

  • How and when will our data be deleted or returned? Including backups and annotators' working copies.
  • Will you confirm deletion in writing?
  • What do you keep, if anything and why? For example, invoices for accounting, but never the data itself.

Red flags

Red flag Why it matters
Won't sign a DPA, or wants data before signing Basic GDPR requirement missing
Can't say where data is stored or where annotators are They may not know themselves
Annotators can freely download data High leak risk
Unknown subcontractors or crowd platforms Your data may pass through many hands
Vague answers on deletion Copies may live on indefinitely
"We have strict security" without specifics A claim, not a control

Ask Us Anything on This List

AnnoTrue answers every question here in writing before you share any data. We sign an NDA and a data processing agreement first, store data in the EU and confirm deletion in writing at the end. Start with a free pilot.

Sources