AI annotation platform reviews: verified Reddit evidence for 11 platforms
One evidence-based review per platform, built the same way each time. This page lists them alphabetically with each review's claim counts shown, so you can judge the platforms rather than take our word for it.
These are worker reviews of 11 platforms that post AI evaluation, AI training and data annotation work. 9 of them were built from a stored Reddit corpus of 19,483 comments in total, and every claim printed had to be stated by at least two different accounts in two different threads before it appeared; the counts below are how many claims cleared that bar in each direction. The reviews are listed alphabetically, not ranked. Last review update: October 21, 2026.
Annotation Academy is independent of every platform listed. No platform has paid for a review, a listing or a position on this page. Platforms can respond to anything in a review through the contact page; corrections are made on the page with the date.
The 11 reviews
Each card shows the review's own opening sentences, the size of the corpus it was built from and how many claims cleared the evidence bar: positive, neutral and negative.
AfterQuery
Updated Oct 2, 2026AfterQuery Experts is a real platform, and people on Reddit say they have been paid by it: 16 accounts across 10 threads describe receiving payment. The same corpus shows the friction that matters before you put in hours.
- Corpus
- 327 comments, 44 threads, 227 accounts (March 2025 to September 2026)
- Claims that cleared the bar
- Not counted for this review
Alignerr
Updated Oct 12, 2026Yes, Alignerr is legit. It is Labelbox's contributor-facing brand and it does pay people.
- Corpus
- We read 62 Reddit threads and cross-checked them against the 60 Alignerr roles live on our job board.
- Claims that cleared the bar
- Not counted for this review
DataAnnotation
Updated Sep 15, 2026DataAnnotation is real and it pays for approved work. Workers with a year or more on the platform report being paid for every task they submitted, on time (5 accounts across 2 threads, 2025 to 2026).
- Corpus
- 7,514 comments, 41 threads, 2,313 accounts (July 2023 to July 2026)
- Claims that cleared the bar
- 67 verified: 11 positive, 39 neutral, 17 negative; 22 did not clear it
Ethos
Updated Sep 15, 2026Ethos is real, and contributors report being paid for completed work (5 accounts across 4 threads, 2026). The mechanism to understand before you start is that logged hours go into a quality review with no fixed deadline, and nothing is paid until that review clears (5 accounts across 4 threads, 2026).
- Corpus
- 918 comments, 33 threads, 435 accounts (April 2026 to September 2026)
- Claims that cleared the bar
- 22 verified: 5 positive, 8 neutral, 9 negative; 28 did not clear it
Handshake AI
Updated Oct 21, 2026Yes, in the sense that matters: Handshake AI is a real platform and it pays people, weekly through a linked Stripe account (7 accounts across 7 threads, 2025 to 2026; 3 accounts across 3 threads, 2026). The thing to understand first is that you are not hired, you are matched to a project, and everything else follows from that match.
- Corpus
- 1,615 comments, 38 threads, 995 accounts (June 2025 to July 2026)
- Claims that cleared the bar
- 37 verified: 5 positive, 14 neutral, 18 negative; 3 did not clear it
Mercor
Updated Oct 12, 2026Mercor pays, and that is not seriously disputed. The risk is continuity.
- Corpus
- combining first-hand experience with public reports.
- Claims that cleared the bar
- Not counted for this review
Micro1
Updated Oct 12, 2026Micro1 is real and it pays people. Contributors placed on projects describe timely payments and steady work (6 accounts across 4 threads, 2025 to 2026), and accounts with direct experience call it legitimate but highly competitive and selective (4 accounts across 2 threads, 2026).
- Corpus
- 2,450 comments, 51 threads, 1,243 accounts (December 2024 to July 2026)
- Claims that cleared the bar
- 37 verified: 10 positive, 8 neutral, 19 negative; 4 did not clear it
Mindrift
Updated Sep 15, 2026Mindrift is a real platform, and the threads we read include people describing tasks approved and paid. The record we could verify, though, is mostly complaints, and most of them come back to one mechanism: getting through onboarding does not mean tasks arrive, because availability is tied to individual client projects rather than a steady stream of work.
- Corpus
- 496 comments, 42 threads, 285 accounts (December 2023 to July 2026)
- Claims that cleared the bar
- 15 verified: 0 positive, 4 neutral, 11 negative; 0 did not clear it
Outlier
Updated Sep 15, 2026Outlier is real and it does pay people. It is the contributor-facing brand of Scale AI, and contributors report being paid the full amount shown in their earnings tab, without missing or altered payments: 5 accounts across 2 threads, 2024 to 2025.
- Corpus
- 4,212 comments, 40 threads, 2,151 accounts (May 2024 to July 2026)
- Claims that cleared the bar
- 58 verified: 18 positive, 7 neutral, 33 negative; 33 did not clear it
Prolific
Updated Sep 23, 2026Prolific is a real research-participant platform: researchers post studies, participants complete them for payment, and Prolific's own rules require researchers to pay at least £6/$8 per hour (it recommends £9/$12). Beyond that, the Reddit evidence is thin.
- Corpus
- 1,141 comments, 30 threads, 559 accounts (July 2019 to September 2026)
- Claims that cleared the bar
- 1 verified: 0 positive, 1 neutral, 0 negative; 176 did not clear it
Welocalize
Updated Oct 12, 2026Welocalize is a real company. It is a localisation and AI-data services business that hires contractors for rating, review and translation work, it publishes a rate on its listings, and long-standing contributors describe being paid without incident: 5 accounts across 2 threads, 2025 to 2026.
- Corpus
- 810 comments, 40 threads, 379 accounts (October 2022 to July 2026)
- Claims that cleared the bar
- 16 verified: 2 positive, 4 neutral, 10 negative; 0 did not clear it
Counts are generated from each review's stored corpus and verified-claim files and update when a review is re-harvested. A low verified count can mean a quiet platform as much as a clean one: Prolific's single verified claim comes from 1,141 comments. Glassdoor, Trustpilot and Indeed scores are not part of the corpus and are not shown here. Pay figures, where a review prints any, are the platform's own published rates with their date; see the earnings disclaimer.
How every review is built
The same procedure for each platform, so the reviews can be read side by side.
We read the platform's own pages and listings, then a set of Reddit threads inside and outside the platform's own subreddit. A claim is printed only when at least two different accounts in two different threads state it, and a negative claim about the company only when at least five do; the counts in each review say how many accounts and threads stand behind every sentence. We never estimate pay, we show both sides when reports conflict, and we say when the evidence is old. The full rules, including what we refuse to print, are on the methodology page.
Common questions
Are AI annotation jobs legit?
The platforms reviewed here are real companies, and in each review people describe being paid for approved work. What the reviews also show, platform by platform, is where the friction sits: unpaid qualification tests, waits before a first project, work that appears and disappears, and accounts closed without a stated reason. Read the review for the platform you are considering before you put in hours.
How do I tell whether an annotation opportunity is legitimate if it asks for an unpaid test or personal information?
An unpaid qualification test is normal on the real platforms; a request for money, for a payment to unlock work, or for banking details before any contract exists is not. Apply only through the platform's own domain (each review links the platform's own statement), check that the recruiter's email matches that domain, and compare what you are told with what the review documents about how the platform onboards people.
Which platform pays the most?
We do not estimate pay. Where a platform publishes a rate on its own listings, the review shows that figure with its date, and our dated table of advertised rates by platform is kept on the AI training platforms comparison. Forum pay figures cannot be checked, so no review prints them as fact. Advertised rates by platform.