Robots · Methodology v1.0
How we rate robots (and why most get no score)
We rate a model only when at least two independent sources have looked at it, including at least one field trial or independent test; otherwise it says "Not enough evidence to rate". The score (0-100) adds independent evidence (up to 60 points, weighted by kind and by what the evidence found), spec transparency (up to 20: how many of the key specs for its type the maker publishes) and spec strength (up to 20: how its published specs rank against other catalogued models of the same type, only where at least 3 publish the figure).
What counts as evidence
- Government or academic field trial: 20 points.
- Independent third-party test: 20 points.
- Independent certification or government vetting list: 10 points (at most 1 counted).
- Independently documented deployment: 5 points (at most 2 counted).
The source has to be independent of the maker: a university, a government lab or agency, a standards body or certification lab, or a named operator's published record. A maker's brochure, press release or case study is not evidence here; we show it as a maker claim. A study run "in collaboration with" the maker still counts, and we say so.
Points are scaled by what the evidence found: supports the job fit × 1, mixed result × 0.5, shows a limitation × 0.25. Evidence on an earlier version of the same machine counts at 50%. Evidence points are capped at 60.
Certifications (for example EU drone class marks listed by EASA, or a safety certificate) show that a machine meets safety or airspace rules. They say nothing about how well it does the job, so they add points but never unlock a score on their own.
When we give a score
Only when a model has at least 2 independent sources, including at least 1 field trial or independent test. Otherwise the page says Not enough evidence to rate and lists what we found. Today that is 50 of 53 models.
The other two parts
- Spec transparency (0-20): the share of the key specs for its robot type that the maker publishes. Buyers can't compare what isn't published.
- Spec strength (0-20): for each key spec where "more" or "less" is clearly better (runtime, payload, coverage, noise…), where it ranks among catalogued models of the same type, as a percentile; we average them. Only specs published by at least 3 catalogued models count. If none qualify, this part is not scored and the score is out of 80 points, rescaled to 100.
Bands
- 75+ : Strong evidence of fit
- 55+ : Good evidence of fit
- 35+ : Some evidence, with limits
- 0+ : Weak evidence of fit
What the score is not
It is a whirrwork assessment, not a lab test. We have not tested these machines. Coverage rates and similar figures are maker claims. A high spec-strength score only means the published figures are high compared with the other models we've catalogued, which is a short list. Fit for your site depends on things no spec sheet covers: your floor, field, building, weather, rules and people.
Job fit
Each job in the directory lists the robot types used for it. The first one is its primary type; the rest are alternatives. Model pages list the jobs their type does. We don't score a model per job, because that would need job-specific thresholds we can't source yet.
Specs and prices
Every spec comes from the maker's official spec page or datasheet and carries the link and the date we checked it. If two pages disagree, or a value applies to an accessory rather than the robot, we leave it out. Published list prices are also recorded in Whirrwork's append-only price history.
Corrections
Spotted a wrong spec or a missing study? See corrections. We fix the data, not the wording.