In short
Ayonix publishes no accuracy percentage, because a figure without its threshold, dataset, gallery size and demographic breakdown cannot be reproduced. Accuracy for a deployment is established by a pilot on the site's own cameras, measuring both error types and reporting where the threshold was left.
Why an accuracy percentage is not a fact a buyer can use
A percentage is only a fact if it can be reproduced, and reproducing one requires the threshold it was measured at, the dataset it was measured on, the gallery size, the image quality and the demographic breakdown. None of those appears in a headline number, and changing any one of them moves the result far enough to describe the same system as excellent or unusable.
The practical test of any published figure is whether a buyer could repeat the measurement from what is printed next to it. Where they could not, the number is a marketing claim in the shape of a result.
| Variable | What it is | What happens if it is omitted |
|---|---|---|
| Threshold | The setting that trades false accepts against false rejects. | The most commonly omitted variable, and the one a vendor has most freedom to choose flatteringly. |
| Dataset | What the images were: visa portraits, mugshots, corridor camera captures. | Identical software produces wildly different figures on different data. Without it the number describes nothing. |
| Gallery size | How many identities the comparison searched against. | An identification error rate measured at a thousand enrolments does not transfer to a million, and not linearly. |
| Demographics | How error rates differ across age, sex and skin tone. | An aggregate averages the variation away — and the variation is usually the number that matters most. |
| Capture conditions | Pose, lighting, occlusion, motion, compression. | These move results more than most differences between algorithms do, and they are the part a site controls. |
The method
How accuracy is established for a specific deployment
Six steps. The output is a number that describes one installation — which is the only kind of accuracy figure a buyer can act on, and is not comparable between sites.
Establishing a deployment accuracy figure
Run in this order. Steps one and two bound everything that follows, which is why shortcutting them produces a measurement of the wrong thing.
-
Survey the cameras
Measure the pixels across a face at the point people actually pass, not at the centre of the frame. This is the ceiling on everything downstream, and it is measurable in a morning.
-
Enrol under the conditions the site will use
Not in a bright meeting room if the door is lit from behind. Enrolment quality that does not reflect the real process produces a measurement of a system nobody will deploy.
-
Run for a defined period on live streams
Recording every comparison, not only the alerts. A log of successes is not a measurement; the denominator is where the information is.
-
Count both errors
People the system failed to match, and people matched to the wrong identity. One without the other describes nothing, because either can be driven to zero at the expense of the other.
-
Break the result down by group
Age, sex and apparent skin tone, using the population that will actually use the system. An aggregate hides the variation that matters most for fairness, and acting on it requires seeing it.
-
Tune the threshold against what each error costs
At this site, for this door. Record where it was left, the reasoning, and who is permitted to change it. An unrecorded threshold cannot be defended when a decision made at it is challenged.
-
Measure throughput and exceptions too
End-to-end time including retries, the retry rate, the exception rate and how long an exception takes to clear. Accuracy alone does not tell you whether the hall flows.
-
Record how often operators overrule the system
Logged separately from the system’s output. If operators essentially never disagree, review has become a signature rather than a control — and that is only visible if you measure for it.
-
Repeat after any change
New camera position, new lighting, a changed enrolment process, a software version. Deployments drift, and a figure from a building that has since changed is not a current figure.
Terminology
The four error rates, and why vendors quote whichever flatters
Two for verification, two for identification. Knowing which is which is most of what it takes to read a vendor claim critically.
| Rate | Applies to | What it measures | Behaviour |
|---|---|---|---|
| FMR — false match rate | 1:1 verification | How often two different people are called the same. | Falls as the threshold rises. Does not change with gallery size. |
| FNMR — false non-match rate | 1:1 verification | How often the same person is not recognised. | Rises as the threshold rises. Does not change with gallery size. |
| FPIR — false positive identification rate | 1:N identification | How often someone not in the gallery is returned as a match. | Rises with gallery size. This is what drives watchlist alert volume. |
| FNIR — false negative identification rate | 1:N identification | How often someone in the gallery is missed. | Also moves with gallery size, which is why 1:N cannot be inferred from 1:1. |
FAR and FRR are the looser, older terms for the verification pair. They mean roughly the same thing and are less precise about what was being compared.
What independent evidence does exist
The strongest independent evidence in this field comes from the evaluations run by the US National Institute of Standards and Technology, which measures submitted algorithms on sequestered data under fixed conditions and publishes the results whatever they show. NIST does not endorse vendors and issues no certification, so a report is evidence about an algorithm rather than a recommendation of a supplier.
A laboratory result and a site result answer different questions. The evaluation says how an algorithm behaved on that data; the pilot says what this installation will do with these cameras, this lighting and these people. Both belong in a procurement decision, and neither substitutes for the other.
The Ayonix NIST record is published in full, with links to NIST’s own report cards rather than a summary of them.
Procurement
Six questions for any face recognition vendor
Including this one. A vendor that can answer all six has evidence; a vendor whose answer to any of them is a percentage has restated the claim rather than supported it.
- 01 Which algorithm identifier did you submit to an independent evaluation, and on what date?
- 02 At what threshold is your quoted figure measured, and on which dataset?
- 03 What is the demographic breakdown behind the aggregate you publish?
- 04 Which presentation attack instruments has liveness been tested against, and by which laboratory?
- 05 Will you run a pilot on our cameras, and will you report the failures as well as the matches?
- 06 Where do templates live, and what happens to recognition when the internet connection drops?
These are published so they can be used against Ayonix as readily as against anyone else. Because this site publishes no accuracy percentage, it cannot answer question two with a number either — the honest answer there is a pilot on your cameras, which is what we would propose.
Frequently asked questions
How accurate is Ayonix face recognition?
Answering that honestly requires naming a threshold, a dataset and a deployment, which is why the honest answer is a pilot rather than a number. This site publishes no accuracy percentage for any vendor including Ayonix. What we will do instead is run a measurement on your own cameras with your own population and report both error types — the people missed and the people matched to the wrong identity.
Why does Ayonix not publish an accuracy rate?
Because an accuracy percentage without its threshold, dataset, gallery size and demographic breakdown cannot be reproduced, and a buyer cannot act on a figure they cannot reproduce. The same algorithm can truthfully be described as 99.9 per cent or 95 per cent accurate depending on those four variables. A number that survives none of those questions is a marketing claim in the shape of a result.
What is the difference between FAR and FRR?
The false accept rate is how often the system accepts someone it should have rejected; the false reject rate is how often it rejects someone it should have accepted. They move in opposite directions as the threshold changes, so quoting one alone describes nothing — an algorithm can be made to look excellent at either by being poor at the other. In biometric evaluation these are more precisely reported as FMR and FNMR for verification, and FPIR and FNIR for identification.
How do I choose a matching threshold?
By deciding what each kind of error costs at that specific location, then measuring on your own data to find the setting that balances them accordingly. A false accept at a server room door and a false accept at a staff canteen are not comparable events. There is no universally correct threshold, and any vendor supplying one without asking about your application has not understood the question. Record the value and the reasoning, and name who may change it.
How many test subjects does a meaningful measurement need?
Enough that the error you are trying to measure occurs often enough to count. If you expect an error rate around one in a hundred, a test with fifty subjects and one pass each cannot distinguish that from one in fifty or one in five hundred. The practical approach is to define the smallest error rate that would change your decision, and size the test so that rate would produce enough events to be countable within the measurement window.
What should a pilot measure besides accuracy?
Throughput end to end including retries and exceptions; the retry rate; the exception rate and how long exceptions take to resolve; the distribution of results across demographic groups; and how often operators overrule the system. That last one is the measurement that reveals automation bias, and it is almost never collected.
Does a NIST result predict how the system will perform at our site?
No. It measures an algorithm on sequestered data under controlled conditions, which is strong evidence of independent scrutiny and weak evidence about your building. Your camera placement, lighting, mounting angles and population are not in the report, and they move results more than most differences between algorithms do.
How often should accuracy be re-measured?
After any change to camera placement, lighting or the enrolment process, and periodically regardless. Deployments drift: a tree grows in front of a window, a light fitting is replaced with a different colour temperature, an entrance is reconfigured. A system measured once at commissioning and never again is being trusted on the basis of a building that no longer exists.
Next step
Measure it on your own cameras
A pilot with acceptance criteria agreed before it starts, measuring both error types on your population, and reporting the result whatever it shows. That is the only accuracy figure anyone can plan against.