What automated accessibility testing finds, and what it misses (the 57% figure)
Published: 7 min readThe AccessibleOrNot team
Automated accessibility testing scans a page's code against a set of rules and finds WCAG failures without human input. According to a Deque study based on more than 2,000 audits, these tests catch about 57% of the accessibility issues found on real sites, by volume. The remaining 43% needs a person to check the page with a keyboard, a screen reader and their own eyes.
In short
- 57% is a share of the number of issues found, not of WCAG criteria: more than 2,000 audits, more than 13,000 pages, about 300,000 issues (Deque, March 2021).
- Common issues (contrast, missing alt text, unlabelled form fields) can be found automatically, which pushes up the share by volume.
- Automated testing can't tell whether alt text is accurate, whether keyboard flow makes sense or whether an error message is clear.
- No automated tool can prove on its own that a site meets WCAG.
Where does the 57% figure come from?
In a study published in March 2021, Deque Systems looked at data from sites audited for the first time: more than 2,000 audits, more than 13,000 pages, about 300,000 accessibility issues. The audits combined automated testing (axe-core rules) and manual testing. The result: 57% of all issues had been found by automated testing.
Why is the gap so large? Because issues aren't evenly distributed. Errors that can be found automatically, such as low contrast, missing alt text and unlabelled form fields, repeat hundreds of times across sites. In WebAIM's 2026 scan, 83.9% of home pages had low contrast text and 53.1% had images with missing alt text.
One more caveat: the study is based on audits by a single vendor, using its own tool. The figure is a guide; you won't get the same share on every site.
Source: Deque Systems, WebAIM
What does automated testing find, and what does it miss?
| Issue | Automated testing | Why |
|---|---|---|
| Image has no alt text | Finds it | Whether the attribute exists is visible in the code |
| Alt text exists but is wrong ("IMG_2041.jpg") | Partly | Catches some patterns, can't judge meaning |
| Form field has no label | Finds it | The label–field relationship is visible in the code |
| Button has no accessible name | Finds it | Name computation can be expressed as a rule |
| Text contrast below 4.5:1 | Mostly finds it | Can't be sure about text on images and gradients |
| Page language not set | Finds it | The lang attribute is visible in the code |
| Custom menu doesn't open with a keyboard | Misses it | Behaviour has to be tested by using the page |
| Keyboard trap in a pop-up | Misses it | Where focus goes has to be followed |
| Illogical focus order | Misses it | Logic needs human judgement |
| Unclear error message | Misses it | The meaning of the text has to be judged |
| Wrong video captions | Misses it | The content has to be compared |
Engines like axe-core mark cases they can't be sure about as "needs review", not as failures. This is deliberate: to avoid false alarms. So a clean automated report doesn't mean "no issues"; it means "no issues that can be found automatically".
Source: Deque Systems, Deque Systems, GitHub
If it's limited, why start with automated testing?
- Volume: it lists more than half of the issues found in minutes. You don't spend hours of manual testing on them.
- Repetition: it repeats the same checks at no cost after every theme update and on every campaign page. You notice a broken fix the next day.
- Location: it shows each issue with the page, the element and the WCAG criterion, which makes it easy to report to a developer or platform.
- Prioritisation: a list sorted by severity shows you where to focus the remaining human checks.
A 20-minute manual test for the other 43%
It doesn't replace an expert audit, but it finds the most critical issues automated testing can't see. Do it on your most used flow (checkout, application, booking).
- Keyboard (8 min): put the mouse away. Complete the flow from start to finish with Tab, Shift+Tab, Enter, Space and the arrow keys. Is focus always visible? Do menus, pop-ups and the side cart open and close? Do you get stuck anywhere?
- Zoom (3 min): zoom your browser to 200%. Is text cut off, or do buttons overlap?
- Forms (4 min): submit the checkout form with deliberate mistakes. Is each error stated in text, next to the field, or is there only a red border?
- Screen reader (5 min): turn on the free NVDA on Windows, or the built-in VoiceOver on Mac and iPhone. On a product page, are the image, price, sizes and add to cart button announced clearly?
These four checks cover a large part of the Level A criteria that can't be found automatically (2.1.1 Keyboard, 2.1.2 No Keyboard Trap, 2.4.3 Focus Order, 3.3.1 Error Identification), which every WCAG-based law includes.
If a tool says "your site is compliant"
No automated tool can prove on its own that a site meets WCAG, the EAA or any other law. In April 2025, the US FTC barred an accessibility tool vendor (accessiBe) from claiming without evidence that it makes any website WCAG-compliant. We cover this in our post on overlays.
Our own scan has the same limit. It lists the issues that can be found automatically, shows where each one is fixed and gives you a manual checklist for the rest, much like the one in this post. It never says "your site is compliant".
Frequently asked questions
Is automated accessibility testing enough?
No. According to Deque's study, it finds about 57% of issues by volume. Keyboard use, focus order, the meaning of error messages and the accuracy of alt text need a human check.
What's the difference between 57% and 20–30%?
57% is calculated from the number of issues found. 20–30% is the industry's accepted estimate of how many WCAG criteria can be fully tested automatically. They measure different things.
Which free tools can I start with?
For automated testing: the axe DevTools browser extension, WAVE or the accessibility section of Lighthouse. For manual testing: NVDA on Windows, and VoiceOver on Mac and iPhone, are free.
Does a Lighthouse score of 100 mean my site is accessible?
No. Lighthouse's accessibility score is based on a subset of automated checks; 100 only means no failures were found in those checks.
Sources
- Automated testing study identifies 57 percent of digital accessibility issues · Deque Systems · 10 March 2021
- The WebAIM Million, 2026 report · WebAIM · February 2026
- axe-core rule descriptions · Deque Systems, GitHub · current
- Presidential Circular No. 2025/10, Official Gazette No. 32933 (in Turkish) · Lexpera · 21 June 2025
- WCAG 2.2 · W3C · 5 October 2023 (current version)
- FTC Approves Final Order Requiring accessiBe to Pay $1 Million · US Federal Trade Commission (FTC) · 22 April 2025
Related articles
Close the 57% that can be found automatically first.
Enter your URL. We list the automated findings in order of severity and add a manual checklist for the rest to your report.