Skip to content
CS Capstone Handbook

Accessibility Testing

Your team has a working interface: a form that submits, a dashboard that loads, a flow you can demo. Some of the people who use it will do so with a screen reader, with a keyboard and no mouse, with the browser zoomed to 200%, or with low vision. This page covers the three checks that find what a mouse and a good pair of eyes miss: automated rules, a manual pass you run yourself, and a session with someone who uses assistive technology every day. It doesn’t teach you how to write accessible markup; the pattern libraries under Additional Readings do that.

Teams often leave accessibility to the last sprint, as polish. Treat it like any other correctness property instead: a missing form label breaks the page for a screen-reader user the same way a null pointer breaks it for everyone, so the check belongs in CI next to your tests.

The reference standard is WCAG 2.2 (Web Content Accessibility Guidelines) at level AA. Its success criteria turn “accessible” into statements you can test, such as a 4.5:1 contrast ratio for body text, a text alternative for every meaningful image, and every function operable from a keyboard, so a team checks its work against a list instead of an impression. Laws often name an older version, 2.0 or 2.1, and the W3C intends content that conforms to WCAG 2.2 to conform to 2.0 and 2.1 as well. Targeting 2.2 AA covers them, with one gap: 2.2 removed the 4.1.1 Parsing criterion, which a policy bound to an older version may still ask you to test.

Three kinds of check sit behind that target, and each answers a different question. Automated rules tell you whether the mechanical requirements are met. Your own manual pass tells you whether the software can be used without a mouse or without sight. A session with someone who relies on assistive technology every day tells you whether it works for the people who need it, which the first two can’t, because both are run by people who don’t depend on the tools they’re trying out.

The Testing Strategy guide covers the rest of the testing picture, and these checks run in the same CI pipeline and the same pre-demo routine as everything there.

Automated Accessibility Testing

Section titled “Automated Accessibility Testing”

Automated checks run on every pull request without anyone remembering to run them, which makes them the right home for the mechanical rules. They come in three layers, cheapest first, and each sees something the one before can’t: source code, a rendered component, and a whole running page.

  • Lint level. eslint-plugin-jsx-a11y reads your JSX before anything renders. Its rules flag missing alt text (alt-text), labels with no associated control (label-has-associated-control), links and headings with no content, a page with no lang, and click handlers with no keyboard equivalent (click-events-have-key-events). It runs in the editor, so it’s the cheapest layer, but it sees only source: anything that depends on computed styles, such as contrast, is invisible to it.
  • Component and page level. axe-core runs inside your existing test suite against the rendered DOM and fails a test when a component violates a rule. Its maintainers state that it reports no false positives, and it marks what it can’t decide as “incomplete” for a person to review. Its color-contrast rule doesn’t work under JSDOM, so check contrast in a real browser through an end-to-end test rather than in a JSDOM unit test.
  • Whole-page budgets. Lighthouse and pa11y load a running page and report on it. Lighthouse’s accessibility score is a weighted average of pass-or-fail audits, weighted by axe’s user-impact ratings. Lighthouse CI can fail a build below a minimum score, and pa11y-ci fails when the count of errors, warnings, and notices passes a threshold you set.

Run these as a CI gate on every pull request. A violation caught in review touches one component while its author still has it open. The same violation found shortly before a demo has usually been copied into every screen that reused the component, and a one-attribute fix becomes a sweep across the codebase. The UK government’s Service Manual makes the same argument to its teams: test from the first production code, because issues are much more expensive to unpick and fix later.

The layers target the failures that are most common in practice. WebAIM’s annual industry report on the top million home pages, which runs the WAVE engine against each rendered page, found that six error types made up 96% of everything it detected: low-contrast text (on 83.9% of home pages), missing alternative text for images (53.1%), missing form labels (51%), empty links, empty buttons, and a missing document language (WebAIM, 2026). All six are mechanical, and axe-core has a rule for each (see its rule list).

Manual Accessibility Testing

Section titled “Manual Accessibility Testing”

Automated tools find a large share of the issues but cover at most half of the rules, and the figure you hear depends on which one is counted. Deque, the company that maintains axe-core, published an industry report on first-time audits covering more than 13,000 pages and page states and nearly 300,000 issues: axe-core’s automated tests found 57.38% of the issues by volume (Deque Systems, 2021). Counted by WCAG success criteria instead, the same data had automated findings for 16 of the 50 WCAG 2.1 AA criteria, which the report acknowledges supports the widely quoted figure of 20 to 30% coverage. The report was written to argue for the by-volume count, so read it as the tool vendor’s case. An independent benchmark of six tools against WCAG 2.0 by Vigo et al. (2013) found that they caught between 14% and 38% of the violations on the success criteria the study analyzed, and tested at most 50% of the success criteria at all.

Tools are reliable on rules a machine can decide (is there alt text, is the contrast ratio at least 4.5:1) and blind to rules that need judgment: whether the focus order makes sense, whether a label is meaningful rather than merely present, whether the flow can be completed at all without a mouse. Keyboard and focus problems show it most clearly in Deque’s data. Automated tests found none of the 9,553 issues against Focus Order or the 7,312 against Focus Visible, and 234 of the 9,412 against Keyboard; manual testing found the rest (Deque Systems, 2021).

The manual pass has four steps, none of which needs special equipment. Run them before every demo and every release, next to your smoke test.

  • Keyboard only. Unplug your mouse and complete your main flow with Tab, Shift-Tab, Enter, Space, and Escape. Watch for two failures: a control you can’t reach or operate, and a focus indicator you can’t see. People who can’t use a mouse, including screen-reader users on a desktop, move through the page this way, so a step you can’t complete is a step they can’t complete.
  • Screen reader. VoiceOver ships with macOS (Command-F5 turns it on), NVDA is free on Windows, and Orca is the screen reader for the GNOME desktop on Linux. Spend ten minutes on your main flow listening to what it announces: a button with no accessible name is read as just “button”, and an image with no alt text may be skipped or read out as its file name.
  • Zoom to 200%. Text should stay readable rather than clip, overlap, or push content off-screen. WCAG’s Resize Text success criterion requires text to scale to 200% without losing content or function.
  • Contrast. The element inspector in Chrome and Firefox shows a text element’s contrast ratio. WCAG AA asks for 4.5:1 on body text and 3:1 on large text. Check text over images and gradients by eye, where a tool often can’t compute a ratio; in Deque’s data, about one contrast issue in six came from manual testing (Deque Systems, 2021).

Testing with Assistive Technology Users

Section titled “Testing with Assistive Technology Users”

A session with a daily assistive-technology user tests whether the people who rely on those tools can finish their tasks. Power et al. (2012) ran task-based evaluations with 32 blind users on 16 websites and logged 1,383 problems; only 50.4% of them were covered by a WCAG 2.0 success criterion. Where WCAG did cover a problem, the page had already met the relevant success criterion in 16.7% of cases: the developers had followed the guideline and the user still hit the problem.

A daily user brings what a teammate with a screen reader switched on doesn’t have: shortcuts and navigation strategies learned over years, and a sense of how a well-built page behaves. They notice where yours departs from it, including problems that no success criterion names.

Where you can, recruit one participant who uses a screen reader, voice control, or switch access as part of their normal day, and run the same task-based session you’d run with any other user: a short list of realistic tasks, you watching and taking notes, nobody helping. Campus disability services, local accessibility meetups, and the organization you’re building for are the usual routes to a participant.

When that isn’t possible on your timeline, the closest substitute is a task-based session under simulated constraint: a teammate who didn’t build the feature completes the flow keyboard-only or with a screen reader on. A simulated session finds broken things, such as a focus trap or an unlabeled control, but its tester is learning the tool while testing, so it can’t tell you what daily use is like.

Test with an Assistive Technology UserThe session itself: recruiting, the task list, and what to write down.

Best Practices for Accessibility Testing

Section titled “Best Practices for Accessibility Testing”
  • Start at the lint layer. It runs before a browser is involved and catches missing alt text, one of the most common errors in WebAIM’s report. Its shipped rules don’t check for missing form labels, so leave those to axe-core.
  • Run axe-core in the tests you already have, and check contrast in a real browser rather than in JSDOM.
  • Do the keyboard pass on every flow you demo. It takes a few minutes, and keyboard failures are the ones automated tools almost never report.
  • Judge the work by whether the flow can be completed. A page can pass every automated rule and still be impossible to finish without a mouse.
  • File a violation as a bug on the same board as the others, so it’s prioritized against the same deadlines as every other defect.

Some Truths About Accessibility Testing

Section titled “Some Truths About Accessibility Testing”

A team can skip all of this and see no consequence, because the people who build, review, and demo the software usually all use a mouse and a screen. That’s why it gets skipped: the cost lands on people who aren’t in the room.

It’s also what most of the web looks like. WebAIM’s million-home-page report found detectable WCAG failures on 95.9% of the pages, up from 94.8% the year before, which reversed six years of small improvements (WebAIM, 2026). Because only automatically detectable failures were counted, WebAIM says this suggests the rate of full WCAG 2 A and AA conformance was certainly lower than 4.1%.

Automated tools are oversold and still worth running. A clean run means the mechanical rules pass, and the coverage figures show how much that leaves unchecked whichever way you count. Run them anyway, because they catch the most common failures on every pull request, and the only human cost is reviewing what they flag as incomplete.

If you run only one check on this page, make it the manual pass, because it costs nothing but the laptop you already have and it targets the failures automation misses, keyboard and focus above all. Put it on the pre-demo checklist where someone has to tick it, because unlike a CI gate, nothing turns red when nobody runs it.

Recruiting a daily assistive-technology user on a short project timeline is hard. The people you need are few, busy, and reasonably wary of being asked to sign off on someone else’s product, and often you won’t find one. Say so in your write-up: a reader can weigh a simulated session they know was simulated, but a simulated session described as a real one misleads them about what was tested.

Accessibility Testing in Industry and Academia

Section titled “Accessibility Testing in Industry and Academia”

For much of the industry, accessibility is a legal requirement rather than a quality preference. In the United States, Section 508 of the Rehabilitation Act requires federal agencies’ technology to be accessible, and the Access Board’s standards under it incorporate WCAG 2.0 AA. The Department of Justice’s guidance on web accessibility states that the Americans with Disabilities Act (ADA) applies to the websites of state and local governments and of businesses open to the public, and its rule for state and local governments sets WCAG 2.1 AA as the standard for their web content and mobile apps, public universities included. In the European Union, the European Accessibility Act sets accessibility requirements for products and services that include e-commerce, banking services, e-books, and smartphones.

Buyers bound by those laws need evidence that a product conforms, and the usual format is an Accessibility Conformance Report: the Information Technology Industry Council’s VPAT template, filled in with test results against Section 508, the European standard EN 301 549, and WCAG.

For native mobile apps, the platform owners publish their own guidance alongside WCAG: Apple’s Human Interface Guidelines on accessibility and Android’s guide to building accessible apps.

The UK government’s Service Manual shows what making the checks hard to skip looks like. It sets WCAG 2.2 AA as the minimum for government services, tells teams to run both automated and manual testing because automated testing alone misses issues, asks teams to test with assistive technology and with disabled users, and requires an accessibility audit before a service moves into public beta. Its Design System supplies accessible styles, components, and patterns, and states plainly that using them doesn’t make a service accessible on its own: teams still owe their own research, design, development, and testing.

Research software built for one lab may fall under none of these rules. Run the lint layer and the keyboard pass on it anyway, because once the artifact is published, someone who didn’t build it may need it to reproduce the result, using whatever assistive technology they rely on.

References

Section titled “References”

Additional Readings

Section titled “Additional Readings”

Sources and further reading

Activities that exercise this