How to organize a testing with users
“If we are WCAG compliant, we are accessible” - in reality, even with very high levels of compliance, there can still be real usability issues that only appear when people actually use the product.
There are two complementary approaches for testing with users:
➡️ Collecting feedback from your users (support tickets, surveys, direct messages): this is useful, but it is reactive and often incomplete.
➡️ Structured user testing: this is where you can proactively identify issues before they impact a large number of users.
If you want to run meaningful accessibility tests with users, here is a practical approach:
1️⃣ Start with a clear testing goal
Define what you want to evaluate: navigation, forms, content comprehension, specific user journeys. Without a clear goal, results are difficult to interpret and act on.
2️⃣ Recruit participants with relevant profiles
Testing requires diversity, because different users reveal different issues.
You can work with specialized services such as:
dedicated accessibility job board A11y Jobs
LinkedIn with the usage of specific accessibility keywords
active accessibility community WebAIM mailing list
professional association IAAP member network
These platforms help you recruit people with:
screen reader usage (NVDA, VoiceOver, JAWS)
low vision
motor impairments
cognitive differences
3️⃣ Use realistic tasks
Instead of asking users to “explore”, give concrete scenarios:
find specific information
complete a task
interact with a feature
This reflects real usage and reveals actual friction.
4️⃣ Observe real behavior, not just feedback
What users do is often more informative than what they say.
Look for hesitation, repeated actions, unexpected navigation patterns.
Research has shown that many usability issues are only detectable through observation, not self-reporting.
5️⃣ Combine automated testing and user testing
Automated tools (like Accessibility Insights or Wave tool) are useful, but limited. Studies show that automated tools typically detect around 30% of accessibility issues, especially missing more complex usability problems. So user testing is really essential.
6️⃣ Test with assistive technologies
Accessibility testing should reflect real environments.
This means testing with:
screen readers (NVDA, VoiceOver)
keyboard navigation
zoom and magnification
The World Wide Web Consortium explicitly recommends combining technical testing with real user evaluation.
7️⃣ Turn findings into actionable improvements
Testing only creates value if it leads to change. Prioritize issues, document them clearly, and integrate them into your roadmap.
