Accessibility testing is crucial when building a website or application to ensure that what you’re building will work for all users. This includes disabled users, as well as people with temporary and situational limitations (such as the colleague who broke his arm while skiing, or the customer with a bright light on his screen while using his phone outdoors).

We will share how to “layer” accessibility testing by using various tools and methods at different stages of the digital product life cycle to catch accessibility problems earlier when they are easier and cheaper to fix. Taking a layered approach to testing your site’s accessibility also improves the usability of your site – which in turn increases your customer base and reduces customer service queries. It can make you money or save you money.

We’re going to use the metaphor of a tiered cake to talk about the different layers of accessibility testing and when to use them. Food analogies have become quite popular in the world of accessibility! This approach has worked well for both of our companies.

It worked for both of us. Mike is an experienced accessibility advocate and senior strategist at CivicActions, while Kate is the service lead at Fable, an accessibility testing platform.

Mike looked at accessibility testing from a technical perspective early in the development phase and scanned the actual site for compliance, while Kate focused on the user experience. Both of us realized that incorporating multiple types of accessibility testing throughout the product development cycle is a powerful way to improve overall product accessibility. In this article, we’ll share some of what we’ve learned.

Most organizations deal with accessibility in three main ways.

  1. Run tools to review your code and/or user interface. This is often referred to as “automated testing” because you use software to automatically test many accessibility issues at once.
  2. Use your computer in a different way than you normally do.

    For example,No Mouse.Enlarge the browser to 200%), orSwitch to Windows High contrast mode.

  3. Use assistive technology and disabled users to check for usability issues. This is often referred to as a “manual test” because it requires a person to assess accessibility problems.

Too many organizations rely entirely on a single accessibility solution to validate their sites. There is no tool or process that gives an organization confidence that they can truly meet the needs of as many people as possible.

How to ensure the purchasing power of accessibility

In many organizations, in order to conduct accessibility testing, you need management to prioritize and support this effort. If you don’t already have accessibility support, here are tips on how to make it happen.

  • Check whether your organization has legal requirements for accessibility. “Accessibility law” and “disability law” are search terms and should be found in most countries. For some organisations, sharing legal risk may be the right incentive.
  • Know what your competitors are doing. Check their website for accessibility claims. Most organizations are eager to stay ahead of the competition, and knowing that others prioritize accessibility can make a difference.
  • Contact customer service to find out if there is an accessibility complaint. If possible, contact customers directly to hear their experiences and share those stories with company leaders. Hearing from unhappy customers can be a huge motivator. If you can get the customer’s permission, record a demo of the challenges they face using your product. Videos like this can be very convincing.
  • Explain financial costs and returns. Many companies think they can’t afford to do accessibility, but it’s more affordable when it’s integrated into everyday work, rather than an afterthought. There is also the potential income from people with disabilities — globally, they represent more than $3 trillion in disposable income.

  • Find the right advocate. It is possible that there is already someone at the top of the organization who cares and does the right thing. This could be a leader in diversity and inclusion, a person fighting for environmental sustainability, or other issues. Maybe someone with a disabled friend or family member. Making them aware that accessibility may be all it takes to add a new focus to their efforts.

Gather your materials

Accessibility should be incorporated into your process as early as possible. One place to start is the procurement process. You can include accessibility as part of the review process for any technology system you buy or build. DisabilityIN has some excellent resources on frictionless IT procurement.

Looking for a vendor’s accessibility claim or a product’s VPAT can help you, but also do a quick review with some of the tools mentioned in the recipe below. Not all software is created equal, so make sure you work with vendors who actively contribute to tools and processes to help you prioritize accessibility from the start.

If you are creating or updating a design system, another way to incorporate accessibility early on is to select a component library that already has accessibility in mind. Look for libraries that have explicit accessibility statements and open question queues that allow you to review issues.

Example.

  • The Angular component team has built accessibility into the Material UI library. For example, a radio button component uses a radio group with an aria-label. Each radio button is selected or unchecked for screen reader users, and the button can be selected using the arrow keys like a standard HTML radio button, with the focus state clearly visible.
  • Reakit for React describes an accessibility warning feature on its accessibility page, which tells developers when an RIA-label is needed.
  • The Lion Accessibility Web Component Library uses the A11Y tag on GitHub to flag accessibility issues so you can see what’s being improved and open your own issues as needed.

Another way to embed accessibility into your process is to update one of the roles your team uses to include disability. Many people have more than one disability, so creating a character with at least several disabilities will ensure you stay focused on your audience in all of your early design work.

To flesh out the role, talk to real people with disabilities — both temporary and environmental constraints — to help you understand how they use technology, websites and apps in the real world. One in five has a permanent disability, but 100 percent will face visual, hearing, motor or cognitive impairment at some point in their lives. Our roles reflect that.

  • People with allergies, insomnia or broken bones.
  • People who use outdated technology or computers outdoors; Or even
  • Change the people using their technology based on their location (for example, disable images when they need to save Internet bandwidth).

Small changes like this can have a big impact on the way your team thinks. One way to communicate this change to leadership and the team is to talk about how it will make your persona more reflective of your actual users — that’s what a persona is all about. They have to be realistic.

One of the most powerful ways to involve people with disabilities is to get them to help design services and products together. Australia has a free training kit on how to design with people with disabilities. There is also a great case study of how a company co-designed with people with learning disabilities on behalf of the UK government.

The legacy of the IT

Whether we like IT or not, most decisions about organizing IT were made months, if not years, ago. Even in the procurement process, accessibility is often only one of many considerations. This is to be expected – even in organisations keen on accessibility.

For legacy technologies, the first step is simply to raise vendor or team awareness of the importance of accessibility. If you can specify the accessibility issues that you want to fix using automated tools, this will help fine-tune vendors’ ranking of their problem queues. There isn’t always a community portal to post such questions, but there might be a community on Twitter or Reddit where you can bring them to light.

In addition, there may be a customizable theme that can be adjusted to solve some problems. Some solutions may provide an application programming interface (API) that allows developers to build a frictionless user interface around it.

If a vendor has a competitor, it may be useful to emphasize the accessibility features included in the product. It can be helpful to remind suppliers that you do have a choice.

If legacy IT is an in-house built product, a good way to quickly evaluate IT is to use only the keyboard. If you can’t use the product with a keyboard (for example, there’s no visible focus or the user interface can only be clicked with a mouse), you’ll probably have to put a lot of effort into improving the product’s accessibility.

Consider providing alternative means of service (for example, telephone support, in-home service, or email) so that those who cannot access the product digitally because of an accessibility barrier can still get what they need.

Consider the organization’s roadmap and when upgrading or decommissioning products is feasible, and weigh the cost and effort of accessibility. If you have other newer products that are not accessible, it may be more productive to focus your efforts on those products if a legacy tool is nearing the end of its life.

formula

Here is an example of a comprehensive accessibility testing method, and the five-tier accessibility testing cake is delicious. Figure out what your budget is, and then price all the different testing methods. Some are free, some cost money. In the next section, we’ll provide advice on where to start if all of these levels of testing don’t fit your budget.

  1. Research user needs and make sure that the questionnaire you use to screen potential study participants asks about assistive technology use. This will make it easy to include people with disabilities in your existing research process at no additional cost. If you can’t find participants this way, try contacting disability organizations.

    You can also modify your existing user personas to include disabled users. If you need to do this quickly and cheaply, you can borrow aspects of the user profile from gov.uk. If you have the budget, include people with disabilities in your prototype and design reviews. This is probably easiest to do if you hire a provider that provides this service, so you need to have a budget. Alternatively, you can pay participants directly.

  2. Improve your processes Encourage developers, designers, and content writers to include accessibility checks as part of their processes. Here’s how to use the free automated testing tool.

    • Download free browser extensions/plug-ins to do specific page tests (WAVE or Accessibility Insights) for design reviews.
    • If you’re using continuous integration testing as part of your developer build pipeline, make sure you’re evaluating accessibility (there are free open source tools for this, such as Axe Core and Pa11y)
    • Provide content authors with tools in the WYSIWYG interface to identify obstacles they have added (HTML code sniffers).
    • Make sure you crawl your site regularly to catch accessibility issues. If possible, run crawlers in both staging and production environments (Purple Hats is a free open source option

  3. Manual QA You don’t have to add extra people to do QA, just integrate it into your existing process. If you’re only doing one thing, stop using the mouse in your regular QA. You catch accessibility errors along with other functionality errors. If you want to do more, you can also test it with a screen reader and magnifying glass.

    There are various ways you can do manual accessibility QA without having to buy any tools.

    • Can you access your site without using a mouse? Use simple manual keyboard-only tests to evaluate new components and content.
    • Use your browser’s built-in magnification tool (Ctrl + +) to browse your site with magnification set to 200% or more.
    • Flip your browser or operating system to dark mode to see if your site works well for light-sensitive people.
    • Sprint level testing with developers and designers using assistive technologies (VoiceOver, Microsoft Anna, and NVDA are free options).

  4. User testing In a large enterprise environment with a dedicated accessibility budget, you can pay assistive technology users to test features in your staging environment before launching.

    There’s nothing that makes you more sure that your product is useful to people with disabilities than user verification. Even a perfect WCAG compliance score doesn’t guarantee you as much as someone who actually uses the product.

    Disabled people are often asked to work for free, which is problematic because many are already at an economic disadvantage. If you’re working on a personal project and don’t have a budget, check your network to see if there are some people who might be interested in helping out in exchange for an equal favor.

  5. If your organization has a accessibility team, have them do pre-release user acceptance testing. At this point, you can get detailed feedback on WCAG compliance that you may have missed in the previous steps.

    Think of it as a final check; Your accessibility group doesn’t do everything about accessibility; everyone has a role to play. Accessibility teams are most effective when they set standards, provide training, give guidance, and assess compliance. They support access work, but they’re not the only ones doing it. That way, no one person or team becomes the bottleneck.

    If you don’t have a team, you can hire accessibility experts to review prior to release.

Where to start

Start with your position. The goal is not perfection, but continuous improvement. Implementing all levels at once is not necessarily the goal. Rather, it’s about starting with one or two layers and gradually adding more layers as your team gets better at accessibility testing. A little cake is better than no cake.

personal

  • If you’re new to accessibility testing, start by adding a free browser extension to find accessibility problems and learn how to fix displayed errors. WebAIM’s WAVE toolbar is great in this regard.
  • Start sharing accessibility information you find useful. It might just be on Twitter or Reddit, but you can also start a newsletter to help raise awareness.
  • Sign up for a webinar or event focused on accessibility so you can learn more.

team

  • A team with a strong user-centric design approach might want to start with the first layer: interviewing people with disabilities as part of a user study.
  • A team with a strong IT compliance process might invest in tight integration of automated tests during their continuous integration process, or crawl the entire site first.
  • Look for ways to incorporate accessibility early in the design/development process.

institutions

  • Make sure you have meaningful accessibility statements that reflect your organization’s commitment to removing barriers for people with disabilities.
  • Build a network of champions and allow communities of practice to grow and learn from each other.

Limitations of automation tools

Every baker needs to have a library of tools they can rely on. There are some proprietary accessibility tools worth considering, but there are also good open source tools, including the free ones we mentioned in the “recipe” above.

In modern dynamic sites, it is important to use automated tools to catch accessibility errors early before they are published to live sites. It is also important to crawl the site to see if all the pages are still up to date after publication and continuous updates.

The challenge is that designers and developers often assume that if tests don’t report any bugs, the site is fine. When you give people a test, people tend to write towards it. Unfortunately, many designers and developers stop when they fix bugs they see with WAVE or Axe.

To put it plainly, even a small number of teams do this, but we have to do better if we want websites to be sensible, actionable, and understandable to a wider range of people using different types of technology.

Automation tools are great, but also limited. Even the best automation tools catch only about 30 to 40 percent of wCAG-compliant accessible errors. An automated tool can tell you if an image lacks an alternative description, but what it can’t tell you is if the description is completely inaccurate or used in the wrong context and therefore useless. It still takes one person to evaluate.

To overcome these limitations, it is important to recognize that accessibility does not automatically mean availability for people with disabilities. Consider accessibility as the lowest standard; It can be used with assistive technology, but to go beyond “it can be used” to “it’s enjoyable and easy to use,” you need to test it with real users.

Many organizations already do usability testing, but most do not include people with disabilities. If you’re having trouble recruiting more participants, consider partnering with an organization that has an assistive technology user community and platform to make testing quick and easy.

Let’s bake!

When working on an inclusive website, use a layered accessibility testing approach. Don’t rely on just one test to find disabilities.

  • Test your idea with assistive technology users early in the construction process
  • Incorporate regular automated code checks into the website building process
  • Manual testing using assistive technology as part of QA
  • Test with people with disabilities before release
  • A comprehensive accessibility review should be carried out for phased work

Remember that the goal is not to score high on the test tool, or even to meet the WCAG guidelines, but to make your content widely available, including to assistive technology users.

Ultimately, the accessibility statement is the icing on the cake. Include an accessibility statement with contact information on your website to provide a feedback loop. Your users are experts, and everyone should be a part of making your site better over time.