Accessibility Testing Overview

[00:00:00]
>> Marcy Sutton: So, I'm really excited, we're gonna talk about testing. And this is something that I've really grown to see value in as a developer in my career. Once upon a time I was working at an agency, I'm working on a widget over here, and this one page that we were supposed to update, yeah, I forgot it was in use on another page and didn't tell our QA team to go test that page.

[00:00:21]
So, yeah, guess who shipped broken things to production? And it really wasn't the QA person's fault, they weren't looking for it on another page, it was pretty much my fault. So, I think testing is a way to prevent those failures, those failed builds, and staying late on a Friday at the office trying to fix your mess you made cuz you deployed on a Friday, and you don't have continuous deployment.

[00:00:46]
So testing is just so powerful to get that peace of mind, to bake that quality into your software. And for accessibility there's so much potential, because you can bake that stuff in for accessible patterns, these are supposed to work with the keyboard, and if you break it, you break the build.

[00:01:04]
So there's a lot of value in having automated testing in general. And there's some great Frontend Masters courses just on testing, but we're gonna look at it from an accessibility lens. And we'll talk about Linting, unit tests, integration and end-to-end test. Accessibility test API's, and how they kind of factor into all of those.

[00:01:26]
We'll touch briefly on continuous integration, and then some manual and user testing, because we can only automate so much. It's really amazing, and we can catch some low hanging fruit, and automate it, and fail things from getting deployed. But that only goes so far, and we can test manually, like we did with debugging, and to start the day.

[00:01:48]
And then user testing is a huge part of that as well. There's no substitute for getting your product, or site, or app, in front of a user with a disability early, because they might tell you something that's a huge game changer. You're like, wow, I wish I would have known that earlier, [LAUGH] well,you can do user testing.

[00:02:08]
So Linting, this is a big area for developers, because when we're coding, we hit save on a file, or even just, typing in a file. I mean, we were using Linting to look at that spread operator in the last section to see what is valid, and it can give us some of that feedback of like, I typed this wrong, there's an error here.

[00:02:27]
And I can figure that out before I hit save and have to go over to the browser and figure that out. So Linting has a role in helping us catch errors early. And for accessibility we can use the ESLint-plugin for JSX and accessibility. It is limited to sites that use ESLint and JSX, but a lot of sites do these days, a lot of React applications use this.

[00:02:51]
Gatsby ships with an ESLint config that comes with that installed. You might need to reconfigure some of the rules. If you look in the Gatsby develop, when I'm running that on the server locally, it's complaining about things that I actually need to go report to them, but sometimes I don't agree with the findings.

[00:03:12]
And so, what I can do about that is reconfigure it. And I have some resource here from the Gatsby site, I wrote about how to reconfigure ESLint, it would apply to other sites too, not just Gatsby sites. Linting is great in the JSX or HTML space. Most times, you're not loading CSS at that time, so you maybe aren't seeing that, or even having other components, you might not be aware that the rendered state of it is maybe fine.

[00:03:43]
Maybe your complaining about something and it's not a big deal, or it's missing something that is a bigger deal. So, we can use Linting for some low hanging fruit, but rendering does make a difference, and so, you kinda just have to take that with a grain of salt.

[00:03:59]
So some accessibility units. So Linting, that's a tool that you can just install, so I'm not gonna go into it, I'd rather spend the time working on automated tests. So I'm gonna talk to you kind of about the line between unit test and integration test, and kind of when would you write what.

[00:04:18]
And, some of that's very subjective, where that line for integration and end-to-ent test is, but let's look at accessible units in unit test. These are mostly component level concerns, like the API for a component, you pass in a prop, it puts that label into some button buried in the tree of that component somewhere.

[00:04:39]
That contract for that component should have some unit tests. So, in isolation, when we have that component, it has some tests that, it doesn't have any outside inputs, it's not going and hitting the network, it's not depending on some other component. We either need to stub things that it does depend on, so that the test can still be isolated.

[00:05:02]
That's probably the best approach, I would say, is just try and limit all those dependencies, and then you can just pass in inputs. So the inputs are what you pass into a test, the variables that change its outcome. And then we're usually asserting the outcome, the state that it ends up being when we've passed in those specific inputs.

[00:05:22]
An example would be, in our drop down widget when I passed in the activator text. In a test I could pass in any random string, and that's what we'll do in a test. And then I can assert that that string matches, so it's very specific to the test.

[00:05:38]
It is pretty robust, because the test data is something I define, and it's not depending on a real implementation. These unit tests are often headless, which means that they're not technically rendered in a browser. So using things like JS DOM, that's kind of an in memory representation of a web page, that means you might have limited DOM APIs.

[00:06:01]
It means that sometimes styles, a certain focus styles might be a little funky because you aren't rendering it like a normal page would, but it does enable fast changing of state. I can change the state of a component, make sure that focus is going into it correctly, that it's getting labeled correctly, things like that.

[00:06:20]
So for accessibility unit tests, it's really about component-specific behavior. You could test interaction and focus APIs, so, like the drop down, making sure focus is send to the right elements. Things like text alternatives, if you have a custom icon button component, or a custom forum widget that's styled, and you wanna make sure that the props you're passing in are making it to the right elements.

[00:06:45]
You wanna toggle ARIA states, that might be a good thing to test for state. Cuz really what we're concerned about is the outcome. So not the implementation, so not the internal state, but the component that gets output on the other end. Does it have ARIA selected when it should?

[00:07:02]
Does it have ARIA disabled when it should? And that sort of thing. And I have an article that I wrote linked here called Writing Automated tests for Accessibility that goes much deeper into a lot of these kind of the line between different styles and tests.