Skip to content

Accessibility test results aren't checked #120

Description

@medmunds

[Stumbled across this while trying to implement the test case for #119.]

The test suite attempts to use @open-wc/testing's Chai A11y aXe assertion to verify accessibility. But that assertion is async, and because the test suite drops the returned promise any aXe failures are ignored:

    afterEach(function () {
      // Check to make sure we still have accessible markup after the test finishes running.
      expect(document.body).to.be.accessible()

      document.body.innerHTML = ''
    })

    it('has accessible markup', function () {
      expect(document.body).to.be.accessible()
    })

A solution is to either await or return the promise in all places:

    afterEach(async function () {
      // Check to make sure we still have accessible markup after the test finishes running.
      try {
        await expect(document.body).to.be.accessible()
      } finally {
        document.body.innerHTML = ''
      }
    })

    it('has accessible markup', function () {
      return expect(document.body).to.be.accessible()
    })

That will expose a new problem: as written, the tests do not create an accessible document.body. The <tab-container> in the fixture appears outside a landmark, so fails aXe's "region" rule.

Three possible solutions:

  • [preferred] Test only the tab container element, not the entire body: expect(tabContainer).to.be.accessible()
  • Wrap each fixture in <main> or a similar landmark element: document.body.innerHTML = `<main><tab-container>...
  • Ignore the region rule as not applicable to component-level testing: expect(document.body).to.be.accessible({ignoredRules: ['region']})

Finally, even with all that I ran into some cases where failing tests would time out the test run rather than report the failure. (With my own tests, or even just modifying existing tests to fail.) AI suggests that Mocha really isn't happy with assertions in the after cleanup functions, but I haven't looked into it or tried to narrow down reproduction. Just noting it here in case someone else runs into the same problem.

(AI also noted that web-test-runner.config.mjs is trying to run tests on src/index.ts, which doesn't contain any tests.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions