You’d be forgiven for not recognizing the time period “testability,” regardless of its central significance to the W3C’s new Internet Content material Accessibility Pointers (WCAG 2.0). There’s little point out of testability in WCAG 2.0 paperwork—and given the verbosity of the rules, the absence of details about testability appears virtually purposeful. Certainly, testability is one among WCAG 2.0’s large secrets and techniques: whereas many of the public complaints about WCAG 2.0 have been about know-how neutrality, jargon, and the dearth of consideration to folks with cognitive disabilities, the underlying trigger behind these points—testability—has taken a again seat.
Article Continues Beneath
So what’s testability, and why does it matter? Earlier than we are able to reply that, we have to return to the start.
The World Huge Internet Consortium (W3C) fashioned the Internet Content material Accessibility Pointers Working Group in late 1998. In Could 1999, the Working Group launched a set of accessibility growth and content material tips, known as the Internet Content material Accessibility Pointers (WCAG 1.0). Virtually instantly, the Working Group started engaged on the second model of the rules, WCAG 2.0. I joined the Working Group in Could of 2000 as an Invited Professional and was energetic within the Working Group, with two notable absences, till August of 2006. Based mostly on my expertise as a member of the Working Group—and of the bigger accessibility neighborhood—I imagine that lots of the issues related to WCAG 2.0 will be attributed to testability.
So as soon as once more, what’s testability, precisely? Though testability is talked about within the summary of the current WCAG 2.0 working draft paperwork and expanded within the “Conformance” part, a full definition sits not within the glossary however within the Necessities for WCAG 2.0 Checklists and Strategies, dated 7 February, 2003. Inside this doc, you will discover the one definition of testability because it applies to WCAG 2.0. Right here’s that definition:
Definition: Testable: Both Machine Testable or Reliably Human Testable.
Definition: Machine Testable: There’s a identified algorithm (no matter whether or not that algorithm is thought to be applied in instruments) that may decide, with full reliability, whether or not the method has been applied or not. Probabilistic algorithms will not be enough.
Definition: Reliably Human Testable: The method will be examined by human inspection and it’s believed that no less than 80% of educated human evaluators would agree on the conclusion. The usage of probabilistic machine algorithms could facilitate the human testing course of however this doesn’t make it machine testable.
Definition: Not Reliably Testable: The method is topic to human inspection however it isn’t believed that no less than 80% of educated human evaluators would agree on the conclusion.
In less complicated phrases, the Internet Content material Accessibility Pointers Working Group defines a testable success criterion as one that’s:
- machine-testable or
- “reliably human testable”—which implies that eight out of ten human testers should agree on whether or not the location passes or fails every success criterion.
Testability first entered the scene in 2000 as a response to criticism directed at WCAG 1.0—particularly, that among the tips had been being ignored as a result of they had been too broad or imprecise. One instance is the WCAG 1.0 checkpoint, “Use clear and easy language.” The overall consensus was that this checkpoint was open to interpretation and folks didn’t know how you can comply. Cue testability.
At first look, testability appears not solely affordable however integral to the event of a profitable WCAG 2.0—how else will builders know they’ve complied with an accessibility requirement? The WCAG Samurai Errata are however one instance of a set of tips that don’t depend on testability however do give builders clear directions on how you can adjust to related necessities. And in any case, the Working Group was created to jot down success standards that help folks with disabilities. Success standards which might be integral to serving to folks with disabilities use the net are being outlawed as a consequence of WCAG 2.0’s testability requirement; their definition as in any other case legitimate success criterion isn’t at situation. As soon as an insistence on testability begins outlawing in any other case helpful success standards, it must be reconsidered.
The issue with testability is that even essentially the most affordable of success standards will be non-testable—and if a hit criterion isn’t thought of testable, it isn’t included in WCAG 2.0. Whether or not the criterion is an in any other case helpful method that improves accessibility is now irrelevant as to if it will get included in WCAG 2.0. Because of the testability requirement, many helpful success standards have been faraway from WCAG 2.0, and others watered down.
For instance, it has been argued that the accessibility specialist’s outdated devoted, alt attributes for photos, fails the testability requirement—and the tangled logic required to make them appear testable has made the rule of thumb weaker. I lodged the next touch upon the Final Name Working Draft on Guideline 1.1.1, which reads:
For all non-text content material that’s used to convey info, textual content alternate options establish the non-text content material and convey the identical info. (Emphasis added).
In my remark, I argued {that a} machine can by no means take a look at whether or not an alt attribute conveys the identical info as a picture, and that eight out of ten human testers couldn’t agree whether or not the textual content conveys the identical info. I gave the next instance:
…within the Reside in Victoria website (www.liveinvictoria.vic.gov.au) there may be a picture underneath the heading “Enterprise Migrants”. After I labored on this website, a number of folks stated this picture ought to have a null ALT attribute because it conveyed no info. A number of different folks steered ALT attributes of “A few enterprise migrants chatting at work” or “Guys chatting at work”.
Whereas the ALT attribute that I really helpful was “There’s a wealth of alternatives for Enterprise Migrants in Victoria”.
Though I obtained a roundabout response from the Working Group on my remark1, their public on-line remark tracker dated 12 January, 2007 proves extra insightful:
With regard to 1.1 the success standards don’t require that ALT textual content supplied by completely different folks be the identical. Actually the enough strategies solely require that ALT textual content be current that may be construed to be ALT textual content. The requirment [sic] is for alt textual content to be current. Because the high quality of the alt textual content can’t be measured, there is no such thing as a particular criterion for high quality. (Emphasis added.)
It appears like insistence on testability has introduced us again to the nice outdated days of alt=“picture”, besides that we now have no tips to level to after we inform builders that the outline is improper. To be truthful, the Working Group has tried to get round this specific downside by including a couple of clauses to Guideline 1.1.1, for instance to permit decorative photos to have null ALT attributes. Nonetheless this specific clause appears inherently untestable.
…if non-text content material is pure ornament, or used just for visible formatting, or if it isn’t offered to customers, then it’s applied such that it may be ignored by assistive know-how.
What sort of assistive know-how? What variations? Ignored as a default, or provided that the person chooses to disregard it?
There are lots of cases in WCAG2 the place success standards are literally not testable—and the Working Group is aware of it. In Bugzilla, the Working Group’s situation monitoring system, there’s a tracked situation lodged by three Working Group members that reads: “Specifically, the present wording [of WCAG2] doesn’t appear testable. Phrases comparable to, “key,” “constant,” “predictable,” “inconsistent,” and “unpredictable” are subjective.” But these phrases have been used all through WCAG2—there’s even a whole guideline that rests one among these subjective, non-testable phrases:
Make Internet pages seem and function in predictable methods.
The place doable, the Working Group has tried to narrowly outline success standards to make them testable: success criterion 1.1.1, with 4 sub-sections, is equal to WCAG1 Checkpoint 1.1: “Present a textual content equal for each non-text aspect.” However this simply highlights one other downside with testability—it will increase the complexity of the success standards. As a result of WCAG2 is technology-neutral, the rules must be testable in a technology-neutral method, a scenario that produces prolonged and jargon-heavy tips. In distinction, the WCAG Samurai Errata are an instance of the kind of tips that may be developed with out the constraint of testability (and know-how neutrality).
Cognitive disabilities uncared for#section3
One criticism of the primary model of WCAG was that many of the cognitive-disability–associated checkpoints had been relegated to Degree AAA, a stage hardly ever tried. Just one checkpoint devoted to the wants of individuals with cognitive disabilities was within the minimal stage (Checkpoint 14.1: “Guarantee language is obvious and easy”). Nonetheless, with the introduction of testability, this checkpoint was faraway from WCAG2 in April 2004. It was this checkpoint that originally piqued my curiosity in testability and when it turned clear that this checkpoint was being eliminated—not as a result of it wasn’t a legitimate checkpoint, however as a result of it merely wasn’t testable—I proposed the removing of testability. As a member (in good standing) of the W3C Internet Content material Accessibility Pointers, on April 22, 2004, through teleconference, I argued that:
in case you lock out tips [because] we can’t outline them in a testable method, then we run the danger of locking out tips that individuals discover helpful and that enhance the accessibility of content material … [non-testable guidelines] shouldn’t be relegated to highest stage (3) as a result of we can’t outline them in a sure method … they need to be outlined in method that’s most assistive to folks with disabilities.
—http://www.w3.org/2004/04/22-wai-wcag-irc.html
At that time, the destiny of testability—which wasn’t but utilized to all success standards—was put to vote, and many individuals voted in opposition to its removing. Throughout the identical teleconference, the Working Group held one other vote to determine whether or not testability needs to be a required attribute of all success standards, and I used to be the one particular person on the Working Group who voted in opposition to this modification. Every week later, my standing as a member (in good standing) was revoked as a consequence of “non-participation.” On the subsequent teleconference, the inclusion of testability was handed unanimously.
Many different strategies to help folks with cognitive disabilities, from error prevention to abstract info, have additionally been deleted from WCAG 2.0 or moved to Degree AAA or to the advisory strategies. Actually, as WCAG 2.0 does so little to help folks with cognitive disabilities {that a} formal objection was lodged (co-signed on my own) and a taskforce created to debate the matter. Sadly the Working Group’s foremost response to the formal objection is to preface WCAG 2.0 with a press release that declares the rules not enough to help folks with cognitive disabilities. It’s troubling {that a} set of tips geared toward aiding folks with disabilities ought to solely neglect the massive variety of net customers with cognitive disabilities.
Nonetheless, it’s probably not a shock. The Working Group minutes are suffering from varied feedback warning in opposition to utilizing testability for the whole thing of WCAG 2.0—one thing it was by no means initially meant for. Even the group’s chair, Gregg Vanderheiden, stated “if we require a take a look at for each checkpoint, life shall be tough within the realm of cognitive accessibility.” And that’s exactly what has come to go.
Calling for an finish to testability#section4
There are lots of explanation why testability was launched and stays a tenet of WCAG 2.0. A few of these causes could also be legitimate and vital—the W3C is in search of ISO certification, the WAI need WCAG 2.0 enshrined in regulation—however none needs to be allowed to attract consideration away from the core purpose of immediately enhancing the flexibility of individuals with disabilities to entry web sites.
When success standards are eliminated as a result of they aren’t testable—even when they’re in any other case legitimate and helpful success standards—the Working Group has misplaced its method, and we have to information them again to the correct path. The Working Group has made vital modifications to WCAG 2.0 after the Final Name Working Draft; amongst different issues, one of the vital contentious points, baseline, has been considerably modified. The Working Group must go one step additional and take away testability, lest they danger alienating each builders and accessibility specialists. With the publication of the WCAG Samurai Errata, the net neighborhood lastly has a selection—and if WCAG 2.0 continues to be unworkable, builders will merely flip to a different set of tips.
The Working Group has requested for feedback on their newest WCAG 2.0 Working Draft by June 29, 2007. Now’s the time to name for the removing of testability.
