My experience with the Section 508 Trusted Tester exam

Whew! I just passed the Section 508 certification for accessibility auditing. It was rough! But, I have to say, really not as rough as I expected, and I learned some really neat things along the way...

Some background

I've been consulting on accessibility in digital media (websites, marketing emails, etc) for a few years. A few months ago I took the CPACC exam and passed. I studied for that one for about two weeks and it was about as difficult as I expected, and I was relieved — but not too surprised — to pass it on my first try.

I decided to tackle 508 next. Reading about the exam online, I kept running into the same scary-sounding details:

1. A lot of people have failed the exam many times — including people who are currently digital accessibility professionals! That's always a very discouraging thing to hear.

2. A lot of people report that the training materials, quizzes, and the exam are poorly designed and poorly written. I believe this explains that first point above — partially, at least. A common complaint I encountered was folks saying that the exam does a poor job of successfully handling what can be tested, what should be tested, and how these things should be tested. After my own experience with the training, quizzes, and exam, I agree with this, regarding certain parts.

I'll add, some of the questions and answers are worded in a way that's very difficult to mentally parse. For a small percentage of the questions, I had to re-read the sets of answers many, many times in order to feel sure I was choosing the answer that actually meant what I wanted it to mean. For certain questions it almost felt like the challenge was in being able to parse the very complex grammar of the answers to choose the right one, rather than actually having the knowledge to answer a question about the given topic. (And a thing I think about all the time is: what if English weren't my first language? I think this exam would be next to impossible, just because of the complex wording of some of the questions and answers.)

And lastly,

3. A lot of people report that the quizzes and the exam have straight-up errors in them; things marked as wrong when they should be correct, and things marked as correct when they should be wrong. And, reader, this is absolutely, unbelievably, maddeningly true. I'll get into that more below.

Despite how intimidated I was by all of the above, successfully training for and passing the 508 exam ultimately wasn't as hard as I'd feared! So don't let all of this discourage you from trying!

Preparing for the test

The 508 Trusted Testers program consists of four training modules (with many dozens of quizzes along the way), then a practice exam of one hundred questions, and then the final exam of one hundred questions. I worked my way through all of these things in about one week.

The four training modules start off really breezy and then they get longer and more intense as they go; the last one is by far the longest and most complex, but they're all broken down into small chapters with smort quizzes. It's all pretty digestible in these bite-sized pieces, but as I worked my way deeper and deeper into the training I started to worry that my brain was probably forgetting the things I learned earlier on. There are a million little separate topics for your brain to hold on to. For this reason, my strategy was to try to get through the whole thing as quickly as possible, and not allow myself to wait a day or two in between sessions of working on it — I figured the longer I waited the more I'd forget some of the earlier things I'd learned. For me it was important to try to keep it all fresh in my mind as I approached the final exam. This approach worked for me, and I would recommend it to anyone who's interested in doing this training and taking the exam. So this may mean you'll need to find a period of a week or so where you'll able to devote a lot of time to this thing every day.

When I finished the last bit of training, I immediately took the practice exam that same day, and on that first attempt I failed by a couple points. You need to get 85 questions correct out of a hundred, just like for the real exam. I then reviewed the things I got wrong in the practice exam, and, for a few of them, I was able to figure out why I got them wrong. A couple of them were related to small topics I'd simply missed in the training, but seeing that they were wrong, and looking a bit deeper, helped me understand why they were wrong, and this made me better prepared for my next attempt.

I took a very short break and then took the practice exam again that same day. I did much better on this second try and passed with flying colors. I reviewed my answers to see what I got wrong, and had a couple more of those "a-ha" moments. Very useful.

I then took another short break and decided to just go for it and do the real exam. I passed that on my first try! Woohoo!

So, to be clear, I finished the last bit of the training and took the 100-question practice exam (twice!) and the 100-question final exam, all in one day! I was done with all of it by around 3pm.

I think I spent a total of about forty hours on the training and exams, over the course of about a week. I've heard some people say they've spent as much as eighty hours, and I can totally see that.

Some notes on how the test is formatted

Unlike the CPACC test, this is one that you can do at home on your own time, with no real moderation. They have you read some rules about not cheating, which are completely reasonable. Other than that, you're on your own and can do as you please.

A few times while taking the final exam, I wondered if it would be possible to go back into the training modules to check on something, or if the website would recognize that I was doing that — my account would have been recorded as having started the session for the final exam and then visiting these training pages that maybe I shouldn't be visiting. I'm honestly not sure if it's allowed or not. It certainly could be tracked if DHS CXD wanted to. I don't know if they do or not. So I didn't tempt it. I just relied on the newly-acquired knowledge in my head for the practice exam and final exam.

Here's an important thing! The practice exam and final exam draw from the fourth training module only! That's the big one, Trusted Tester Training for Web. The things you'll learn in the first three modules do not directly figure into the big exams. (And for whatever it's worth, the first and third aren't very relevant, but you might assume, like I did, that the second one, Section 508 Standards for Web, would be important for the test. It's not. I wrote down and studied a ton of notes about P.O.U.R. that turned out to not be necessary for the exam. Still, good knowledge to have!)

Most of the questions in the exam ask you to look at a sample webpage and answer one question about that page. As you work your way through dozens of quiz and test questions, you'll see the same webpages many times, with different questions applied to them. For example, a particular webpage may have an audio player that auto plays, and you have to answer something about that, and for another question later on you see the same webpage again, but this time they're asking about color contrast in a different section of the page. So you'll end up seeing some of these pages a million times, to the point that it's a little annoying, especially the ones with auto-play audio. No big deal.

One thing to look out for is that there are different versions of some of these sample webpages. They may look the same, but they may be coded differently, or have slight differences in content and functionality, so when you see a webpage that looks familiar, you can't assume that certain things that were true last time are true this time. This really isn't a big deal if you're aware of it and if you're not rushing through. Just take your time and make sure you're really evaluating what the question is asking you to evaluate and you won't have any problem. You'll only screw up if you assume something is the same this time as the last time you saw the page. (I'll make up an example: you may see a certain webpage and be asked something about the title tag, and it turns out there isn't one, which obviously is something you're supposed to flag as a problem. Later you might see the same page again, with a similar question about the title tag — and so, you can't just assume there's no title tag; this may be a different version of the page that does have one.)

Some of the exam questions don't involve looking at a sample webpage, but this is a very small percentage of them.

Infuriating problems

As mentioned above, the quizzes and exams definitely have some problems where the correct answer is misidentified in their database. This is incredibly unfair to us users because it means we can answer a question correctly, get it wrong, and then this hurts our overall grade. It's awful. One bad question can mean the difference between passing and failing. It's just unfair — even if (of course) the organization included these wrong items in error.

The first couple times I encountered this kind of problem, I wasn't totally sure that this was the case; maybe I really did mess up! But eventually I was able to prove it. In my first attempt at the practice exam, I got a question wrong that I couldn't believe was wrong; I reviewed it afterward and was just baffled that it was wrong. When I took my second attempt at the practice exam, I happened to be served this same question again, and I again selected the answer that I was sure was correct — like, my thinking was, maybe the answers were worded a little differently last time, and I got tricked? Well, I was sure this time, so I went with what I knew was correct. And it was. But then I decided to go back and find this question in my first attempt, to see if the wording was different, and it was not. I took screenshots of both versions; they're identical — and, again, I selected the same answer both times, and one time it was marked wrong and the other time it was marked right. I also clicked to look at the sample webpages associated with the two instances of the question, and they are the exact same webpage, with the same unique url — so, they're not two slightly differenet versions of the same page; they are literally the same page. I've got screenshots of all of this. So, the question itself must live in the database two separate times, each time with the same sample webpage and the same set of answers, but with one instance where the correct answer is marked incorrect.

(I don't feel like I should post the screenshots here, since they come from the testing materials and they show the answers. But I've got 'em; if anyone from DHS CXD is reading this, feel free to get in touch.)

You can certainly imagine how, after having this happen just one time, one completely loses one's ability to trust these exams at all. How many points am I going to lose to wrong data every time I take it? There's no way to know, but the thought was always in the back of my head.

There's another question that came up multiple times that was equally infuriating. The question asks you to look at a sample webpage and check if there's a keyboard trap, and there absolutely is. You can't miss it. You can tab forever and you're just stuck in a loop between a few focusable elements. And when you select the answer that says that there is indeed a keyboard trap, it's marked wrong. The answer that gets marked correct is, "There is no keyboard trap."

Some of the other things that I thought were wrong could be considered gray areas. But not that one.

Speaking of gray areas, there are a few questions about whether a webpage's title tag is accurate and descriptive, and some of those feel very subjective. When I saw what they marked as right or wrong, it was like, okay, I get what they're going for. It still really felt like bs though, in some cases. While things like that aren't straight-up errors in their database, they fall into the category of: is this a good test question? Can this be tested well in a black-and-white, right/wrong context? I would argue that these gray-area questions are just not well-designed test questions. (Now, a title tag that just says, "Page," sure, that's undeniably wrong.) To take this even further: accessibility consultants are not copywriters; is it really up to us to evaluate the quality of the writing of the title tag? I can see the case for the argument that, yes, it is. But it doesn't seem quite fair to include that in a black and white test question like this. It's the kind of thing that, in real life, would be better resolved together with the copywriter and team. "Are we doing our job with the title tag correctly here? Let's discuss and explore options."

Okay, one more example of how the test may be considered poorly designed. It seems that the answers are served in a random order each time, which is fine; you may get the same question more than once (during multiple attempts), with the same five answers, but they're displayed in a different order. Totally fine. However, a few of the questions have "All of the above" as one of the answers — and since the answers are displayed in a random order, "All of the above" may not be the last choice! So, do we assume that "All of the above" really means "All of these answers"? Or should we take it literally, and if "All of the above" is option C out of five answers, then it's referring only to A and B; the two answers that are above C. See what I mean? I'm certain they mean "All of these," but then the wording is poor, and misleading. We shouldn't have to guess what they mean.

For whatever it's worth, while you can review your attempts at the practice exam, and see what questions you got wrong and what the right answers were, you cannot review your answers after you've completed and submitted the final exam. They just tell you how many you got right.

You cannot proceed to the real exam until you've succeeded (with 85% or better) on the practice exam. I assume you can take the practice exam as many times as you want before moving on to the real one; I didn't see any mention of a limit.

You can take the real exam three times, and I don't know what happens if you get it wrong all three times. I imagine you have to wait a certain period of time before taking it again. Maybe you have to start over and do all the training and quizzes again; this would be pretty easy but would take a lot of time.

Certification

This was a laugh: when I passed the test, the website showed me my passing grade and instructed me to take a screenshot of it and email it to a .gov email address with "PASSED FINAL EXAM" in the subject line! Talk about janky! Heh. I received an email with my certificate exactly one week later. They say it can take seven to ten days.

What the 508 exam didn't entail

One thing that I'd imagined I would have to do during the 508 training and exam was to use Accessibility Compliance Report Template (ACRT) or something similar to document accessibility issues for webpages. However, that was not part of the training nor the exams. You're just answering multiple choice questions throughout.

I also imagined I'd have to review full (small) websites and look for issues on my own, which would feel pretty open-ended and intimidating. This was not the case; as I mentioned above, for each question you're just shown one sample webpage and asked to review something related to one specific accessibility topic. So they're sort of pointing you in the right direction. That was a bit of a relief.

I'd even imagined I might have to evaluate a sample website and use ACRT to write up notes for remediation, and then submit that for review by a real person. This was not the case.

Thoughts about 508 and CPACC

I studied for the CPACC certification for a longer period of time than I did for 508, and I also took extra training for CPACC through Deque University, and ultimately I would say that learning all of those materials was harder — however, everything related to CPACC felt clearer and better explained than the 508 materials. I didn't encounter any errors or wrong answers for CPACC, so I didn't have the same feeling of trepidation and doubt (and frustration!) throughout the whole process. I also didn't find the CPACC materials to be worded in a needlessly complicated way. And so I was able to walk into the CPACC exam and take the test with a decent level of confidence that my training would all pay off as expected.

Anyway, I'm glad I've got both certifications now! It was worth the time and effort. It's a real relief to have completed the 508 training and pass the exam! I was nervous about it for a long time and am very happy to have been able to pass it without too much trouble. I'm proud to have these two certs now, and I'm hopeful that they'll lead to some good opportunities. If you're considering taking the Section 508 exam, I say don't be intimidated; go for it!

– Manning

Back to top  |  Articles list

Questions/comments? Feel free to contact me at manning@manningkrull.com. I update these articles pretty frequently — best practices evolve over time as the world of digital quickly changes, and I always welcome insights from others.