The Certification Game

Spin the Wheel

Certainty is the enemy of growth. Nothing is for certain until it has already happened—and even then, it’s still debatable.

— Mark Manson, The Subtle Art of Not Giving a F*ck

I am not a fan of certification. Or perhaps it’s more accurate to say: I’m not a fan of the way our society treats certification.

I just got certified. I had actually been avoiding it for a few years. But now I have a piece of paper (actually, it’s digital — I haven’t even received the physical copy yet) that has my name and the name of the company that provided the educational content and exam, and it has the signature of someone I don’t know from Adam. I never met this person; they never taught me; they did not look at my exam answers; they do not likely even know I exist, and yet their signature certifies that I have passed.

Perhaps you’re already starting to smell something. Something smells a little off, doesn’t it? What does this piece of digital paper actually mean, what did I really do in order to get it, where does this issue originate, and how can we do better?

Testing Trouble

You are probably already aware that most tests don’t do what they say they do.

A test is supposed to show knowledge of a subject. You get a bunch of questions, maybe they’re multiple-choice, or require a short essay. The multiple-choice questions have one “correct” answer. The essay questions are up to the interpretation of whoever grades the exam (in many cases, not the teacher — if there even is one; we’ll get to that later). This might seem fine on the surface but for anyone who has taken a test of any sort, you know that there is inevitably that one question that just doesn’t make any sense. Maybe it’s an essay prompt that is so vague you’re not sure what to write about. Maybe it’s a multiple choice question where two of the answers could easily fit but you know only one is “correct”. Oh, did I say “one question”? Sorry, I meant to say, “there are inevitably many questions that just don’t make sense”. Every test, exam, or quiz I have ever taken has had this issue. Perhaps you can relate. Actually, there are a couple of unique exceptions but we’ll look at those in a bit.

So, if not knowledge, what are these tests testing? Memory.

You might now be nodding sagely, having already thought of that answer. Well done! But wait just a moment. Whose memory?

The issue we have come across here is that these tests are not really testing the student’s memory but the memory of the test author. Allow me to elaborate.

A person wants to make a test. They must first have access to the material that the people they are testing also have access to. Hopefully, they’ve read it. They then produce questions based on one of two things:

1. Their recollection of a “fact” that they believe is covered in the material and matches their existing understanding, or

2. A bit of trivia directly pulled from the material

A number of things can go wrong in both of these cases. For the first, the author may misremember whether or not a fact is in the material; they may misremember the specifics of the fact, having replaced them as they appear in the material with their own advanced understanding; they may be remembering old material that has since been updated in ways they did not notice; or they may actively disagree with a fact in the material and not recall that the material presents a different answer.

For the second, the author may be missing context with their fact that appears later in the material and may even contradict it; they may pull a fact which is seemingly irrelevant or was not covered in class (if there was a class); they may pick a fact which they know to be wrong and even corrected during class but which will then confuse students as to whether they need to pick the answer in the book or the answer given by the author; or the material may contain internal contradictions that were never resolved, making the resulting question a guessing game.

I may have missed some, but you get the idea. Now that questions have been put to paper, each one with a definite “correct” answer, the student must access the author’s memory in order to answer the questions “correctly”. Not such an easy task.

You might be tempted to say, “Well, if the teacher got something wrong just go to their office and discuss your grade with them. They’ll fess up and correct it if they messed up.” Sure, that is an option. If you have a teacher, that is.

Bring it Online

The internet is a wonderful tool for learning. Any subject imaginable has instructional YouTube videos, blog posts, and encyclopædia entries. You can even take free courses with recorded seminars, quizzes, and exams. What’s missing in most of these though, is a teacher. Maybe you get an “assigned tutor” or a “team of experts” to help answer questions. Maybe you get a live video session with your whole class every Tuesday at 3 (hope you can make it!) taught by one of the “student success team” staff. But that final exam is still a dead list of multiple-choice questions with no way to even make a note in the margin for the teacher to see; then, it’s graded by automatically checking your answers against the list of “correct” ones that sit in the computer behind the scenes. Also, have you noticed that these are always Pass/Fail? Today’s world is so black-and-white, isn’t it.

The digital environment, produced in such a way as to encourage as many people as possible to sign up, and run on as small of a budget as possible to keep the investors happy, is not a great place to certify knowledge. The lack of direct, instant, personal communication with the one who has the knowledge you are trying to gain is crippling. It becomes so easy to disconnect emotionally from the process and emotions are what keep memory in place; they are uniquely tied to learning, and without them functioning via a social connection the results are people with certificates who have no deep understanding of the material they were tested on. Of course, having that connection isn’t automatic just because there’s a physical teacher present, as many a highschooler can attest to.

I got my certification online. I was provided with a textbook, a few videos, quizzes, and optional video meetings. As I went through the learning process, I was distinctly aware that I didn’t know what I didn’t know. The exam was likely to throw a complete non sequitur at me, and it did. But I still passed. Without ever once speaking to a real person about the material. I could now wave the certificate around as proof that I know something; I have, after all, been certified. But this certificate doesn’t have anything to say about my working knowledge. It doesn’t certify what I know. It certifies that I have correctly guessed the memory of the exam author at least 85% of the time.

But Why?

How did we get here? How is it that a certificate (of any kind, mind you, this applies to everything from driver’s licenses to PhDs) can be proudly displayed as an accomplishment to be lavishly praised; can be used as a stand-in for authenticity; can be taken as a sign of authority? And what’s the alternative?

A certificate declares that you knew at the time of the exam a sufficient percentage of what the exam author believes you learned, in their words.

A certificate is a way to show that you have passed some sort of test. It does not say anything about the nature of the test. You are supposed to rely on the reputation of the author or the author’s institution to back up the value of the certificate. In this way, a certificate’s purpose is as a banknote of social currency. I do not believe that they deserve this use. The way that many people use the fact that any certificate at all is held by a person as an appeal to authority grinds my gears.

After all, someone who has been certified is an expert and you want to trust the experts, don’t you?

Poppycock.

As distasteful as this whole charade is, what can we do instead? We can’t expect everyone to know enough to personally verify the knowledge of a specialist … can we?

Well, no, of course not. That would be absurd because if you know enough to completely verify someone’s knowledge, that would mean you have (at least) the same knowledge as them! The reason “experts” get into arguments about facts is that they both fail each other’s validation, so they necessarily do not have the same knowledge, though the Venn diagram may mostly overlap.

Instead, we can do “good enough”. The current “good enough” is a certificate. I suggest an alternative “good enough”: learn enough about a subject yourself so that you can smell bad fish. You don’t need to know that much, even. You don’t need to know details. All you need is to have a working, functional, integrated understanding of the fundamentals of a subject. The fundamentals are going to be things which are a consensus within a field. If a field does not so much as share a consensus about the most basic fundamentals then it is so blurry as to make expertise, and thus the point of certification, laughable.

There’s another option available but it’s not as reliable: Trust. If you feel that you can trust someone, be it that they are a friend (read: someone you already trust), or were recommended by a friend, or give off super chill vibes, then listen to what they say. But do not substitute a certificate for actual social currency.

The Good Ones

I promised you some examples of exams that don’t fall into this sort of trap, that actually test knowledge and not memory. Unfortunately, I don’t have much to put in this section. Maybe I’ve just been unlucky. If you can think of any that you’ve run into, I’d love to hear about them.

One truly good example that comes to mind is the quiz that Mensa uses. It provides you with two examples of a complete three-step pattern and then one partially completed pattern with the last step missing. Your duty is to fill in the last step from a list of choices. There’s nothing to memorize. There is some pattern (even if it’s extremely obtuse). It is only testing one very specific knowledge, that of what patterns are; and one specific skill, that of finding patterns. But it does its job well.

Another similarly good exam I recall seeing is the DLAB or Defense Language Aptitude Battery test by the Defense Language Institute, a US military-associated organization. The DLAB provides you with examples from a constructed language and then asks you to answer a question, using your acquired knowledge from the examples. It is another pattern-matching test, like Mensa’s, but rather than testing arbitrary pattern recognition, it tests linguistic knowledge (embedded or explicit).

Curmudgeon

After nearly 2,000 words of complaint, am I just being curmudgeonly? Perhaps. I have always avoided certification because, though I want the knowledge I can get by going through the learning process to get the certificate, I both don’t want to be forced to give questions answers which I know are incorrect and I don’t want an interlocutor to use my certificate as an excuse to appeal to authority. As you already know, I finally gave in. Was it worth it? I think so.

For me, getting a certificate serves two purposes: one is that now I can’t feel superior to someone because they have a certificate and I don’t like them, and the second is that people who haven’t read this article yet might use my certificate as a substitute for social currency and end up listening to my opinion (which, of course, I consider to be a good thing).

The reason this post is titled The Certification Game is because it truly is a game. The rules are simple: Guess what the author of the exam thinks you learned from the material they supplied you with, then get a piece of paper. So, I’m going to go collect some more paper and maybe learn a thing or two along the way. If you’d like to see what I’m collecting, head to my website http://aon.ink#certs where I’ll be updating what certificates I have or am working on currently. The one I just got is for the International Sports Association’s Certified Personal Trainer course. If you have any interest in receiving personal training, nutritional guidance, or meditation counseling, respond to this email and I’ll let you know when I have finished getting my paperwork in order and can accept new clients.

And now that you’re done reading, might as well subscribe.