Saturday, April 12, 2014

Interviewing with HAL

In a recent post, The New Teacher Project (TNTP) asks an interesting question. And by "interesting," I mean "dumb."

Can Better Questions Lead to Better Teachers?

By "better," TNTP means "very specific multiple choice questions asked by a computer." As it turns out, I'm pretty sure I know the answer to this question, so let's end the suspense right away. Jessica Cardullo, contributing writer to the TNTP blog, here is the answer to your question:


Or possibly, "for the love of God, no." I was also tempted to go with "No, dummy," by I do not know Ms. Cardullo and for all I know, she's a highly intelligent person who was assigned to write about a dumb question.

TNTP is the big-boys-and-girls version of TFA; they take people who have an actual history of working for a living in some non-teaching job and try turning those folks into teachers. It's actually a far less ridiculous concept than TFA, and if TNTP didn't have the same corporate weenie connections as TFA, I might be inclined to take it seriously. But instead, they are run by the kind of people who come up with ideas like this one.

First we have to figure out what "better" means when applied to teachers. Cardullo starts out by contrasting broad questions such as "Why do you want to be a teacher" with focused questions such as "Which of these statements best describes you" followed by multiple choice answers from which to select. Or they could be asked agree-or-disagree statements such as “All students should be held to the same grade level standards,” and “teachers in high-need schools must be given equal resources to teachers in wealthier schools if they are going to be evaluated by the same standards.” And I'm starting to see their problem, because by "better," maybe we mean "fully aligned with the biases of this organization."

Cardullo assures us that lots of standardized test questions such as these will elicit a picture filled with "nuance" and "texture." Because I think we can all agree that when we really want a nuanced and textured picture of a person, we hand that person a set of standardized test questions.

But TNTP is really looking at this "interview pointillism."

One interesting possibility: using computer-based interviews, at least at the outset of someone’s application to one of our training programs, to ask these sorts of questions. It’s an interview model that sounds inhuman and even a little scary, like the quest for machine-based efficiency gone too far. But we’re looking into computer interviews not because we’re trying to skimp on time, but because we think they might actually predict future teacher performance better than our old model, a daylong in-person interview.

Well, yes. If by "interesting" you mean "inhuman." Now, it's not that they never believed in rigorous interviews and screening; they've got training screening and interview screening and even phone interviews. It's just that "when we took a hard look at whether performance during the screening process was connected to classroom performance, we found very little relationship."

Now, I can definitely draw a conclusion from that, and my conclusion is that TNTP's screening people are very, very bad at their jobs. Either they don't know what to look for in prospective fellows, or they make it really easy to game their system, or they just don't know how to draw impressions about the character and ability of other carbon-based life forms. I might send them for training. I might reassign them. I might fire them. But the brain trust at TNTP has a better idea.

Multiple choice questions.


Rather than relying on whatever meaning interviewers ascribed to an open-ended conversation, these “forced-choice” computer survey questions could give candidates—and us—a clear, specific and common set of terms and ideas to review regarding their skills, experience, aspirations and expectations for teaching as a career. They could replace part of our existing application screen to allow us to drill down on the handful of particular skills we know we’re looking for, like professionalism, critical thinking and receptivity to feedback, which our experience tells us that most effective teachers possess.

You know, my ordinary approach is to exaggerate outrageously in order to make my subject look foolish, but today I am stumped. And by "stumped" I mean "saddened and astonished." In addition to the virtues of drilling down and attempting to determine critical thinking with a bubble test, Cardullo goes on to tout the virtues of computer assessment because it can't be biased (only the person who writes the assessment questions can be biased-- seriously, what is this magical belief that when a person says something or writes something it has human bias, but as soon as you type his words into a computer, magical elves dance out and suck all the personal bias away, leaving nothing but sweet, sweet perfect objectivity. And by "sweet" I mean "imaginary"). (And yes, I know we're practically describing the Praxis test here, but I'll save curmudgeoning about that for another day.) There are so many things wrong here. Sooooooo many. Let's just pick two.

1) You know what the best way to gauge someone's ability to interact with other human beings? Have them interact with other human beings. Yes, all the human beings involved in that process will bring their own personal biases, ideas, personalities, human foibles, hair styles and histories into the process. So will all the students who walk into a classroom. This is how human beings are. Stop trying to create systems that don't allow for humans to be human! If your humans in charge of talking to the humans can't handle talking to humans, you need to hire different humans.

2) You know what's really easy to lie to? A bubble test. Your questions will be biased and you will be testing your candidate's ability to figure out what you want him to say. What you won't have is the benefit of seeing him roll his eyes and flip the bird at the computer screen. Your standardized test will tell you exactly nothing except how good he is at navigating bureaucratic baloney. That is, of course, excluding the candidates who will decide that any organization that tries to screen humans for a job that's all about working with humans without involving any humans-- well, that organization is to be avoided.

Cardullo acknowledges this last problem. It is possible that this approach might shrink the talent pool. "The idea of a computer predicting which candidates have the potential to become effective teachers might sound a little crazy." If by "a little crazy" you mean "unbelievably stupid," then yes, you are correct. But in reform land, they have fallen so far down the rabbit hole, they cannot see how far they've fallen and how little sense they make. And by "rabbit," I mean "stupid."


  1. Wow! Thank you for this post!!!! Several years ago, I was applying for positions in reform-y Indiana, and of course, I came up against the initial online bubble tests... forced choice. As one who tends to "think outside the 3-d cube" (or in this case the bubble test), I answered the questions as honestly as I could; however, given the choices, it was impossible to truly express my thoughts. One of the questions that sticks out in my mind had to do with what to do if a child came into class with a hostile affect (not the exact words... but along those lines). The choices I had left me with questions. One of the questions I had was, "Well, my response will depend on the personality of the child and the resources available at the school. If we have a school social worker, then the appropriate response would be to quickly direct the child to the social worker before the child's behavior escalates. Or... depending upon the child, if the child is easily calmed, then there are simple responses that can taken care of in the classroom (like a quiet space / sensory table area). If there IS a social worker, then valuable class time doesn't need to taken from the other children if the upset child is able to have their issues dealt with outside of the classroom. If there IS no social worker, then the teacher (me) has to put on that social worker hat and help the child de-escalate... but again, it depends on if the child is a disturbed child (having worked in sp.ed.) or if this is just an average kid having a bad day. Soooo... this was my thought process... and yet I was forced to choose from A, B, C, or D. I based my decision on my (at that time) current position which included a school social worker. So, naturally, I wasn't considered "child centered" enough, because I should have chosen one of the other responses... and yet, if someone had personally asked me that question, they would have found the response child-centered for ALL children involved... not just the one who was having a rough time.

  2. This has nothing to do with your post, but I have a funny anecdote about Ms. Cardullo and TNTP in Memphis. Ms. Cardullo was once the interim director of Memphis Teaching Fellows. When I was with the program, we had 3 directors in less than a year. During that time I asked often asked for information concerning the partnership with CBU for a M. Ed. I often heard, "I'll get back to you on that." and then a new director would come. I then went to their offices to ask for the information. Ms. Cardullo then said she wasn't sure where that information was and then said, "Wait a second." literally reached in the cabinet above her desk and pulled out a packet with all the information I was looking for. There were stacks of them.

  3. I really love reading and following your post as I find them extremely informative and interesting. This post is equally informative as well as interesting . Thank you for information you been putting on making your site such an interesting. I gave something for my information. Interview Preparation