- Topics
- Aerospace
- Animals
- Anthro and Archaeology
- Bio and Medicine
- Brain and Behavior
- Business and Economy
- Computers and Electronics
- Education and Outreach
- Energy and Environment
- Geoscience
- Internet and Communication
- Media and Entertainment
- Nanotech, Chem and Materials
- Physics and Numbers
- Security and Defense
- Software
- Space
- Transportation
- Reader Blogs
- Commerce
- Register/Login
- RSS
Why admissions interviews should be banned
An important part of the admission process to a competitive college is the admissions interview. I'm against it. And that isn't just because interviews were originally instituted to keep Jews out of Harvard. It's because they are poor predictors of future performance and, even worse, they are poor predictors that people weight very heavily.
I was first clued into this by none other than Google. Google recently revamped the way it chooses new hires, and an important part of the overhaul was minimizing the importance of the interview. As Laszlo Bock, Google's vice president for people operations said, "Interviews are a terrible predictor of performance."
This stands to reason. We all know people who make great first impressions but then turn out to be lousy employees/students/friends/etc. Similarly, we know people who originally struck us as dull but turned out to be our best employee/student/friend/etc. However, it would be nice to have something quantitative to back up this observation, and so I've been on the lookout ever sense.
It is in this context that I read this following quote from a classic Science paper by Tversky and Kahneman:
It is a common observation that psychologists who conduct selection interviews often experience considerable confidence in their predictions, even when they know of the vast literature that shows selection interviews to be highly fallible. The continued reliance on the clinical interview for selection, despite repeated demonstrations of its inadequacy, amply attests to the strength of this effect.
Tversky and Kahneman probably did not think this was a problem with the clinical interview per se. They give several other examples, including a study in which participants read a short description of a particular lesson a student teacher gave. Some participants were asked to evaluate the quality of the lesson, giving it a percentile score. Others were asked to guess the percentile score of that student teacher's overall abilities 5 years in the future. The judgments in both conditions were identical. That is, the participants believed that the quality of a single lesson fully predicted how good a future teacher would be. They don't take into consideration that the student teacher might be having a bad or good day.
Tversky and Kahneman have an explanation for why people care so much about interviews. Across the board, people believe that small samples are much more reliable than they are. I recommend the original paper if you want the full argument, but they bring up many examples. For instance, participants believe a random sample of 10 men is just as likely to have an average height of 6 feet as a random sample of 1000. This is not mathematically possible, but even experts in statistics can, under the right circumstances, fall for this.
This is why I think the admissions interview, as well as the job interview, should be scrapped. It takes place over a short period of time, which means it is an inherently unreliable predictor of future performance. It's unreliable, but, even knowing that, the information gleaned from it irresistible.
Tversky, A., Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases. Science, 185(4157), 1124-1131.
Submitted by coglanglab on Fri, 2008-05-09 08:50.
- coglanglab's blog
- Printer-friendly version
- 1575 reads
Interview prediction-Lockheed?
I'm living proof that job interviews, pigeonholing, and measuring success and company profile fitness have very little to do with ability, motivation and success. As a matter of fact, we have surpassed NASA's wildest expectations and abilities in planetary science, and for every one cent I spent in the lab they probably spent a million dollars on the same project and I am still 6-8 years ahead of them in planetary science. No doubt about it.
SRD
www.bccmeteorites.com
The job interview
In the IT world, the job interview is also used as a means to enable age discrimination in hiring and enforce a more homogeneous work force. Many people/companies erroneously believe that a homogeneous work force performs and gets along better.
Google has been sued for age discrimination in the past. A couple of years ago I read that the average age of a Google employee was under 31. Has Google changed its bias against older employee's by deemphasizing interviews? Or perhaps they have found that they can simply use metrics such as year that you graduated college, whether you have a Facebook/MySpace profile, an IM ID or just a lack of work experience??
Job ads, particularly for small or start-up companies will sometimes use keywords like a "fun workplace" or "recent college graduate" to hint that the company is looking for younger people.
Job Interviews
Early in my working career I found that I would always get a job offer if I was at least asked in for an interview. This was true until I had a series of interviews with Lockheed-Martin. :-(
The job was referred to as a "data analysis" position, though they claimed they wanted a Ph.D. I noticed during the interview process that it appeared that I was being classified as a "modeler". Unfortunately, I didn't redirect their impression, and I later received a call from their HR department saying that they were wanting someone that was more of a "data analyst".
At this point there was nothing I could think of that an HR person would have understood in terms of trying to get myself out of this "cubby hole" in which they had filed me.
In my mind, of course I'm a "modeler". Isn't all of science about "models"? What is a theory but a "model" that has withstood all tests thus far? Isn't "data analysis" about "modeling" the data, applying the appropriate methods for which the data "model" apply?
Sure, if all they wanted was someone to apply "black box" algorithms to their data, I wasn't their man. I wouldn't even want such a job. But if what they wanted was someone that could come up with new data analyses, based upon new "models", then I was their man.*
So, yes, interviews can certainly mislead.
Unfortunately, what's a good alternative? For academia there is past academic performance (jobs do have somewhat similar data, provided the candidate has had similar positions), and standardized test scores (what standardized tests would one suggest for jobs?).
Perhaps it's not a fair comparison to lump academic admissions interviews with job interviews, even though they both have very similar reliability characteristics.
David
* The data they wanted "analyzed" was synthetic-aperture radar data. They wanted to be able to create maps and surfaces from the data. I had already thought of some novel approaches to such issues, but I wasn't going to tell them what I was thinking until I got the job. (Maybe that was a mistake, maybe not. I wouldn't be surprised if the person they eventually hired never thought of what I was thinking, since it was way outside of any standard "data analysis" methodology, and relied upon a "model" of what the data was of, and what information should be available within synthetic-aperture radar data, so long as the data wasn't "preprocessed" to the point that such information was lost.)
Thin Slicing
"thin-slicing" which is a huge factor in the interview process as it appears is the main culprit of the potential failure of the interview process.
See "Blink: The Power of Thinking Without Thinking" by Malcolm Gladwell for more incite.
Post new comment