Psychological tests are important instruments used in courts to aid legal decisions that profoundly affect people’s lives. They can help determine anything from parental fitness for child custody, to the sanity or insanity of a person at the time of a crime, to eligibility for capital punishment.
While increasingly used in courts, new research shows the tests are not all scientifically valid, and once introduced into a case they are rarely challenged, according to Tess Neal, an assistant professor of psychology at Arizona State University.
“Given the stakes involved one would think the validity of such tests would always be sound,” Neal said. “But we found widespread variability in the underlying scientific validity of these tests.”
The problem is made worse because the courts are not separating the good from the bad.
“Even though courts are required to screen out ‘junk science,’ nearly all psychological assessment evidence is admitted into court without even being screened,” Neal said.
Neal was speaking today (February 15) at the annual meeting of the American Association for the Advancement of Science in Seattle. She presented her findings in the talk “Psychological assessments and the law: Are courts screening out “junk science?”
In a two-part investigation, Neal and her colleagues found a varying degree of scientific validity to 364 commonly used psychological assessment tools employed in legal cases. The researchers looked at 22 surveys of experienced forensic mental health practitioners to find which tools are used in court. With the help of 30 graduate students and postdocs, they examined the scientific foundations of the tools, focusing on legal standards and scientific and psychometric theory.
The second part of the study was a legal analysis of admissibility challenges with regard to psychological assessments, focusing on legal cases from across all state and federal courts in the U.S. for a three-year period (2016-2018).
“Most of these tools are empirically tested (90%), but we could only clearly identify two-thirds of them being generally accepted in the field and only about 40% as having generally favorable reviews of their psychometric and technical properties in authorities like the Mental Measurements Yearbook,” Neal explained.
“Courts are required to screen out the ‘junk science,’ but rulings regarding psychological assessment evidence are rare. Their admissibility is only challenged in a fraction of cases (5.1%),” Neal said. “When challenges are raised, they succeed only about a third of the time.”
“Challenges to the most scientifically suspect tools are almost nonexistent,” Neal added. “Attorneys rarely challenge psychological expert assessment evidence, and when they do, judges often fail to exercise the scrutiny required by law.”
What is needed is a different approach. In their open-access paper in Psychological Science in the Public Interest, Neal and her colleagues offer concrete advice for solving these problems to psychological scientists, mental health practitioners, lawyers, judges and members of the public interacting with psychologists in the legal system.
“We suggest that before using a psychological test in a legal setting, psychologists ensure its psychometric and context-relevant validation studies have survived scientific peer review through an academic journal, ideally before publication in a manual,” Neal explained. “For lawyers and judges, the methods of psychologist expert witnesses can and should be scrutinized, and we give specific suggestions for how to do so.”
Two reasons: Psychology is a not a science because experiments can’t be duplicated. Why not? Because, unlike neutrons, every human is different. I have “professionally” talked to several psychologists. It was like being on the “Bob Newhart” show.
Second, because statistics and probability do not demonstrate a cause and effect.
Why not? Think bell curve and experiments. Bell curves are actually standard deviation curves. Most experiments will fall within 1 SD of median 87% (? my memory for numbers was never good) but in a run of identical experiments, the first experiment several runs have an equal chance of being more than 4 SD inaccurate.