Tuesday, September 18, 2007

Stats on Remote Viewing Tests

There are lots of bogus stats studies on parapsychology tests, but this looks more promising: UC Davis statistician analyzes validity of paranormal predictions. Pity it's in something called "The California Aggie."
In 1995, Utts was hired by the American Institutes of Research, an independent research firm, along with psychologist Ray Hyman from the University of Oregon to analyze data from a 20-year research program sponsored by the U.S. government to investigate paranormal activity.

After doing initial research, Hyman and Utts found statistical support, she said.

"The two of us did this review and we both concluded that there were really strong statistical results there, but [Hyman] still didn't believe that it could be explained by something psychic - he thought there would be some explanation [that he] can't provide," Utts said.

The research program involved remote viewing, in which test subjects were asked to describe or draw an unknown target. The target could be anything and could be located anywhere. According to Utts' meta-analysis of the 966 studies performed at Stanford Research Institute, subjects could identify the target correctly 34 percent of the time. The probability of these results occurring by chance is .000000000043.

In contrast, statistical support for the effect of aspirin on heart attacks: "The results demonstrated that aspirin reduced the number of heart attacks in people likely to have heart disease by 25 percent, with a probability of it occurring by chance equaling .0003."

Hyman's concern is valid, of course; the stats don't tell us causation, just that there's a pattern in the data that's unlikely to be due to chance. All sorts of biases could have been introduced during the experiments to produce the results.

But it's still provocative!


karl said...

I'd worry about confirmation bias in the arbitrary-image remote viewing case. With an infinite description space in "identifying" the object, how do you define a hit or not?

I read the study (http://www.stat.ucdavis.edu/~utts/air2.html) and was more interested in the immediately following test: remote viewers were asked to send two bits of zeros and ones, with three following bits for error correction. This is much more black and white in its scoring, and the results were indistinguishable from random chance.

Lynn said...

Sure, it's more black and white in the scoring. But it may have little relationship to the remote viewing test results. The "skill" involved in comprehending/sending/receiving strings of digits may have nothing to do with the skill of sending images or receiving images.

Having a predisposition to assume it's all crap is another bias of interpretation, too, karl.