"Some say they see poetry in my paintings; I see only science." -Georges Seurat



Wednesday, December 5, 2012

Why Eye Witness Accounts Are Not Good Scientific Evidence


People think that because there are thousands of eye witness reports of Bigfoot sightings (or UFO's, or angels, or Loch Ness monsters, or unicorns, or whatever) there must be some truth to the matter. They admirably want to remain open-minded about things. Surely all those eye witnesses can't be wrong, right?


Let's use some simple math to show where this line of thinking leads people astray. Suppose we are saying to ourselves: "There were something like 3,500 'credible' Bigfoot sightings in the U.S. over the last several decades. Even if 90% of these eye witness reports are wrong or inaccurate, that still leaves 10% or around 350 accurate reports. Therefore, Bigfoot likely exists, and there are probably hundreds or maybe even thousands of them! Yay!!"

The problem here is that people are working with the wrong set of numbers. What we should be looking at instead as our starting number is the number of events in which a Bigfoot sighting is at least theoretically possible, and from that starting point figure out how many might be likely to be credible or reliable (and how many not). As an example using real data, we can check how many black bears there are, use this to estimate the number of black bear sightings by humans, and then use an estimated misidentification rate to see how many mistaken eye witness reports we might expect to see in which 'squatches are mistaken from black bears. We will try to use conservative numbers when estimating. Then we will compare our results with the actual count of eye witness reports...

There are at least 735,000 black bears in North America (excluding Alaska, Texas, and a few other states due to lack of data). If we assume that on average each of these bears gets sighted once per year (some get sighted multiple times, others don't get seen at all), then there are 735,000 events per year with the potential for a mis-identification by a human viewer. Over the last 60 years, this means that there were roughly 44.1 million sighting events of black bears by humans with the potential for a mis-identification.

Let's be generous to witnesses and assume that 99% of the time a black bear is seen by a person, it is correctly perceived as a black bear. But 1% of the time, a misperception occurs, perhaps due to environmental factors like low light levels, fog or rain, far viewing distance, tree coverage, "eyes playing tricks", or whatever else. Sometimes, the black bear is misperceived as a brown bear, other times as a deer, still other times as other mammals, maybe as people, maybe as an alien. Occasionally, though, a black bear is misperceived as a non-human primate (let's say 1% of these misperceptions).

So taking 1% of 1% of 44.1 million viewing events leaves us with the prediction that there should be about 4,410 reported instances over the last six decades of someone seeing a non-human primate when in fact what they really saw was a black bear. It's an interesting coincidence that there are just about this many "credible" Bigfoot sightings reported; while the numbers aren't identical they are certainly close and of similar magnitude. And it's also a strange coincidence that the hypothetical Bigfoot habitat, which has been geographically-determined based on the spatial locations of eye witness reports, coincides almost exactly with the habitat regions for black bears, to a degree that strains credibility.

The nice thing about this analysis is that it makes absolutely no assumptions regarding whether Bigfoot exists or not. It simply shows that there are more than enough expected mis-identification possibilities to account for all the 'credible' Bigfoot sighting reports, without any of them actually being true. And it seems likely that black bears are the likely culprit driving many (if not most) mis-identifications.

...........................
I'm an experimental psychologist by training, specializing in perception. If there is anything close to a scientific law in psychology, it's that what people report seeing, hearing, feeling, remembering, or experiencing is just not reliable. Perception is not a reflection of reality, it's a model, a mental representation constructed by our brains. Usually this model is fantastically accurate, but sometimes it's not. If we bring people into the lab, sit them in a dark room, and have them press a button whenever a dim light is flashed in front of them, their performance will almost never be perfect over repeated trials. Accuracy may be high, but it's rarely perfect.

So even on the simplest possible perceptual task we can imagine (seeing a flash of light in a dark room), people still make mistakes and misperceptions about what was experienced (sometimes seeing the light when it is not there, sometimes missing the light when it really is there). They might even be absolutely certain and 100% confident that their perceptions were accurate, and still be wrong, I see it all the time just in the lab. Now imagine how many more perceptual errors we might expect when identifying an animal through trees, from a distance, in possible low light levels, with maybe poor eye glass prescriptions; it could be foggy or rainy, plus a million other intervening variables that complicate visual processing ability when identifying a moving blob.

The legal and law enforcement professions know all about this eye witness reliability problem. Eye witness inaccuracies are the single greatest cause of wrongful convictions, contributing to something like 75% of the legal cases that are later overturned through DNA testing. Unfortunately, you just can't totally trust what people say, or report seeing. You can certainly believe them when they say they saw something; I don't think most people are lying or are stupid or anything like that. I think they literally saw what they say, in the sense that their perceptual system reported to their consciousness that a visual identification was made, a match was found in their heads, even if it happened to be the wrong one. But again, perception is not reality, it's just our brain's best attempt at a representation of reality given limited, noisy, and sometimes poor-quality data about what's out there in the world. This is well-understood in the psychological sciences, neurosciences, legal, and law enforcement professions. By now it should be common knowledge, but often isn't.

Sadly, we must acknowledge that eyewitness reports are not good or reliable evidence, and they certainly don't count as good scientific evidence for proving the existence of mythological creatures for which no other good quality scientific proof can be fostered.

3 comments:

  1. You say "there should be about 4,410 reported instances over the last six decades of someone seeing a non-human primate when in fact what they really saw was a black bear. It's an interesting coincidence that there are just about this many 'credible' Bigfoot sightings reported; while the numbers aren't identical they are certainly close and of similar magnitude."

    I see the case you are making but I consider it too specific for correlating numbers of black bear sightings with those alleged to be Bigfoot.

    Black bears inhabit approx. 35-40 US states while Bigfoot has been "sighted" in every state except Hawaii. So BF sightings in certain states cannot be misidentified black bears.

    Also, many BF "sightings" are hoaxes, tricks of light & shadow (optical illusions), hallucinations (due to alcohol or substance impairment), or misidentifications of animals other than black bears (deer, elk, etc.).

    You say "People think that because there are thousands of eye witness reports of Bigfoot sightings (or UFO's, or angels, or Loch Ness monsters, or unicorns, or whatever) there must be some truth to the matter" from which I might reasonably infer that such "people" are a random sampling of the general population.

    I don't think they are. (We are talking now about the folks who believe the eyewitnesses, a separate group.) I think that those who steadfastly maintain that BF is real, the true believers, are mostly of a certain type. This type might significantly comprise those with Fantasy Prone Personality or related disorders of perception & thinking. BF believers, I have noticed, tend to buy readily & uncritically into conspiracy theories (9/11 is an inside job, alien bodies + spacecraft kept at Area 51, JFK's grassy knoll), UFOs, etc.

    The believers explain the absence of BF carcasses by variously asserting that BFs bury their dead, that the govt confiscates any BFs killed by cars & hunters, & that BFs are simply too smart/cryptic in their habits to die where humans might find them. (These, of course, are purely speculative "theories" of behavior pulled out of thin air in support of a beast not demonstrated to exist. Fallacy piled upon fallacy.)

    And they insist that hunters, as a group, choose NOT to shoot a BF because he is too humanlike, too majestic. (By that reasoning, gorillas, chimps, orangutans ought never to get shot either, but they do, & by the thousands. Believers also don't account for the statistical exception to the rule: the rogue hunter who WILL elect to pull the trigger on a BF.)

    Which is to say that diehard BF believers are delusional, disordered in their thinking, & purveyors of faulty logic. They *believe* on the mere say-so of "eyewitnesses". "Not all of them can be lying," they insist. And they assert BF is real because I can't empirically prove he isn't. (But then I can't empirically prove Santa Claus & the Tooth Fairy don't exist either.) They appear incapable of objectively weighing one side of a case against the other. They seem unable to surrender a prior tenaciously held position & adopting a view that counters it, however superiorly reasoned.

    So while there are problems (to say the least) with eyewitness accounts per se, the larger issue is with the people who believe the eyewitnesses.

    My mother is the most honest person I know. If she said she saw Bigfoot last night, I would absolutely believe that she believed what she saw. But her *fact* is not objective & not provable to anyone else, even to me. She's not lying so I conclude that she was a little too tipsy on Nyquil or that she saw something else (the neighbor in shadow). But not a Bigfoot, since he doesn’t exist until it’s proven he does.

    Cheers,

    Windigo ~ Somewhere in the Rockies, USA

    ReplyDelete
    Replies
    1. Thanks for reading and commenting, Windigo. I don't think that all Bigfoot sightings are accounted for by black bear mis-identifications, although I think many (if not most) are. The point was to show that ONLY considering black bears, we can easily account for all BF sightings without a single one being true. We can add as many other animals to this analysis as we like, just for a comfortable margin of safety in our numbers.

      It may very well be the case that belief in one unlikely thing tends to go with another, but this in itself is not a compelling argument for or against a particular thing. While I agree with you, it won't sway any believers to your cause or convince an open-minded reader who hasn't yet decided which way the evidence seems to point. So I'm treading carefully so as not to unnecessarily insult anyone's beliefs, simply because many people will just instantly stop listening to an argument once their sensibilities are insulted. Maybe not the path to truth, but a common human quirk nonetheless. Take care ~ wishing I were in the Rockies.

      Delete
  2. "While I agree with you, it won't sway any believers to your cause or convince an open-minded reader who hasn't yet decided which way the evidence seems to point. So I'm treading carefully so as not to unnecessarily insult anyone's beliefs . . ."
    ------

    I think you are wise to be diplomatic & respectful in tone, especially when maintaining a website like this, where you are discussing a hot-button topic such as the existence of Bigfoot while being the skeptical party. Once you get disrespectful or present as superior in such a forum, it becomes a game of insult exchange where no one wins, nothing is learned, & both sides come out even more entrenched in their positions.

    While I am direct & clinical in my assessment of the BF believers' basic rationale (which tends to be commonly held), I, like you, have no intention of starting up flame wars & getting all ad hominem. But I am also not interested in persuading anyone to come over to my side, to see things as I do. I don't care. People are free to think & believe as they choose. (Besides, once a person has *decided* that BF is real, he actively resists attempts to be persuaded otherwise. The same is typical of 9/11 "truthers" & UFO believers. It is a complete & utter waste of time to engage them in debate if your purpose is to win & gain converts.)

    What DOES intrigue me is the phenomenon of belief in things for which there is no supporting physical evidence. I am quite amazed at how certain people can be so steadfast, even absolute, in holding to positions when there is zero objective basis for doing so & where the far better reasoned view is in opposition.

    So while debating the existence of Bigfoot per se holds little interest, understanding WHY people believe as they do, analyzing it, is eminently eye-opening. There is little doubt in my mind that it is largely a specific personality type that exhibits the tendency to uncritically entertain & adopt the believer's view - which requires one to engage in some remarkable gymnastics of logic & reason during the process. It truly baffles me, especially since such people are often otherwise ordinary & well-adjusted in their personal & professional lives.

    I do wish Bigfoot existed. The world would be infinitely more wonderful & fascinating for sure. But wishing won't make him real, nor will sincere, fervent belief in the accounts of "eyewitnesses." I've got to have a body but I'm not holding my breath in anticipation of a dead one showing up anytime soon, whether as a roadkill specimen, a poaching victim, or a Lyme disease casualty.


    Windigo ~ The Wasatch Range of the Rockies

    p.s. I would agree that the majority of Bigfoot "sightings" involving actual misidentified animals are of black bears.

    p.p.s. We have Bigfoot in our part of the Rockies, too. Just 10 miles from my house! See "bigfoot provo canyon" on YouTube. 7,000,000 views in one month. (I can't tell whether it's a black bear or a dude in a suit.)

    p.p.p.s. I like your basic statistical approach in addressing Bigfoot questions. It should nudge the more math-minded of questing agnostics toward the skeptical view.

    ReplyDelete