Not So Fast on the UFO Warm Fuzzies

by Guest Blogger,
Ellen Tarr, Ph.D.

On Feb. 19, Motherboard posted an article by Daniel Oberhaus entitled, “This Neuroscientst Wants to Know Why People Who See UFOs Feel So Good.” The interview with Dr. Bob Davis discussed recent results of the FREE (Foundation for Research into Extraterrestrial Encounters) study. Information about the study as well as results from the Phase 1 and Phase 2 surveys can be found on the FREE website. The finding highlighted in the article and in their first paper is that of 3,057 people studied who have conscious memory of “contact with a physical craft associated with some form of non-human intelligence,” approximately 85% “are being transformed in a very positive behavioral or psychospiritual way.” This is an interesting finding, and even Davis points out that is wasn’t what was expected. However, there are a number of reasons these claims should be interpreted with caution and at least a little skepticism.

First, it is unclear from the information available how many people were really included in the study. Phase 1 and Phase 2 results show 2,658 and 1,792 respondents, respectively, although most questions were not answered by this many individuals. It is unclear how they arrived at the 3,057 people mentioned. The total from both phases would be 4,450, but it is likely some of the respondents participated in both phases. This could result in the lower number, but the section of the paper describing the methods does not include this information. It is standard practice when reporting survey results to report how many people answered the survey as well as how many surveys were used in the data analysis. It isn’t unusual to have exclusion criteria, but they need to be stated. The Appendix gives an N of 2,990, also with no explanation of how the number was obtained. To give benefit of the doubt, let’s assume for the remainder of this article that surveys from 3,057 unique respondents were analyzed in the study.

Second, the authors need to differentiate between the “study population” and the number of people responding to a given question when reporting their results. For example, 85% of the study population would be 2,598 individuals. This “major positive behavioral transformation” was represented by an answer of “strong increase” to items, including: concern with spiritual matters, desire to help others, compassion for others, appreciation of the ordinary things in life, ability to love others, concern for ecological matters, an understanding of “What is Life all about,” understanding of others, and conviction that there is life after death. I identified 28 items in the Phase 2 study that seemed to directly address these issues. These had an average “strong increase” response rate of 49%. Similarly, “strong decrease” responses to some questions was also evidence of a positive transformation: concern with material things, interest in organized religion, and fear of death. The average “strong decrease” to six items I identified was 42%.  It is possible that different respondents answered “strong increase/decrease” for different items, such that 85% of individuals responded this way to at least one of the questions, but the provided data don’t address this. The average overall response rate for these questions was 77% of the 1,792 Phase 2 respondents, so even if all of them had answered “strong increase,” it would not be enough to be 85% of the study population (3,057 respondents).

In the paper that is available from the website, they specifically report state that, “Overall, 50.9% reported a ‘Highly Positive Effect’ and 21.7% reported a ‘Slightly Positive Effect’ on ‘changing their life’ directly from their UFO-NHI interaction. In contrast, only 4.3% reported a ‘Highly Negative Effect” and 6.7% a “Slightly Negative Effect’.” These appear to be the responses for Q415 of the Phase 2 survey, which had only a 74% response rate. Combining the two positive responses, the reader might think that 72.6% of the 3,057 respondents (2,219) reported a positive impact on their life when instead, it is 72.6% of 74% of 1,792 (Phase 2 respondents), which is 963 individuals. Q99 on the Phase 1 survey was similar, and approximately 64% gave a “positive effect” response; the overall response rate for the question was 48%, so this was only 31% of the Phase 1 respondents.

A third major issue is the lack of controlling which respondents answer follow-up questions. The results are supposed to be for people “who have reported to have had unidentified flying object (UFO) related contact experiences with non-human intelligence (NHI).” However, it isn’t clear that analyzed responses were only given by people who claim to have had this type of experience. Only 924 people claimed there was a craft or ship associated with the ET contact experience (Phase 2, Q41), and only 553 and 708 claimed to recall being on UFO in Phase 1 and Phase 2, respectively (Phase 1, Q97; Phase 2, Q42). Only 1,275 claimed to have observed and NHI entity/ET (Phase 1, Q79), and 1,850 claimed to have seen an intelligently controlled craft in the sky or on the ground (Phase 1, Q14). There are numerous cases within the survey where more people responded to follow-up questions about a specific type of experience than had claimed to have had the experience. For example, 211 respondents reported having sex with an ET and 236 gave answers regarding what type of ET they had sex with. The likelihood that many items include responses from people who did not have the experience calls many results into question. Survey programs (such as SurveyMonkey used in the study) provide the ability to use skip logic to allow only those who respond a certain way to one question to see related questions. This problem could have been avoided if the authors had just taken the time to design the survey appropriately.

My fourth point relates to exclusion criteria. The methods section states that, “All subjects included in this study reported that they have never been diagnosed with a mental illness by a licensed mental health professionl [sic].” This is the only exclusion criterion mentioned, although there is no explanation of how many respondents were excluded for this reason. It seems unlikely that no one with a mental health diagnosis even tried to take the survey. While it is understandable to exclude those with mental health issues, it has direct implications for interpreting the results regarding positive impact of contact. Individuals who have had UFO/ET contact that has been very negative, even traumatic, may be more likely to have PTSD, anxiety (including a panic disorder), and/or depression. Excluding these individuals has a high probability of biasing the results toward individuals whose experiences were less negative/traumatic.

For my final point, I’m going to overlook the myriad problems with the survey itself and the analysis (much of it not discussed here), and accept for the moment that the “positive transformation” finding is valid. A positive transformation can, however, occur following a negative experience. One can imagine a renewed appreciation for life and what it has to offer following a serious accident or major illness. Should the accident or illness itself then be interpreted as a positive experience? I hesitate to advocate for that interpretation.

While it is tempting to conclude from these survey responses that contact experiences are overwhelmingly positive, negative experiences should not be ignored. As SNL points out, for every two people having these positive experiences, there is a Ms. Rafferty saying, “Yeah, a little different for me.”