The gloves are off, the pants are on
A lie gets halfway around the world before the truth has a chance to get its pants on. - Winston Churchill
(...Not really, though)
I collect links that are good examples of "how much BS is floating around out there unchecked." Though in many cases I haven't run everything to ground, and it could be the debunking that's BS, or both the original and the debunking ... so I do want the lesson to be "Don't trust things you've heard," as opposed to "Trust this debunking." (I will probably make a future post devoted to debunking debunkings.)
Nice piece debunking various memes that are supposedly based on studies: "the less someone knows about a subject, the more they think they know" (not actually Dunning-Kruger at all), "money doesn’t make people happy" (it seems like it does when making some basic adjustments to the data - I think this point is well known by now), "people bounce back from setbacks (as well as positive events) and return to a fixed level of happiness" (guess not) and "type systems help in programming" (don't know what this one is about).
You may have come across most of these, but here in one place are debunkings (of varying convincingness) of pretty much all of the famous old social psychology experiments that blew my mind when I was in my 20s:
- Disappointing replication of the "marshmallow" experiment.
- One person argues that the Prison Experiment was a case of subjects behaving as their experimenters clearly wanted them to.
- The Robbers Cave experiment backstory sounds particularly dicey: the experimenter tried to get two bands of boys to fight each other ala Lord of the Flies, failed miserably, tried again, succeeded, never mentioned the first time, and gained fame (despite even the 2nd try having the same issue as the Prison Experiment mentioned above).
- Finally, someone reports that many of the Milgram participants' own reports of why they had administered electric shocks provides more support for "they didn't think the person was really being harmed" than for Milgram's theory (link). I think this is the least compelling of critiques listed here, as people will rationalize their behavior with all kinds of stuff.
Via an old Marginal Revolution post, here's a study claiming that none of the famous findings about happiness are robust. As far as I can tell, the central claim is basically this general idea: say that Group A consists of two people who each rate their happiness 6/10. And Group B consists of one person rating their happiness 4/10, and another 7/10. In some fundamental sense, we don't know which group has higher 'average' happiness, because for all we know, each increment on the 1-10 scale could represent an extra '10 units' of happiness or an extra '10 times as many units' of happiness, or something else. Now, sometimes we might be able to know which group has higher average happiness despite this issue (for example, a two 7's and a 6 vs. two 8's and a 7), but the authors here argue that the famous happiness findings are not robust in this way. Which I think makes sense, though I hope to read this more closely later on.
One of the studies I've found most mind-blowing (with video evidence!) was this seeming demonstration that chimpanzees have better working memory than humans (at least for a particular, surprising task). But oops, here is a very believable-sounding debunking claiming that chimpanzees were trained extensively on the task, and similarly trained humans keep up just fine.
Finally, here's a 2008 study questioning the connection between exercise and certain mental health benefits in a convincing-seeming way. "Regular exercise is associated with reduced anxious and depressive symptoms in the population at large, but the association is not because of causal effects of exercise." The usual result is that exercise does cause such benefits, but this one looked at twin pairs and over-time changes: "Cross-sectional and longitudinal associations were small and were best explained by common genetic factors with opposite effects on exercise behavior and symptoms of anxiety and depression. In genetically identical twin pairs, the twin who exercised more did not display fewer anxious and depressive symptoms than the co-twin who exercised less. Longitudinal analyses showed that increases in exercise participation did not predict decreases in anxious and depressive symptoms."