Saturday, August 17, 2013

Bad Science

I love science. It's not just a hobby for me, but a way at looking at the world to see the intricate beauty that permeates everything. In the same regards, I loathe pseudoscience. I despise it with a passion. It is detrimental and disgusting. As someone that prizes our ability as humans to rationally attack an argument, pseudoscience is one of the most filthy concepts that I can think of.

To note, I am not talking about science fiction (a literary field that I love very dearly and has no pretenses about being fictional) or theoretical science (which we have no way of properly testing as of yet, but offers interesting explanations) but rather ideas that wear the guise of legitimately tested science without the actual rigor or repeatability.


Show me the Citations

When looking at a supposedly scientific document, look at what it cites as its sources. Nothing cited? That's extremely suspicious. Never trust anything that doesn't have citations if it purports itself as a scientific document or scientific article. Notice that this document does not cite things. It is an opinion piece, not a scientific article. You can safely ignore opinion pieces as they are opinion and not fact. Lots of people have opinions and very few people have tested facts. Myself included in this matter.

A good rule of thumb : if a news report has some shocking scientific breakthrough, you can safely ignore it unless they cite some other sources of information or papers. This is doubly true if the title is a question.

"Does the newest research show that wallpaper causes bronchitis?"
Probably not, but if it were true they might have written
"Newest research shows that wallpaper causes bronchitis."

Note that I put probably there. There are legitimate cases where these tabloidesque articles are true and full of legitimate, factual information. The chances that they will not cite relevant papers and have their title as a question while still being accurate and relevant are extremely low.


Legitimate Citations Please?

Ok, you've looked it over and it cites three different articles on three other websites. Good, right? Not so much. Take a look at what those sources actually are. Let's say you've got a link to the main page of The Society of Homeopathy, a video of a perpetual energy machine, and the ever lovely theory of Time Cube. They are all bogus. These range from statistical tricks, to sheer impossibility, to... I'm not actually sure what Time Cube theory presents. I just really love their font choices.

(I really do truly swear that anything that purports homeopathic effects is lying to you. It's just placebo. Diluting anything does not make it more effective in the way that they are claiming. It literally does nothing.)
(Likewise, I promise that if perpetual energy machines weren't just garbage and flashing lights, and even if the US government was hiding it all from us, every other country in the world would be jumping on it like nobody's business.)
(You can believe the Time Cube stuff if you want. It's pretty trippy.)

A paper is only as good as its sources. If it has shitty sources, the science held within is also highly dubious. A good way of searching for the credibility of sources is looking up the exact opposite of what the article is purporting and seeing what the other side is saying. Please, do research. Don't just take my word on this. Look up stuff and see what people are actually throwing money at.

Look at what scientists actually think about it, though really only trust ones that are in that field. If you ask me about some advanced quantum effect that manipulates spacetime and results in a cubed field of time that swirls around the planet, I honestly don't have any idea. I'm a biologist for god's sake. Go talk to a planetary physicist. They do that stuff for fun.

Now the real problem isn't when you've gotten really spectacularly bogus stuff like above. The real problem is when you have semi-credible sources, or the science they are showing you isn't shown in the papers they are citing. This is fine if they show their own research and provide detailed methods for replicating it, but then you actually have some insight into what they are doing to determine if it's good science.


Do we really have to bring statistics into this?

If you're a lay person, you're good. You've got an article. It's got data and credible sources. It's got revered sources even, stuff that people have been saying is the cornerstone of the field. From an outside perspective, it looks good and it checks out. If it's got a spot in Nature or one of the other big scientific publishing houses, you should be good.

Just know that people suck at statistics. There is no way of certifying things one hundred percent. People also make stupid math errors, and people who have a lot to lose will sometimes make stuff up. It is for these reasons that I hate pseudoscience. Not because science is so great, but because science is so very fragile.

We can have findings that aren't statistically significant. Our best methods typically have a certainty factor of ninety five percent. We build a web of knowledge on strands of trust and probability. After enough time and enough trials of experiments, we can hold fast onto some knowledge as truth, and each fact is hard won.

When people mock science and intellectualism, they don't realize that everything we have is a delicate victory against a difficult and hostile universe that we must fight as a species to gather knowledge about. I do not hate pseudoscience because it dares to question the authority of some greater group, but because it destroys the fragile trust that we as scientists try to establish in our brief academic lives. It mocks the rigorous methods that have stood the test of time and it uses our hard work as a thin veil to push unsupported opinions as fact.

This is merely my opinion. As someone who has done promising work only to have it break apart upon scrutiny of statistics, I can only say as personal anecdote how frustratingly difficult science can be. It is fickle and temperamental. It will never show you the results you want, but it will show you what is true.

Whether you recognize the truth when you are staring at it is another matter entirely.

3 comments:

  1. Neil deGrasse Tyson disagrees:
    http://i.imgur.com/d2eJ4zK.jpg

    --Wes

    ReplyDelete
    Replies
    1. In all seriousness though, something that bothers me more than pseudoscience is when people scrap papers where their results don't match accepted theories. I didn't think it was something people did until my physics professor freshman year (Michael Thoennessen, associate director of the NSCL) explained to the class that part of his job was to make sure people didn't do that. It bothers me more than pseudoscience because some people are always going to be opinionated, some people are always going to lie, and some people are always going to misunderstand things.

      Not trying to make known the results (regardless of their alignment with accepted theories) of a scientifically rigorous study is tantamount, in my opinion, to saying that we already understand everything about a field. As you worded much more elegantly than I ever could, we have to strive to understand a universe this complex. Results being different from someone else's isn't about someone doing it wrong, it's about the both of you simply doing it differently, and the remedy is to take careful notes of everything you do, and I mean everything, so we can all further our understanding.

      Seacrest out,
      --Wes

      Delete
    2. Oh definitely. There's an entire movement that is trying to have people publish null results or in more basic terms, stuff that didn't work. The stuff that doesn't work, the stuff that goes against the current model, and the stuff that has statistically significant data that just doesn't match what we know, those are all things that we need to see in our publications. If the world worked just as we thought it would, then it would be a very boring world indeed.
      I think that diverse and often controversial research needs to be done, but it also needs to be done right. Having interesting results but not having the statistical power to back up claims is honestly heartbreaking. I want research that breaks the mold and with the correct setup and controls, it should be given equal weight.
      My rant on current scientific publishing process will have to wait for another time however. It is so messed up.

      Delete