## Friday, April 8, 2016

### Red(shirt) Alert: Stats of expendable crewmen

Statistics-first research planning is a relatively new concept to me, though one that I’m increasingly seeing as important. Over the past few weeks, Dr. Murphy has mentioned several times that a researcher should know exactly which tests they want and use them to guide their experimental design. Scientists are often trained, however, in an experiment first, analysis later manner. Rarely have I seen a lab class include the use of a statistical test to examine data besides linear regression. Instead, statistical testing is often taught in standalone classes where data is contrived. Even working in a non-academic laboratory, the mentality was data now and we’ll ask the staff statistician what to do with it later.

Though, through several stats courses, I’m familiar with most of the standard statistical tests used often in lab work, they are still an afterthought. Combatting years of procedural habits is difficult, but there’s time to get into the habit of statistics-first planning before I really begin in my thesis work. The first step to that is to integrate the use of statistical tests into my daily thoughts and, since many of my musings revolve around one topic in particular, I thought it best to begin there. Ahead, full impulse, Mr. Sulu!

One well-known joke arising from Star Trek: The Original Series is that of the ominous red shirt. (But only TOS red shirts, not any subsequent series) Kirk, Spock, and Bones beam to a planet with a small force of red-shirted security and within a few minutes those escorts are victims of the latest alien/ plant/ disease/ energy/ weather mishap. Just observationally, it seems that wearing a red uniform increases chances of death. But how does that stand up to the rigors of statistics?

In 2013, Matt Barsalou wrote a post for Significance Magazine exploring that idea and several people used his ideas as a springboard for further analysis. Of the 40 on-screen crew deaths (of people wearing uniforms), 24 of them wore red shirts. But Barsalou pointed out that viewing those numbers on their own isn’t necessarily fair considering that there are varied numbers of crewmen of each of the three uniform colors and that multiple departments wear the same colors. One of those departments using red is the security team, which is often included in more dangerous work than, say, the similarly red-shirted engineers. Using Bayes Theorem, he showed that there’s a 64.5% chance that any given red-shirt casualty is on the security team despite security making up a minority of red shirts. Of the remaining non-security red shirts, he calculated that they have an 8.6% chance of being that casualty. Rather than the color of the uniform being the issue, Barsalou concluded, it’s actually the occupation. This makes more sense than a universe-wide aversion to a uniform color.

Jim Frost, an author on the Minitab blog, also confirmed Barsalou in another post where he examined the same data with different tests. When looking at whether red-shirt deaths exceed the overall average of deaths, he found with the chi-square test that red shirts actually have a rate of death that is about average of the three groups, as they die more often but derive from a larger pool of crewmen, whereas gold uniforms die with a higher rate than average due to being such a minority overall. Noting, as Barsalou did, that shirt color isn’t the correct variable to examine, Frost used the two proportions test to look at occupation. With a very low p-value, he found that the rate deaths of red shirts in the security department far exceeded those of other red shirts. Thus, being in security is dangerous, not your shirt color.

Other people on the internet have looked into other Trek-related topics using statistics, including one post using the Mann-Whitney test on Trek film ratings to see if the odd- or even-numbered films are better. Realizing how applicable these statistical tests actually are brought statistics out of the realm of rat stressors and confusion and into the streamlined Federation star ships that I know and love. Spock might agree that statistics-first organization is the logical choice to make, if not the natural choice, and better-designed experiments and better-informed test choices will be the result.

//The video is a song about red shirts performed by the Enterprise Blues Band, a group made of actors from Trek that I saw in person last month. Unless they're all security team, stats says they don't have much to worry about.

1. There's nothing I love more than reading about using the scientific method to describe pop culture (as recently done using math to determine who is the main character in Game of Thrones: http://www.iflscience.com/editors-blog/mathematicians-create-game-thrones-social-network-work-out-who-rules-westeros). It's one of those things that helps to make something as difficult as statistics interesting and fun to learn.

I agree that it's been weird to think about science as stats first, data later. It has made me realize how much more rigorous I need to be with my experimental design. The example presented here about red shirts in Star Trek is interesting to think about. It is such a well known trope, and I've never stopped to really question it. But when pulled apart and really probing with statistics, it's clear that there are a lot more important variables impacting the observations that red shirts are more prone to dying on the show. An odd way to learn something as important as learning how to properly design an experiment and analyze data.

2. This comment has been removed by the author.

3. Really interesting points! I've obviously heard the red-shirt jokes over the years since my dad was a Trekkie, but I hadn't read the more in depth analysis of it being related to occupation. It does in fact make far more sense then it simply being a color bias on the part of the production team. Though maybe that's still a factor! Red does hide blood better than almost any other color (except black) so perhaps they chose red to make filming battle scenes simpler.

Pop-culture aside, what struck me the most from your post is actually was this: "Rarely have I seen a lab class include the use of a statistical test to examine data besides linear regression." My physical chemistry class did require more complex statistical analysis than most of my course work, but never did ANY of my classes require doing statistics a priori. Never once did we have to do a power analysis of the number of flies necessary for the genetic test we were doing in lab, or predict the effect of a ligand on protein kinetics and plan how we would determine if the effect was significant. We did the experiment as it was laid out on the protocol and then messed around with some analysis until it looked nice enough to put in the lab report.

Surely we are capable of teaching undergrads better than that? Just think how much more rigorous our statistical analysis would be if the habit and process of choosing the correct experimental set up and statistical methods were ingrained from the very beginning of our scientific careers!

4. I liked how you related what we have been learning in class to a cultural reference that pretty much everyone can understand.

I was never taught to statistically design your experiment before performing the experiment, so I feel like I am personally going to have to put in extra effort to make sure that I do this in my thesis work. When thinking about this concept, it makes perfect sense to determine what statistical tests you want to use beforehand because it allows you to avoid bias. However, the actual implementation will probably prove to be more difficult.

I thought your blog was a great example of how these tests can be used to discover findings, and even disprove common beliefs.

5. Thank you for explaining the bias against the Red shirts as actually just a reflection of occupational hazards. I had a discussion about this a while ago with a friend and we came up with many reasons for the prevalent death of red shirts, including occupation, but we never got around to actually investigating with statistics. I always find it interesting when someone approaches a pop culture reference with a scientific method.

The other thing that really stood out about this post was the mention that we do not learn proper statistic in college lab science classes. When completing an experiment in lab often the teacher or TA would just tell you how to analyze it or just want it graphed with the standard deviation. Until i actually too a statistics class, I did not even realize that there was other ways to analyze data and that the t-test would not work in every situation. We really need to do a better job teaching statistical analysis while we are teaching science. Statistics goes so closely with science that it is a bit ridiculous that statistics is not taught in conjunction with lab sciences.

6. As a total fiction lover I found this blog post extremely interesting. Like everyone else I had heard about the red shirt jokes and memes (http://s2.quickmeme.com/img/5e/5ed8f8b75fb1a9b33b072519ae1b560c518c4c2c071723a52c4e2544f0072e09.jpg) like the one in this link.
However, I find it fascinating that this concept applies to science and pop culture. It is true that we might think we found the answer or in this case the solution to preventing death in Star Trek just do NOT wear a red shirt but we are biased and sometimes ignore other factors that come into play. Furthermore, it is a prime example of how the appropriate statistic analysis method makes a huge difference in the conclusions that can be made from data.
Being premier weekend for GoT (game of thrones) when I saw Alice's post I got really excited and read the article. I found it extremely interesting that we can mathematically calculate the main character of such a complex story line. Specially, where the perceived or usual main characters are so often killed. It is a great reminder that statistics can be applied to our daily lives.

7. What a great post! Like the other commenters, I too am a big fan of science fiction, and of Star Trek in particular. I think it's an interesting side note that this phenomenon only occurs in the original series - maybe the writers got wise to the joke by the time The Next Generation came around?

But what I really liked about your blog post was that it took the time to tease apart all the other potential factors that could be the cause for the famous casualty ratings on the Starship Enterprise. I feel that too often in science we can fall victim to seeing the trend that we want to see in the data we have collected. As the first figure you included in your post attests to, even by using the power of statistics we can fall prey to this trap. If we know by our preliminary study (i.e. casually watching a ton of Star Trek) that a lot of crew members wearing red shirts die, we can decide before conducting our official 'experiment' (i.e. taking notes on a ton of Star Trek) to analyze our data by looking into the number of deaths associated with each crewman's shirt color. This would give us the first figure you provided, and decent evidence that there is a trend. Thus, even by having a well-designed experiment using well-thought-out statistics, we are not safe from making the wrong or an incomplete conclusion. Statistics cannot save us from missing an underlying connection in our study - death by occupation, rather than attire. While statistics is certainly powerful, it is still our job as scientists to not use it as a crutch; it is up to us to apply our critical eye and constant skepticism to uncover the whole story.