Tuesday, April 5, 2016

The color of the dress should give you confidence... intervals

A bit over a year or so ago, an otherwise unremarkable photo of a dress in poor lighting became an internet sensation when it was discovered that different people saw the colors of the fabrics in the dress in dramatically different ways. There were two camps with little middle ground in between: the white and gold camp and the blue and black camp. People, including myself, were adamant that the other camp needed to have their eyes checked. Personally, I saw the dress as white and gold, and thought that people claiming that they saw the dress as blue and black were part of some elaborate hoax set up by ophthalmologists to generate checkup business. To investigate this, I set up a poll and submitted the link to r/SampleSize, which contains a small subpopulation of the popular website Reddit that are interested in taking surveys. I received a total of 45 replies, 27 responses for white and gold and 18 responses for blue and black This indicates that only 60% of responders saw the dress as white and gold, which was certainly not the hallmark of some great conspiracy, unless the secret ophthalmologist societies were in on my small-scale survey. The 95% confidence interval calculated from these results for the proportion of the population that sees the dress as white and gold was 45.45% to 72.98%. Phrased differently, there's a 95% chance that actual value for the percentage of the whole population that sees the dress as white and gold is in the range of 45.45-72.98%.

Another poll regarding the dress colors was conducted on a more visible platform. This survey received 432 respondents, 172 who saw the dress as white and gold. This is a much smaller percentage (39.81%) than that observed in my poll. The 95% confidence interval from this survey came out to 35.30-44.50%. Comparing my results to the results from this survey, there are two different 95% confidence intervals that do not overlap. So what does this mean? One possible reason for the disparity could be due to response validity, or lack thereof. I had created my poll well before taking this statistics class, thus it fell victim to a few different flaws in design. For one, my sample population was not random nor representative of the total population. Posting the poll on Reddit alone limited my sample population to the part of the population that uses Reddit, which is demographically weighted towards young males that spend a lot of time in front of a computer. The poll on Survata included a much larger age range of participants, though data on gender and time spent in front of screens was not recorded. Additionally, I did not limit the poll to one response per person, so responses may not have been independent. I also did not provide a choice other than white and gold or blue and black, which the Survata poll did include a choice for "neither". This may have skewed responses, as perhaps respondents who didn't see either color scheme chose to select the color scheme closest to what they saw, or just selected the first option on the survey, which was white and gold. These errors within my poll violated several assumptions that are necessary when analyzing confidence intervals, and thus weaken the meaningfulness of confidence intervals in the analysis of the data from the poll. However, it should be noted that it's entirely possible that confidence intervals from two different completely valid data sets do not overlap, or even do not contain the true population value of the measured parameter, due to the nature of what the confidence interval truly means.
The infamous dress

3 comments:

  1. When you conducted this survey, did you consider that listing the color options could provide some sort of response bias? Could less bias have been introduced if you instead showed a picture of the dress and asked the participants to list what two colors they thought were most prominent?

    When I first saw this picture, I saw a normal looking white and gold dress and did not understand what all of the fuss was about. After reading about the color controversy, however, I went looked at the picture and saw the dress as black and blue! Since seeing the dress in that light (pun intended) I have been unable to see the dress as white and gold anymore.

    Perhaps the best way to set up a survey of this nature would be to ask the observer to list the two colors that they see to avoid influencing their response. This design would definitely be more difficult to execute (especially since just about everyone has seen or heard of this dress) but I believe it would provide more unbiased data than the two surveys you discussed in your post.

    Also, great choice of a topic to write on. Seeing this dress as two different color schemes the first two times I saw it definitely made me question my brain's ability to process visual stimuli.

    ReplyDelete
  2. When you conducted this survey, did you consider that listing the color options could provide some sort of response bias? Could less bias have been introduced if you instead showed a picture of the dress and asked the participants to list what two colors they thought were most prominent?

    When I first saw this picture, I saw a normal looking white and gold dress and did not understand what all of the fuss was about. After reading about the color controversy, however, I went looked at the picture and saw the dress as black and blue! Since seeing the dress in that light (pun intended) I have been unable to see the dress as white and gold anymore.

    Perhaps the best way to set up a survey of this nature would be to ask the observer to list the two colors that they see to avoid influencing their response. This design would definitely be more difficult to execute (especially since just about everyone has seen or heard of this dress) but I believe it would provide more unbiased data than the two surveys you discussed in your post.

    Also, great choice of a topic to write on. Seeing this dress as two different color schemes the first two times I saw it definitely made me question my brain's ability to process visual stimuli.

    ReplyDelete
  3. Very cool of you to tie in a pop culture phenomenon with this class, Sara. I believe that this non-overlap between confidence intervals speaks to the multifaceted nature of statistics and can be attributable to Simpson's paradox, the idea that a trend that appears in different groups of data may disappear or reverse when groups are sorted in different ways. Upon looking into this further, I found that data scientists broke down the results of this color debate from a Facebook-administered survey into factors like gender, age, and viewing medium. Since men are more likely to be color-blind than women, this gender split proved to be interesting - 6% of men were more likely to see the dress as black and blue. Age-wise, younger users were more prone to viewing the dress as black and blue. Specifically, age groups 13-17, 18-24 and 45-54 saw black and blue. White and gold was dominant in the 25-34, 35-44 and 55-64 groups. "Neither" was the most selected option for those 65 and over, perhaps because they couldn't entertain the idea that such trivialities were permeating the world of younger generations. The outcome that I found the most intriguing was that data suggested users who took the survey on the phone (both iPhone and Android) were more likely to see the dress as white and gold as compared to those who viewed the image on their computer. It is clearly no surprise that depending on the way the data is grouped, statistics can illustrate varying relationships between data. And it is precisely this reason that the field holds a substantial amount of inherent bias.

    ReplyDelete