Showing posts with label big data. Show all posts
Showing posts with label big data. Show all posts

Monday, January 22, 2018

Bias: an unavoidable word related to Science and Research

Research bias, defined as the process where the scientific researcher influences results for a specific outcome, is a loaded phrase that has become an important topic to address in scientific studies. Unfortunately, the cyclic process necessary to become a successful, funded scientist leaves many opportunities for the scientist to introduce bias into their research. This includes, but is not limited to, research design, hypothesis development, previous research results, choice of analysis, and getting a grant to keep a job.

A relatively new field in science, mainly because of advanced technology and resources, is big data research. Big data research, regardless of subject, relies on the scientist either to process data with previously generated algorithms/codes or to create their own processing method. When picking or creating the “correct” algorithm or code to use, it is easy to unconsciously choose a method that best agrees with one’s proposed hypothesis or desired results. For example, it is hard to decipher whether the data has no effect because there was indeed no effect or if the wrong algorithm/methods of analysis were used.

There needs to be accountability. One way to do this, specifically with big data research, is by sharing raw data, coding algorithms, and specific data analyses methods through open source websites such as GitHub: Science. Not only will this encourage other scientists to reproduce results, it will also create an environment that encourages collaboration. If raw data and codes are made available through open source, anyone can provide feedback or ways to better the analysis process. Another important aspect is allowing people to upload or provide data on negative results. In most cases, negative results are not publishable. This would give a research group an opportunity to get feedback from other people in the community on their negative results and perhaps even help others that are pursuing the same research question. 

Now, more than ever, it is necessary to expect experimenters to produce unbiased and reproducible results. We have the resources to do this. It is up to the science community to create new standards that allow for accountability and reproducible results while encouraging collaboration. 

Monday, May 2, 2016

BIG brother regulating BIG data







http://www.sciencebusiness.net/news/79758/UK-government-plans-ethics-council-to-guide-big-data-researchers

The government in the UK recently passed legislature to regulate what they are calling ‘big data’. This is an umbrella term to describe all of the major meta data being collected recently such as the data that might be collected from Facebook and Microsoft concerning usage of their products by the public. To provide ethics and regulations for this type of data, a new committee will be put into place at the Alan Turing Institute.


I thought this story was an interesting take on ethics. We always talk about ethics in terms of potential (physical) harm to the individual. With the advances in technology and the ability to quickly access and analyze large sets of personal data it makes sense that different countries are attempting to put regulations in place to try and curb misuse. I think it is great that they are trying to put rules in place now before things get out of control. This would be in contrast to the type of bioethics committees in the US that were largely formed due to the public exposure of unethical experimentation such as the Tuskegee experiments. One aspect that I am not sure about is how long these different regulations can last before reform is needed to keep up with the times. Technology is always changing and surely new ways of mining personal data will be developed. It will be interesting how these different committees interpret these new technologies and how the rules are adjusted to protect the public. 

Tuesday, April 12, 2016

The Trees for the Forest


            I picked up my first pipette in 2009 in a virology lab that focused on a single domain of a single protein of a single virus. The majority of our assays were in vitro, and even cell culture was used sparingly. We essentially studied a single cog of a machine outside the context of that machine, and this was standard practice in the field.
            Today I am sifting through massive piles of RNA sequencing, proteomics, and metabolomics data in an effort to see the machine more clearly. As our high throughput and analytical methods have progressed, scientific research has gained the ability to investigate not only the intricacies of biology, but also the opportunity to examine the bigger picture. With that opportunity, however, come significant challenges.  We can spend months chasing down artifacts or trying to piece together seemingly contradictory data with little success. Statistics is our only guide in this process, and many of us have only rudimentary training in the nuances of analysis necessary to navigate these mountains of data.
            The days of “your favorite protein” may be coming to an end, replaced by a more complex and wholistic approach at the bench. Amin et. al. describe this new approach in an excellent article about systems biology and the rise of big data. They summarize the “omics” approach in the figure below, and argue that research must move away from any one aspect of biology and begin to integrate all approaches into the investigation of a scientific question.

            With this more complex approach comes a need for more advanced statistics in graduate and even undergraduate education. Big data has become an integral part of our scientific lives and increasingly an important aspect of our personal lives as well. In the midst of this contested presidential election, popular interest in big data is incredibly high, and interested onlookers would do well to familiarize themselves with the art and science of statistics.

Big Data, Big Challenges

Did you know that every day, we create 2.5 quintillion bytes of data? Data from everywhere one could possibly imagine - social media, retail industry forecasts, healthcare software, lottery analysis, performance efficiency, business intelligence, surveillance, product generation, and so much more. Termed big data, an explosion of available information, this trend of incessant data production and storage is likely to continue advancing. Big data lies at the center of a 4 circle venn diagram - high volume, high velocity, poor veracity, and large variety. However, with these inherent characteristics come colossal challenges for conventional statistical approaches. 

First, in this era of scaling up, statistical analyses often fall short. In other words, many popular algorithms that may provide sound statistical significance for a smaller n pool are unable to scale up to the big data power and run terribly slow on terabyte-scale data sets. These outdated statistical tools are simply no match for the massive data they must compute. For example, a basic operation of computing the median of a data set proves to be far more complex when big data is scaled up exponentially. Ultimately, the underlying philosophy of statistics in this context must change from getting the best solution to getting a good answer quickly. To effectively play a role in this high-speed, technologically-charged world, statistical quality control must master a trade-off between speed and accuracy. 

Second, with big data comes large data volume and no random sampling, as contexts are often purpose-driven (i.e. Macy's assessing customer preference on the Women's Shoes section of their website). Traditional statistics models (Frequentist/Bayesian) inherently assume a random sample drawn from a population. However in the big data context, there is an urgent need for statistical methods that are either capable of incorporating non-random samples without presenting inaccurate results, or have built-in abilities to adjust for sample selection. 

Third, big data sets are heterogeneous, meaning they are created via combining several data sources from diverse subpopulations (i.e. a database of SNP sequences).  A huge challenge lies in the fact that most current statistical methods are designed to develop inferences for single populations. In fact, many statistical methods that are successful with low-dimensional data are unable to retain their validity with the high-dimensionality aspect of big data, leading to spurious correlations, noise accumulation, and incidental endogeneity. Statisticians need to take strides towards developing more flexible algorithms such that single populations within a mixture population data set can be identified and individual inferences can be drawn. 

In revolutionizing the world we live in, big data has posed challenges to statisticians where they are forced to grapple with the reality that "concise insights but faster data" may be more relevant/valuable than "deeper insights but slow data." For further reading on this topic, I highly suggest this phenomenal paper which not only discusses the pitfalls of specific statistical algorithms, but also devises ways to reform them. 

Fan, Jianqing, et al. "Challenges of Big Data analysis." National Science Review vol 1: 293-314. (2014)


Friday, April 1, 2016

Big Data and Big Bias

Contemplating big data?

Here's a good thread worth thinking through. h/t @ereinerts

My advice in terms of how to begin thinking about this: Complex data coupled with sophisticated analytics isn't some magic woo-woo soap that mysteriously washes away our biases.

Click here for a relevant kerfuffle (warning, website slow to load).