Statistical Agreement - Duh!

           I recently experienced a situation in which I was faced with what I believed to be a statistical impossibility of sorts (Granted I had yet to embark on TJ's biostats course at the time). This conundrum was exposed when attempting to establish equity of quantitation between two entirely dichotomous experimental methods aimed to provide similar readout metrics. Explicitly, the change in measurement from an isotope release assay to that of a single cell based analysis.  While ample statistical tests could be run internal to each individual experiment, cross comparisons between methods attempting to glean “significance” were determined to be illegitimate in theory due to the contrasting experimental designs.

            Briefly, cell mediated lympholysis (CML) has traditionally been enumerated by loading peripheral blood mononuclear (PBMC) target cell populations with the radioactive isotope ⁵¹Cr that diffuses into live cells and is released upon the induction of apoptosis and subsequent lysis as mediated by alloreactive cytotoxic lymphocyte (CTL) effector cells. The goal of this study was to develop an isotope-free approach to the quantification of CML in the context of transplantation immune monitoring, by moving from ⁵¹Cr release as the metric of target cell lysis to determining the total loss of healthy target cells as measured by flow cytometry. To further complicate matters, the combination of isotope decay and natural day-to-day variation of both the donor and recipient PBMC, along with minor histocompatibility antigen discrepancies prevents the normalization of individual experiments. While these differences may seem negligible to a third party - years of alloreactive CML assays run with chromium said otherwise leading to an adoption of the method to simply provide a binary "responsive" or "unresponsive" result ignoring magnitude all together. This aptly illustrates the reason for my ambition to establish a more modern approach.    

             In order to provide the journal reviewers with a visual representation of correlation I resorted to contacting my bosses go to biostatistician to try to make sense of how this could be done (Don't worry TJ I was still at MGH at the time). After discussing the matter with the biostatistician I was absolutely dumbfounded that I had not thought of graphing the correlation as a simple agreement between the two methods. Anyways, to accurately graph the data correlation we plotted the result from each method for a single experiment with a "perfect fit" line containing a slope of 1.0 to show gross deviation and relative magnitudes of agreement.