The paper, “Homeopathic Treatment of Migraine in Children: Results of a Prospective, Multicenter, Observational Study” published in 2012, examines the use of different homeopathic treatments to treat migraines in children under the age of 15. The researchers evaluated the effectiveness of the treatments by comparing the number and severity of migraine attacks before and after homeopathic treatment. In addition, the researchers evaluated the effect of these treatments on the children’s education by determining the number of missed school days before and after treatment.The overall findings of the study found that the use of these different treatments decreased the number of migraine episodes and had a beneficial effect on children’s education, by decreasing the number and severity of migraine attacks and by decreasing the number of sick days. However, this paper is built upon a badly designed experimental plan and used incorrect statistical tests in order to “verify” its statistically significant events.
The main problem with this paper is the overall design of the study. The subjects included were children under the age of 15 who had been diagnosed with migraines by the doctors involved in the study. The subjects included in the study were chosen on a “first-come, first-serve” policy, so the researchers did not impose additional criteria upon which subjects should be included in the study. This may not have been a major problem, but it probably decreased the possible significance that could be found from this study by narrowing the possible criteria for acceptance into the study. The study was also not randomized or blinded, which allows for the introduction of bias.
Another major problem was that the doctors involved in the study participated voluntarily. This suggests that the doctors who willingly became involved in the study had the preformed belief that homeopathic treatments could be beneficial in treating migraines. This suggests the doctors could have introduced bias into the study, and this bias could have played in a role in the results that they reported back to the researchers. In addition, each doctor in the study prescribed different homeopathic treatments to their patients, so the study was noncomparative. Based on this, the researchers could not say if one treatment worked better than the other. In addition, the dosage and the timeframe of treatment varied between each individual. This means that the individual patients within the study really can’t be compared to each other due to the great variation in treatment between all the subjects. The researchers also only included one follow-up time point three months post-treatment to test the effect of the treatment on the patient. As a result, the researchers could not test the effect of the treatments long-term.
A large problem with this paper is that the majority of data collected was subjective, determined by the individual doctors or from the guardians of the patients. As a result, different doctors may have interpreted the suggested scale differently. In addition, guardians may exaggerate the severity of migraine attacks. In addition, the majority of the criteria that the doctors and the guardians used were not included in the paper. The few examples were very vague, referencing the presence of an “aura” with the patient; the characteristics of this aura never are explained in the paper.
One of my main problems with the paper was the fact that no control or placebo group was included into the study. The researchers even admit that previous studies have shown that the placebo effect can play a big role in studies like this. This means that any significant results could be due completely to the placebo effect.
When examining the effect of the treatment on the subject’s education, the researchers chose to use the number of the days of school each patient missed. However, this is not the best way to determine whether the treatment had a beneficial effect on the patient’s education. I believe that examining the change in grade performance before and after treatment, in addition to days absent, would give a better indication of whether the treatment helped to improve the patient’s performance in school.
When determining whether the results of their study were significant, the researchers chose to use the Chi-square test and the Student’s t-test. However, based on the design of the experiment, it is clear that the researchers should have used a paired t test when analyzing their data since they are comparing results before and after treatment from the same individuals. However, the researchers did not include their data in the paper, meaning that I could not reanalyze the data with the paired t-test to re-evaluate the study's significance. In the small amount of data and figures provided, the researchers failed to include any error bars into their graphs. Upon closer examination, I discovered this was because values such as a mean of 10 reported migraine attacks, had a standard deviation of 14.1. As a result, instead of having a graph with decisive results, you get a graph like you see below.
The results of the graph above show a great amount of variability in the study’s results, indicating that the overall findings of the study aren’t as significant as the researchers would like us to believe. All of the figures included in the study lacked error bars based on the standard deviation, due to the fact that the standard deviation was always greater than or equal to the calculated mean.
Overall the findings in the paper cannot taken as significant due to a terrible experimental design and a misuse of statistics to analyze the data.