The Issues in Undertaking a Study of Autism Incidence/Prevalence in Vaccinated vs. Unvaccinated Populations

Generation Rescue's JB Handley, January 5, 2011 in a CNN interview with Elliot Spitzer and Kathleen Parker

More importantly the science that has been done is what we like to call "tobacco science". You take a group of kids who all got vaccines but got a little less mercury and compare them to a group of kids who all got vaccines but a little more mercury and find there's no difffenced in autism and then claim that vaccines don't cause autism. The only appropriate study to do would be to look at a group of childrren who never got vaccines and a group of children who got all of them, and see if there's a difference in autism rates and that study has never been done despite many people trying to call for it.

In 2007, Mr. Handley's organization commissioned and published a telephone survey, asking respondents their children's vaccine and neurocognitive status.  The survey was rubbish and was rightly ridiculed, as the links below show. 

• June 26, 2007 Kev at LeftBrain/RightBrain Generation Rescue Survey Results http://leftbrainrightbrain.co.uk/2007/06/generation-rescue-survey-results/

  This is a disaster for Generation Rescue and the whole ‘vaccines cause autism’ debacle. Generation Rescue’s data indicates that you are ‘safer’ from autism if you fully vaccinate than partially vaccinate. It also indicates that across the spectrum of autism, you are only 1% more likely to be autistic if you have had any sort of vaccination as oppose to no vaccinations at all – and that's only if you are male. If you are a girl your chances of being on the spectrum are less if you have been vaccinated! Across both boys and girls, your chances of being on the spectrum are less if you have received all vaccinations.

• June 27, 2007  Survey Says...Nothing Prometheus at Photon in the Darkness http://photoninthedarkness.blogspot.com/2007/06/survey-says-nothing.html

  Generation Rescue just published their long-awaited telephone survey results that showed - surprise! - that vaccinations cause autism. Well, actually, it doesn't really show that vaccines cause autism, what it shows is that vaccines cause "neurological disorders", loosely defined as autism (or autistic spectrum disorder) and/or ADD/ADHD. Except that it really doesn't show that, either. So what does the survey show? Well, let's just take a look.

• June 27, 2007 Orac at Respectful Insolence Fun With Phone Surveys and Vaccines http://scienceblogs.com/insolence/2007/06/fun_with_phone_surveys.php

  Not surprisingly, given the source, the "study" turns out to be totally underwhelming, nothing more than a phone poll really. (Amusingly, David Kirby has said that he doesn't consider phone surveys to be "data.") Even so, expect to see it trumpeted all over antivaccination websites and blogs as "proof" that vaccines cause autism or, at the very least, as "evidence" that compels a study. It might be, if it weren't so poorly designed and analyzed and if it actually showed what GR claims that it shows.

• September 3, 2007 Joseph at Natural Variation A Simple Selection Bias Model Explains Generation Rescue's Survey Results http://autismnaturalvariation.blogspot.com/2007/09/simple-selection-bias-model-explains.html

  What I want to do in this post is go over a model that explains the survey results, including discrepancies between its findings and those of prior phone surveys. Data of this nature can sometimes result in knowledge that wasn't expected, as I will hopefully demonstrate.

Generation Rescue and other anti-vaccine activists then switched to asking for a study comparing autism rates in vaccinated vs. unvaccinated populations.  Handley's January 6, 2010 is just the latest in a long string of demands.  There are particular barriers to doing cuch a study, which has been widely discussed by science bloggers.  

What follows is a list of blog posts that have addressed the issues: This issue has been discussed at length.

• July 19, 2007 Let's Do a Study! Prometheus at Photon in the Darkness http://photoninthedarkness.com/?p=100

  What GR [Generation Rescue] and other advocacy groups have been agitating for is a study comparing groups that don’t vaccinate their children (e.g. the Amish, although they actually do vaccinate their children) with vaccinated children. The folks at GR are pushing for this despite the fact that their baloney “survey” showed no connection between autism and vaccination.

• October 20, 2007 We Want a Study! Prometheus at Photon in the Darkness http://photoninthedarkness.com/?p=121

  Let me run you all through the process of designing a study to show that thimerosal does or does not cause autism. Perhaps you will see why “nobody” has “done the study” that these parents are yearning for. And why probably nobody ever will.

• November 3 2008 Let's Put on a Study! Prometheus at Photon in the Darkness http://photoninthedarkness.com/?p=154 

  The interest in “doing a study” seems to stem from the idea that the only way to find if vaccines are connected with autism is to study unvaccinated children and compare their autism prevalence with vaccinated children. This is not true and is - in fact - not even the most effective way to study the postulated connection, as I will show later.

• April 14, 2009 Feasibility study of Vaccinated/Unvaccinated/Alternatively Vaccinated Children National Vaccine Advisory Council (NVAC) Vaccine Safety Working Group Draft Report www.hhs.gov/nvpo/nvac/.../NVACVaccineSafetyWGReport041409.pdf   (full text is at the bottom of this post)
• April 27, 2009  Human subjects protections and research ethics: Where the rubber hits the road for science-based medicine David Gorski MD at Science-Based Medicine http://www.sciencebasedmedicine.org/?p=467  An in-depth, but accessibly to lay readers, discussion of ethical limitations in medical research.
• June 2, 2009 Recommendations on the Centers for Disease Control and Prevention Immunization Safety Office Draft 5-Year Scientific Agenda National Vaccine Advisory Committee (NVAC) www.hhs.gov/nvpo/nvac/.../nvacrecommendationsisoscientificagenda.pdf 

  The NVAC endorses the Writing Group’s recommendation for an external expert committee, such as the Institute of Medicine, with broad methodological, design, and ethical expertise to consider “strengths and weaknesses, ethical issues and feasibility including timelines and cost of various study designs to examine outcomes in unvaccinated, vaccine delayed and vaccinated children and report back to the NVAC.

• June 06 2009 A Vaccinated vs. Unvaccinated Study Kev Leitch at LeftBrain/RightBrain http://leftbrainrightbrain.co.uk/2009/06/a-vaccinated-vs-unvaccinated-study/ Discussion of the 2007 Generation Rescue phone survey.
• August 24, 2009 The perils and pitfalls of doing a “vaccinated versus unvaccinated” study  David Gorski MD, at Science-Based Medicine  http://www.sciencebasedmedicine.org/?p=939 

  On the other hand, antivaccinationists should be very careful what they ask for. They may just make enough of a pain of themselves to get it. True, getting the resources necessary to do a study the like of what Prometheus described initially would monopolize autism research funding for years, but the anti-vaccine movement doesn’t really care about that, because it’s always been all about the vaccines more than helping autistic children.

• June 08 2010 Generation Rescue's proposed vaccinated vs. unvaccinated study Sullivan at Leftbrain/Rightbrain http://leftbrainrightbrain.co.uk/2010/06/generation-rescues-vaccinatedunvaccinated-study/ In-depth analysis of a study proposal published by Generation Rescue, including analysis of study design and qualifications of the researchers.
• December 10, 2010 Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims? Kimball Attwood at Science-Based Medicine http://www.sciencebasedmedicine.org/?p=8874  A discussion of ethical limitations in medical research.

 

In short, there are significant barriers to performing the sort of study Generation Rescue is demanding. 

In November 2010 The American Academy of Pediatrics (AAP) published a annotated bibliography of studies to date on vaccine safety,Vaccine studies: Examine the evidence, noting

The concerns regarding vaccine safety have received a great deal of attention by parents, doctors, vaccine manufacturers and the media. Dozens of studies have been performed in the United States and elsewhere. The purpose of this document is to list those studies and provide links to the publications to allow parents and all those who administer or recommend vaccines to read the evidence for themselves. The studies provided have been published in peer-reviewed medical journals. These studies do not show any link between MMR vaccine, thimerosal and autism. This is not an exhaustive list and it will need to be updated frequently since vaccine safety studies are ongoing.

  Click the link to download a paper copy of this paper. Download Vaccinestudies-1

April 14, 2009 National Vaccine Advisory Council (NVAC) Vaccine Safety Working Group Draft Report Feasibility study of Vaccinated/Unvaccinated/Alternatively Vaccinated Children

www.hhs.gov/nvpo/nvac/.../NVACVaccineSafetyWGReport041409.pdf

Feasibility study of Vaccinated/Unvaccinated/Alternatively Vaccinated Children

Members of the public, stakeholders, and the Interagency Autism Coordinating Committee (IACC) have articulated interest in a study of vaccinated vs. unvaccinated children to determine if there are differences in health outcomes between groups with varying exposures to vaccines. The Working Group considered drafting a recommendation for an IOM review of the science, epidemiology and feasibility of studies of unvaccinated, vaccine delayed, and vaccinated children. The Writing Group Draft Document on Gaps in Research Agenda further developed this idea. The Working Group wishes to clarify several points on this topic. First, the Working Group believes that the strongest study design, a randomized clinical trial that includes a study arm receiving no vaccine or vaccine not given in accord with the current recommended schedule, is not ethical, would not pass IRB review, and cannot be done. The type of study that is being suggested would be an observational study of populations looking at natural variation in vaccination schedules including some children where vaccination is declined through parental intent. All children in the study should be recommended to receive the standard immunization schedule. The Working Group endorses the Writing Group’s recommendation for an external expert committee, such as the Institute of Medicine, with broad methodological, design, and ethical expertise to consider “strengths and weaknesses, ethical issues and feasibility including timelines and cost of various study designs to examine outcomes in unvaccinated, vaccine delayed and vaccinated children and report back to the NVAC.”

The Working Group does not necessarily agree with all of the language in the Writing Group’s statement, but with its general intent. The process should be open and transparent, engaging individuals from a broad range of sectors. Considerations as outlined by the Writing Group and modified by the Working Group are as follows:

• This review should consider strengths and weaknesses, ethical issues and feasibility including timelines and cost of various study designs and report back to the NVAC
• Consideration should be given to broad biomedical research including laboratory studies, and animal studies.
• Consideration should also be given to study designs comparing children vaccinated by the standard immunization schedule with unvaccinated children (by parental intention), and possibly partially vaccinated children or children vaccinated by alternative immunization schedules
• Outcomes to assess include biomarkers of immunity and metabolic dysfunction, and outcomes including but not limited to neurodevelopmental outcomes, allergies, asthma, immune-mediated diseases, and other developmental disabilities such as epilepsy, intellectual disability and learning disabilities.
• The inclusion of autism as an outcome is desired. This review should also consider what impact the inclusion of Autism Spectrum Disorders (ASD) as an outcome would have on study designs and feasibility, as referenced in the IACC letter to NVAC.
• This review should be conducted expeditiously, in a transparent manner, and involving broad public and stakeholder input.

Comments

[Squillo]

Thanks for collating these, Liz.

It's a case-study in how reducing an issue to a series of talking-points does a disservice to anyone who wants to understand it in any useful context.