By Lisa Stoddard, M.A., BCBA
From the time our children are born, we parents are faced with a seemingly endless stream of decisions. Disposable or cloth? Cow’s milk or soy? Montessori or Waldorf? But for parents of children with special needs, decisions about treatments and therapies can be daunting. When facing a sea of conflicting theories, treatment models and news reports touting the latest study, how does a parent determine the most effective plan? And, more importantly, how can parents avoid wasting precious time and money on ineffective treatments?
The Evidence-Based Movement
Scientific evidence can often offer some guidance. From medicine to education, special-needs services and beyond, experts are increasingly recommending only those interventions that are “evidence-based.” Lawmakers are even writing this language into policy, including the No Child Left Behind Act of 2001 and California’s SB 946, which requires health insurers to fund evidence-based behavioral therapies for the treatment of autism.
While there is no universally accepted definition of “evidence-based practices,” the term generally refers to practices that are shown to be effective based on the results of high-quality research. But in a world where virtually anyone can publish a journal or present a study, parents must put on their “scientific research analyst” hat to determine whether the evidence supporting a treatment or therapy is sound. A good place to begin is by asking:
- How many studies seem to support this treatment or therapy?
- What is the quality of the individual studies?
One single published study is not enough to suggest that a given treatment is evidence-based. Rather, parents will want to look to see if multiple studies, conducted by different researchers, have found the same effects. Treatments supported by the findings of at least three independent studiescould be considered reasonably well supported by scientific evidence. However, this is definitely a case where “more is more.” Each additional study supporting a treatment lends greater confidence in its possibilities.
Looking At Individual Studies
But evaluating therapies isn’t as easy as just adding up supporting studies. The quality of the research matters as well, because even a treatment with 10 supporting studies would not be considered “evidence-based” if those studies were poorly designed. The next time you hear about the latest and greatest study, use the following questions to help you determine whether that study is a reliable basis for decision-making:
Is there a control group? To show that a treatment is effective, researchers need to be able to compare what happens to one group of patients who received the treatment with another group who did not (or maybe with a group who received a different form of treatment). The untreated group is called a “control” group. When a study includes a control group, researchers can be sure that improvements in patients who received the treatment were actually caused by the treatment – and not by some other factor such as getting older, other treatments being received outside the study, or changes in diet or environment.
Were the patient and control groups randomly assigned, and are they comparable? When putting together the group of patients who receive the treatment and the control group, researchers should work to equalize or eliminate any factor (other than the treatment itself) that might possibly be responsible for the treatment effect. For example, if the treatment group is made up of people who sought out the intervention, and the control group is recruited from the community, those receiving the treatment might have more invested in the treatment working. Assigning people randomly to either the patient or control group, called “random assignment,” can eliminate this possibility. There is still a chance, however, that differences such as the age of the participants, severity of symptoms or socioeconomic background can affect treatment outcomes, so you will want to be sure the groups were as similar as possible.
Lisa Stoddard is the mother of a rambunctious toddler, a Board Certified Behavior Analyst, and Clinical Director of FirstSteps for Kids, Inc. PHOTO BY DAWN BOWERY (www.dawnboweryphotography.com)
What was the sample size? The number of people who received each treatment in a study is referred to as the “sample size.” Studies that describe positive treatment effects on one or two participants would generally not be as strong as those with larger samples – another case of more is more.
Are the measures objective? You can place greater confidence in research findings that are based on information that can be easily, reliably measured as fact. This can include pre-treatment and post-treatment tests or counts of observable behaviors (such as duration of sustained toy-play or total number of play initiations observed in a defined period of time). Be cautious of studies that base their findings on more indirect measures, such as rating scales or questionnaires in which parents give their opinion of how effective the treatment was.
Was the study peer-reviewed? In the internet age, anyone can publish anything. But scientific journals rely on a peer-review process where an author’s peers (other established researchers in the field) read and evaluate the merits of a study’s design and findings before it is published. You can be more confident in the findings of studies that have been published in scientific journals using the peer-review process.
One important point to remember, though, is that not all interventions found to evidence-based will necessarily be effective for every child. Individual differences and varied responses to treatment will likely always play a part. But when it comes to making treatment decisions, starting with those that, through high-quality research, have been found to be effective for many is an excellent place to start when making choices for your child’s future.
Lisa Stoddard is the mother of a rambunctious toddler, a Board Certified Behavior Analyst, and Clinical Director of FirstSteps for Kids, Inc. an early intervention program providing behavior intervention services to children with special needs in Los Angeles and Ventura counties.
