I got this email yesterday:
NYSPA E-ALERT: CONTACT YOUR LEGISLATOR TODAY REGARDING AUTISM BILL
All members are strongly encouraged to contact their local Assembly member or Senator today to oppose the passage of S.3044-A (Carlucci)/A.1663-A (Abinanti), a bill that would amend the Insurance Law and the Mental Hygiene Law to codify the DSM-IV definition of autism as the official definition of autism for the purposes of New York State law. The proponents of the legislation seek to freeze the definition of autism because they are fearful that the new definitions in DSM-5 may diminish or eliminate eligibility for special education services in schools and/or health insurance coverage for community services. This is simply not true and would be an improper intrusion of the Legislature into the realm of medical science. Medical professionals must have the ability to update and revise clinical diagnoses according to new scientific evidence and advances in medicine.
I'm gonna ignore, for now, the topic of what role the government should play in medicine, and focus on that last sentence: Medical professionals must have the ability to update and revise clinical diagnoses according to new scientific evidence and advances in medicine.
Nu, so show me the science.
According to the DSM-5 description of the changes in criteria for Autism Spectrum Disorder:
The DSM-5 criteria were tested in real-life clinical settings as part of DSM-5 field trials, and analysis from that testing indicated that there will be no significant changes in the prevalence of the disorder. More recently, the largest and most up-to-date study, published by Huerta, et al, in the October 2012 issue of American Journal of Psychiatry, provided the most comprehensive assessment of the DSM-5 criteria for ASD based on symptom extraction from previously collected data. The study found that DSM-5 criteria identified 91 percent of children with clinical DSM-IV PDD diagnoses, suggesting that most children with DSM-IV PDD diagnoses will retain their diagnosis of ASD using the new criteria. Several other studies, using various methodologies, have been inconsistent in their findings.
There didn't seem to be any reference or link to the "real-life clinical settings", and it seems like, if a "real-life" study was done, it would have been published, or pending publication, at least cited. But I did check out the Huerta paper, published in October, 2012. Here's the method:
Three data sets included 4,453 children with DSM-IV clinical PDD diagnoses and 690 with non-PDD diagnoses (e.g., language disorder). Items from a parent report measure of ASD symptoms (Autism Diagnostic Interview-Revised) and clinical observation instrument (Autism Diagnostic Observation Schedule) were matched to DSM-5 criteria and used to evaluate the sensitivity and specificity of the proposed DSM-5 criteria and current DSM-IV criteria when compared with clinical diagnoses.
I don't really understand what was done, and I'm not paying for the full article to figure it out.
These were the results:
Based on just parent data, the proposed DSM-5 criteria identified 91% of children with clinical DSM-IV PDD diagnoses. Sensitivity remained high in specific subgroups, including girls and children under 4. The specificity of DSM-5 ASD was 0.53 overall, while the specificity of DSM-IV ranged from 0.24, for clinically diagnosed PDD not otherwise specified (PDD-NOS), to 0.53, for autistic disorder. When data were required from both parent and clinical observation, the specificity of the DSM-5 criteria increased to 0.63.
Okay, sensitivity 91% or less ("remained high"), specificity 0.53 to 0.63. I'm a little confused. You have 100 patients who were diagnosed with DSM-4 PDD, and 91 of them were diagnosed with DSM-5 ASD. Normally, you would use a sensitivity of 91% to establish that the new standard you're proposing is adequate, since it's almost as good as the old standard. So the diagnostic standard you're comparing to is DSM-4, and you're saying that since DSM-5 is almost as good as DSM-4, it's better than DSM-4.
What would make sense is if there were a third standard, the Autism Standard, which was the basis for diagnosing Autism. If DSM-4 had, say, 80% sensitivity, and DSM-5 had 91% sensitivity, compared with the Autism Standard, then you could conclude that DSM-5 was more sensitive than DSM-4.
Alternatively, if they're saying that DSM-5 only picked up 91% of cases because 9% of DSM-4 cases are inaccurately diagnosed, then you need to question the basis for DSM-4 criteria, so you can't use it as a standard to compare DSM-5 to.
To belabor the point: Suppose you're testing lexapro vs. prozac, and comparing both to nortriptyline. If lexapro helped 80% of patients who were helped by nortriptyline, and prozac helped 95% of patients helped by nortriptyline, you could reasonably conclude that prozac works better than lexapro. But if you compare lexapro directly with prozac, and you find that lexapro helps 91 of the 100 patients that were helped by prozac, you can't conclude that lexapro works better than prozac.
And if you then claim that lexapro does work better than prozac because prozac didn't really help all the 100 people it claims to have helped, then you have no idea what prozac does and doesn't do, so what does it mean to say lexapro works better?
See what I'm getting at?
You could argue that DSM-5 has better specificity than DSM-4, but the whole concern is that people who have a DSM-4 diagnosis of PDD won't all have a DSM-5 diagnosis of ASD, so some will lose needed services/treatment. So the concern is about missing a real case, not misdiagnosing someone who doesn't have PDD. In other words, specificity isn't the issue.
Moving right along.
I looked up some of those "other studies" that have been "inconsistent in their findings".
This study, by McPartland, et al, published in April, 2012, found these results:
When applying proposed DSM-5 diagnostic criteria for ASD, 60.6% (95% confidence interval: 57%-64%) of cases with a clinical diagnosis of an ASD met revised DSM-5 diagnostic criteria for ASD. Overall specificity was high, with 94.9% (95% confidence interval: 92%-97%) of individuals accurately excluded from the spectrum. Sensitivity varied by diagnostic subgroup (autistic disorder = 0.76; Asperger's disorder = 0.25; pervasive developmental disorder-not otherwise specified = 0.28) and cognitive ability (IQ < 70 = 0.70; IQ ≥ 70 = 0.46).
The study concludes that:
Proposed DSM-5 criteria could substantially alter the composition of the autism spectrum. Revised criteria improve specificity but exclude a substantial portion of cognitively able individuals and those with ASDs other than autistic disorder. A more stringent diagnostic rubric holds significant public health ramifications regarding service eligibility and compatibility of historical and future research.
Another study found lower sensitivity and greater specificity, with sensitivity improving, although still less than DSM-4, if one DSM-5 criterion was relaxed.
Yet another study found lower sensitivity.
Let's see. We have "real-life" clinical settings without a linked study. We have one cited study demonstrating that some of the people with DSM-4 PDD diagnoses would lose those diagnoses under DSM-5, we have a statement that, "This is simply not true," and we have a sneaky little sentence dismissing, and not citing, those "other studies".
No, I don't think I will be contacting my local assembly member or senator.