Whose Planet Is It Anyway?

Friday, October 27, 2006

Number Needed to Treat

The title of this post refers to a way of measuring the effectiveness of pharmaceutical drugs and other medical treatments. The concept of Number Needed to Treat (NNT) was developed by epidemiologists to help consumers make more informed decisions about medical care by advising them, not of relative risks, but of the number of people who must get a treatment for a single person to benefit.

Here's an illustration of how the NNT concept works: Let's suppose somebody invents a new drug to prevent teenagers from becoming obsessed with gangsta rap. The manufacturer advertises that teenagers who are treated promptly with the drug when their parents first catch them listening to gangsta rap are 50 percent less likely to develop GROD (Gangsta Rap Obsessive Disorder). Sounds pretty good, doesn't it? Parents who value their hearing, their sanity, and household peace should all rush their teenagers to the doctor, right?

Well, no. Not exactly.

In this hypothetical scenario, what the manufacturer hasn't mentioned in the advertisement is that, out of 100 teenagers who listen to gangsta rap, only 4 of them become obsessed with it. Taking the new wonder-drug reduces that number to 2, making a gangsta rap obsession "50 percent less likely." As you can see, that statistic would seriously mislead parents because it was never very likely that any particular teen would become obsessed with gangsta rap. Because 100 teenagers had to be given the drug to prevent 2 instances of GROD, we would calculate the NNT for this particular drug as 50, meaning that the drug had to be administered to 50 people in order to benefit one.

You can find a more detailed explanation of the NNT concept in a recent article on the Slate website.

What does all of this have to do with autism? Well, we often see claims by biomed proponents that a particular treatment, supplement, or what-have-you is 50 percent effective at "rescuing" children from a lifetime of being trapped in their autism and unable to communicate. Leaving aside the awkward fact that not a single one of these claims has ever been supported by legitimate peer-reviewed studies, let's just wander back into the Land of Unlikely Hypotheticals for the moment and imagine that someone actually did perform a valid study confirming such a claim. What would it mean for parents who had just been told that their young child was on the autistic spectrum?

As I mentioned in a recent post, research has shown that approximately 90 percent of children on the autistic spectrum develop speech by age nine. (C. Lord et al, "Trajectory of language development in autistic spectrum disorders," in Developmental Language Disorders: From Phenotypes to Etiologies, 2004.)

Several older studies were discussed (thanks, Joseph and David) in the comments to that post, indicating that the percentage of Kanner autistics who developed speech by adulthood was between 30 percent and 50 percent. Not all autistics who lack functional speech as children are in the Kanner category, but many are. And we know that in past years, such children often were institutionalized at a young age, with no speech therapy and no reasonable opportunity to learn speech. According to the National Autistic Society, the ratio of Kanner autistics to the entire autistic spectrum is about 5:91 (this is based on studies conducted in the UK). So I think it's reasonable to estimate that at least 3 percent of today's autistic spectrum children will learn to speak after the age of nine.

There are also autistics who rely on alternative methods of communication. Facilitated speech has gotten a significant amount of media attention, but there are many methods. Unfortunately, some non-speaking autistics who could benefit from augmentative communication devices have been unable to get the devices as a result of being "written off" by ignorant people who assume that they are incapable of learning. As more advanced technology becomes widely available and society's attitudes toward autistic people improve, I expect that more autistics will be able to benefit from these devices. At present, however, I do not believe there are any accurate statistics on how many non-speaking autistics can learn to communicate by alternative means. Considering how often we see the topic mentioned on the Internet and elsewhere, though, we're probably talking about another 3 percent of the total autistic population (that's a conservative estimate).

Now let's add these percentages together. If 90 percent of all autistics learn to speak before age nine, and 3 percent begin speaking at a later age, and 3 percent do not speak but can communicate by other methods, what this means is that 96 percent of all autistic adults can communicate. Just as in my NNT illustration of teenagers and gangsta rap, only 4 percent of the children who would be receiving the hypothetical scientifically-proven biomed treatment would fall into the non-communicative category from which it was intended to "rescue" them, and only 2 percent actually would be "rescued." Once again, the NNT for this treatment would be 50, if such a treatment existed.

To put it another way, even if there really were a biomed treatment that had valid research studies supporting a claim that it could effectively "rescue" half of the autistic children who would otherwise spend their lives being unable to communicate, for every child who was "rescued" with this treatment, another 49 children would be dosed unnecessarily with drugs that could endanger their health. Those wouldn't be good odds even if there were legitimate scientific backing for the biomed peddlers' claims. Given the fact that no such proof exists—it's just lunacy to put children at risk like that.

Labels:

16 Comments:

  • ABFH: "As I mentioned in a recent post, research has shown that approximately 90 percent of children on the autistic spectrum develop speech by age nine. (C. Lord et al, 'Trajectory of language development in autistic spectrum disorders,' in Developmental Language Disorders: From Phenotypes to Etiologies, 2004.)"

    Um... yeh.

    Which would explain quite well why a certain ignoramus' son is now speaking.

    Good post, ABFH... good post.

    David N. Andrews MEd
    Applied Educational Psychologist
    Kotka, Finland

    By Anonymous Anonymous, at 11:37 AM  

  • Joseph: "In that case, the natural 'cure' rate for Kanner autistics is over 50%"

    And this actually occured after they stopped doing psychotherapy on the kids...


    David N. Andrews MEd
    Applied Educational Psychologist
    Kotka, Finland

    By Anonymous Anonymous, at 12:19 PM  

  • Just like the 80% effective rate for the SSRIs with OCD - out of that 80% some only reported as little as a 15% improvement. Not much of a tradeoff considering some of the side effects.

    LB

    By Anonymous Anonymous, at 12:20 PM  

  • We have 2 Kanner autistics in our home. The oldest is 15. She began speaking between 8 and 9 years of age. She can now hold a conversations. Not a good one, and very high pitched, but she will answer questions. Our youngest is 5 and still quite non-verbal. But, as of this week, is typing on the computer keyboard. He will definitely have no problems in his communication skills, no matter what they are.

    My question with the biomedical drugs is why there is so much regression? We never have seen regression in our home. I am wondering if the biomedical influences cause these regressions!
    Just my 2 cents....

    Yeast die off??? Stomach aches???
    Leaky Gut??? Aggression??? rashes??? Whatever.....

    By Blogger Mom26children, at 11:20 PM  

  • Great post ABFH. I like the NNT idea, and this is perhaps one of the clearest autism-specific descriptions of biomed "regressive" or "regression to the mean" fallacy.

    The "medical illness" analogy in the embedded link could be adapted to being autistic (not an illness) and rewritten as follows:

    "One of the most common occasions for the Regression Fallacy is developmental delay. Parents are most likely to seek treatment for a developmental delay—especially experimental treatment—when their child is very young, that is, when their differences are the most noticeable. They pursue treatments, and the child develops and learns due to regression to the mean, but they attribute this development and change the effect of the treatments. This is one reason why some people will swear by such worthless treatments as homeopathy and chelation for autism."

    By Blogger Do'C, at 11:29 PM  

  • I've never met an autistic person who absolutely does not communicate. People find at least some way of doing so, even in the absence of speech or writing. I have, however, met a large number of people described as non-communicative. I think that reflects the problem of the people using that word.

    By Blogger ballastexistenz, at 4:10 AM  

  • Good post.
    I agree with Do'C that the time when parents worry the most about what their child will become, is when they are very young and the differences with typical children seem the greatest. That is also often the time just after diagnosis when parents are at their least knowledgeable and most vulnerable to talk of cures.

    My son is another one who I can safely say is developing speech due to natural maturation and education with no medical 'treatment' at all.

    By Anonymous Anonymous, at 7:21 AM  

  • ABFH,

    "As I mentioned in a recent post, research has shown that approximately 90 percent of children on the autistic spectrum develop speech by age nine. (C. Lord et al, "Trajectory of language development in autistic spectrum disorders," in Developmental Language Disorders: From Phenotypes to Etiologies, 2004.)"

    Does it mention anything on the quality/appropriateness of speech or behavioural issues?

    By Anonymous Anonymous, at 8:12 PM  

  • This is a good post, a logical one and well thought out.

    I have often thought that kids with "developmental differences" or autism follow a different path to maturity, that they have more in common with each other than the average child. I wish I had known that when Ben was first diagnosed.

    As we learn to give them what they need to mature most successfully...the "disability factor" will become less and less. Like how we are figuring out Dyslexia can also be a gift if we quit punishing the differences and build on the abilities.

    By Blogger Usethebrains Godgiveyou, at 7:34 AM  

  • rb

    I don't understand your logic
    "Like how we are figuring out Dyslexia can also be a gift...."

    How can a disability be a gift?

    Who is "punishing the differences"?

    By Anonymous Anonymous, at 7:43 AM  

  • David, Joseph and LB: It's just a blind shot, indeed.

    Mom26children: Yay for Kiernan! It won't be long before he is hogging the computer every time you look around. I'd think the most difficult part of raising so many autistic children would be policing all the squabbles about computer time...

    Do'C: Thanks, you've given me an idea for another post. :)

    Ballastexistenz: That's a very important point. Even if a person communicates only by simple gestures, caretakers and others should make an effort to understand what he or she is trying to say.

    Sharon and RB: Thanks. I agree that parents are often misled in the early years because there is so much ignorance in society, and I think you're both doing a great job of giving people helpful and realistic information with your blogs.

    Anonymous: The researchers measured speech in various ways, including parents' reports, spontaneous speech, and standardized vocabulary tests. Of course, with any method of assessing the "quality" or "appropriateness" of speech, it's not possible to avoid cultural biases and subjective expectations about what is good or appropriate. In my view, that issue is similar to the controversy about ethnic minority children getting lower scores on vocabulary tests because they grew up speaking a ghetto dialect. Speaking in dialect doesn't mean that a person is less capable, and autistic speech patterns aren't necessarily inferior just because they are different; what's problematical is figuring out how mainstream society can best integrate people who communicate differently.

    By Blogger abfh, at 9:54 AM  

  • Thanks ABFH,
    Kiernan has never had a problem getting what he wants..even if it is getting a sibling off of the computer when he wants it.
    Caitlin and Deirdre have their own computers. The other 4 share the family computer.
    The computers were gifts from friends who were upgrading. They were blessings to our children who love the computer.
    Also, they are given no more than 1 hour a night on the computer and ONLY after homework is done. Yes, they do their own homework. And all honor roll students, by the way.
    I have been asked to write a book about our family. I am considering doing this..the only problem is finding the time. Wish me luck...

    By Blogger Mom26children, at 12:23 PM  

  • Trouble with older studies and the 50% meme it is like the 75% meme in its turn, a matter of apples and pears because no two studies of autism between say the 50's and the mid 80's are even talking about the same thing, no standardisation, no concept even of it but researchers thinking that they are beyond compare and immune to criticism and that a word means what they think it does even if someone else uses the same word to mean something else.

    If you say that autism is X and only X then only Y percent of the population will be X, but if you say autism is P and not X, then Y percent of the population will not be X but they will be P, however Z percent of the population will be neither and yogh of the population will be somewhere in between the two supposing X and P bear at least some slight relation to each other, and thats not maths cos yogh ain't greek its saxon

    By Blogger Larry Arnold PhD FRSA, at 7:08 PM  

  • ABFH: "Of course, with any method of assessing the 'quality' or 'appropriateness' of speech, it's not possible to avoid cultural biases and subjective expectations about what is good or appropriate."

    It may be possible, depending on how a qualitative researcher is prepared to work on the definitions.

    A good definition of how communication can be considered 'appropriate' could include the intentionality of an utterance, and a criterion-referenced test of how close to the mark the communication was in terms of fulfilling the purpose that such an utterance would have.

    For example... does an utterance wilfully exert an influence on:
    a) the listener's behavioural state?
    b) the listener's cognitive state?
    c) the listener's emotional state?
    d) nothing?

    By comparing the result level on this scale with the usually expected level at which the utterance might be expected to function would give a reasonably objective measure, without too much room for subjectivity error/unreliability.

    Just a thought... actually, based on a recently acquired MEd ...

    By Blogger David N. Andrews M. Ed., C. P. S. E., at 8:37 PM  

  • Mom26children: I am sure you have enough determination and self-discipline to write a book. A few words of advice... be careful about your choice of a publisher. Make sure you're working with someone who respects autistic people, so that your family won't end up being described in derogatory terms on the book jacket. That has happened to some autistic writers.

    Larry: I agree, the numbers in the older studies don't mean much at all.

    David: I see that you're registered on Blogger now... I'm looking forward to reading your blog when you get it set up. :)

    I like your idea of measuring communication in terms of how well the speaker's intent is conveyed. Maybe you should create your own speech assessment test; I'm sure it would be better than a lot of what's out there!

    I was going to write more in this comment about how autistic speech patterns are both neurological and cultural, but I think I'll turn that into a post instead.

    By Blogger abfh, at 9:11 AM  

  • ABFH: "David: I see that you're registered on Blogger now... I'm looking forward to reading your blog when you get it set up. :)"

    Yeh... had to do it because that pisswit in Spew Spampshire....

    ABFH: "I like your idea of measuring communication in terms of how well the speaker's intent is conveyed."

    Thanks... I'm not sure as to how much it's my idea per se, but certainly it's my understanding of what was suggested as elements of what communication is about.

    ABFH: "Maybe you should create your own speech assessment test; I'm sure it would be better than a lot of what's out there!"

    Fuck it... I think I'd like to, you know. Seriously. problem is, I'd need an attachment to a university education or psychology dept to do it. Finnish psych departments are notoriously 'racist' in their rejection of outsiders (as most of Finland is). But I would like to get into ipsative assessment particularly... criterion referenced and qualitative... standardised tests tell about performance only on relation to a group, but fail to tell much that is actually that useful for development of any meaningful way of working to support any aspect of development. Even what I'm able to get from qualitative observation during standardised testing is a bare minimum for making educated guesses at what is going off for any given examinee.

    By Blogger David N. Andrews M. Ed., C. P. S. E., at 12:32 PM  

Post a Comment

<< Home