Interesting paper. I have encountered patients in a hospital sending tending to gain weight when mirtazapine is added, perhaps due to a combination of a sedentary lifestyle and mirtazapine’s purported effects on shifting dietary carbohydrate preferences (see Hennings et al., 2019)
signal to noise ratio of sum score outcomes in psychopharmacology research is just too high to really take effect size magnitudes literally. do you really interpret them quantitatively, i.e. "augmentation leads to 0.3 SD less depression? I usually take it as a qualitative hint, i.e. it works in general and is a good idea to try or not. Maybe ordinal ranking is feasible but even that is hard former (thinking of Cipriani)
I try and at least understand what effect sizes mean quantitatively, because that helps me understand if the mean effect even looks vaguely like it has clinical significance. I've warmed up to a lot of vague-but-about-function scales like the CGI for this reason. I try to avoid using effect sizes as a qualitative hint from the start because I think psychiatry is filled with too many small/small-medium effect sizes that translate into borderline meaningless benefits (e.g. an additional improvement of 2 points on something like the HAM-D, woo)
Leucht has some good papers like "What does the PANSS mean?" or "What does the MADRS mean?" that I think have been helpful in refining my thinking and acting as a reference point.
Very good piece, but I still feel that doing clinical trials on “MDD” is about as useful as doing the same for “fever.” Why would we expect good results?
I tend to agree, though the question is what construct do we replace it with? My (admittedly more shallow than I'd like) impression is that attempts to find subgroups of patients diagnosed with MDD with distinct patterns of response has not been terribly fruitful.
I am a fan of the idea of dimensional models like HiTOP, though I think that will just lead us to a purely symptom-driven approach. Maybe that's what we want, but it doesn't seem like it will be terribly different than what we're currently doing!
I had not heard of Ban before, though reading what you linked and going down the rabbit hole of Wernicke–Kleist–Leonhard nosology, I am sorry that I am just learning about it now. Thank you very much for sharing!
The Astrup and Fish paper is certainly interesting, but I probably don't need to point out that it is not blinded, dose controlled, etc. etc. so there is an enormous potential for bias in this study. I don't think we can take it at face value as something that works, but I do think it could be valuable to try and do some modern studies using these subtypes and see what falls out!
We also have to content with studies like this one (https://www.nature.com/articles/s41380-019-0502-5) that suggest that there are not unique subgroups of psychotic individuals who respond differentially to treatment.
I agree the study needs follow-up, but I’m not sure an RCT would answer the same questions. This study isn’t trying to measure the statistical significance of a modest or marginal effect; it’s looking at whether there are differences in treatment response between relatively homogeneous clinical subgroups that are large enough to be obvious without statistics. It’s essentially an interocular trauma test. It’s not looking for a three-point change on a fifty-point scale. In this case they identified entire diagnostic groups that show no meaningful response to any of the drugs. That kind of finding is groundwork that arguably needs to happen before RCTs, not after. It’s separating the wheat from the chaff, looking for the large pieces that split away easily.
The WLK is interesting, as is much of the old German literature. That isn’t to say I think Leonhard was right nor Kleist. He was certainly eccentric and arguably got into counting angels on the head of a pin. Fish's study suggests he overshot the mark on his divisions by quite a margin. However, it also demonstrates that "over-splitting" could be experimentally useful, and that "the attempt" to carve diseases at their joints might be almost as good as the real thing if you're willing to be meticulous. Psychiatry often complains that its categories are too heterogeneous for research purposes; this work shows that deliberately creating more homogeneous groups, even when etiology is uncertain, could help triangulate the true underlying substrate. I think the reason the DSM doesn’t achieve this same result is because it is splitting by committee and horse trading, whereas Leonhard is splitting by pedantry. Horse trading is not the same as counting angels on the heads of pins.
The study you linked to looks like GIGO to me at a glance. I'm not sure if g = 0.47 supports the authors conclusions, it seems consistent with heterogeneous groups and weak effect. Also, I'm not sure if there is a serious scientific question about the existence of a non-responsive group. I mean, it's sort of logically impossible, you only need a handful of people to demostrate the existence of non-response. I suppose non-response raises diagnostic questions but how do you have no non-responder groups? Its sort of extraodinary. That is just off the top of my head though. Actually it makes me think of that Mitchell and Webb sketch "Can People Levitate?". It just seems... like I bet if I read each of the 30 studies individually it would quickly become obvious that the conclusion of the meta-analysis is logically impossible. I could be wrong though, and I'm definately being lazy, but I'm actually taking my first unit of statistics right now and I'm supposed to be doing an assignment, so I'll just assume I'm wrong about this since I only had time to read the abstract.
How would an RCT not answer the same questions? Just split patients into subtypes and see how they do with different drugs vs. placebo. If you don’t like the PANSS, the CGI is just a better defined “how does the clinician think the patient looks” test.
Your reading of the McCutcheon paper I think is because of a common misunderstanding of statistical populations. Imagine we have two groups of schizophrenics that we have mixed into a single group: treatment resistant and non-treatment resistant. They are indistinguishable from one another except for the fact that when you give them antipsychotics, the treatment-resistant group’s PANSS scores change on average by 2, while the non-treatment resistant group’s PANSS scores change on average by 20.
In at the start of a study, you wouldn’t know this. Is there anything that would tell us that our one group was actually two groups with very different treatment responses at the end of the study? Well, yeah, the variance. If we have two subgroups with very different treatment responses, we should expect measures of variance in our treatment group to be large, especially relative to placebo.
What the McCutcheon paper finds is that variance in treatment arms is actually smaller relative to placebo, which suggests that there is a relatively uniform effect of treatment on PANSS scores across patients with schizophrenia.
This does not mean that there are not people who experience only modest benefit from antipsychotics.
I particularily like your last point about modest benefits. I think it actually has a bearing on why most studies don't get dramatic variances. Antipsychotics seem to have perhaps even a bit more than a modest benefit in just about everybody if you're looking to calm someone down and make them more docile. This is an important point. I would even go so far as to say that it is a good justification to use antipsychotics quite routinely in acute settings. I certainly don't disagree with such use, I think the benefits are quite plain. This becomes more evident when you consider that antipsychotics can be quite useful in states other than schizophrenia. John Cade gives a superb description of there use in a hospital in his short but excellent book on the history of psychiatry. Its very down to earth in a way I think only a fellow Australian could write.
However, I think Astrup and Fish where looking for something other than the same effect antipsyhotics have on almost everyone. They were communicating with colleagues who perhaps would have found comments on a tranquilizing effect trite. It is hard to assess this, but it was a very different time and academic writing seems to me to have been more of an insider group thing back then, you were writing for your peers. Therefore, perhaps they felt no reason to repeat Deniker's observations to an audience already familiar with them.
I wouldn’t say that an RCT, or even several RCTs on these subtypes, wouldn’t be a good idea. There is a difference though between how we would like RCTs to be done and how they are often done in practice.
Keeping in mind that I'm currently doing undergrad stats for the first time, these are my thoughts:
Studies that mix all of these subgroups will often have similar variances in each group. On a PANSS scale, relatively large variances are routine, so I'm not so sure that anything unusual would be noticed.
We would also be assuming the distribution of subtypes is roughly equal. There’s no reason real diseases should behave this way, and in any given trial you could easily have one subtype dominate an arm. This doesn't seem unlikely given the possible disappearance of hebephrenia. We're also assuming the studies contain a similar mix of subtypes. Rather than assuming heterogeneity, it's possible that some homogeneity is occurring also.
I like RCTs and I do agree that the results of other studies have to be reckoned with. I suppose I like it when all of the evidence can be "accounted" for. I would be very reluctant to shrug this off as a fluke. Bias is a very unsatisfying explanation in this instance, and I think it is worthy of a great deal of further investigation. Something doesn't add up.
I suppose, just on a heuristic basis, if this difference was real, why wouldn't more experts be talking about it? By the same token though, if heterogeneity isn't an issue, why do experts who know far more about stats than I do think it is? If heterogeneity can be solved just by looking at group variance, why do experts talk about it like it's the number one issue?
I'm not sure if either of our viewpoints quite reflect the mainstream. I don't hear the mainstream discussing Leonhard's subtypes but I've also never heard any expert say "oh don't worry, the variance will pick up subtypes". You know, I suppose neither of us have run an RCT, I'm not sure, have you? I suppose there is a great deal I don't know.
Anyway, what would be far more interesting would be to investigate Fish's findings. I'm not sure if I could resist regardless of where other evidence pointed. I'm just too curious.
I think I might go and find some experts to hassle about this paper.
This may be your best writing yet Nils! Based on some of the polls Scott has posted previously, I would not be surprised if most MDs lack the statistical knowledge to correctly interpret this study as you have done. It also seems completely wild to me to assume Mirtazapine is just an "a2 antagonist" and pharmacological equivalent to trazadone and others. From my understanding, Mirtazapine has similar affinity for 5HT2a
Thanks kev; this is the first one I've written in a while that I'm proud of. It's very satisfying to teach yourself something and then understand it well enough to translate it into (relatively) less technical language.
I think that it was fair for the authors to put together mianserin and mirtazapine together - they're pretty similar drugs from a receptor-affinity point of view. The inclusion of trazodone is most baffling to me given that it has direct effects at SERT.
The complete normalisation of the mass labelling and drugging of children and adults has got to be Psychiatries most destructive era yet. Given its history thats really saying something.
Idk about you but I’d much rather people get handed SSRIs — which they have the choice to decline — than get absolutely tanked from the get go on clomipramine. The era of routine lobotomization and dehumanization was clearly worse, get some perspective.
Get some perspective, ok. By the way, we are still hacking into brains and blasting them with ECT today. While obviously brutal and insane, even at its peak lobotomy and related brutal madness, only managed to brain damage tens of thousands. However, in the modern era we are mislabelling with a range of DSM twaddle and drugging tens of millions of children and adults the world over on the back of these erroneous labels.
The damage to human health is incalculable. So many drugs are now prescribed that the unmetabolized junk being urinated out into rivers and seas is changing the behaviour of marine life.
Lobotomies for intractable seizures and ECT for profound treatment-resistant depression are lifelines for people who would otherwise be crippled interminably by organic disease. For the vast majority of the “tens of millions” you’re talking about, the sum total of these “incalculable” negative effects are basically feeling a little off on SSRIs, dealing with the (admittedly often very unpleasant) withdrawal syndromes of something like an Effexor, or not being able to get your dick hard. It takes a whole lot of ignoring what your eyes can very clearly see to think that the effects of these medications approach anything resembling “drugging”.
I am trying to work out if the ignorance in your comment is just ignorance or wilful ignorance? The negative effects are many, varied and catastrophic, hence my wondering about your ignorance. To minimise the significance of sexual dysfunction is ignorance in action. I suggest you go explore than many ‘survivor’ groups and support services for people harmed and often killed by the impact of these drugs. Its amazing to me that terms like ‘antidepressants’ and ‘SSRI’s are both misleading. The first is purely a marketing term based on corrupt research and the latter another untruth. These drugs have wide ranging effects across the body and brain and no one really knows exactly what they are doing because no one has bothered to do detailed long term studies.
Interesting paper. I have encountered patients in a hospital sending tending to gain weight when mirtazapine is added, perhaps due to a combination of a sedentary lifestyle and mirtazapine’s purported effects on shifting dietary carbohydrate preferences (see Hennings et al., 2019)
signal to noise ratio of sum score outcomes in psychopharmacology research is just too high to really take effect size magnitudes literally. do you really interpret them quantitatively, i.e. "augmentation leads to 0.3 SD less depression? I usually take it as a qualitative hint, i.e. it works in general and is a good idea to try or not. Maybe ordinal ranking is feasible but even that is hard former (thinking of Cipriani)
I try and at least understand what effect sizes mean quantitatively, because that helps me understand if the mean effect even looks vaguely like it has clinical significance. I've warmed up to a lot of vague-but-about-function scales like the CGI for this reason. I try to avoid using effect sizes as a qualitative hint from the start because I think psychiatry is filled with too many small/small-medium effect sizes that translate into borderline meaningless benefits (e.g. an additional improvement of 2 points on something like the HAM-D, woo)
Leucht has some good papers like "What does the PANSS mean?" or "What does the MADRS mean?" that I think have been helpful in refining my thinking and acting as a reference point.
Very good piece, but I still feel that doing clinical trials on “MDD” is about as useful as doing the same for “fever.” Why would we expect good results?
I tend to agree, though the question is what construct do we replace it with? My (admittedly more shallow than I'd like) impression is that attempts to find subgroups of patients diagnosed with MDD with distinct patterns of response has not been terribly fruitful.
I am a fan of the idea of dimensional models like HiTOP, though I think that will just lead us to a purely symptom-driven approach. Maybe that's what we want, but it doesn't seem like it will be terribly different than what we're currently doing!
Have you Tom Bans's idea about “homotyping”? It worked for Astrup and Fish.
a-neuropszichofarmakologiai-kutatas-klinikai-modszereirl.pdf https://share.google/oMUYVbCMX6dQI0ili
The response of the different Leonhard subgroups of schizophrenia of psychotropic drugs https://share.google/2NgCU8iNfM8P0hPoH
I had not heard of Ban before, though reading what you linked and going down the rabbit hole of Wernicke–Kleist–Leonhard nosology, I am sorry that I am just learning about it now. Thank you very much for sharing!
The Astrup and Fish paper is certainly interesting, but I probably don't need to point out that it is not blinded, dose controlled, etc. etc. so there is an enormous potential for bias in this study. I don't think we can take it at face value as something that works, but I do think it could be valuable to try and do some modern studies using these subtypes and see what falls out!
We also have to content with studies like this one (https://www.nature.com/articles/s41380-019-0502-5) that suggest that there are not unique subgroups of psychotic individuals who respond differentially to treatment.
I agree the study needs follow-up, but I’m not sure an RCT would answer the same questions. This study isn’t trying to measure the statistical significance of a modest or marginal effect; it’s looking at whether there are differences in treatment response between relatively homogeneous clinical subgroups that are large enough to be obvious without statistics. It’s essentially an interocular trauma test. It’s not looking for a three-point change on a fifty-point scale. In this case they identified entire diagnostic groups that show no meaningful response to any of the drugs. That kind of finding is groundwork that arguably needs to happen before RCTs, not after. It’s separating the wheat from the chaff, looking for the large pieces that split away easily.
The WLK is interesting, as is much of the old German literature. That isn’t to say I think Leonhard was right nor Kleist. He was certainly eccentric and arguably got into counting angels on the head of a pin. Fish's study suggests he overshot the mark on his divisions by quite a margin. However, it also demonstrates that "over-splitting" could be experimentally useful, and that "the attempt" to carve diseases at their joints might be almost as good as the real thing if you're willing to be meticulous. Psychiatry often complains that its categories are too heterogeneous for research purposes; this work shows that deliberately creating more homogeneous groups, even when etiology is uncertain, could help triangulate the true underlying substrate. I think the reason the DSM doesn’t achieve this same result is because it is splitting by committee and horse trading, whereas Leonhard is splitting by pedantry. Horse trading is not the same as counting angels on the heads of pins.
The study you linked to looks like GIGO to me at a glance. I'm not sure if g = 0.47 supports the authors conclusions, it seems consistent with heterogeneous groups and weak effect. Also, I'm not sure if there is a serious scientific question about the existence of a non-responsive group. I mean, it's sort of logically impossible, you only need a handful of people to demostrate the existence of non-response. I suppose non-response raises diagnostic questions but how do you have no non-responder groups? Its sort of extraodinary. That is just off the top of my head though. Actually it makes me think of that Mitchell and Webb sketch "Can People Levitate?". It just seems... like I bet if I read each of the 30 studies individually it would quickly become obvious that the conclusion of the meta-analysis is logically impossible. I could be wrong though, and I'm definately being lazy, but I'm actually taking my first unit of statistics right now and I'm supposed to be doing an assignment, so I'll just assume I'm wrong about this since I only had time to read the abstract.
How would an RCT not answer the same questions? Just split patients into subtypes and see how they do with different drugs vs. placebo. If you don’t like the PANSS, the CGI is just a better defined “how does the clinician think the patient looks” test.
Your reading of the McCutcheon paper I think is because of a common misunderstanding of statistical populations. Imagine we have two groups of schizophrenics that we have mixed into a single group: treatment resistant and non-treatment resistant. They are indistinguishable from one another except for the fact that when you give them antipsychotics, the treatment-resistant group’s PANSS scores change on average by 2, while the non-treatment resistant group’s PANSS scores change on average by 20.
In at the start of a study, you wouldn’t know this. Is there anything that would tell us that our one group was actually two groups with very different treatment responses at the end of the study? Well, yeah, the variance. If we have two subgroups with very different treatment responses, we should expect measures of variance in our treatment group to be large, especially relative to placebo.
What the McCutcheon paper finds is that variance in treatment arms is actually smaller relative to placebo, which suggests that there is a relatively uniform effect of treatment on PANSS scores across patients with schizophrenia.
This does not mean that there are not people who experience only modest benefit from antipsychotics.
Good points.
I particularily like your last point about modest benefits. I think it actually has a bearing on why most studies don't get dramatic variances. Antipsychotics seem to have perhaps even a bit more than a modest benefit in just about everybody if you're looking to calm someone down and make them more docile. This is an important point. I would even go so far as to say that it is a good justification to use antipsychotics quite routinely in acute settings. I certainly don't disagree with such use, I think the benefits are quite plain. This becomes more evident when you consider that antipsychotics can be quite useful in states other than schizophrenia. John Cade gives a superb description of there use in a hospital in his short but excellent book on the history of psychiatry. Its very down to earth in a way I think only a fellow Australian could write.
However, I think Astrup and Fish where looking for something other than the same effect antipsyhotics have on almost everyone. They were communicating with colleagues who perhaps would have found comments on a tranquilizing effect trite. It is hard to assess this, but it was a very different time and academic writing seems to me to have been more of an insider group thing back then, you were writing for your peers. Therefore, perhaps they felt no reason to repeat Deniker's observations to an audience already familiar with them.
I wouldn’t say that an RCT, or even several RCTs on these subtypes, wouldn’t be a good idea. There is a difference though between how we would like RCTs to be done and how they are often done in practice.
Keeping in mind that I'm currently doing undergrad stats for the first time, these are my thoughts:
Studies that mix all of these subgroups will often have similar variances in each group. On a PANSS scale, relatively large variances are routine, so I'm not so sure that anything unusual would be noticed.
We would also be assuming the distribution of subtypes is roughly equal. There’s no reason real diseases should behave this way, and in any given trial you could easily have one subtype dominate an arm. This doesn't seem unlikely given the possible disappearance of hebephrenia. We're also assuming the studies contain a similar mix of subtypes. Rather than assuming heterogeneity, it's possible that some homogeneity is occurring also.
I like RCTs and I do agree that the results of other studies have to be reckoned with. I suppose I like it when all of the evidence can be "accounted" for. I would be very reluctant to shrug this off as a fluke. Bias is a very unsatisfying explanation in this instance, and I think it is worthy of a great deal of further investigation. Something doesn't add up.
I suppose, just on a heuristic basis, if this difference was real, why wouldn't more experts be talking about it? By the same token though, if heterogeneity isn't an issue, why do experts who know far more about stats than I do think it is? If heterogeneity can be solved just by looking at group variance, why do experts talk about it like it's the number one issue?
I'm not sure if either of our viewpoints quite reflect the mainstream. I don't hear the mainstream discussing Leonhard's subtypes but I've also never heard any expert say "oh don't worry, the variance will pick up subtypes". You know, I suppose neither of us have run an RCT, I'm not sure, have you? I suppose there is a great deal I don't know.
Anyway, what would be far more interesting would be to investigate Fish's findings. I'm not sure if I could resist regardless of where other evidence pointed. I'm just too curious.
I think I might go and find some experts to hassle about this paper.
I share your pessimism but I'm not sure there's any other option but to keep trying to sub-phenotype with new methods.
This may be your best writing yet Nils! Based on some of the polls Scott has posted previously, I would not be surprised if most MDs lack the statistical knowledge to correctly interpret this study as you have done. It also seems completely wild to me to assume Mirtazapine is just an "a2 antagonist" and pharmacological equivalent to trazadone and others. From my understanding, Mirtazapine has similar affinity for 5HT2a
Thanks kev; this is the first one I've written in a while that I'm proud of. It's very satisfying to teach yourself something and then understand it well enough to translate it into (relatively) less technical language.
I think that it was fair for the authors to put together mianserin and mirtazapine together - they're pretty similar drugs from a receptor-affinity point of view. The inclusion of trazodone is most baffling to me given that it has direct effects at SERT.
The complete normalisation of the mass labelling and drugging of children and adults has got to be Psychiatries most destructive era yet. Given its history thats really saying something.
Idk about you but I’d much rather people get handed SSRIs — which they have the choice to decline — than get absolutely tanked from the get go on clomipramine. The era of routine lobotomization and dehumanization was clearly worse, get some perspective.
Get some perspective, ok. By the way, we are still hacking into brains and blasting them with ECT today. While obviously brutal and insane, even at its peak lobotomy and related brutal madness, only managed to brain damage tens of thousands. However, in the modern era we are mislabelling with a range of DSM twaddle and drugging tens of millions of children and adults the world over on the back of these erroneous labels.
The damage to human health is incalculable. So many drugs are now prescribed that the unmetabolized junk being urinated out into rivers and seas is changing the behaviour of marine life.
Lobotomies for intractable seizures and ECT for profound treatment-resistant depression are lifelines for people who would otherwise be crippled interminably by organic disease. For the vast majority of the “tens of millions” you’re talking about, the sum total of these “incalculable” negative effects are basically feeling a little off on SSRIs, dealing with the (admittedly often very unpleasant) withdrawal syndromes of something like an Effexor, or not being able to get your dick hard. It takes a whole lot of ignoring what your eyes can very clearly see to think that the effects of these medications approach anything resembling “drugging”.
I am trying to work out if the ignorance in your comment is just ignorance or wilful ignorance? The negative effects are many, varied and catastrophic, hence my wondering about your ignorance. To minimise the significance of sexual dysfunction is ignorance in action. I suggest you go explore than many ‘survivor’ groups and support services for people harmed and often killed by the impact of these drugs. Its amazing to me that terms like ‘antidepressants’ and ‘SSRI’s are both misleading. The first is purely a marketing term based on corrupt research and the latter another untruth. These drugs have wide ranging effects across the body and brain and no one really knows exactly what they are doing because no one has bothered to do detailed long term studies.
I really enjoyed the statistics overview in this essay! Fascinating. Thank you