Critique of the Evidence Base for Mental Health Treatments
Critique of the Evidence Base for Mental Health Treatments
Issue 34 – February 2013
Authors: Aisling Mannion (aisling.mannion@nottshc.nhs.uk)Key points
- Randomised Control Trials (RCTs) are considered the ‘gold standard’ for research, and clinical guidelines are based on the findings of these studies. However, RCTs have a number of disadvantages
- A vast literature exists highlighting the importance of non-specific factors, such as a good therapeutic relationship, over the choice of treatment modality
Implications for practice
- Commissioners, clinicians and patients should be made aware of the methodological flaws in RCTs, so that informed decisions can be made
- There are many factors to clinical work that go across different treatment approaches. Clinicians should focus on these in their work with patients
Problems with the evidence
Randomised controlled trials (RCTs) are considered the ‘gold standard’ of research methodology. These types of studies require groups of patients, defined on the basis of strict inclusion and exclusion criteria, to be randomised to one of two or more treatment or control conditions. The assumption of RCTs is that by controlling for all possible variables except for treatment condition, any differences found between groups can be attributed to the treatment itself. NICE guidelines typically only include RCT studies, as these are considered to be the most methodologically robust form of research. However, there are numerous disadvantages with this approach, as outlined below.Lack of generlisability(i.e. external validity)
RCTs use strict inclusion and exclusion criteria to define patient groups. Hence, patients with a range of difficulties (so-called ‘comorbid diagnoses’) are typically excluded from studies. Many have argued that these studies then use groups of patients which typically are not representative of the types of patients that present at mental health services. For instance, a recent Cochrane review for Generalised Anxiety Disorders1 excluded patients diagnosed with psychosis and those with issues related to substance abuse. Drop out rates are another issue that lead to very selective samples, with typically only data on those individuals who engaged in the majority of treatment sessions being used to make conclusions.Adherence to the medical model:
NICE guidelines recommend treatment approaches for specific diagnoses, often based on DSM-IV criteria. The focus, therefore, is on symptom reduction. Many individuals, including psychiatrists, have highlighted the lack of scientific credibility of the diagnostic system. Furthermore, quality of life and social factors are often more important to service users than symptom reduction. Despite this, the medical model and the DSM-IV have an incredible amount of power when it comes to the funding of research, which influences clinical guidelines and the provision of services.Lack of focus on non-specific and client factors:
In the field of psychotherapy, numerous studies have found that regardless of treatment type, there are common non-specific factors across therapies which are found to be responsible for outcome. These factors include the therapeutic relationship and therapist competence. However, as RCTs are focused primarily on standardised factors such as number of sessions attended and the use of self-report outcome data to measure symptoms, these non-specific factors are rarely given much attention. Hence, the qualitative aspects of therapy are ignored and instead the mechanical and operational aspects are favoured. Smail argues that RCTs become a vehicle for ‘churning out’ results, with a lack of focus on the actual content of the research2.Conflict of interest:
Many clinical trials are funded through private sponsors who have a vested interest in the results of the trial. A significant example of this in the field of mental health is the influence of the pharmaceutical industry. This raises ethical concerns regarding the objectivity of the data. Perlis et al (2005)3 conducted a review of RCTs and found that 60% of the 397 trials identified reported receiving funding from a drugs company or other interested party, and 47% included at least one author with a reported financial conflict of interest. The authors also found a significant association between the reporting of a conflict of interest and the reporting of positive results. Similar results were found in another review, in which four articles were found comparing industry-sponsored and non-industry-sponsored RCTs. In all of the articles there was a correlation of industry sponsorship and positive study outcome4. It is important for commissioners, clinicians and patients to be aware of such problems with clinical trials and to be able to use this information to make judgements about the reliability of the results. Smail used the idea of ‘interest-base’ as opposed to evidence base. By this, he was highlighting the tendency for researchers to pursue their own professional interests. There are several other disadvantages with RCTs that have been discussed in the literature, including statistical error, difficulty in studying rare events, and the large cost of RCTs. Blinding, in which both the patient and the experimenter are unaware of what treatment is being administered, is also not possible in psychological therapy. Furthermore blinds and double blinds are often broken in pharmaceutical research trials, which can artificially enhance the effect of psychiatric drugs (e.g. Kirsch, 20096).Are RCTs the only way to assess efficacy?
Some authors have found that when observational studies are compared with RCTs on the same clinical topics, the results are actually very similar5. A range of methodologies could be usefully employed to investigate clinical questions regarding of psychological and pharmaceutical treatments, including research on the subjective experience of people taking psychiatric drugs. It is the responsibility of commissioners and of clinicians to view research evidence critically, so as to be able to make informed decisions regarding what would work best for the patients they are providing a service to. It is also useful to hold in mind the vast literature on non-specific factors, which highlights that developing a good therapeutic relationship is more important than using a particular psychotherapy modality.REFERENCES
- Hunot, V., Churchill, R., Teixeira, V. & Silva de Lima, M. Psychological therapies for generalised anxiety disorder. Cochrane Database of Systematic Reviews 2007, Issue 1. Art No.: CD001848.
- Smail, D. (2006). Is clinical psychology selling its soul (again)? Clinical Psychology Forum, 168, 17-20
- Perlis, R.H., Perlis, C.S., Wu, Y., Hwang, C., Joseph, B.A. & Nierenberg, A.A. (2005). Industry sponsorship and financial conflict of interest in the reporting of clinical trials in psychiatry. American Journal of Psychiatry, 162, 1957-1960
- Bekelman, J.E., Li, Y. & Gross, C.P. (2003). Scope and impact of financial conflicts of interest in biomedical research: a systematic review. Journal of the American Medical Association, 289, 454-465
- Concato, J., Shah, N. & Horwitz, R.I. (2000). Ranodmised, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine, 342, 1887-1892
- Kirsch, I. (2009). Emperor’s new drugs. London: Bodley Head.
Trackback from your site.