Did the evidence really not support the introduction of low fat dietary guidance before 1983?

diet

Kamal Mahtani

Kamal Mahtani: Deputy Director of CEBM

David Nunan, Senior Research Fellow

David Nunan, Senior Research Fellow

As part of our work in the Centre for Evidence-Based Medicine we take an active interest in the quality of systematic reviews of topics of public and patient importance, particularly when they generate bold headlines.

Perhaps the most common, and recent, eye catching headlines relate to diet and nutrition.

For example, there has been recent widespread media reports with headlines such as “Eating fat is good for you: Doctors change their minds after 40 years” and “Saturated fat ‘ISN’T bad for your heart’”. If correct, these headlines challenge current dietary recommendations from a number of organisations such as the American Heart Association, the British Heart Foundation, and the World Health Organization, all of which advise limiting dietary saturated fats.

So should there be a major shift in the public health advice we receive and give out on this topic? For this to happen the evidence underpinning these headlines should be at least equivalent with current evidence in terms of its quality, reliability and trustworthiness.

Unfortunately, the quality of the evidence does not appear to justify such bold headlines.

As one example, a recent systematic review in the BMJ Open journal concluded that, based on a summary of their evidence, current recommendations to lower blood cholesterol for cardiovascular protection was wrong. However, the study itself highlighted several competing interests amongst the authors which may limit the interpretation. Furthermore, our post-publication analysis of the study highlighted several weaknesses in the methods, limiting the conclusions drawn from the study. The review authors, despite a press release to the public, did not convey the weaknesses we highlighted.

In a similar vein, a 2015 systematic review in BMJ Open Heart emphatically concluded (in its declaratory title) that “Evidence from randomised controlled trials did not support the introduction of dietary fat guidelines in 1977 and 1983”. The review set out to specifically analyse the association between saturated fat intake and mortality due to all causes and coronary heart disease (CHD) from RCTs that had been carried out before key national recommendations were made in the USA (1977 and the UK (1983).

Interestingly, a Cochrane systematic review, also published in 2015, looked at reduced saturated fat intake for cardiovascular disease from RCTs. The review also included studies published before the introduction of the 1977 USA and the 1983 UK guidelines.

You might think that both reviews would come to similar conclusions. However, not all systematic reviews are equal. A recent analysis concluded that non-Cochrane reviews report larger effect sizes with lower precision than Cochrane reviews, a systemic difference that indicates that such reviews (non-Cochrane) “may be overstating the evidence,” the author’s comment.

A quick look at some of the differences between the 2015 BMJ Open Heart systematic review (hereafter referred to as ‘non-Cochrane review”) and the Cochrane systematic review reveals some interesting findings.

For example, authors of Cochrane reviews are required to publish their protocols before starting their review. They must outline their methods for searching, including, and appraising the studies that will be included (and excluded). Cochrane reviewers also have to outline the outcomes they will look at, and categorise them as to their clinical relevance/importance. The 2015 Cochrane review identified eight outcomes that they considered to be clinically important.

Stating and then publishing systematic review protocols in this way is important, as it reduces the risk of reporting only selected outcomes when the review is published (known as outcome reporting bias). It also reduces the risk that the reviewers will simply run lots of analyses for lots of different outcomes and find a statistically significant difference for one of them by luck (known as “p-hacking”). For this reason researchers of non-Cochrane reviews are also encouraged to prospectively register their systematic review protocol on the PROSPERO database.

The 2015 non-Cochrane review did not appear to have a pre-defined, published protocol. And because the final published review only included two outcomes – deaths due to any cause or due to coronary heart disease (CHD) – the reader has little idea if these were the outcomes the reviewers set out to examine from the start.

The 2015 non-Cochrane reviewers stated in their review, that they only combined studies published at or before the USA (1977) and UK (1983) recommendations on saturated fat intake were made. They showed there was no statistical difference between people randomised to a low-fat diet and those randomised to a “usual” diet for death from any cause (relative risk [RR] 0.996, 95% confidence interval [CI] 0.87 to 1.15) and death due to CHD (RR 0.99, 95% CI 0.78 to 1.25). This finding was used to underpin their (definitive) title.

However, as the Cochrane review also included studies published before 1983, it’s possible to compare this data directly with those of the non-Cochrane review.

For all-cause and CHD deaths, the Cochrane review showed similar results but pooled risk estimates were between 1-3% larger (i.e. higher risk of death in the usual diet group).

On a closer look, we have observed discrepancies in the data extraction between the two reviews that might explain these differences in risk estimates (see figure 1). For example, lets compare the data extracted on all cause mortality from the “Rose Olive Oil (1965)” study.

That’s a 2-fold difference, between the reviews, in the data extracted from the same study. Such discrepancies can start to put doubts into a reader’s mind, especially given the evidence for less precision in non-Cochrane reviews.

nunan1

Figure 1. Relative risk estimates for all-cause mortality from included studies in the two reviews (Harcombe = lead author of the non-Cochrane review; Hooper = lead author of the Cochrane review). NI = study not included in the review.

 

Although it was not obviously pre-specified, the non-Cochrane review only looked at deaths as their main outcome.

However, the number of cardiovascular events (such as myocardial infarction, stroke, heart failure) a patient is at risk of having is also a critically important outcome to measure.

Therefore, we reanalysed data from studies published before 1983 that were included in the Cochrane review. This showed a statistically significant 21% reduced risk in cardiovascular events that was in favour of participants randomised to a low saturated fat diet (RR 0.79, 0.69 to 0.91; figure 2). This roughly translates to a number needed to treat of 14, meaning that for every 14 people who follow a low saturated fat diet, 1 additional cardiovascular event would be prevented.

Although we would still need to take into account the quality of the evidence, the data available at the time of the 1977 US and 1983 UK guidelines appear to support a low saturated fat diet for the reduction of major cardiovascular events.

So why the discrepancy? This may be difficult to fully elicit.

The non-Cochrane review initially stated that there were no competing interests amongst the authors of the review. However, the journal later published an accompanying correction to this, which highlighted that the corresponding author receives income from writing diet books. The correction was also accompanied by an editor’s comment, which stated that the competing interests should have been made from the outset.

Figure 2. Meta-analysis for cardiovascular events based on studies available pre-1983 from the Cochrane review.

Figure 2. Meta-analysis for cardiovascular events based on studies available pre-1983 from the Cochrane review.

 

While they are often used to stimulate scientific debate, contrasting findings and conclusions can be confusing to wider audiences.

Our preliminary results appear to suggest that there may be elements in at least one non-Cochrane review that contradicted the definitive statements its authors have made. This warrants an independent and more detailed analysis.

We’ll be sure to let you know what we find, whatever the results.

Dr David Nunan  – Departmental Lecturer in Evidence Based Medicine
david.nunan@phc.ox.ac.uk

Dr Kamal R. Mahtani – GP and Deputy Director
kamal.mahtani@phc.ox.ac.uk

Centre for Evidence Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford.


Disclaimer

Dr David Nunan and Dr Kamal R. Mahtani are both members of the Royal College of General Practitioners (RCGP) steering committee to support the new Physical Activity and Lifestyle clinical priority.

Dr Nunan and Dr Mahtani have received funding for research from the NHS National Institute for Health Research School for Primary Care Research (NIHR SPCR) and the RCGP for independent research projects related to physical activity and dietary interventions. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, the Department of Health, the RCGP or any other institution named in this post.

They declare no other relevant conflicts of interest.

About David Nunan

View more posts by David Nunan

8 comments on “Did the evidence really not support the introduction of low fat dietary guidance before 1983?

  1. There is a BIG question about saturated fat you have to answer though. All the slicing and dicing is fine, giving small effects, but the question is this.
    If saturated fat is bad, why does France sit at the top for cardiac health, and in particular, why is the area of max sat fat consumption in France (Gascony), have half the French chd rate? On top of that, the Kitavans ate 18% saturated fat and had no heart disease at all.
    You have to conclude that the effects of saturated fat must depend on what is being consumed with it. Probably sugar.

    • “If saturated fat is bad, why does France sit at the top for cardiac health, and in particular, why is the area of max sat fat consumption in France (Gascony), have half the French chd rate? On top of that, the Kitavans ate 18% saturated fat and had no heart disease at all.”

      The answer is simple: you confuse ecological correlations with controlled trials. For instance, the “French paradox” is old news. Welcome to the 2010s.

    • Perhaps it is not so much a case of what the saturated fat is consumed with, but rather what type of saturated fat it is and where it comes from. Over 10 types of saturated fatty acid are commonly consumed in the diets of developed countries. Experimental trials indicate these have different effects on CHD risk, some increase the risk, some decrease the risk. To make matters more complicated we don’t generally eat fatty acids of any type individually. we eat foods that contain different amounts of different fatty acids, at different concentrations (high fat/low fat foods). The French eat a lot of cheese which whilst high in sat fat also contains nutrients such as calcium, which appear to have a cardio-protective effect (see the DASH diet). So perhaps the French paradox isn’t dead and buried.

      • but but but, in what study is satfat associated with an increase in MORTALITY? Er, in none. Major cardiac events are not death. At least, the last time I checked they weren’t. Stop trying to explain the French Paradox. Paradox it ain’t. When this Diet Satfat hypothesis was first floated in several countries, including Ireland and Italy, they couldn’t find ANY relationship between satfat and heart disease. Our Prof of Social Medicine at the time, that’s 1983 went through all of this with us. His message was, it’s not satfat. We of course listened to demented Americans.

        • George Henderson

          I’ve heard the calcium explanation before. Of course this would also apply to meat stews cooked with the bone in.
          However, hypothetically, if there is an effect of saturated fat on CVD but it’s so small that it’s easily reversed by a slight difference in dietary calcium, does this really justify a health warning on saturated fat?
          The old idea was that SFA increased risk because it increases ApoB concentrations.
          The problems with this are –
          Carbohydrate, which is present in the usual diet in much larger amounts than SFA, also increases ApoB to the same extent.
          SFA increases Apo A1, which is beneficial – carbohydrate does not, neither does PUFA.
          PUFA and MUFA lower ApoB – all fats that supply SFA also supply significant amounts of MUFA and/or PUFA.
          Ergo, replacing carbohydrate with fat is beneficial, even if that fat supplies much SFA.
          There is additional benefit in that fat will have less effect on insulin, and therefore lipogenesis and blood pressure, than will carbohydrate.
          All this is theoretical, but it uses modern data and the same theory that the diet-heart hypothesis was based on.
          It’s consistent with modern epidemiology showing lower mortality, especially CHD mortality in males, with higher fat intakes (e.g. Malmo Diet and Cancer Study, Health Professionals Follow-Up Study, EPIC Netherlands cohort) with no strong or consistent association with SFA.

  2. When you discuss Rose Corn oil, you don’t tell us which of the 2 meta-analyses got the data extraction for death right.

    You point about p-hacking may not be valid, for 2 reasons

    1) both these teams know the source material well, and have run prior meta-analyses. It is reasonable to assume they could predict the trend of outcomes with some certainty before registering a protocol if they wanted to.

    2) the end-points that the non-Cochrane team used, CHD mortality and especially total mortality, are the only ones that should be relevant to a question of public health policy. The Government does not have a mandate to exchange one cause of death for another. The Cochrane team, by using a meaningless combination of minor events, heart attacks, and deaths as a primary outcome, served up their conclusions in such a way as to conceal the important conclusion, that the intervention had no effect on heart attacks or deaths.
    For the non-Cochrane team to use death as their primary outcome – the outcome diagnosed with most accuracy – is not p-hacking – this should be the primary outcome for all public health interventions, and when it is not we should ask why.

    In your re-analysis you arrive at a significant reduction in events, but you do not say what the death rate is. You would think that preventing heart attacks, strokes, and heart failure in one in 14 cases would decrease the age-adjusted death rate.
    Did your analysis disagree with the non-Cochrane conclusion that this was RR 0.996,in other words that preventing these major events resulted in no such decrease?
    Of course you will prevent major events if they are overall more fatal, because the same person is likely to experience more events the longer they survive. This may be one possible way of explaining this discrepancy; that there were benefits for some, offset by harms to others, resulting in equal mortality overall.

  3. There was a program on recently comparing world wide diets and which are the healthiest http://www.channel4.com/programmes/the-worlds-best-diet/videos/all/the-worlds-best-diet-clip-1/3652004866001. They went to various countries where the populations are considered very healthy and have low morbidity and mortality: South Korea, Sicily, Iceland etc. The only thing that correlated between these different diets was the lack of processed foods. In every other respect e.g. how much veg, how much fat, how much wholegrain, how much meat, fish etc they differed markedly. The conclusion of the program makers was simply that eating natural foods grown/raised locally was the key to staying healthy. Processing kills food. Processing by definition is anti-life because by making sure the microbes can’t consume it you automatically ensure the microbes in our guts cannot consume this dead food. So how can we gain any nutrients if our gut biome is unable to break it down? Sugar of course is a preservative and therefore a killer.

Leave a Reply to George Henderson Cancel reply

Your email address will not be published. Required fields are marked *