Meta-analyses and why I don't fear saturated fat

"The great tragedy of Science (is) the slaying of a beautiful hypothesis by an ugly fact"

 - Thomas Henry Huxley

I eat butter.  Plenty of it.  And I also started rendering lard and beef tallow for cooking.  Not many people my age (26) have ever seen someone render lard, let alone use it readily.  The reason I use these fats is because they are delicious, and because I can buy them at the farmers market.  The conventional wisdom is that I should be scared - or nearly terrified - of saturated fat.  Just take a look at this YouTube video from the U.K (see below).*  The eerie lighting and tone of the television announcer is enough to make me worry, but let's take a look at some recent evidence regarding saturated fat and heart disease to see if the sink analogy is apropos.

The reason that saturated fat has been demonized by the nutrition community is because it is the cornerstone of the Lipid Hypothesis.  The Lipid Hypothesis, which is more of a concept than a working hypothesis, proposes that dietary saturated fat elevates cholesterol in the blood, specifically LDL cholesterol, which in turn causes atherosclerosis.  Cardiovascular disease (CVD) is characterized by atherosclerosis (plaque in the arteries) and includes coronary heart disease (CHD; atherosclerosis of blood vessels in the heart) and cerebrovascular disease (atherosclerosis of blood vessels in the brain leading to stroke).

Source:  Wikipedia.  Myristic acid, a saturated fat

The Lipid Hypothesis has always had its critics, but it has generally been accepted as fact.  However, the totality of evidence is a bit fuzzy, as many studies are contradictory or only show a small benefit from restricting saturated fat.  But since CVD is the leading cause of death in the U.S., public health authorities argue that any intervention, even if it's small or somewhat uncertain, will be beneficial to the health of the population.  These circumstances are ideal for what medical researchers call a "meta-analysis."

Simply put, a meta-analysis is a single study that combines the results from numerous smaller studies to form an artifical mega-study that will have enough statistical power (ability to detect a difference between the control and experimental group) to determine what the true impact of an intervention is.  The goal is to walk away with an actual number, such as a relative risk or mortality statistic.  For a meta-analysis to be valid, the included studies must be sufficiently similar and test the same exposure-outcome hypothesis, e.g. dietary saturated fat causes heart disease.  And researchers will further restrict their inclusion criteria to well designed studies.  But even with these criteria, you might ask: if I lump all these studies together, doesn't that falsely give equal credibility to both good and not-so-good studies?

Researchers address the issue of good, better, and best studies by "weighting" the different study results.  This means that each study will have more or less impact on the final outcome measurement - again, such as the relative risk for heart disease - depending on the quality of the study.  What makes one study better than another?  Usually the size of the study (10,000 subjects is likely more accurate than 1,000), the number of confounders adjusted for (older studies might only correct for age and smoking status, whereas a newer study might have adjusted for age, smoking, socioeconomic status, cholesterol, fasting glucose, etc.), and the quality of the methods used (in a diet study, a trial that provided all of the food for the subjects is much more reliable than giving subjects a questionnaire to determine what they ate).  Once you have all the studies tabulated and weighted, then you can get a summary outcome measurement and a neat graph that looks something like this:

Fig 1.  Anatomy of a meta-analysis

Every meta-analysis has a graphic like this (Fig. 1), and usually they have several more depending on how many hypotheses are being investigated.  The x-axis represents the relative risk.  If you remember from my last post, a relative risk of 1 means no difference in risk between groups, whereas a relative risk greater than 1 indicates that the "experimental" group has more risk than the control group.  Each study included in the graph is represented by a hash; the length of the hash represents the 95% confidence interval for the relative risk of that study.  If you are unfamiliar with statistics, the 95% confidence interval shows the range of numbers that we are reasonably confident includes the true effect of the experiment.  A smaller interval means you are more confident of the real number.  All you need to know is that if the confidence interval intersects the vertical line, then our safest bet is to conclude that there is no difference between the groups, since the relative risk is likely to be 1.  If the confidence interval does not intersect the vertical line, then we can conclude with reasonable certainty that there is a difference in risk between the groups.  At the bottom of the graph there is a diamond that represents the confidence interval derived from all the (weighted) studies included in the meta-analysis.  As you can see in the example, the diamond does not intersect the vertical line, and so the relative risk of all the studies combined is 0.85.  This would mean that the totallity of the evidence, based on this meta-analysis, indicates that the treatment reduces the risk of whatever outcome by 15%.

This is virtually all that you need to know in order to interpret a meta-analysis.  And if you are still reading this post, now it's time to talk about saturated fat.  Several large meta-analyses have been published in the past couple of years, and they all seem to give roughly the same answer.

In 2010, Siri-Tarino et al., published a meta-analysis on prospective cohort studies that evaluated the assocation of saturated fat with cardiovascular disease.  Based on 21 studies, they find no difference in the risk of CVD (the confidence interval contained 1), and conclude that "there is insufficient evidence from prospective epidemiologic studies to conclude that dietary saturated fat is associated with an increased risk of CHD, stroke, or CVD."  Interestingly, they assert that there is evidence of publication bias.**  But as the authors correctly point-out, this meta-analysis was limited to cohort studies and not powered enough (not large enough) to analyze the effect of replacing saturated fat with specific nutrients, such as carbohydrates or polyunsaturated fats (PUFA; think walnuts and seed oils).  Fortunately, other meta-analyses have.

Mozaffarian et. al performed a meta-analysis on randomized controlled trials that replaced dietary saturated fat with PUFA.  They only looked at myocardial infarction (heart attack) and CHD death; these are known as "hard endpoints," as heart attacks and death are not usually mis-diagnosed.  They show that increased PUFA intake (from 5% of daily calories to 15%) in place of saturated fat reduces the combined risk of heart attack and CHD death by 19% (Fig 2).  However, when the analysis isolated people who did not have pre-existing CHD, the aforementioned benefit disappeared (became statistically insignificant).  And there was no benefit seen in total mortality.  That is, replacing saturated fat with PUFA did reduce CHD and CHD death, but the risk of dying from all causes remained the same.

Fig 2.  Source:  Mozaffarian et al. PLoS Medicine

What saturated fat is replaced with is not trivial.  Mozaffarian et al. also analyzed studies that replaced saturated fat with carbohydrates and monounsaturated fat (think olive oil).  The single randomized controlled trial showed that replacing saturated fat with carbohydrate had no benefit, and in cohort studies, carbohydrates appear to increase the risk of CHD.  Monounsaturated fat is expected to lower the risk of CHD because of its beneficial effects on the cholesterol profile, but this has not been tested in a randomized controlled trial, and pooled analysis of available cohort data show a borderline increased risk of CHD (Fig. 3).  Weird, huh?

Fig 3.  Source:  Mozaffarian et al.  PLoS Medicine

And lastly, The Cochrane Collaboration has recently published an updated meta-analysis on the effect of dietary fat reduction and/or modification (PUFA instead of saturated fat) interventions, in randomized controlled trials, on cardiovascular outcomes.  Similar to their previous study and the aforementioned meta-analyses, they find that reducing and/or modifying dietary fat intake, for greater than six months, reduces the risk of CVD (events, not deaths) by 14%.  This decrease is attributable to:

"studies of fat modification and reduction (not studies of fat reduction alone), seen in studies of at least two years duration, in studies of men (and not those of women), and in those with moderate or high cardiovascular risk at baseline (not general population groups)."

This means that there is a small benefit from replacing some dietary saturated fat with unsaturated fats, but this may only apply to men and those who are at risk of or already have CVD.  And again, with a "high quality of evidence" given the shear number and size of the studies included, reduction of fat intake or modification of fat intake did not decrease the risk of CVD mortality or total mortality.

To be a bit critical, meta-analyses are far from perfect.  Remember, they are simply a pooling of results that improves statistical power in order to weed-out a result.  They do not improve the quality of the data or the individual studies themselves.  An accurate colloquialism is that a meta-analysis of garbage is still garbage.  Given their difficulties, I wouldn't go out and replace all of my butter with vegetable oil and expect a precisely 14% decrease in my risk of CVD.  But they give a nice summary of the evidence.

In the case of saturated fat, there is consistency between these analyses.  Total dietary fat is irrelevant to heart disease.  Replacing saturated fat with polyunsaturated fat modestly reduces the risk of cardiovascular disease, whereas replacing saturated fat with carbohydrate has no effect and may be harmful (if it's refined carbohydrates or sugar).  But at the end of the day, modifying or decreasing saturated fat likely does not decrease the risk of dying from heart disease and certainly has no effect on total mortality.  So after looking over these meta-analyses and bouncing it off of my current understanding of diet and diease, here is my conclusion: there is clearly no over-whelming evidence that saturated fat is bad, and in fact, there doesn't really seem to be any evidence.  And if it replaces sugar (butter instead of jam on toast), then it might actually be "healthy."  And yes, I'm aware of how crazy that notion sounds.  So what is one to do?

Source: Wikipedia, photo by Steve Karg

There are plenty of people who have given up butter and whole fat milk because of trepidation about saturated fat bringing them to an early grave.  Or in the words of Michael Pollan from In Defense of Food, "over the last several decades, mom lost much of her authority over the dinner menu, ceding it to scientists and food marketers (p. 3)."  Since the message to restrict saturated fat was loud enough to disrupt dinner, it is shocking that the evidence seems to have vanished.  And this is why the conventional wisdom will not change overnight.  Marion Nestle, a nutrition professor whose schtick I otherwise like, wrote a post on her blog to acknowledge these recent publications, but she inexplicably fell short of saying that saturated fat is probably harmless.  So in my opinion, it seems that the facts have ruined yet another hypothesis, because clearly, butter isn't out to get you.

This post is shared on Real Food Whole Health's Traditional Tuesday's Blog Hop.

___________________________

*   Ignore the impossibility that the "saturated fat" in the video is liquid in the refrigerator but solid at room temperature in the drain.  Saturated fats (coconut oil, beef tallow, butter) are solid in the refrigerator AND at room temperature.

**  Smaller studies showed an increased risk of CVD from dietary saturated fat, but larger studies, which will always be published since they are well-known and anticipated, showed an equal distribution of increased, decreased, and neutral risk.  The implication is that smaller studies that showed a detrimental effect of saturated fat were published, whereas smaller studies that showed no effect or a beneficial effect of saturated fat were either not submitted for publication or not accepted for publication.