Chocolate, heart disease, and selective skepticism

I've recently developed an obsession with 88% dark chocolate that I can find on sale at Sprouts.  It has about as much fiber as it does sugar, and it's bitterness verges on espresso.  I've prescribed myself to a near-nightly dose of it after dinner.  And one night, while I was medicating, I came across a new study and some responses to the study that got me thinking about how easy it is to be critical about data that do not conform to our preconceived notions.

A meta-analysis was recently published on the effects of chocolate consumption and cardiovascular disease (the full text article can be found here).  The analysis only included 7 studies: 6 cohort studies, one cross-sectional study, and no randomized controlled trials.  Needless to say, I was surprised by the relative strength and consistency of the findings.

Source:  Buitrago-Lopez et. al British Medical Journal 2011.

The various levels of chocolate intake were associated with a 37% decrease in the risk of any cardiovascular disease, and a 29% decrease in the risk of stroke.  This is indicated in the figure by the diamond-shaped confidence intervals, which represent the risk of the combined studies, stratified by cardiovascular outcome, hovering around estimated relative risks.  There was no association between chocolate intake and heart failure, as you can see by the respective diamond crossing the relative risk axis of 1.  Unfortunately, none of these studies measured CVD mortality or total mortality, so we don't know if this reduced CVD risk was associated with a longer lifespan.

If you've been following my previous posts, then hopefully you've taken note that all of these studies are observational rather than randomized and interventional.  This makes this type of data prone to confounding variables (although many of these studies corrected for numerous confounders) and bias.  The bias is especially worrisome given that chocolate is somewhat of a luxury that probably associates with discerning palates, and because of all the marketing buzz regarding cocoa's abundance of antioxidants.

Even the authors are dubious of their findings since chocolate has more than just antioxidant-rich cocoa.  "The high energy density of commercially available chocolate (about 500 kcal/100 g) means excessive consumption will probably induce weight gain, a risk factor for hypertension, dyslipidaemia, diabetes, and cardiometabolic disorders in general."  And they didn't even mention sugar, which I'm more concerned about than calories.  And others are quick to point out that the association may very well not be due to the chocolate, as I mentioned above.  Marion Nestle, at FoodPolitics, concurs: "wisely, the authors point out that much more research is needed to confirm these benefits, not least because the studies were observational, not clinical trials."  


I agree with all of these issues.  And personally, I need substantial evidence to convince me that a food is not just food, but is actually therapeutic.  But if we are going to be skeptical about chocolate because of insufficient evidence, shouldn't we be just as skeptical of other foods or nutrients with similar insufficient evidence?


My last post on meta-analyses featured a study by Siri-Tarino et al. that concluded that prospective cohort studies show no association between saturated fat intake and CVD*.  Chocolate, on the other hand, does show a benefit, even in studies with similar design.  Furthermore, the reduction in the risk of stroke appears greater than might be expected from abstaining from processed meat.  Despite these difference, recommendations to reduce saturated fat abound, while poor chocolate only gets criticism.


Again, we have to be skeptical about all of these studies until there is either sufficint clinical trial data, overwhelming epidemiological data, or preferrably, a combination of clinical trials, epidemiology, and laboratory experiments.  It's imprudent to disparage a study because chocolate "just can't be healthy" or quickly agree with an association becuase processed meats "must be bad for you."  We have to be equally critical of all studies of the same methodology, regardless of the outcome. 


As with coffee, I'm not convinced that chocolate is a health food or will prevent heart disease; although if you can restrain yourself, it certainly doesn't appear harmful.  And with more investigation, perhaps chocolate will be definitively healthy.  So until we have more conclusive evidence,  eat chocolate because it is divine, because we can mull over the richness of it will friends at the dinner table, and because it is the perfect end to a meal.  And be equally skeptical of claims for all foods, regardless of how sweet, sweet, delicious they may be.


*As I described in the previous post, and to be accurate, there have been clinical trials to investigate the benefit of saturated fat reduction.  Replacing saturated fat with polyunsaturated fat does indeed lower the risk of cardiovascular disease.  Although it does not appear to reduce CVD mortality or total mortality.

Be skeptical of small numbers

A nuclear bomb is far scarier than a fire cracker.  Both are dangerous, but a nuclear bomb is clearly more destructive.  Not exactly rocket science.  In science-speak, the magnitude of this destruction is called the effect size.  Researchers spend a lot of time determining if an effect is real and how big the effect actually is.  Unfortunately, this information tends to distill down to "there was an effect" or "there was no effect."  This post is inspired by a lunch conversation with the girlfriend's parents, as it seems that nearly every food is out to get us.  It is one thing to say that a food has an effect on our health, but it's just as important to ask how big the effect actually is.

Effect size is an easy concept to measure in the laboratory.  A treated neuron can depolarize 5 times per second while a control neuron can depolarize 2 times per minute - an increase of 3 times per minute.  Differences in blood concentrations of a hormone, weight gain in rodents, and increased muscle mass are all easily recognized as an effect size.  In nutritional epidemiology, and epidemiology in general, the effect size is the strength of the association between an exposure (a food) and an outcome (a disease or mortality).  This is often measured as a relative risk.

Before I can talk about relative risk, I should explain absolute risk.  Absolute risk is the probability that an individual will develop a health outcome during a stated period of time (Fig 1).  Absolute risk, often measured as an incidence rate, is only meaningful if we have the number of outcomes AND the size of the population at risk AND a period of time.  The statements "4 men had heart attacks" and "4 out of 10 men had heart attacks," do not contain enough information to draw meaningful conclusions.  Rather, we need to know that "4 out of 10 men had heart attacks over the 5 year study period."  If we have the valid rate information about one group of people, we can compare it to another group's.  Absolute risk is vital for the real world impact of some exposure, but we rely on relative risk to get a grasp on the effect size of an exposure.

Fig 1.  Absolute Risk

Relative risk is simply the ratio of the absolute risk in the exposed group compared to a non-exposed group (i.e. control group).  If there is no difference in incidence rates of disease, then the RR will be 1.  If the exposed group has a higher rate, then the RR will be greater than 1.  And if the exposed group has a lower rate, then the RR is less than 1.  They are often discussed as percents (e.g. an RR of 1.3 means a %30 increased risk in the experimental group compared to the control).  Scientific journals will report rate ratios, hazard ratios, observed-to-expected ratios, and odds ratios - all of which are permutations of relative risk that are particular to different study designs.  Now that we're up to speed on relative risk, let's talk about effect size.

Fig 2.  Relative Risk

Effect size can help determine if an association seen in a study is causal.  Provided that the study is reasonably well conducted, a large relative risk suggests a causal association between the exposure and the outcome.  But how large is large?  Smoking and lung cancer are a textbook example of this principle.  Lung cancer is exceedingly rare in populations that do not smoke, especially if there are no industrial hazards.  Based upon an average of relative risks derived from several cohort studies (remember the limitations), men and women who smoke more than 20 cigarettes per day are 16 times more likely to die of lung cancer than non-smokers.  That's a whopping 1,500% percent increase in the risk of dying from lung cancer!  More moderate smokers have a considerably lower risk than the heaviest smokers, but are still far more susceptible with a relative risk of 5.0 and 9.0 for women and men, respectively.  The shear size of the effect provides evidence that smoking can cause lung cancer.  So what about not-so-large effects?

Because nutritional epidemiology relies heavily on observation rather than randomized controlled trials, the strength of an association can be distorted by confounding variables.  In fact, chances are that every observed effect is confounded by myriad unmeasured variables; many are insignificant, but some are important.  A study from the Health Professionals Follow-Up cohort demonstrated that men who consumed the most sugar-sweetend beverages had a 25% increased risk of developing type 2 diabetes over the 20 year follow-up.  Men who drank the most artificially-sweetend beverages (e.g. diet soda) were 91% more likely to develop the disease compared to those who drank the least.  However, after adjusting for the known confounding variables, the sugar-sweetened beverages still increased the risk by 24%, whereas the risk seen in the diet-rinkers was completely abolished.  It is easy to see how a relatively large effect size suggests causality, but does not prove it.  But what if the effect persists after adjusting for confouners?

Source:  Wikipedia:  Processed meat

A relatively recent article in the American Journal of Clinical Nutrition reported that men who reported eating the most processed meat (2 ounces or greater per day) compared to those who ate the least (less than 0.7 ounces per day) had a 23% greater chance of having a stroke over the course of the 10 year study.  Fresh red meat had no effect.  23% sounds fairly alarming; should we go to our fridge and throw out all of our salami and deli meat?  Looking at it another way, the average man in this study had a 6% chance (2409 out of 40, 291 men) of having a stoke over an average follow-up of 10.1 years.  By eating the highest amount of processed meat, his chances now increase to 7.4% (6% x 1.23).  His absolute risk increased by 1.4%*.

This may seem like a lot to you.  But also bear in mind that obesity and heavy smoking increase the risk of stroke by 100% compared to lean persons and non-smokers, respectively.  Using the average Swedish man above, each factor would increase the risk of stroke  from 6% to 12%.  Trading processed meat for fresh meat surely doesn't cause any harm, and this potential risk may simply be worth avoiding.  But think about how we need to approach this as scientific evidence.  Given that this has all the standard caveats of a prospective cohort study; and that the food record was based on a single survey given at the beginning of the study; and that you can never meaasure all of your confounders (they forgot sugar); are studies like this actually capable of detecting a true 23% increase in the risk of a specific mortality from a single type of food?  And is it worth constantly changing our diets when we're presented with these kinds of results?

Next time you hear a claim about a foods effect on health, or read another headline, make sure you find out how strong the effect actually is.  More often than not, you will only have access to the relative effect.  So keep in mind that if a disease is exceptionally rare, it will take a very high relative risk to have any real impact.  The risk of non-Hodgkin's lymphoma is .003 per 1,000 people over 1 year, which is so unlikely that an increased risk of 15% probably doesn't reflect a true association, and even if it does, it is virtually irrelevant.  The relative risk allows us to better comprehend the effect, but the absolute risk is what matters to the individual.

The problem with nutrition is that when you change something in your diet, it has to be replaced by something else.  How can you know you are making a change for the better?  And enjoying your food is important as well.  There are few things better than salami with cheese and wine, and bacon is arguably the best food there is.  The goal is not to disparage every study, but for the sake of health and culture, be skeptical about small numbers.



*The baseline risk I am using for this example is a crude estimate.  By simply using the number of strokes dived by the number of study participants over 10.1 years, I am ignoring the fact that some men were followed for less while some where followed for more.  However, this crude estimate approximated stroke statistics in the U.S. that I came across.  So don't hate!

On Cohorts and Coffee

Coffee beans, courtesy Elvis John Ferrao on Flickr.

Are you a big coffee drinker?  How many cups per day?  2? 4?  Well, if it's six or more, and if you're a man, then you may be lowering your risk of lethal prostate cancer.  That's great news for those of us who liberally indulge.  But if you don't drink coffee, should you?  Headlines that tout the benefits of individual foods are common.  Many of these findings are produced by observational epidemiology, so making an informed decision about integrating these foods into your diet requires an understanding of how these studies are designed, their strengths, and their weaknesses.  As in the aforementioned headline, much of the science of nutrition is derived from cohort studies.

A prospective cohort study, or longitudinal study, is an epidemiological study that defines two or more groups of people with various exposures (e.g hormone replacement therapy, coffee drinking), and then follows this cohort to measure any differences in outcomes (disease) between the groups in order to infer a causal association (see figure below).  Ideally, researchers ascertain a breadth of exposures and characteristics to discover associations and to improve the validity of such discoveries.  If an exposure is rare in the general population, a "special exposure cohort" can be used to follow a uniquely exposed group, such as vegetarianism in Seventh-day Adventists, and compare the special group's outcomes to a similar non-exposed group or the general population.  There are a few major prospective studies in nutritional epidemiology that warrant some attention.

 Source: Wikipedia.  Note that the investigator ascertains the exposures (black/white) prior to the unknown outcome.

One of the most influential diet studies is the Nurse's Health Study.  This study is technically composed of two phases, NHS I and NHS II.  NHS I began in 1976 to identify potential long-term complications of oral contraceptives that many women had begun to take.  It was later expanded to include diet and quality of life data.  NHS  II began in 1989 and recruited younger nurses for the purpose of collecting data on oral contraception, diet, and lifestyle factors that began earlier in life.  Major findings from this study include: smoking has a strong positive assocation with cardiovascular disease that reduces with smoking cessation, obesity increases the risks of several chronic diseases, and a Mediterranean-type diet appears protective.  However, the spurious idea that hormone replacement therapy would prevent coronary heart disease in all post-menopausal women was also produced by this study.

The Health Professionals Follow-Up Study (HPFS) began in 1986 as the male complement to the NHS.  And it produced the coffee-prostate cancer study above.  It is comprised of roughly 51,000 non-medical doctor health practitioners; over half of them are dentists and the vast majority are white.  Here is an example of the long form survey sent to participants.  Both the NHS and HPFS recruited motivated healthcare practitioners because this population is expected to accurately report disease outcomes and has the occupational commitment to maintain follow-up.  In fact, the NHS has retained a 90% response rate.

The Eurpoean Prospective Investigation into Cancer and nutrition, or EPIC, is a European equivalent.  This study has recruited over half of a million people from ten European countries, and studies the general population rather than healthcare practitioners.  Here's a neat infographic depicting the reported diet of "health conscious" and general population groups; it's a nice example of how types of people do not just aggregate around one food choice or the other, but rather a whole pattern of eating.

Cohort studies can be prohibitively expensive and are generally restricted to relatively common diseases or outcomes. But they offer substantial benefits over other types of observational studies for establishing a causal assocation.  If putative exposures and outcomes are measured at the same time, such as in cross-sectional studies, one cannot say with absolute certainty which one preceeded the other.  Cohort studies are better capable of  determing this information, or more technically, establishing the direction of causality.  Additionally, cohort studies often use real-time medical records, physical examination, or biological tests, and sometimes all three, to provide valid measurements of the exposures rather than relying on subjective recall.  However, unlike randomized controlled trials, the exposure status is chosen by the subjects and not the researchers.

In a prospective cohort study, the investigator ascertains the exposure status of the subjects and then groups them accordingly.  If science was easy, then these populations would just so happen to be the same with the exception of the exposure of interest.  But science can be cruel, and there is myriad reasons why individuals "choose" different exposures, which biases the results.  In our coffee study, it is possible that men who were developing lethal prostate cancer avoided coffee due to subclinical symptoms related to the impending prostate cancer diagnosis, which would bias cancer prone individuals away from coffee exposure.  This is called self-selection bias and can only be avoided by assigning exposure.  The investigators attempted to correct for this reverse causation by doing a sub-analysis with urinary symptoms to ensure that these type of symptoms were not associated with lower coffee consumption.  But such a bias could still have occurred from an unknown non-urinary symptomology or "drive" for cancer prone men to drink less coffee.   As such, the causal association between exposure and outcome from a cohort study is only inferred (e.g "heavy coffee consumption protects against lethal prostate cancer"), and can technically only be interpreted as "people who choose, or are otherwise driven by unknown factors, to consume large amounts of coffee tend to have a lower risk of lethal prostate cancer."  And then there's the issue of what we failed to measure.

The second major problem with the validity of cohort studies is the effect of confounding variables.  Because the groups are not randomized, the population with the exposure of interest may also have another exposure that associates with the outcome.  The classic example is the apparent positive association between coffee consumption and lung cancer.  This association is entirely explained by the fact that coffee drinkers also tend to smoke.  In the coffee and prostate cancer study, the invetigators made a Herculean effort to control for confounding by adding numerous potential confounders into their risk model.  These included: race, BMI, smoking, multivitamin use, PSA test history, and many more for a total of seventeen variables.  While adjusting for more and more confounders does enhance the validity of the association, remember that this type of manipulation is limited to "prostate cancer risk factors previously identified in this cohort and in other studies."  We cannot know what we have not measured, and it is always possible that there is at least one unknown variable that confounds our association of interest.

For what it's worth, I have a soft-spot for cohort studies.  The idea of a "natural experiment" is somehow quaint and very appealing.  They offer a lot to validity over other types of observational studies, but they always have important flaws, namely selection bias and potential confouding variables.  At the risk of pessimism, good science requires that we highlight the flaws of each experiment.  Read the headlines (and preferably the whole article!) with a critical eye.  Take note of the study design, and always ask how it fits into the greater scheme of the evidence.  As the authors concluded, "it is premature to recommend that men increase coffee intake to reduce advanced prostate cancer risk based on this single study."  Given the nature of this study, I will remain skeptical that coffee is therapeutic, although I am more confident that it is harmless.  But keep in mind that my opinion is heavily biased, as I've invested too much into my habit to stop any time soon.