Shahar Madjar, MD, Journal columnist

The ideal way to solve such a question is to conduct a randomized study in which the participants are divided by chance into two or more groups that compare different treatments or interventions.

Here is an example of a ‘recipe’ for a randomized study on the effect beef consumption would have on health and mortality: Take a very large group of people and randomly divide them into two groups–red meat eaters and red meat avoiders.

Instruct one group to consume beef in moderate or high amounts and the other group to avoid beef altogether. Contact the two groups regularly–by phone, mailed questionnaires, or online–to verify that they are following the directions you gave them, and check on their health status. Did they experience a heart attack or a stroke? Are they still alive?

Conducting a randomized study like this is a very tedious task. It’s costly. It would take many years to complete. In most cases, conducting such a large-scale, intense study isn’t feasible.

The other option is to conduct studies in which researchers observe large populations over time, or in different geographical locations, to identify patterns of association between beef consumption and health. The trouble with observational studies is that the results are often inconsistent. One study finds that beef is good for you, the other claims that beef will bring about your early demise. What is one to do?

One solution is a meta-analysis–a study of several studies–in which scientists combine data from several studies to reach a single conclusion. The idea goes something like this: If one study is too weak to draw a meaningful conclusion because of, for example, fewer than the desired number of participants, combining two or more studies may have a greater statistical power.

The problem with solutions is that they often come entangled with new problems. In the case of meta-analyses, the new problem is that of too many choices. Here are a few examples of some of the choices researchers performing a meta-analysis need to make: which research articles should be included in the meta-analysis and which should be left out? Which of the participants in any given study should be selected for the analysis? And, which statistical model should be used to analyze the results?

How much of an effect do these analytic choices have on the conclusion? A recent study led by Dena Zeraatkar examined the effect such choices have when the association between red meat and mortality is studied. The title of the article itself is entertaining: “Grilling the data: application of specification curve analysis to red meat and all-cause mortality.”

Instead of deciding on one particular way to analyze a large dataset, the researchers chose to conduct their analyses in thousands of different ways using computers. The statistical technique they used is called specification curve analysis, and nicknamed multiverse analysis.

Zeraatkar looked at 15 studies on the effect of red meat on mortality. Their first analysis included only 70 ways of examining the data and was a bit inconsistent. The hazard ratio varied from 0.63 (a reduced risk of early death) to 2.31 (almost 2.5 times higher risk of mortality in red meat eaters). The median risk hazard was 1.14 (which means that the risk for mortality in red meat eaters is 14% higher).

Zeraatkar then calculated that there would be 10 quadrillion possible unique ways to calculate the hazard risk associated with red meat. This is how this number looks when written in its long form: 10,000,000,000,000,000. It a number of analyses that was beyond the calculating reach of any available computer. Therefore Zeraatkar generated 20 random unique combinations of covariants which narrowed the number of analyses to about 1,200.

As to the conclusion, only 4% of the analyses were statistically significant. In the end, the median hazard ratio for the effect of red meat on all-cause mortality (mortality from all causes) was 0.94. In other words: according to Zeraatkar’s final analysis, red meat consumption is highly unlikely to affect the risk of mortality.

In the media, in print and on screen, we are attacked daily by reports on observational studies and their conclusions. These are often “spam calls.” In these articles and news feeds, we are offered guidance. We quickly jump to a rushed conclusion. We do this, we avoid that, we stick to this diet or another.

In most cases, we would do better by waiting for a more comprehensive analysis, one that looks into the question in search of more cohesive patterns. Meanwhile, remember this: your health isn’t so much about any particular food item. It’s more about eating habits, a wholesome diet, and portion control.

EDITOR’S NOTE: Shahar Madjar, MD, MBA, is a urologist and an author. He practices at Schoolcraft County Memorial Hospital in Manistique, and at Baraga County Memorial Hospital in L’Anse. Find his books on Amazon. Contact him at smadjar@yahoo.com.