Hurricane Pali at its peak intensity on January 13, 2016

The study landed just in time for the 2014 hurricane season, and it created quite a weather system of its own.

A team of university researchers had found that female-named hurricanes are deadlier, and they posited that this was due to sexism – people didn’t take hurricanes with female names as seriously. They concluded that changing a severe hurricane’s name from Charley to Eloise “could nearly triple its death toll.”

The findings, published in the Proceedings of the National Academy of Sciences, drew international media attention. They also drew skepticism from some observers and academics who questioned the methodology.

Pomona College Economics Professor Gary Smith digs more deeply into those doubts in his new paper, “Hurricane Names: A Bunch of Hot Air?,” published in Weather and Climate Extremes. In addition to questioning the methodology, Smith uses new data to provide the most extensive look at the controversial findings so far.

Smith finds the hurricane names conclusion is “based in a questionable statistical analysis of a narrowly defined data set” and does not hold up when looking at a more inclusive set of data or at a fresh set of data.

His skepticism was heightened by the study’s conclusion that there is no female-male effect for less severe storms. If the sexism theory is true, he says, it ought to be most apparent for storms of questionable danger. 

“It is implausible that an imperiled public’s response to a potential storm of the century—with catastrophic warnings broadcast by news media that feed on sensationalized reporting—depends on whether the name Sandy is perceived to be a feminine or masculine,” notes Smith.

Smith found that the original study’s conclusions depended on the inclusion of pre-1979 data, a period when all tropical storms were given female names. Hurricanes happened to have been stronger during these years and it is likely that infrastructure was weaker and there was less advance warning. It is more scientifically valid to analyze storms since 1979, when weather officials started assigning alternating female and male names before the hurricane season begins.

Smith also found that the statistical analysis was flawed and that the authors estimated at least a dozen models, which he calls a sure sign of tortured data. Smith tried to replicate the original research by 1) looking at a wider set of data and 2) looking at a fresh set of data.

The original study, Smith notes, excluded tropical storms that did not meet the wind-speed threshold to be labeled hurricanes, and also excluded storms that stayed off the coast and did not make landfall in the U.S. (as well as excluding deaths that occurred outside the U.S.) When a wider set of data is considered, the study’s conclusions don’t hold up, says Smith.

For a second test, Smith looked at Pacific storms – the original research only considered Atlantic storms – and again found no difference in fatalities from female-named and male-named storms.

The Fletcher Jones Professor of Economics at Pomona College in Claremont, California, Smith teaches finance and statistics and pursues research on topics such as housing prices and stock prices. Author of “Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics,” Smith also has a penchant for looking more deeply at seemingly implausible research findings, such as the report (published in the British Medical Journal) claiming that Japanese and Chinese Americans are susceptible to heart attacks on the fourth day of every month because in Japanese, Mandarin and Cantonese, the pronunciation of four and death are very similar.

Smith laments that, “Statistical analyses are indispensable for evaluating competing claims and making good decisions. Unfortunately, the credibility of useful analyses is undermined by studies that torture data.”