James P. Scanlan, Attorney at Law

Journalists and Statistics

(Dec. 19, 2010; rev. Dec. 27, 2011)

Prefatory note:  This is one of several sub-pages to the Vignettes page of jpscanlan.com principally addressing presentation issues.  Other such sub-pages include the Times Higher and the Percentage Points sub-pages.  The former addresses the way that scientists and other observers incorrectly describe a rate such as 3% as three times higher than a rate of 1% and certain related issues, including that most dictionary definitions of multiplication are incorrect.  The latter addresses the way medical and health policy researchers fail to distinguish between percentage and percentage point differences even when the two yield opposite patterns of changes in the size of health and healthcare disparities over time.  The Mortality and Survival page of the site, which treats the way that medical journals discuss relative differences in mortality and relative differences in survival interchangeably without recognizing that the two tend to change systematically in opposite directions as survival generally increases or decreases, involves a presentation issue as well.  But that page principally concerns the more substantive matter of the failure generally to recognize that relative differences in experiencing an outcome and relative differences in avoiding the outcome tend to change in opposite directions as the prevalence of an outcome changes or that neither relative difference is a useful measure of the size of a disparity.

Prior to my adopting the practice of publishing on-line letters to medical and health policy journals, such as may be found here, rarely did I submit letters to editors of newspapers or magazines.  But sometime in the 1970s or 80s I sent a letter to the editors of Newsweek on some statistical point.  The magazine did not print the letter.  But it sent a note thanking me for the letter and observing that while the magazine could only print a small number of letters, it nevertheless reviewed them all carefully with an eye toward learning ways to improve its reportage.  A form letter no doubt, but it still seemed a nice touch.

It would be ten or so years before I was prompted to write the magazine another letter.  That occurred when I was struck by an illustration in an article on the plight of black youth that appeared in Newsweek’s October 15, 1990 issue.  Styled “Can the Boys be Saved?,” the article, which identified three authors, contained a sidebar styled “An American Dilemma,” verbally describing four sets of statistics about black men and black families and presenting a horizontal bar chart styled “Who Goes to College.”  The bars were styled as follows: 

Black Males 4%

Black Females 6%

White Males 39%

White Females 46%

A bar chart like this would typically be read to mean that 4 percent of black men and 6 percent of black women attend college, compared with 39 percent of white men and 46 percent of white women. In other words, white men were almost ten times as likely to attend college as black men, and white women were almost eight times as likely to attend college as black women.

Because I knew that blacks comprised about 10 percent of college students at the time, compared with their being something like 12 to 14 percent of the population, I knew that those figures could not be right.  Moreover, one should always be suspicious of figures on bar charts that sum to anything close to 100%.

So it seemed clear enough that, though the authors thought they were presenting the proportion of each group that attended college, they were in fact presenting the proportion each group comprised on all college students, something that would typically appear in a pie chart.  And it was easy enough to imagine just how the error occurred, given that practically no one, including a vast array of statistical experts, ever states anything about proportions very precisely.  Probably, while wishing to see figures on the proportions of black and white men and women who were in college, an author asked a researcher for figures on the proportion of black and white men and women in college and the researcher understood the request to mean the proportion black and white men and women comprised of college students.  And the authors never grasped the difference.  Anyway, the end result was a chart that said that white men were about 10 times as likely as white men to attend college when white men were in fact perhaps 1.5 times as likely to attend college as black men. 

I wrote Newsweek and explained the problem. I added that there surely was cause for concern in the fact that black men were six percent of the population (a figure gleaned from a statement elsewhere in sidebar itself) and only four percent of college students, but racial disparities in college attendance were by no means as great as the bar chart had suggested.

While I considered the point to be of some significance, I could see one reason for not printing my letter or what I imagined to be some number of other letters saying essentially the same thing. The title "Who Goes to College" could just as easily mean (a) the proportion each group made up of the total college population as (b) the proportion of each group that attended college (though, to be sure, the latter would be the more revealing information). Thus, if technically the use of a bar graph connoted the latter, possibly the chart would not mislead most readers. This is hardly a satisfactory reason for failing to correct an error that would mislead at least some readers.  But the editors might deem it a sufficient reason to justify devoting valuable letters-to-the-editor space to other purposes.

Newsweek did not print my letter or any like.  But in the November 19, 1990 issue, it did print three other letters about the  October 15, 1990 article.  None was unthoughtful. One of these, however, specifically addressed the data in the bar chart.  The writer had read the figures in the chart to reflect what they typically would mean in such an illustration –  specifically stating that “your chart shows that only 4 percent of all black men attend college” –  and then questioned what she regarded as  the article's emphasis on the plight of black men to the neglect of black women.  Thus, while not finding room to print information showing that the chart could not possibly mean what many readers would interpret it to mean, Newsweek found room to print a confirmation of the false impression created by the chart.

Provocative statistics can take on a life of their own. The Newsweek sidebar reappeared in February 1991 issue of Black Enterprise.  The bars in the bar chart were now vertical, however, and the wording had been modified in a notable respect.  Possibly recognizing some ambiguity in the title "Who Goes to College," the editors had changed the heading simply to "College Enrollment" and had added an explanatory footnote to indicate that the numbers reflected the "percent of each ethnic group enrolled in college by gender." Thus, if Newsweek had not, this publication made it absolutely clear that the numbers were intended to mean what could not possibly be true.

I gave the matter little further thought until reading a piece styled "Race on Campus: Failing the Test?" in the May 6,1991 issue of Newsweek.  Another sidebar presented some interesting data on black college attendance, including the fact that in 1989, 23.5 percent of blacks age 18 to 24 were enrolled in college. This time, I thought, Newsweek probably got the numbers right, whether or not it ever knew it had gotten them wrong before.  

Throughout this period, I had not received from Newsweek an acknowledgment of my letter.  Perhaps Newsweek no longer acknowledged letters, I thought, or no longer even devoted staff time to reading all of them.

But in the middle of June 1991, just about eight months after I had written Newsweek about the error in the sidebar, the editors responded, and this time not with a form letter.  Apologizing for the lateness of the reply, a representative of the editors explained that numbers on the bar chart were not, as I had maintained, the proportion each group made up of total college students, but the proportion of each group that attended college.  The chart was correct as published, the editors maintained, concluding: "Only four percent of black males have attended--whereas 39 percent of white males have. That's a radical disparity." The writer thanked me, "nonetheless," for writing.

I wrote back, attaching copies of varied sources, including the May 6 piece from Newsweek itself, showing that the editors could not conceivably have been right

on what is, after all, a very simple point. I even explained that the best readily available data on college attendance for both races showed (for 1988) a black rate of 25.6 percent

compared with a white rate of 36.4 percent –a substantial disparity but hardly the eight-to-ten fold disparity that Newsweek continued to assert was in fact the case.[i] I never received a further reply.

The episode is illustrative of the fog within which journalists present figures that are supposed to be important.  None of the three authors grasped the problem with the numbers.  And the editorial staff, presumably after giving the matter some thought, adhered to the impossible interpretation, and chose to be condescending in doing so.  So the episode does not speak well for the ability of journalists to grasp the most elementary statistical concepts.  Of course, as demonstrated on many other pages of this site, scientists experience considerable problems in that regard as well.