Methodology

This study compares the news stories from user-determined news aggregators with those that were part of PEJ’s News Coverage Index. In this study PEJ chose to look at three different independent user-news sites: Digg, Del.icio.us and Reddit. In addition, PEJ looked at the three components of Yahoo News that depend on user input. These are Yahoo News Most Emailed stories, Yahoo News Most Recommended stories and Yahoo News Most Viewed stories. For each Web site mentioned, PEJ studied the top-ten stories on the homepage. This study looks at how topics in user-news aggregators compare with the larger, mainstream news media.

PEJ’s News Coverage Index is a thorough content analysis study of the topics that appear in the news media every week. The universe for the Index consists of 48 media outlets including newspapers, network television, cable television, online news sites and radio. A team of trained coders captures and analyzes each of the outlets. We code segments of these outlets that serve as a purposive sample of the larger media universe. The Index’s methodology details the rigorous standards followed for its coding.

The News Coverage Index is weighted according to media sector, and stories are presented as percentages of the entire newshole (click here for NCI Methodology).

For the purposes of this study on user-news aggregators, we focused on the number of stories alone and presented them as percentages of the total number of stories. Digg, Del.icio.us and Reddit link outside the host site and the size of a story is not an editorial decision made by the site itself. So we only measured how many of the total stories were about a particular topic. This particular study diverged from the News Coverage Index in that it did not measure the newshole.

To avoid discontinuity, we recalculated the percentages for each story in the News Coverage Index so that it would exclude weighting, word and time counts. For the purposes of this study, variables in the News Coverage Index were re-tabulated to measure how many from the total of 1,395 stories were devoted to a particular geographic focus, big or broad story topic. Since calculations about the News Coverage Index were done without weighting, statistics that appear in this study may differ from the numbers that appeared in other PEJ studies.

The time period for this comparative study is from a randomly selected week beginning Sunday, June 24, 2007 and ending on Friday, June 29,2007. The universe for this particular comparison consists of the top ten stories featured on Digg, Del.icio.us, Reddit and the Most Emailed, Most Viewed and Most Recommended stories from Yahoo News. A team of professional coders analyzed the top stories that appeared on the homepage (homepage refers to the main news page for each site). All Yahoo stories are hosted by Yahoo, but each sector—Most Emailed, Most Viewed, and Most Recommended—has its top ten stories listed on a different homepage. The content analysis of this study was conducted under the supervision of PEJ’s methodologist.

Each site was downloaded every day between June 24 and June 29 at 9am and at 5pm. A total of 644 stories were coded and used for this study.

For Yahoo News, the main page for each of the three sectors listed the top ten stories, so that there were a total of 30 Yahoo stories per download.[1]

At Digg.com, the homepage listed all the highest recommended stories. Of these the top ten were included in the study.

For Del.icio.us, the page lists all the most recommended stories, and from this the top ten were coded and included in the study.

Reddit’s homepage listed 25 most recommended stories. Of these, the top ten were coded and included in the study.

The protocol designed for this study was derived from that used for the Weekly News Coverage Index to maintain consistency. Eleven variables were coded including coder ID, story number, source, date (on which the story appeared on the homepage), download time, origin (story format), geographic focus, broad story topic, big story topic, sub story topic and story describer.

Variable source includes all three Web sites that were part of the study and the source of the story refers to the Web site (Yahoo News Most Emailed, Yahoo News Most Viewed, Yahoo News Most Recommended, Digg, Del.icio.us, or Reddit) that listed the story coded.

The origin variable refers to the format of the story, tracking whether the linked story appeared on the Web site server, a blog post, a news outlet, whether it was a wire story, or if it linked to a completely different site, which is not generally regarded as a news source.

Geographic focus concerns the location where the story originated, measuring whether a story was a U.S. story, a U.S.-international story or an international story that did not include the U.S.

The variable for broad story topic determined the general topic categories that were addressed by the story. The list of broad story topics used for this study was the same as those used by the Weekly News Coverage Index.

Big stories are particular news stories that recur frequently in the news during the time period understudy.

Variable sub story refers to developments, aspects and features of those long-running big story topics that occur most often in the media. Stories in this study were coded using the same list of big stories and sub stories that was used for the News Coverage Index for the week beginning June 24.