For a PDF of the topline data, click here.
This is a joint report by the Project for Excellence in Journalism funded by the Pew Charitable Trusts and the Joan Shorenstein Center on the Press, Politics and Public Policy at the Kennedy School of Government at Harvard University.
PEJ and Shorenstein together designed the study, analyzed findings and wrote the report. The content analysis was conducted at PEJ by PEJ staff, with the financial support of both the Shorenstein Center and the Pew Charitable Trusts. Marion Just was the lead researcher from The Shorenstein Center.
Sample Design The content studied is based on coverage originally analyzed for PEJ’s weekly News Coverage Index (NCI) from January 1-May 31, 2007. Each week the NCI examines the coverage from 48 different outlets in five media sectors, including newspapers, online news, network TV, cable TV, and radio. Following a rotation system, 35 outlets each weekday are studied as well as 7 newspapers each Sunday. The media outlets examined are as follows:
Newspapers (Thirteen in all, Sun-Fri) Five large circulation newspapers, four regional major metropolitan dailies, and four smaller circulation papers. At the suggestion of our academic advisors, one paper, the New York Times, was captured every day to have one “paper of record.”
NY Times every day
Code 2 out of these 4 every day
Washington Post
Los Angeles Times
USA Today
Wall Street Journal
Code 2 out of these 4 every day
The Boston Globe
Star Tribune (Minneapolis)
Austin American-Statesman
Albuquerque Journal
Code 2 out of these 4 every day
The Sun Chronicle (Boston, MA)
Star Beacon (Ashtabula, Ohio)
The Chattanooga Times Free Press
The Bakersfield Californian
Web sites (Five in all, Mon-Fri)
CNN.com
Yahoo News
MSNBC.com
Google News
AOL News
Network TV (Seven in all, Mon-Fri)
Morning shows
ABC – Good Morning America
CBS – Early Show
NBC – Today Evening news
ABC – World News Tonight
CBS – CBS Evening News
NBC – NBC Nightly News
PBS – Newshour with Jim Lehrer
Cable TV (Fifteen in all, Mon-Fri)
Daytime code 2 out of 3 every day*
CNN
Fox News
MSNBC *From Jan. 1 to March 16 coded from 1-1:30 p.m. EST; from March 19 on, coded from 2-2:30 p.m. EST
Nighttime CNN – code 3 out of the 4 every day
Lou Dobbs Tonight
Situation Room (7 pm)
Paula Zahn Now
Anderson Cooper 360
Nighttime Fox News – code 3 out of the 4 every day
Special Report w/ Brit Hume
Fox Report w/ Shepard Smith
O’Reilly Factor
Hannity & Colmes
Nighttime MSNBC – code 2 out of the 4 every day
Tucker (6 pm)
Hardball (7 pm)
Countdown w/ Keith Olbermann
Scarborough Country
Radio (Eight in all, Mon-Fri)
Headlines every day
ABC Radio headlines at 9am and 5pm
CBS Radio headlines at 9am and 5pm
NPR Morning Edition every day
Talk Radio
Rush Limbaugh every day
1 out of 2 additional conservatives each day
Sean Hannity
Michael Savage
1 out of 2 liberals each day
Ed Schultz
Randi Rhodes
From that content, the study included all campaign related stories:
- On the front page of newspapers
- In the entirety of commercial network evening newscasts.
- The first 30 minutes of network morning news, the PBS evening news, and all cable programs
- The top 5 stories on each website at the time of capture
The basic NCI codebook codes for topic at three different levels, and also includes date coded, Story ID number, story date, source, broadcast start time, broadcast story start timecode, headline, story word count, placement/prominence, story format, story describer, and broadcast story ending timecode. The complete methodology for the weekly NCI has further details on the coding system and inter-coder reliability.
Story CollectionFor this analysis, we began by pulling all stories from January 1 – May 31, 2007 originally coded as election-campaign stories. The resulted in the following: 1,742 news stories were analyzed, including 1,468 stories (68.2 hours) of broadcast content (454 stories with 16.7 hours from network TV, 722 stories with 37.9 hours from cable, 292 stories with 13.6 hours from radio), 168 stories in newspapers, and 106 stories from news websites.
Coding Design
A coding protocol was designed for this project based on PEJ’s and Professor Just’s previous related studies and the particular aims of the PEJ-Shorenstein study. In addition to the existing variables in the NCI, the campaign study included a variety of variables designed to probe about the election. These included dateline, recurring campaign theme, primary figure, tone for primary figure, secondary figure, tone for secondary figure, general topic category, story trigger, and story impact.
In particular:
Dateline designates the state/city dateline of the story. Variable recurring lead designates the recurring lead, or “big story” element, if any, within each story.
Primary figure is the candidate or campaign organization, or other figure that dominates the story.
Primary figure tone reflects whether the journalist’s tone is constructed in a way, via use of quotes, assertions, or innuendo, which results in positive, neutral, or negative coverage for the story’s primary figure as it relates to the topic of the story. While reading or listening to a story, coders tally up all the comments that have either a negative or positive tone to the reporting. Direct and indirect quotes are counted. In order for a story either positive or negative, it must have 1.5 times the amount of positive or negative comments (with an exception for 2 to 3, which is coded as neutral). If the headline or lead has a positive or negative tone, it should be counted twice into the total value. Also counted twice for tone are the first three paragraphs or first four sentences, whichever comes first.
Election topic measures the broad election-related topic, or what the story is about “on its face.” In addition, each broad topic was also segmented into several “sub-topic” categories for further specification.
Story trigger indicates who initiated the story—the action, event or editorial decision that makes this news, thus triggering its publication.
Story impact designates the individual or group whose interests are at stake or were affected by events in the story. They could be citizens, politicians, interest groups, non- U.S. citizens, other. A story had citizen impact if it conveyed information that would be useful to voters in determining how someone would govern. At least 50% of the story had to relate directly to that group. Coders were instructed if they could infer a citizen impact to default to that category.
Coding Team & Process
Using the existing data in the Index and adding the codes for new variables, the team responsible for performing the content analysis is made up of the five trained coders, a coding administrator, and a senior research methodologist on the PEJ staff.
In coder-training, inter-coder reliability tests were conducted for all variables. For the variables derived from PEJ’s weekly Index, the average level agreement was 96%. For the new variables, initial tests found levels of agreement of 80% or greater for all but two variables (election topic and story impact). For those two variables, clarifications were made to the codebook and additional training implemented. An additional test was conducted later in the process and both variables reached a level above 80% agreement. In total, testing was conducted on 5% of the sample. In addition, the coding administrator monitored coding throughout the process to ensure levels were maintained.
The specific levels of agreement for the variables in this study were as follows:
Dateline: 92%
Recurring Lead: 83%
Primary Figure: 92%
Tone for Primary Figure: 86%
Story Trigger: 83%
Story Impact: 85%
Election Topic: 83%
For a PDF of the topline data, click here.
