Methodology

This study was conducted in two parts. The first was a replicate study, using the same methodology and sample as in PEJ’s 2001 study “The First 100 Days: How Bush Versus Clinton Fared in the Press.” This was done to allow us to make accurate comparisons between the coverage of Obama’s first days in office and the same time periods for the preceding two presidents.

The second part of the study added additional news reports beyond those included in the 2001 study. For the expanded universe we utilized data collected as part of PEJ’s News Coverage Index during the same 60 day time period in order to examine additional media outlets.

As a result, this study includes two samples. The first sample, referred to here as the comparison sample, is modeled after the previous 2001 study. The second sample, referred to as the broader sample, includes all the stories that were in the comparison sample plus additional media outlets and sectors.

All the data about the coverage of President Clinton and President Bush come from the 2001 study. The methodology for that study can be found here.

Sample Design

Comparison Sample

The comparison sample was duplicates for 2009 the sample used in PEJ’s 2001 study. This sample consisted of seven media outlets—2 newspaper, 1 news magazine and 4 broadcasts. The time period was January 21 to March 21, 2009.

The specific outlets, selected to develop a sample of coverage provided by the national press, are as follows:

Newspapers
The New York Times
The Washington Post

Weekly Magazine
Newsweek

Evening Network TV
ABC World News Tonight
CBS Evening News
NBC Nightly News
PBS Newshour

Newspaper stories were drawn from section front pages (i.e. national news, style, business, metro, Sunday review), editorials and op-ed pieces. Complete newscasts and complete issues of Newsweek were the basis for the sample of television and magazine stories. It should be noted that the PBS Newshour only airs from Monday through Friday, while the three broadcast networks air newscasts every day of the week. Also, there were a number of weekend network broadcasts that were pre-empted for other programming. CBS Evening News did not air on February 1, February 8, February 15, February 22, March 15, and March 21. NBC Nightly News did not air on February 1, February 8, and March 15.

Story Inclusion
For the newspaper and television outlets, this sample included stories published or broadcast every 2nd day and all stories published or broadcast on Sundays. All Newsweek issues published during the period were used in the study except for the Commemorative Inaugural Edition which differed in style from the typical Newsweek edition.

Stories were only included in this study if 50% of the time or space was devoted to Barack Obama or his administration. Print stories less than 100 words long, broadcast stories 30 seconds or less, 1-on-1 interviews with Obama, or the purest form of straight news were excluded.
The resulting comparison sample consisted of 362 stories.

Broader Sample

The broader sample included all the outlets included in the comparison sample along with many of the additional outlets that are coded as part of PEJ’s regular News Coverage Index.

The complete methodology of the PEJ’s News Coverage Index is available here.

The outlets that were part of the broader sample (in addition to the comparison sample listed above) are listed below. And here, there is an embedded system of rotation, as indicated.

Newspapers (Sun-Fri)
Coded every second day
USA Today
Los Angeles Times
Wall Street Journal
Coded two out of these four every day
Kansas City Star
Pittsburgh Post-Gazette
San Antonio Express-News
San Jose Mercury News
Coded 2 out of these 4 every day
Herald News (MA)
Anniston Star (AL)
Spokesman-Review (WA)
Meadville Tribune (PA)

Web sites (Coded 6 of 12 each day, Mon-Fri)
CNN.com
Yahoo News
MSNBC.com
Google News
AOL News
Foxnews.com
USAToday.com
Washingtonpost.com
ABCNews.com
BBC News (international version)
Reuters.com
NYTimes.com

Morning Network TV (Mon-Fri)
ABC – Good Morning America
CBS – Early Show
NBC – Today

Cable TV (Fifteen in all, Mon-Fri)
Daytime (2:00 to 2:30 pm) coded 2 out of 3 every day
CNN
Fox News
MSNBC

Nighttime CNN – coded 2 out of the 4 every day
Situation Room (6 pm)
Lou Dobbs Tonight
Campbell Brown: No Bias, No Bull
Anderson Cooper 360

Nighttime Fox News – coded 2 out of the 4 every day
Special Report w/ Bret Baier
Fox Report w/ Shepard Smith
O’Reilly Factor
Hannity

Nighttime MSNBC – coded 2 out of the 4 every day
1600 Pennsylvania Avenue
Hardball (7 pm)
Countdown w/ Keith Olbermann
Rachel Maddow

Radio (Mon-Fri)
NPR Morning Edition every day (Rotated daily between the first 30 minutes of the first hour and first 30 minutes of the second hour)

Story Inclusion
The portion of the news product studied different slightly from the comparison sample. For the broader sample content, PEJ analyzed all stories with a national or international focus that appear as follows:

On the front page of newspapers
During the first 30 minutes of network morning news and cable programs
During a thirty minute segment of NPR’s Morning Edition
As one of the top 5 stories on each Web site at the time of capture

Then, as in the comparative sample, stories were included in this study if 50% of the time or space in the story was devoted to Barack Obama or his administration. Print stories less than 100 words long, broadcast stories 30 seconds or less, 1-on-1 interviews with Obama, or the purest form of straight news such as were excluded.

Finally, from the qualifying stories, every second story appearing in newspapers, network TV, online, or radio, every 2nd story was randomly selected for inclusion. Since there were a high number of stories on cable devoted to politics, every 3rd story was randomly selected to be part of this study.
The resulting broader sample consisted of 757 stories (including the 362 stories also in the comparison sample).

Capture and Retrieval
All outlets are captured and included in PEJ’s media archive.

For Newsweek magazine and newspapers that are available in print in the Washington, D.C. area, hard copies are used. For newspapers that are not available for delivery, digital editions of the paper are retrieved either through the newspaper’s own Web site, or through the use of digital delivery services such as pressdisplay.com and newsstand.com. When necessary, the text of articles are supplemented by the archives available in the LexisNexis computer database.

Radio programs are captured through online streams of the shows. Using automated software, we record several local affiliates that air the program in various markets throughout the country. The purpose of this method is to ensure that we have a version of the program in case one of the streams is unavailable on a particular day, and so that we record the show in a manner that represents the way a typical listener would hear the program with commercials and newsbreaks.
Online websites are captured manually by a member of PEJ’s staff. The capture time is rotated daily between 9 am ET and 4 pm ET. The home pages and pages with the top articles for all five sites are saved so that when we reference the material, the format is the same as it appeared online at the time of capture.

Finally, all television shows are recorded digitally and archived for coding purposes. PEJ is a subscriber to DirectTV satellite service and all programs are recorded onto multiple TiVo recording units before being burned onto DVDs for archival purposes. Several of the weekend newscasts used in this study did not air in the Washington, D.C. area at the normal time. In those instances, either copies of those stories were borrowed from the Vanderbilt Television News Archive, or transcripts were accumulated from LexisNexis or Highbeam Research.

All television and radio programs are then coded by a member of PEJ’s staff who watches or listens to the archived version of the program.

Coding Team & Process for Weekly Index Coding

Much of the data in this study was derived from PEJ’s regular Index coding and was conducted by PEJ’s team of 14 trained coders. We have tested all of the variables contained in the regular weekly Index coding and all the variables reached a level of agreement of 80% or higher. For specific information about those tests, see the methodology section for the NCI.

Additional Coding
For all the stories included in this study, additional coding was conducted using the same codes used in the 2001 study of Presidential media coverage.
Stories were coded for topic, frame, and arena. For these three variables, the simple plurality rule was used: researchers coded each paragraph individually, variable by variable, and the code that appeared most often was used to classify the story. Where two codes appeared with equal frequency, the code that occurs earliest in the story was used.

Topic refers to the general subject matter of the story: for example, crime, the environment, or the nomination process.
Frame describes the narrative technique used by the journalist (whether consciously or not) to inform readers or listeners about the subject at hand.
Arena describes to whom the story related the president. For example, a story about the president’s budget (topic) might be written in a way that relates him to Congress (e.g., describing White House staff meeting with Senators) or to interest groups (e.g., reporting on the efforts of lobbyists who support or oppose the plan) or to his cabinet (e.g. how closely did Bush work with his cabinet on final figures compared to other president’s). Another possible relationship is that of the office of the president. These are stories that focus internally and look at the President in his new role.

Finally, stories were coded for Theme and for Tone. Theme refers to what journalists were assessing about the president. Three possible themes of presidential evaluation were developed: leadership, an assessment of the president’s political skills or decision-making ability; ideology, an assessment of the president’s agenda; and character, an assessment of the president’s personal mores and system of values. Researchers coded comments from sources (including the president himself) and journalists’ innuendoes to determine which theme a story belonged to.

Next, researchers coded each comment and innuendoes pertaining to the President for tone: positive, negative or neutral. Extra weight was given to text in the headline or lead paragraph of a story. When the ratio of positive to negative comments, or negative to positive comments, equaled or exceeded 2:1 a story was coded as a positive or negative assessment of the president. All other stories were classified as neutral.

Coding Team & Process for the Additional Coding

A team of five of PEJ’s experienced coders worked with a coding administrator in order to complete the additional coding for this particular study. Of the five coders, all but one had previously coded for tone in a previous PEJ campaign study.
Intercoder testing of the specific additional codes used in this study was conducted using 43 randomly selected stories. Of those, 29 of the stories were coded by all participants while 14 of the stories were coded by two participants.

The percent agreement for each variable was as follows:

Topic: 84%
Frame: 81%
Political arena: 82%
Theme: 80%
Tone: 83%