Methodology

First, broad search terms were used to retrieve a large amount of stories that may have been about the Senate race. The determination was made that it was better to use broader search terms that would result in a larger group of stories to sort through than more detailed search terms that might have resulted in the accidental exclusion of relevant stories.

For the two Boston newspapers, all stories that included the word “Senate” from the dates included in the study were collected. For the two national outlets, the terms “Senate” and “Massachusetts” were used.

In addition, another search was conducted for all four outlets that included the names of all of the prominent candidates who were running (or considered running). This was done to make sure that any stories that focused on a candidate but did not mention the word “Senate” would not have been missed.

Second, members of PEJ’s staff went through each story that was retrieved to determine if it was about the Senate race. A story was considered to be about the race if 50% or more of the text was focused on the campaign. All other stories that were not about the race were then discarded.

This resulted in a sample of 557 stories.

Since the subject of the study was solely the campaign for the Massachusetts Senate seat, stories about the temporary replacement for the position and the rules that governed the process were not included in the sample.

Coding Variables

In addition to housekeeping variables (such as date, source and dateline), each story was coded for the following variables:

Story Format measures the type and origin of stories
Big Story tracks the narrative storylines that were frequently covered throughout the sample period
Topic captures the general subject matter of the story
Frame captures the narrative technique the journalist used to tell the story
Figure Presence determines whether a person was included in 25% or more of a given story. Presence was tracked for the following people:

Michael Capuano (D)
Martha Coakley (D)
Alan Khazei (D)
Stephen Pagliuca (D)Scott Brown (R)
William Coleman (I)
Joseph L. Kennedy (I)
Jack E. Robinson (R)
Stephen Lynch (D)
Andrew Card (R)
Curt Schilling
Ed O’Reilly (D)
Joe P. Kennedy II (D)
Vicki Kennedy (D)

Tone measures whether a story is constructed in a way, via use of quotes, assertions or innuendo, that results in positive, neutral or negative coverage for a given candidate. Tone was tracked for the following candidates:

Michael Capuano (D)
Martha Coakley (D)
Alan Khazei (D)
Stephen Pagliuca (D)
Scott Brown (R)
Jack E. Robinson (R)

Coding for Tone

In coding for tone, PEJ used an established technique to determine whether a story is positive, neutral or negative in coverage for candidates who are present in 25% or more of the story. The unit of measure was the story.

While reading a story, coders tallied up all the comments that have either a negative or positive tone to the reporting. Direct and indirect quotes were counted along with assertions made by journalists themselves.

In order for a story to be coded as either “positive” or “negative,” it must have either 1.5 times the amount of positive comments to negative comments, or 1.5 times the amount of negative comments to positive comments (with an exception for 2 to 3, which is coded as “neutral”). If the headline or lead has a positive or negative tone, it was counted twice into the total value. The first three paragraphs or first four sentences, whichever came first, were also counted twice for tone .

Any story where the ratio of positive to negative comments was less than 1.5 to 1 was considered a “neutral” story.

Coding Team & Process for the Additional Coding

A team of four of PEJ’s experienced coders worked with a coding administrator to complete the coding for this study.

Intercoder testing was conducted for all of the variables used in the study. Each coder was given the same 23 randomly selected stories to make up the intercoder sample.

The percent agreement for the key variables was as follows: