Numbers, Facts and Trends Shaping Your World

Methodology: In Social Media and Opinion Pages, Newtown Sparks Calls for Gun Reform

https://www.pewresearch.org/journalism/2012/12/20/social-media-and-opinion-pages-newtown-sparks-calls-gun-reform/

This special report by the Pew Research Center’s Project for Excellence in Journalism on media reaction to the shootings in Newtown, Connecticut uses data derived from four different methodologies.

Data regarding the tone of conversation on social media (Twitter and blogs) were derived with a combination of PEJ’s traditional media research methods, based on long-standing rules regarding content analysis, along with computer coding software developed by Crimson Hexagon. That software is able to analyze the textual content from millions of posts on social media platforms. Crimson Hexagon (CH) classifies online content by identifying statistical patterns in words.

Data regarding the framing of coverage in the newspaper editorials and op-eds were derived from the PEJ’s in-house coding operation.

Data about the guests on cable and network television news programs were compiled through the use of transcripts and recorded broadcasts of the relevant shows.

Finally, data on the views on YouTube were collected from that site’s publicly accessible lists.

Coding of Social Media Using a Computer Algorithm

For determining the tone of the conversation on social media, the study employed media research methods that combine PEJ’s content analysis rules developed over more than a decade with computer coding software developed by Crimson Hexagon. This report is based on separate examinations of more than 11 million tweets and 45,000 blog posts.

Crimson Hexagon is a software platform that identifies statistical patterns in words used in online texts. Researchers enter key terms using Boolean search logic so the software can identify relevant material to analyze. PEJ draws its analysis samples from several million blogs, all public Twitter posts and a random sample of publicly available Facebook posts. Then a researcher trains the software to classify documents using examples from those collected posts. Finally, the software classifies the rest of the online content according to the patterns derived during the training.

According to Crimson Hexagon: “Our technology analyzes the entire social internet (blog posts, forum messages, Tweets, etc.) by identifying statistical patterns in the words used to express opinions on different topics.” Information on the tool itself can be found at http://www.crimsonhexagon.com/ and the in-depth methodologies can be found here http://www.crimsonhexagon.com/products/whitepapers/.

Crimson Hexagon measures text in the aggregate and the unit of measure is the ‘statement’ or assessment, not the post or Tweet. One post or Tweet can contain more than one statement if multiple ideas are expressed. The results are determined as a percentage of the overall conversation.

The time frame for the analysis began at 10 am ET on December 14 and went through noon ET on December 17.

PEJ used Boolean searches to narrow the universe to relevant posts. Common terminology posted by users varies for each platform. Therefore, PEJ used slightly different search filters for each.

For Blogs, PEJ used the following search filter:

(Newtown OR Newton OR Connecticut OR gun OR guns OR amendment OR Lanza OR “mental health” OR “sandy hook” OR Obama)

For Twitter, the search filter was:

(Newtown OR Newton OR CT OR Connecticut OR gun OR guns OR control OR amendment OR 2nd OR second OR shooting OR weapon OR Lanza OR mental OR health OR sandy OR hook OR Obama)

Human Coding of Newspaper Editorials and Op-eds

The list of eleven newspapers used in this sample included a variety of geographic locations, ownership groups and circulation sizes. The sample is the close to the same one used in PEJ’s weekly News Coverage Index (NCI), the methodology of which explains the newspaper selection process.

The eleven papers used in this study were as follows:

The New York Times Los Angeles Times USA Today The Wall Street Journal The Washington Post The Denver Post Dallas Morning News Orlando Sentinel Traverse City Record-Eagle (MI) The Daily Herald (WA) The Eagle-Tribune (MA)

All editorials and op-eds from the print copies of these papers from December 15 to December 18 were examined. Hard copies of the papers were either delivered to the PEJ offices in Washington, D.C., or digital versions were acquired through the papers’ websites or the web service Pressdisplay.

Experienced human coders determined the frame and tone of each relevant article. PEJ has used similar variables in previous studies, and intercoder testing has revealed that such coding meets rigorous academic standards.

Data Regarding the Guests on Television News Shows

PEJ recorded the guests on television news shows on eight channels: ABC, NBC, CBS, PBS, Fox, Fox News, CNN and MSNBC.

All guests who discussed an issue relevant to the Connecticut shootings were included in the tally. Regular contributors to the programs were excluded, as were people directly involved with the incident such as eye-witnesses and family members.

Broadcasts from noon ET on December 14 through 10 am ET on December 17 were included.

PEJ was able to review recorded broadcasts or transcripts of most of the national news programming on each of the seven channels during that time.

For some channels, transcripts and digital recordings were not available and could not be included in our sample.

The programs and times examined were as follows:

Cable

Friday, December 14

Fox News Daytime shows from noon to 3:00 pm ET All evening shows from 5:00 pm until 11 pm ET

MSNBC All programming from noon to 11:00 pm ET

CNN All programming from 10:30 am to 10:00 pm ET

Saturday, December 15

Fox News America’s News HQ Fox & Friends Weekend Huckabee Greta

MSNBC Up w/ Chris Hayes Elementary School Tragedy Andrew Mitchell Reports Melissa Harris-Perry

CNN All programming from 7:00 am to 11:00 pm ET

Sunday, December 16

Fox News Fox News Sunday

MSNBC The Chris Matthews Show

CNN All programming from 6:00 am to 11:00 pm ET

Monday, December 17

CNN All programming from 6:00 am to 9:00 am ET

Network

Friday, December 14

ABC World News Tonight 20/20 Special Nightline

CBS CBS Evening News CBS News Special

NBC NBC Nightly News NBC Dateline Special

PBS PBS NewsHour

Saturday, December 15

ABC Good Morning America World News Saturday

CBS CBS This Morning CBS Evening News 48 Hours Special Edition

NBC Today Show NBC Nightly News

Sunday, December 16

ABC Good Morning America This Week with George Stephanopoulos World News Tonight

CBS Face the Nation 60 Minutes

NBC Today Show The Chris Matthews Show NBC Nightly News

Fox Fox News Sunday with Chris Wallace

Monday, December 17

ABC Good Morning America

CBS CBS This Morning

NBC Today Show

Data on YouTube Views

On Wednesday, December 19, at 1:30 pm ET, a PEJ staff member captured the list of most viewed news and politics videos on YouTube over the previous week. These videos are categorized as such on the YouTube site and are often a mix of mainstream news reports, raw footage relating to breaking events, or other types of public affairs clips. PEJ determined the most viewed videos as they are listed on YouTube’s page at the time of capture.

Icon for promotion number 1

Sign up for our weekly newsletter

Fresh data delivery Saturday mornings

Icon for promotion number 1

Sign up for The Briefing

Weekly updates on the world of news & information