Analyzing text for distinctive terms using pointwise mutual information
PMI is a quick and easy way to identify words that distinguish one group of documents from another.
A behind-the-scenes blog about research methods at Pew Research Center.
For our latest findings, visit pewresearch.org.
Data Science
-
-
How we built our data science infrastructure at Pew Research Center
After venturing into the world of computational social science in 2015, the Center needed to develop new tools and workflows.
-
Are topic models reliable or useful?
The final post in our series examines how topic models can and can’t help when classifying large amounts of text.
-
How keyword oversampling can help with text analysis
Keyword oversampling can be a powerful way to analyze uncommon subsets of text data.
-
Contending with the 80/20 rule when studying online behavior
The Pareto principle, or “80/20 rule,” holds that in many systems, a minority of cases produce the majority of outcomes.
-
How quantitative methods can supplement a qualitative approach when working with focus groups
In a recent project involving focus groups, we tested out quantitative as well as qualitative research methods.
-
Introducing Pew Research Center’s Python libraries
We’re excited to release a collection of Python tools that we’ve found ourselves returning to again and again.
-
The challenges of using real-time epidemiological data in a public health crisis
Three widely cited coronavirus trackers differ in their methods and in the kinds of information they provide.
-
Exploring the link between European political parties’ ideology and their popularity on Twitter
It can be hard to determine whether parties near the ends of the ideology scale are more popular on the platform than moderate parties.
-
Interpreting and validating topic models
Understanding the exact meaning of a set of words requires an intimate understanding of how the words are used in your data and the meaning they’re likely intended to convey in context.
