June 27, 2014

Why the typology quiz questions are asked the way they are

The Pew Research Center released on Thursday our 2014 political typology, including the newest version of our political typology quiz, in which users can see where they fit in the current political landscape by answering 23 questions about core political values. Within the first 24 hours, more than 90,000 people had taken the quiz, and the last quiz, launched in 2011, had over 1.5 million users over three-plus years.

One of the strongest reactions we have received from some quiz-takers is frustration over the either-or choices each question offers. Many of the comments we’ve received say that the questions feel like choosing “between two extremes,” or that the “right answer is somewhere in between,” or that the options “aren’t really opposites.”

These are all legitimate concerns, and many of our own early users here on the Pew Research Center staff expressed the same frustrations. But there is a reason the questions are asked the way they are: The intent is not to put people “in a box” but rather to understand how their values across multiple political dimensions are related to each other.

FT_Quiz_QuestionTake a question like the one shown here. For many Americans, the two options present a very stark way of thinking about poverty, and it is a fair guess that many people have some level of agreement with both points of view. At the same time, this “forced choice” format reflects a fundamental debate in American politics right now, and most people at least lean one way or the other on this potentially difficult tradeoff. In fact, the American public as a whole splits almost exactly in two on this question – 44% select the first statement, 47% pick the second – and their choice is strongly related to their specific policy preferences and priorities in this realm.

One frustration of a question like this is in its bluntness. It can make a respondent feel as if they are being lumped in with a bunch of people who hold much more extreme views than they do.

But the “forced choice” construction of this question (and others) is part of its design. The question picks up on which of two statements on a core value people lean towards, and does not necessarily presume complete agreement with a question. That is why the instructions ask, “Which comes closer to your view, even if neither is exactly right?”

Typologyquiz-screenshotAnd because the typology quiz includes 23 questions, many of which measure closely related concepts—for instance, there are three more questions in the quiz that pick up on the same theme about the relationship between government and support for the poor—no one statement alone dictates which group you’ll end up in. So even if some questions are not a great fit for your values, the combination of responses should place you in a group that reflects your underlying political type.

Put differently, while individual questions may not allow for nuance, the combination of questions does. In fact, that is part of what the cluster analysis approach we use to define the typology does.

In addition, this year’s typology builds upon the 20-year Pew Research Center history of asking many of these specific questions. Some of these questions use language that is different from how we might ask them if we were developing them for the first time this year. But there is a great deal of value in maintaining identical question wording in order to measure change over time, even if the wording used in the past is not ideal today. Because of the 20 year history with this series of questions, we were able to document the growing amount of ideological consistency in peoples’ thinking over the period.

Finally, it’s worth emphasizing that we do not see these questions as measures of political “extremism.” Instead, these forced-choice questions were designed to capture enduring political values that underlie the policy preferences and partisan choices people make in politics. And our goal is to get a picture of how peoples’ values are organized, not how strongly they hold those views. There is a very important distinction between ideological values and more radical political thinking, and we address this distinction in our earlier report on political polarization.

Topics: Political Typology, Research Methods

  1. Photo of Jocelyn Kiley

    is an associate director of research at Pew Research Center.

18 Comments

  1. Jason1 year ago

    The tendency of this quiz is to paint each person into a box. The phrasing of the questions painted me as a strict conservative when in fact I am far more moderate. Perhaps a bigger problem may be that there are too few boxes. There are socially conservative liberals who simply feel that they should not impose those views on others so they vote for liberals and there are economically liberal conservatives who view moral issues as overriding any economic disagreements they may have with conservatives. I understand what they are saying about the questions, but the boxes they put people into are not nearly nuanced enough.

    1. Anonymous8 months ago

      Agreed it is a lazy quiz

  2. Lisa QUIGLEY-Moon2 years ago

    Even tho I’m okay with my results, I did find a couple of questions that I felt in between the two options. I guess they we loud have still netted the same result. I think the 12% is wrong. I know tons of people who are still very conservative who neve poll & some who (to my Bewilderment) never vote

  3. stan2 years ago

    talking out both side of your face

    Your poll give macro results while you play it up as a micro poll (why typography are you?)

    your poll shows how these things average out on a macro – all results – basis, but that’s not what you were advertising

    humans are way too complicated to fit into extremes

    you talk about leaning one way or another on an issue but then only offer extreme/either of option – “leaning” is not “extreme.”

    do a survey on a sliding scale and get a truer picture – if you want the individual to get a clear picture of their political typography, then offer more answer options

    your results work if you want to look at people groups as a whole but it hardly works for the individual so in that sense it in disingenuous

    simplistic and not at all an indicator of who I am

    1. Linda2 years ago

      My sentiments exactly Stan. This poll provided no additional insight for me on an individual basis. Being FORCED to make a choice between two extremes DOES NOT reflect what I believe on a personal level. It’s like asking whether I prefer to be killed by a car plowing into you or drowning in a pool…uh, how about neither! I’d rather live. But if I ONLY have these two choices I guess I’ll go by car.

  4. Bliss732 years ago

    This is obviously put together by a liberal group. You are not looking for truth – but have structured the questions / answers to extremes to further your agenda. I do not find your research to be a true accounting of how people really think and therefore null and void of the truth.

  5. Donna Thompson2 years ago

    I was surprised at the “Hard Pressed Skeptic” as my outcome. I consider myself as a member of the “Tea Party” which leans toward the Republican Party. Since the Tea Party is in it’s infancy as a recognized Party, maybe another describer needs to be added to the Pew Research Center.

  6. Zogspot2 years ago

    “The intent is not to put people “in a box” but rather to understand how their values across multiple political dimensions are related to each other.”

    Huh? How can you possibly understand how my “values across multiple political dimensions are related to each other,” if you haven’t actually found out what my values are? You want me to choose left or right when my answer is apple. You pretend your purpose is not to put people “in a box”, but all you have are boxes. You pretend that you’re trying to find positions not on the usual right-left spectrum, but all your questions assume a single right-left dimension.

    Then you try to ascribe some value to randomly putting people in these boxes because you have a 20 year history of hammering round people into square boxes. I hope no one actually uses this stuff for anything real.

  7. Public Opinion Student2 years ago

    This quiz is everything that I dislike about Pew wrapped up into one little 23-question package.

    The stances are so badly matched with rationales in each question that it’s laughable to believe this measures political opinion. Why not simply provide the stances with no rationale? Nowhere in this quiz does a question ask the respondent to choose between the same stance based on two different rationales. So, why is it necessary at all?

    These quiz results belongs perhaps on the homepage of Fox News, not on a supposedly respected public opinion data provider. Frighteningly bad research design, Pew– I’m truly disappointed.

    1. Catie Lott2 years ago

      I agree.
      The questions are unanswerable, presumptive and suggestive, too.
      Pew should hire someone who writes better questions or accept essay answers.

  8. walt brinkman2 years ago

    These statements are too extreme and do not allow for ACCURATE assessment of the position the participant wants to express< and that is also why we are in gridlock! you have to be in one extreme or the other! Compromise is not understood because it is too deep for the extremists!

  9. Tom Sharkey2 years ago

    Just wanted to make one other point ….after reading the comments posted prior to mine I thought it important to convey that “Zelig’s” point (about having a 5or7 point scale between the 2 extreme statements) sounded helpful….as having more information rather than less would be the logical way to go when your in the gathering information business.

  10. Tom Sharkey2 years ago

    Thanks Jocelyn for the explanation for how the questions are worded…..I too found choosing between the extremes quite exasperating. Glad however that I took the quiz….as where one is on the political spectrum can feel a bit fuzzy at times. Was glad to see that I was consistent in my “values”.
    Take Care
    Tom S.

  11. Ben Rhodes2 years ago

    I may have been part of this poll, I am not sure. The one question that I remember most is about immigrants. I am really offended that there was no differentiation between legal and illegal immigrants. There is a difference, you know. If this is the poll, there is an obvious lack understanding of true evangelistic beliefs. News flash, we are not religious. Religion is an invention of man, not God. I have a strong relation with Jesus, but I have no belief in religion.
    I realize you might have a difficult time understanding this. Maybe this is because you don’t know Jesus. I assure you there is a difference.

  12. Zelig2 years ago

    This is an interesting argument, and I see it in question wording in a number of surveys. However, in the end I am not convinced that this is the best way to go. The use of extreme paired statements means that a large number of people had to select an option that is incompatible with their true preferences on this issue. My suggestion is the use of a five- or seven-point scale anchored at either end by the two extreme responses.

    One other point: I wonder if the use of the paired question format creates a greater likelihood of polarization in responses. If I am asked to answer a question with two extreme options, I will select the option that is closest to my viewpoint. If I am a consistent moderate liberal, that means that under the current format I will select the liberal extreme each time, even though it does not reflect the intensity of my feelings about a wide range of issues. With a seven-point scale, I might have answered with responses more to the left-middle of the scale; hence I would not be counted as a hard-core left winger with the seven-point scale (or something like it), but I am classified as a hard-core left winger under the current format.

    In my experience it is almost always better to get more information rather than less.

  13. taxpayer2 years ago

    You designated me a “young outsider,” a group of whom you say the majority voted for Romney or Obama in the last Presidential election, which I certainly did not. And I am of Medicare age. Fact is, on many of the questions I could nearly as well have picked the other option, both options including assertions or assumptions with which I strongly disagree.

    1. Jose Sanchez2 years ago

      Can anyone name one question where they strongly disagreed with both options? It would be interesting to know why.

      1. Catie Lott2 years ago

        I strongly disagree with option 1 & 2 of the sample question.
        #1 0% agreement
        #2 14.7% agreement.

        I agree with 5 words.
        The Poor have hard lives, because struggling to survive is hard.