September 30, 2013

Study: Opposition to same-sex marriage may be understated in public opinion polls

About half of all Americans (52%) believe that same-sex marriage should be legal in all states while 43% are opposed, according to a recent survey by the Gallup Organization.

But a new study suggests that opposition to same-sex marriage may be understated in public opinion polls. Using pre-election polling data in states that have voted on same-sex marriage measures, political scientist Richard J. Powell found that pre-election surveys consistently underestimated opposition to these laws by 5 to 7 percentage points.

Blame “social desirability” bias—the tendency of people to give what they believe is the socially acceptable view rather than disclose their true feelings about sensitive topics, wrote Powell in an article to be published in the journal American Politics Research.

Powell, who teaches at the University of Maine, examined all 33 statewide races from 1998 to 2012 where same-sex marriage measures were on the ballot and where statewide pre-election polling was conducted. In all but three of these states, the ballot question was presented as a ban on gay marriage. He only examined polls conducted within 90 days of the election.

He compared pre-election polling estimates on same-sex laws as well as other ballot questions with the actual vote. That allowed him to compare the accuracy of polling on same-sex unions with that of results on other ballot questions, a check to see if any difference was specific to the gay marriage measure.

He also controlled for other factors that might account for differences between polling results and the vote on same-sex marriage, including the amount of time between an individual poll and election day and the share of respondents who expressed no opinion or were undecided.

Detecting social desirability bias in surveys is difficult. To strengthen his claims, Powell took elaborate precautions to control for factors that might be related to “the prevalence of social desirability bias in issues of sexuality.” They included factoring out the influence of education (other studies found social desirability bias more prevalent among better-educated adults), the partisan makeup of each state, and the percentage of a state’s residents who described themselves as “very religious” in Gallup surveys.

Powell compared the results of polls and the vote on “a wide range of non-same-sex marriage ballot measures that appeared on the same ballot with same-sex marriage issues.” He found no pattern to the differences between polling and the vote; in fact, the polls performed well and ”the gaps tend to cluster around zero in a relatively random fashion, as we would expect according to statistical sampling theory.”

Then he did the same comparison with same-sex marriage results and found “pre-election polling has almost always understated election day opposition to same-sex marriage. In fact, in only 9 of the 59 polls did election day opposition to same-sex marriage lag behind the pre-election polling,” he found. Depending on which statistical model he used, Powell found the gap to be 5 to 7 percentage points, which he said is clear evidence—but not proof—of desirability bias at play in attitudes toward gay marriage.

During the period under study, public opinion on this topic moved dramatically. Between 2001 and 2012, the proportion of Americans who favor allowing gays and lesbians to marry legally increased from 35% to 49% – a reality confirmed by the fact that in 2012, the four same-sex marriage initiatives on state ballots were successful, breaking a long losing streak.

Social desirability bias in polling comes in many flavors. Perhaps the most well known is the “Bradley Effect,” named after former Los Angeles Mayor Tom Bradley, a black man, who faced Republican George Deukmejian, who was white, in the 1982 California gubernatorial race. Bradley held a substantial lead in most pre-election polls, only to lose narrowly. His defeat fueled speculation that some white voters had given misleading answers to poll-takers, saying they supported Bradley or were undecided but really favored Deukmejian.

But it wasn’t until 2007 that Harvard political scientist Daniel Hopkins confirmed the existence of the effect. He studied elections between 1989 and 2006 that pitted black and white Senate candidates against each other.  He found that the black candidates polled better than their final share of the vote in contests with white candidates in elections through 1996. But the effect then vanished, for reasons that he said were unclear.

Category: Social Studies

Topics: Gay Marriage and Homosexuality, Polling

  1. Photo of Rich Morin

    is a senior editor focusing on social and demographic trends at Pew Research Center.


  1. the ugly truth3 years ago

    Gay Marriage supporters are very naive. They think it’s just about two people who love each other but it’s more than that. First of all, homosexuals can get the benefits without legalizing their marriages. They need only change policies. However, they want their unions to be legally recognized because it will open the doors of entitlement to other privileges at the expense of taxpayers no doubt. If gay couples want children outside of adoption, they can claim it’s a healthcare issue in order to get subsidized funds for medical procedures involving surrogates. If their marriages are legalized, it means they can sue a church or officiant for denying marriage services to them. It’s more than just because two people love each other. It’s the legal way to impact businesses (like adoption agencies).

    Remember when abortion was supposed to be about helping a down-trodden woman? Suddenly, taxpayers were paying for abortions through organizations like Planned Parenthood. Then there arose legal issues regarding when a fetus could be aborted and so on so forth. Then there were abortion doctors going to jail because of botched abortions and so forth.

  2. Madison3 years ago

    I think people are blowing this out of proportion. It’s there life they should be able to do what they want because when they die they die not the haters so just get over it. I’m not saying I’m with it or against it. I am a heterosexual female and I am in high school. This is mainly when people “come out of the closet” I have gotten used to the fact that one of my best friend recently sat me down and told me she was homosexual,she is in 8th grade. I told her the exact same thing I am telling you. It’s her life, I wouldn’t want to live my life hiding from everyone and lying to everyone. It’s better to be happy doing something than being sad and doing something you hate? I am right.

  3. Nim4 years ago

    Same-sex marriage is not an issue, people just choose it to be an issue. If one is going to marry another, it should be a decision made by the two people in marriage alone and nobody is supposed to mess with it. Same-sex marriage will not hurt others and also will make 2 loving people happy while preventing them would result in the 2 getting hurt.…

  4. dave bowen4 years ago

    I am an unmarried heterosexual male. The biggest threat to traditional marriage is divorce,not gays. Most of my friends have been married and divorced at least once,some twice or more. That come from treating marriage as if it was a tennis match. I have never been married,never will be. Why shouldn’t gay couples have the same right to be as miserable as the rest of you seem to be.

    1. Sam3 years ago

      I think that you don’t get to choose who you love therefore you should be wed to “the one” so to speak…. If it wasn’t meant to be it wouldn’t be. I am a strong believer in God and the Bible but I also believe that God makes everyone the way they are supposed to be for reason. I’m not trying to blame God for man’s free will but honestly it is their heart let them use it……. Fact and logic doesn’t always come in, in this situation, sometimes its just love, not reason.

      1. jackie3 years ago

        The heart is very misleading.The Bible teaches we are not to follow the desires of our heart . We are to follow a well trained conscience. We train our conscience by preying for guidance, studying the Bible and spending time in fellowship with other believes. We are all born with a conscience unfortunately, if not trained it is much like using a compass without the aid of a map. We need the other tools to make an informed decision.

  5. vince4 years ago

    It’s like pressing your wife to believe that sharing her with someone else in bed is good. She might express agreement and even act happily just to please you but inside she will feel miserable. That’s what all these polls never show. The majority of Americans have been persuaded to support SSM by artists who know what strings to pull, like H.Madsen with his book “After the Ball”. But leave them free of social pressure and you’ll get at least 65% against SSM.

    So the word ‘support’ in the case of the poll is misleading.

  6. vince4 years ago

    The author is right: there are different factors, a lot of people would never disclose their true feelings about it. I would say in reality there are only 25% of devoted supporters. But there are not many active opponents either, only about 35%. And the majority is simply apathetic and indifferent, such people often voting randomly just to be left alone but deep inside they feel something is not right. So it’s not that many people care about human rights of others, they just want to stay safe.

  7. Paul Silva4 years ago

    The Federal government has no business in the definition of marriage. Tax every individual equally. Government benefits should be available only to those with children. Leave all marriage issues to individuals and perhaps the States. Sorry married folks, but it shouldn’t be about the money.

  8. Wayne4 years ago

    Let me be clear: if anyone really thinks that there are more people in America that wants SSM than those that do not, you are really drinking serious Kool-Aid. All one has to do is to consider the following:

    1. No Bible Belt State will vote to approve SSM.
    2. The Conservative Mountain States will not vote to approve SSM.
    3. The conservative Mid-Western States will not vote to approve SSM.

    When you add all of this population together, you clearly have most of the 360 million people in America. If this was put to a nation-wide vote, SSM would be defeated easily in a major landslide! No doubt about it and that is why the gays run to the courts and to their liberal politicians. They know that once the people get behind the curtain in the voting booth, the vast majority will say Hell No!

    1. A Real Libertarian4 years ago

      Do you have any clue how little people those states have?

    2. vince4 years ago

      Unless southerners stop feeding on pop culture and media that are full of hidden propaganda 10 years from now they’ll be reprogrammed to accept SSM

  9. Alsee4 years ago

    There’s an alternate explanation that doesn’t seem to be considered. Intensity of motivation has a significant impact on voter turnout. Isn’t it possible, or even probable, that bigots have a 5-7% greater motivation to turn out and vote on election day?

    1. Alex4 years ago

      I considered that too, but I think that would be equally (or even overly) countered by strong supporters or those with homosexual inclinations themselves.
      Also, I don’t think that someone is a bigot for thinking that redefining what marriage means is beyond wrong, and not up to any government, especially not federal. Just because someone doesn’t agree with you doesn’t make them a bigot, much to the disbelief of most social liberals. I do not support gay marriage because the human body isn’t just something to lust after and pleasure, but something that is meant for bringing new life into the world. Other people have their own reasons. I don’t hate anybody for having homosexual inclination in the least, but I honestly do not believe that acting on those inclinations is good for those individuals.

      Every voice matters, and people should not be afraid to speak up, just because they don’t think their opinions are popular. If you stay in your shell and just accept everything going on, no one will even know you have a voice. Everyone should know how the country really feels about these issues.

      1. Alsee4 years ago

        Churches are free to preform, or not preform, whatever religious marriage ceremonies they wish. If they don’t want to preform interracial marriage ceremonies, they don’t have to. If they *do* want to preform a wedding ceremony between a toaster and a cow, they are free to do so. But we’re talking about the government, the law. And I assume you accept that the law should not be discriminating between people on the basis of race, religion, or gender. There is no way for the law to exclude interracial marriages without discriminating between acceptable and unacceptable marriage applicants on the basis of their races. There is no way for the law to exclude interfaith marriages without discriminating between acceptable and unacceptable marriage applicants on the basis of their religions. There is no way for the law to exclude gay marriages without discriminating between acceptable and unacceptable marriage applicants on the basis of their genders.

        Someone can list all the reasons they don’t like interracial marriage, can explain they “honestly do not believe that acting on those inclinations is good for those individuals”, but they can’t write racial, religious, or gender discrimination into the law.

        As for people speaking up, sure. I fully support the right to free speech. I’ll stand up for the KKK’s equal right to a legal parade permit, but I’ll damn well be in the front line calling them evil asshats when their march is approved. People who oppose interracial marriage can speak up, and I will speak up as well, and I’ll call them bigots. They have full legal protection to speak those views, but I’m certainly not going to welcome them at my dinner table. I’m going to avoid socializing with them. And I’m going to avoid putting any of my money in their pocket. People have a protected right to free speech, but there are social consequences to advocating bigoted laws, or advocating for a theocracy, or any other offensive position.

        And like it or not, this war is over. Interracial marriage gained acceptance in public polling at a rate of almost exactly 1% per year. Gay marriage is currently at about 55% support 39% opposition, that support is rising almost exactly twice as fast as interracial marriage gained accepted. It’s been going up by 2% per year. And if you examine the polling demographics, acceptance of gay marriage is completely overwhelming among the younger generation and the primary opposition is among seniors. Exactly like the demographic split went with interracial marriage. The younger generation sees gay marriage as a civil rights issue, just like interracial marriage. The war is over, there is NOTHING that can stand against the force of a generational shift. The younger generation automatically win by the fact that they are literally burying the older generation opposition day by day.

        With gay marriage support at 55% and rising 2% per year, and utterly overwhelming support as a civil rights issue among the younger generation, diehard gay marriage opponents are quickly going to find themselves as unwanted at their children’s Thanksgiving and Christmas family gatherings as that embarrassing old family member who keeps dropping racist comments.

        Like it or not, it’s impossible to halt this social and generational shift. Either figure out how to deal with it and accept other people’s legal right to chose who they want to marry…. or don’t and get used to being Archie Bunker.

        1. Cathaleen Marie Fletcher4 years ago

          All I can say is very well said!
          For those of you who believe this is liberal propaganda please think again and rethink your Sunday school lesson. I am a Southern Libertarian woman who has been married 20 years. I have raised 4 kids. I am not gay, no kids are trying to get married and finding it hard in the Bible belt, what I am is compassionate to others and their rights to pursue happiness as they see fit.
          Hate comes in many forms and when we let it in it takes over and the devil loves hate. I for one believe that I am not able to judge the Bible taught me that. It said for me to love and that I was to talk of God and Jesus and if someone was committing a sin let Jesus enter their heart and he would work with the father to heal that person not me for I am a sinner.
          SO I will love and let others live without judgement and work towards peace in a world where my grandkids might know the meaning of kindness without knowing hate.
          Blessed be to those who believe in peace.

          1. Mark2 years ago

            It’s not just a Sunday School lesson. You are talking about changing a fundamental aspect of civilization that is way older than this entire country.

            Speaking of what is right, it is not right to judge an individual for sin, because we are all sinner, but it is right to judge a behavior that is deemed to be sin itself.

            Hate the sin, not the sinner. They should have taught you that in Sunday School.

        2. k dub2 years ago

          Everyone has the exact same marriage opportunity. Marriage rules apply the same no matter your orientation. But if you spin it to claim a person should be allowed to marry anyone they want, you gave to allow incest marriage. For the argument for ssm applies to them. But if you take a incest marriage poll very few would be in favor and even gays would be against it.

  10. ld4 years ago

    Gay male or female is wrong it is a perversion of what is normal. Marriage is one man and one woman before God and nothing else.

    1. Ash4 years ago

      Actually nobody is normal. We all have our differences. What is normal? I stand up for our rights. I believe almost all the same things you do. We are all made up of the same major and vital organs. Male and female. Is it normal to love? Yes! Is it normal for us to want to marry the person we love and start a family with them? Yes, its all normal. It doesn’t matter if some one falls in love with another person of the same gender. Love is Love and love knows no gender.

      1. Aksu J.1 year ago

        I guess then its ok for people to conduct incest marriages and even marry animals.. Love is love and knows no limits… I’d like to live to see how far people will try to push the limits of marriage… Sad how this world will end up

  11. Thomas R4 years ago

    It makes sense there would be a social desirable influence. Generally speaking those deemed “socially desirable” (celebrities, artists, writers, the “creative class” generally, and to some extent unmarried urbanites under-35) strongly or vastly support same-sex marriage. In the entertainment media opposition is virtually non-existent even among actors I know are Republican or conservative.

    Instead opposition or just skepticism is often from those who are less “desirable” in terms of marketing and media image. (Regular churchgoers who aren’t part of a “liberal” Protestant denomination, rural people, married people over 55, poor people) These people might be seen as lacking much disposable income and creating or inventing little so less “socially desirable.”

    Anyway that it is seen as socially desirable, now, to be for same-sex marriage means it likely will actually increase. If people think they should believe in same-sex marriage, but don’t, the pressure would appear to be more for them to actually start believing it than to openly reject it.

    1. vince4 years ago

      It’s like pressing your wife to believe that sharing her with someone else in bed is good. She might express agreement and even act happily just to please you but inside she will feel miserable. That’s what all these polls never show. The majority of Americans have been persuaded to support SSM by artists who know what strings to pull, like H.Madsen with his book “After the Ball”. But leave them free of social pressure and you’ll get at least 65% against SSM.

      So the word ‘support’ in the case of the poll is misleading.

  12. mungusema4 years ago

    That is to say 52 % of residents in the USA have signed the desolation of what was the United States of America that had a foundation on her creator. North America is finished! A new world leader will soon emerge because USA has rejected herself as the leader and God surely will abandon them on all sides. Unfortunate. No more part or lot in America!!

    1. Jeffsfla4 years ago

      mungusema…please dont say things like this. America is the greatest country because we have always been a melting pot. The only people who do not believe this is religious zealots who want everyone to act and pray as they do. GOD does not take sides between countries and if you believe that how come there is not only one country?

    2. Thomas R4 years ago

      I feel it’s more a sign that

      1: The entertainment media really is a powerful influence on perception. This should be a “no duh”, otherwise why have advertizing, but for whatever reason the media tends to deny or downgrade their propaganda ability. (Generally speaking entertainment shows do not debate this issue, they tell you you should support same-sex marriage and why)

      2: Equality and interchangeability are increasingly important to people. That men and women are identical, even in the purpose of whom they can marriage, might seem absurd or revolting to traditionalists but has been praised in many contexts since the 1970s. So it’s possibly, in an odd way, a fulfillment of feminism.

      3: Civil marriage is increasingly divorced from marriage more widely. I think I’ve seen polls that show some who don’t think of a same-sex marriage as a marriage still support SSM because they feel a civil-status can’t be biased in the way traditions or reality might be. As a Catholic I’ve always thought civil marriage was largely invalid or a “piece of paper” but a focus on its use as a status might well be increasing. If it is merely a government status than arguments for differentiation might seem ridiculous or unfair.

  13. John Mitchel4 years ago

    Why would you be analyzing polls from 1998 in the first place. Those results were taken from a population over a decade ago. People are starting to shift their views on this issue as consistent polling numbers have shown since 2010. A handful of states including Oregon, Nevada and Colorado would vote differently if they were presented with the same question today.

  14. Marco Luxe4 years ago

    While the social desirability effect may be real, it doesn’t support the idea that polls are bad predictors of a trend. On the contrary, it may be evidence that voters know that it is only rank prejudice that is motivating them. It may also support the idea that political “progress” in the direction of the polls is inevitable because people will unlearn their prejudice through further social interaction, the very thing that motivated them to lie when polled in the first place.