An Interview with Joe Lenski
In an exclusive interview, Joe Lenski, co-founder and Executive Vice President of Edison Media Research discusses the special problems in conducting exit polls this campaign season given the unprecedented clustering of primaries and caucuses in the first two months of 2008. Lenski is an expert in the operation and organization of survey research and has been involved in every major exit poll conducted in the last decade for the television networks and the Associated Press. Under his supervision, in partnership with Mitofsky International, Edison Media Research currently conducts all exit polls and election projections for the six major news organizations — ABC, CBS, CNN, Fox, NBC and the Associated Press that participate in the National Election Pool (NEP).
Interviewer: Andrew Kohut, President, Pew Research Center
The following transcript has been edited for length and clarity.
My first question is a simple one:How many primaries and caucuses will the NEP cover this year?
That will depend upon the competitiveness of the primary season.The members have already made the commitment to cover every primary through February 12.They’ll make decisions about the primaries after that some time in January and February.So as of now we are planning to cover 23 primaries and 2 state caucuses-Iowa and Nevada- including all of the primaries through February 12.
So everything is covered on the big February 5 day?
All the primaries are covered on February 5th day.But the only caucuses we are covering are Iowa and Nevada and there are a few caucuses on February 5, like in Idaho and Colorado, that we aren’t covering.
The big question I see in the current situation is what challenges have arisen given the late setting of dates for Iowa, New Hampshire, and other races?I can imagine that the issue is recruitment and training.
We usually need about an 8-week head start on recruiting. It was a little nerve racking until Thanksgiving waiting for New Hampshire and Michigan to set their dates. Also Massachusetts moved its date up at the last minute to February 5. So there was a lot of jockeying around, which kind of played havoc on our schedules, but in terms of the actual work, we were right on track.We have already done all the research and the picking of samples, etc., and we are pretty close to fully hired in Iowa and New Hampshire.We’ll begin our in-depth training next week in both places, and then we are on track to be ready for the February 5 dates.
The real issue with the timing though is how close Iowa and New Hampshire are to the Christmas and New Years holidays.That will, I believe, raise some issues, because you can’t just call up and train people on Christmas Eve or Christmas Day or New Year’s Eve or New Year’s Day, and we still need them to show up in Iowa two days after the start of the year.So that’s one thing that we are going to keep an eye on.But we are going to start training earlier than usual, and hopefully people will still remember what they learned after they eat a lot on Christmas Eve and drink a lot on New Year’s Eve so that they are ready to go on January 3.
In the case of Iowa and New Hampshire, are you sending representatives out to brief interviewers in person or are you doing it by phone?
This is all done by phone.Every individual also gets a training video to watch so they will have a training manual, a training video, a training phone call, and then a rehearsal call to make sure they have learned what they need to learn before the entrance poll or exit poll.
Are you subcontracting any of this in Iowa and New Hampshire?
No, we are doing it all in house.In the last two or three years, we’ve been doing a lot more commercial exit polling at other types of venues — sporting events, shopping malls, concerts, airports, truck stops, plasma centers, and so on.. So we use these interviewers now not just on election or caucus nights, but also for commercial exit polling year-round.
How many locations will be part of the sample for the Iowa caucuses?
For the entrance poll, there will be 40 locations for each political party:40 Democratic locations and 40 Republican locations. In 2004, we did 50 precincts, but that was just on the Democratic side.So the numbers are increasing to 80 total, but split between the two parties.
And do you poll voters both before and after they vote?
No, in the caucuses we just interview participants as they are going in and in the primaries we only interview voters after they leave the polling place. There is a related and important point to make here with respect to the Democratic side. You know the way Iowa and Nevada rules are on the Democratic side.They have this process where they do an initial preference. Then if the group supporting a particular candidate doesn’t make the threshold – say it gets less than 15% in the initial tally — in the second round the members of that group join other groups supporting candidates that are still viable and then they select the delegates to the next level.So what we are measuring on the Democratic side in Iowa is people’s initial preference. As a result our tally is likely not to turn out exactly the same as the final delegate selection and we will try to communicate that when we present the Democratic entrance poll results.On the Republican side it is easier because it is just a straight straw poll.
Do you ask them a second choice?
Yes, we will on the Democratic side.
How many precincts will you do in New Hampshire and elsewhere?
We’re planning to do 50 in New Hampshire. We haven’t determined the final number in other states.
You will be doing a lot of primary polls in places like California and New Jersey, where you don’t have a track record from which to estimate turnout in a competitive environment. What special challenge does this present?
We use the same methodology in trying to predict regardless.Turnout, as you know, is very volatile in primaries even under normal conditions.On the Republican side, many – in fact, most — of these states have gone eight years without a competitive Republican primary. And in some of these states, you’ve got to go back to 1996 or even earlier to find one.So what we look for are other competitive statewide primaries that may have been held in these states over the last eight years — it might have been a competitive Senate or governor’s primary, or even maybe a lieutenant governor’s primary.So at least we get some historic measure of turnout in primaries.Of course, presidential primaries do tend to have higher turnouts than state primaries if they are competitive and especially if there is a lot of attention paid to the candidates. But at least state primaries give us some idea of where Republican primary voters tend to come from.
I can’t remember when there was last a competitive presidential primary in California. In New Jersey, too, it’s been a very, very long time…
In New Jersey on the Democratic side you’d have to go back to Mondale-Hart in ’84 to get a semi-competitive primary.But you do have a fairly recent Corzine-Florio Senate primary that drew a lot of attention and high turnout.So you can use that as kind of an indicator of where Democratic primary voters come from. Remember that in these places where the Democrats and Republicans are voting at the same precincts on the same day — which is most of these primaries — we have to pick a sample that works on both the Democratic side and the Republican side.So we’re picking a sample based on past total vote in the general election, but then we do the analysis based on the size of turnouts in past primaries. We’ll know which precincts historically and on Election Day have more Democrat than Republican primary voters and vice versa.So using that historical information and the current information, we’ll be able to make those adjustments.
In the general election polls, there has been some over-estimation of support for Democratic candidates in recent years.Is there any consistent pattern of overstatement in primaries? For example, do exit polls tend to over-estimate or under-estimate late surges or front-runners or anything like that?
Joe Lenski: Well, that is really the $64 million question on a lot of levels.Let me try to tackle it piece by piece.On average there has been a slight over-representation of Democrats in 2004 and 2006, but unfortunately for us pollsters it has not been consistent.There are certain states where the overestimation is higher, and certain states where it doesn’t exist.And it’s not even consistent election to election: Pennsylvania is a good example.We had a high Democratic Kerry over-statement in the presidential election in 2004, but no over-statement at all in the 2006 Senate election in Pennsylvania.It’s consistent on average, but unfortunately it is not consistent state to state and year to year.If it were it would be a lot more predictable and we could deal with it better.
[Voter News Service]
So that’s the pattern we are going to be looking for, but again we don’t have a lot of historical evidence on that.And each election, as I said, tends to be different and we will probably not find out what the patterns are this year until we start having actual elections.
Are any changes in methods contemplated for the primary polls compared with four years ago — either something that you learned in the presidential election or in the Democratic primary polls back in ’04?
Yes, in the report that Warren Mitofsky and I wrote after the 2004 election we stressed issues involved in hiring, training and recruiting interviewers.All those changes were put into effect in 2006 and did help quite a bit in terms of increasing the age of our interviewers and dealing with the issue of younger interviewers not getting as high a response rate from older voters as from younger voters.
So those improvements have already been put into place and they will be in place again for the primaries.The only methodological change we are making this year is the addition of an adjustment for non-response in the two entrance polls in Iowa and Nevada.In the past, because it is such a compressed period of time in which you can interview the attendees before the caucus starts-we basically have 60 to 90 minutes to interview the rush of people as they come through the door-we’ve never recorded the demographics of those who have refused to answer the questionnaire.There is a lot of evidence that this year there is going to be a real age divide, especially on the Democratic side with Barack Obama leading among younger voters and Hillary Clinton leading among older voters.There is also evidence that younger voters are more likely to respond to both our entrance and our exit poll questionnaires than are older voters.
So we are going to add the recording of the demographics of the refusals for the entrance polls in the Iowa and Nevada caucuses as we already do in the primary exit polls. This will add a little work for our interviewers but it will allow us to adjust for that effect on our entrance poll.
Why do you think that younger people are more likely to respond to exit polls than older people?
Mainly that’s because, a lot of times, we are doing these interviews outside. And, especially with the schedule this year, the weather is going to be an issue in a lot of places. That’s another point I wanted to make about the challenge of this primary season — it is just so darn early, and in a lot of cold climate states, these primaries are going to be held at times when they have never held primaries before. Older people standing outside on a snowy day are probably not going to be as cooperative as they would be in better weather.
Do you find a differential completion rate by race?Race is going to be an issue too.
We’ve already been measuring for that in all the states with primaries where there may be a high racial component and we will do it again.That’s not going to be a major factor in Iowa, obviously, but in other states it could be.But racial differentials tend to be variable — not consistent like the age differential, which tends to be fairly consistent.
Let me do a little bit of follow-up.My concern remains that given what may be a differential age pattern, or gender pattern, or race pattern, not having a strong history of demographics from past primaries, you are going to be struggling to interpret what you get. My concern is that in all of these remote, well not remote, but less familiar places, you don’t have what you need to interpret your data when something unexpected – like a whole lot more women than men – shows up in it in some precinct or other.
Well, Andy, exit polls and entrance polls are different from telephone surveys in that they’re never weighted to any known demographic.We pick the precincts and just weight to the size of the precincts and the past voting record of the precincts.The theory here is if the precinct sampling is correct, if the interviewing sampling within the precinct is correct, and the adjustment we make for the non-response based on age, race and sex is correct, it will take all those things into account and come up with the correct demographics.But you’re right; there isn’t a lot of past history to compare to.
How much higher do you expect your refusal rate to be among Republican primary voters vs. Democratic primary voters, say in a state like New Hampshire?
That is an interesting question and there is no real way to know because we don’t have a past history on that because on Election Day, unlike in a Dr. Seuss story, people do not have a D or an R stamped on their forehead so we can tell which party they belong to.I mean, we will be able to do some analysis after the fact, but on Election Day itself we’ll know the overall response rate, but we’re not going to know the partisan response rate in most places. Of course, when there are separate samples as there will be in Iowa, in Nevada, and in South Carolina where the parties hold their primaries on different days, we will know the partisan differential.But in the other states we won’t. We can make some guesses but we really don’t have any historical or even a lot of current data to back that up.
Are your clients going to observe the same security procedures to restrict leaks of the results of the first and second waves of exit poll data as they did in 2006?
Yes, we thought that the security procedures worked well in 2006 so the so-called quarantine room will still be in operation for all the primaries.
How many partners do you have at this point as opposed to independent subscribers?
The six members of the national election pool are ABC, CBS, CNN, Fox, NBC and the Associated Press, and representatives from those six members will be in the quarantine room until the release time.
Are you getting good participation from subscribers this year?
Yes we are getting many subscribers, especially for the key primaries and caucuses.
Now, what is the most important question that we have not asked at this point?What would you most want to tell an audience that is not made up entirely of researchers but is pretty savvy about politics?
Well I mentioned it before, but I always get a lot of response when I talk about the weather of all things.These primaries are being held on dates they’ve never been held before.Now in some of the states like Iowa and New Hampshire, they are used to early January and February caucuses and primaries and they’re hardy souls who want to go out and participate, so they will probably be there.But take other states, like Michigan. It has a January 15 primary now, and I just don’t know if a snow storm hit the upper peninsula of Michigan, what kind of turnout we’ll have.Same with New York on February 5. If there is a foot of snow in Buffalo, are people actually going to go out and vote?And it also affects our interviewers because a lot of times they are standing outside the polling place and have to deal with the elements.So, I think the weather is a real wild card in this.You saw it last week when snow storms hit Iowa and New Hampshire and the campaigns themselves had to scramble because their candidates couldn’t fly in.So I think you’re going to have weather being a factor to scramble a lot of the scheduling and planning for the candidates, and for the voters, and for us.
Will you have to deal with absentee ballots in any of the primaries?
Yes. Not in Iowa and Nevada where they have caucuses, obviously, but in states like California, Tennessee, Florida and Arizona we would expect to have a third to a half of the votes be cast by mail.
Are you then doing phone surveys?
In the states with significant history of absentees, yes.
In California for example, how many home interviews will you do to find people who voted early? And when will you start?
The state absentee telephone samples will include 600 absentee and early voters in each of the states where we are conducting absentee surveys. We’ll do the phone interviewing during the week before the election.
That’s yet again another wrinkle…
There are a lot of wrinkles here.What makes the primaries and caucuses especially interesting is that basically each state has a separate set of rules with no consistency from state to state. There’s a lot of stuff for us to keep an eye on, with caucuses, different rules on reporting, whether you’re counting a straw poll or delegate numbers.You’ve got ballot issues in Michigan, you’ve got four candidates-Obama, Edwards, Biden, Richardson-withdrawing their name from the ballot, and you don’t know how that will affect the race there.
Here’s an interesting side bar: Those four candidates withdrew, but “uncommitted” is on the ballot; and it won’t be surprising if those four candidates urge their supporters to vote for “uncommitted.”So the race in Michigan may be between Hillary Clinton and “uncommitted” — which will be interesting to report.
And then there are all the different delegate selection rules on the Republican side, from state to state.Many of the big states are winner-take-all, either statewide or by congressional district.That’s part of Giuliani’s strategy: If he can win winner-take-all states like New York, New Jersey and California even by small margins, he gets all their votes. And that might make up for states that he loses but where he receives some delegates because they are allocated proportionally. So the math is more like an electoral college on the Republican side, whereas on the Democratic side is all the delegates are allocated proportional to the distribution of votes.
How are you taking all this data in?You’re not using computer assisted interviews, right?
Not this year.That’s probably where it’s headed in the future, and we’ve done testing with handheld devices, but on the coming election days we will still use paper questionnaires and then telephone interview results to our telephone centers.
Guestimate for me — we’re not going to hold you to this, but readers might be interested –how many cases do you think you are going to process on February 5?
Probably between 40,000 and 50,000 interviews.
And how are you going to handle 40-50,000 interviews, Joe?
Well, we have two phone houses with a couple hundred operators that are working around the clock.And we stagger the calls so they don’t all come in at the same time during the day.And while we take our vote tallies from the full sample, we subsample the full questionnaires that we use for analysis purposes usually at a rate of 50%.So say we do 120 interviews at a precinct to estimate how people voted, we may only read 60 of those questionnaires in full over the phone.
Right, but all respondents will be included on some questions?
On the question of how they voted, yes.
What are the chances of your having trouble falling asleep on the night of February 4?
Well, I’m not really worried about February 4; I’m more worried about sleeping January 2 or January 7.The one good thing about the primaries is that we get a lot of practice before the really big day.And in non-presidential years, when we are doing just the general election like we just did in 2006, it’s like playing the Super Bowl without playing in the playoffs.You’ve got to be ready for that one day when the big wave hits you. In the presidential primaries, you’ve got Iowa, you’ve got New Hampshire, you’ve got Michigan, you’ve got South Carolina.Each week, every 5 to 7 days apart, we play a playoff game. We’ve got to be ready and the system’s got to be ready, but it’s just one or two states at a time.So by the time we get to February 5, the system has been pretty well tested. But you are correct that this will be the largest number of primaries held on one day in history.So, covering 16 state primaries on one single day is a historical first.
