September 23, 2013

The challenges of counting the nation’s unauthorized immigrants

For Updated Methodology

A more recent discussion of our methodology for estimating the unauthorized immigrant population is available here.

In a report released today, the Pew Research Center estimates that there were 11.7 million unauthorized immigrants in the United States in 2012, compared with an estimated 11.5 million in 2011. But there’s a lot more to those numbers than meets the eye, and they’re not comparable to previously issued Pew Research estimates. To help explain how the new numbers were derived and how to interpret them, we spoke with senior demographer Jeffrey S. Passel, who’s been researching the unauthorized-immigrant population for more than three decades. An edited transcript follows:

Explain your basic approach to estimating the number of unauthorized immigrants.

The basic method we used is the same as before: We make an estimate of how many immigrants are in the country legally, and we have a government survey that measures how many immigrants total are in the country. Then we subtract the legal immigrants from the total.

Sounds simple.

Well, it’s a bit more complicated than that. We get numbers on legal immigrants from two federal agencies: First, we get a count of people admitted legally to the U.S. with so-called “green cards” for permanent residence from the Department of Homeland Security. Before the DHS was created, the numbers came from the Immigration and Naturalization Service. We know what countries those immigrants are from, how old they are and a lot of other demographic characteristics. The other group of people the U.S. admits are refugees; we get these counts  from the Office of Refugee Resettlement, which is part of the Department of Health and Human Services. Then we use those counts to develop a population estimate, using standard demographic techniques that account for deaths, departures and new arrivals each year.

The estimates of the total number of immigrants come from two Census Bureau surveys: the Current Population Survey or CPS and the American Community Survey or ACS. Both ask people where they were born, whether they’re citizens and so forth — but not whether they’re here legally. From those, we get a number for what’s called the “foreign-born” population. The big difference between this report and our previous ones is that before we used only CPS data; this time we’re using the ACS data for 2005 through 2011 and CPS data for 2012 and before 2005.

Why the change?

The CPS (which, by the way, provides the data used to calculate the monthly unemployment rate) is a large survey by the standards of the survey world — normally about 55,000 households a month. We use the CPS from March of each year, when they increase the sample size to 80,000 and expand the questionnaire to include questions that get at income and poverty and health-insurance coverage. But when you restrict it to just the immigrants and then look only at some immigrants, it can have pretty sizable margins of error.

So we started using the ACS, which is a very large survey: they end up interviewing more than 2 million households a year. The Census Bureau only started the ACS in 2005, to replace the old “long form” from the decennial census. This is the first time we’ve incorporated the ACS data into our estimates. We get access to a sample that works out to 1% of the entire country, or about 3 million people. Since the ACS is much, much bigger than the CPS, we get much smaller margins of error, so we can have greater confidence in measuring year-to-year changes in the unauthorized population.

But the 2012 ACS wasn’t available when we were putting together this report — the Census folks only released it last week, and we won’t get access to the individual-level data we need for another month or two. But data from the March CPS was available to us, so we used that for the 2012 estimate. At some point we’ll have a 2012 estimate based on the ACS that will replace the one in the report based on the CPS, with a smaller margin of error, and we’ll have a 2013 estimate based on the CPS with a large margin of error.

Using the ACS data wasn’t the only change you made this time around, was it?

No, we also re-weighted prior years’ data. You see, the population numbers you get from a survey are tied to annual population estimates the Census Bureau puts out. What they do is work from the previous census and update the estimates each year; then, every 10 years, they get a new benchmark with the latest census. Last year we didn’t have the 2010 Census as a benchmark, but now we do, so we have a much better idea of what the population looked like in, for example, 2009 than we had before when the estimates were based on the 2000 Census.

However, Census doesn’t go back and re-estimate the CPS and ACS data from past years when it gets the new decennial benchmarks. So we went back and re-estimated the data for every year from 2001 through 2009, based on the 2010 Census data. We do that because we think it’s important that our surveys be consistent over time to talk about year-to-year changes.

Similarly, CPS data before 2000 was not at all comparable to what came afterward, so this time we also went back and re-weighted all the data from the 1990s as well. So this is the first time we’ve released estimates for the 1990s.

In the graphs from the report it sure looks like the line showing the unauthorized-immigrant population is going up from the recent low of 11.3 million in 2009. So why do you say only that “the number may be rising again”? 

unauthorizedThese numbers come from a sample, as all survey estimates do. Surveys don’t interview everyone in the country; they only interview a sample of the population. The sample results may be different from what we would get if we interviewed everyone. Not only that, but the results can vary from sample to sample. That’s what we mean when we talk about a “margin of error.”

In the case of these estimates, the margin of error is pretty large relative to the year-over-year changes. What that means is that what appears to be a change could be just randomness related to the characteristics of the sample. We need to see a big enough change before we can decide that it’s real. For example, when we saw the first signs of a decline in the unauthorized population in 2007 and 2008, we couldn’t say for sure that there was a drop — we had to wait till we had the 2009 data because the CPS sample wasn’t large enough to detect the 2007-08 change.

With the ACS, though, we can say that the 2007-08 change was “statistically significant,” or outside the margin of error. A better reading of the 2011-12 change will have to await the 2012 ACS.

In last year’s report Pew Research estimated the number of unauthorized immigrants at 11.1 million in 2011. Doesn’t that mean there was a statistically significant increase between 2011 and 2012?

No. Because of all the changes in data sources and methodology this year, last year’s estimates have been superseded and aren’t comparable with the current ones.

Topics: Unauthorized Immigration, Immigration Trends

  1. Photo of Drew DeSilver

    is a senior writer at Pew Research Center.


  1. M B NEACE4 years ago

    Your implication is that if ‘everyone’ could be surveyed you have have more accurate information. As a retired professor that once taught marketing research I recall reading several scholarly articles that concluded just the opposite and I shared this with my students. How can Nielsen, who survey 1000+ households regarding TV viewing provide data that hundreds of advertisers believed was good enough for them to make very large $ decisions regarding how they purchased time and space? Those scholarly articles noted that surveying everyone makes huge assumptions about ‘just getting the job done’ plus the amount of error that would come with such a project. That larger the universe – the larger the error. Good sampling trumps every time.

    1. Tea Party+Immigration+Coalition4 years ago

      There are several problems with traditional sampling techniques. We outlined some of them in our comments herein. In addition, the rise of cell phones and unlisted and, more importantly, unanswered phones, means that one can no longer trust telephone sampling. Most recently, the 2012 election shows that Republican polls indicated that Romney would win. On election day, the only people who had some of the phone numbers were the ones who handed out free phones or had collected cell phone numbers previously in a door to door approach. The Republicans were out gunned in 2 ways; their polling was flawed and they had no way of learning the phone numbers of those turning out for Obama.

  2. Tea Party Immigration Coalition4 years ago

    We are shocked that Pew would repeat these phony numbers. There are so many problems with the methodogy or the lack thereof, we scarce know where to begin.
    First off, we have the testimony of many of our friends and members that illegal aliens will not speak to anyone having an official appearance, mein or even those asking questions. Therefore, if one were to ask an illegal Mexican anything about his immigration status, he would not answer. That, of course, would be especially true if the man or woman were a Gringo.
    We have in our organization members who were part of the 2010 Census. We have been told that the Census Bureau had many meetings around the country and attempted to deal with this non communicative style. We suspect that it was nationwide that Latinos, Muslims and Chinese were in particular non responsive.
    Our founder, John Stahl, was on a tv show in 2010 with a member of the illegal hispanic community in California. This man was an organizer and a fairly important person in that group. He stated on air that he was advising his people to not respond.
    We can only conclude that any sampling technique would be of no use what so ever.
    Next, we must talk about Social Security Administration (SSA) no match letters. The latest data we have is that there were 81/2 Million no matches sent out. Now if one takes off a few for wrong numbers and name changes due to marriage and then adds in the suspected number of those illegals working under the table, we believe that the resulting number approaches 11 million at the very least. We could make a coherent argument that the number is closer to 16 million. Nonetheless, neither figure includes the families of the illegal. We suspect that the multiplier should be around 3.5. That is, we suspect that for every illegal worker, there are 2.5 additional family members.
    Recentlly, In a survey of 100 companies(out of the nearly 1 million employers in the country) there were about 21/2 million no matches. That is the latest information we can find.…
    Thus the figure would be between 27.5 and 40 million.
    Thirdly, there is the Bear Stearns study. Using their methodology, one would get to something over 25 million.(…) puts the figure at over 20 million.
    Continuing problems: We should have SSA no match letter current counts. Additionally, the IRS should let us know the current number of Temporary Identification Numbers (TINs). These are isssued to folks who do not have a social security number. While we despise the whole idea of TINs, we do think this number combined with the no matches would begin to give us a saner answer. We would invite Pew to join with us in asking for this information and joining a freedom of information act request, if necessary.
    Other problems with the numbers. We have no idea of the number of illegals collecting welfare. We have estimates. If one uses the total number collecting food stamps and subtracts the number of longterm unemployed and long term pre 2008 welfare recipients, one gets a number of 35 million at the very least. There are some who say that the number is closer to 55 million.
    We should be requiring all these government agencies to report the actual findings. The various state and federal welfare departments are criminally negligent in not asking recipients for proof of nationality. Unless we change the law, this huge loophole will continue to attract illegal aliens as stale and rotting food attracfts vermin.

    In conclusion, we have long said that the figure is around 35 million illegals in this country, (Plus or minus 5). And it matters because every projection is based on the false and misleading 11 or so million. Thus, one should multiply every statistic by at least 3 if we are correct. Our very existence as a nation is at stake here. If we are correct, then having an illegal population at or above 10% is a very real political, economic and cultural danger. With 22 million un and under employed Real Americans, one can easily say that their jobs have gone to immigrants, legal and illegal. That is even more dramatic when one considers the enormity of the real number.

  3. Martin Vega4 years ago

    I would be interested to know if in his tabulations, Passel took into the account the following:

    In the methodology section of the paper in adjustment for undercounts, Passel states that he used “rates for countries of birth based on the predominant race of immigrants from the country—Hispanic and non-Hispanic races for white, black and Asian”. It is perplexing that he does not appear to have incorporated undercount estimates for the “indigenous” population from Latin America, particularly from Mexico, since it is widely acknowledged that this population is not effectively captured by any of the current Census surveys: Decennial, CPS or ACS. The U.S. Census’ own post-Decennial qualitative research shows that they do not respond well either to the ethnicity question (Are you “Hispanic” or “Mexican”?) or the race question (Is the person [surveyed] American Indian or Alaskan Native, translated badly in Spanish to “India Americana). In addition, data from the Colegio de la Frontera Norte shows that a growing share of Mexico’s undocumented population (38% from 2005 to 2010) hails from the country’s indigenous Southern states, including Oaxaca, Guerrero, Puebla, Veracruz and Chiapas. Did he factor these developments into account?

    Since 2000, TRAC from Syracuse University has published the growing number of deported immigrants who have been deported and been prosecuted and imprisoned for illegal re-entry felonies. As more and more Latinos (and indigenous) with deep roots in the U.S. are deported — persons residing in the U.S. five or more years; heads of household with U.S. born children, etc. – these numbers will continue to increase. Did Passel incorporate these numbers in his estimates of the unauthorized population in the U.S.?

    In the past three years, roughly 45,000 migrant children unaccompanied by an adult have been apprehended by U.S. officials. Because they fall under the purview of the Office of Refugee Settlement, U.S. Department of Health and Human Services, are these undocumented children accounted for in the estimates?

    Finally, numerous GAO reports as well as a 2011 RAND study commissioned for Homeland Security have critiqued the fact that U.S. Customs Border Patrol (CBP) apprehension-level data is an imprecise method and a poor proxy to estimate unauthorized inflows. Thus, it begs the question whether Passel formulated any estimates regarding the number of undocumented immigrants who “successfully” entered the U.S. through or between the legal ports of air or land entry.