the variability of small area demographic estimates produced by the ...
THE VARIABILITY OF SMALL AREA DEMOGRAPHIC ESTIMATES PRODUCED BY THE PRIVATE SECTOR
Ken Hodges
Claritas Inc.
Presented at the Annual Meeting of the Population Association of America, Chicago, Illinois, April 2 - 4, 1998.
Background The small area demographic estimates produced by private data firms are among the most widely used in the history of applied demography, but to many the data remain an enigma. Methodology statements range from informative to vague to nonexistent, and questions are often raised about how accurate such mass-produced numbers can be. Amid all the questions, considerable light has been shed on the private sector estimates. In 1982, Donnelley Marketing Information Services (DMIS) was the first to report an evaluation of tract level population and household estimates against census results.1 DMIS evaluated again after the 1990 census,2 and was joined by rivals Claritas and CACI.3 Although noteworthy, these nationwide evaluations made no direct comparisons between the various suppliers.4 Several independent investigations have compared the estimates of different suppliers. Most notable are the comparisons by the International Council of Shopping Centers (ICSC) in 1986, 1991 and 1995,5 and a 1992 study completed by John Pitkin.6 The 1991 ICSC and 1992 Pitkin studies were evaluations--computing error against 1990 census results, while the 1986 and 1995 ICSC studies were mid-decade comparisons investigating how different the suppliers' estimates were at various geographic levels. These investigations were useful, but limited to small numbers of rather large areas. The ICSC studies examined data for a small sample of metropolitan areas and counties, and for large census tract aggregations with populations usually in excess of 40,000. The Pitkin project made some comparisons at the census tract level, but also focused on larger areas, and was confined to the state of California. In the absence of a full comparison of private sector small area estimates, questions persist about how similar they are. One view is that methods and input data differ enough to produce significantly different results. Another view is that small area estimation capabilities are so limited that there is little difference between the estimates offered by the suppliers. Among the users of these estimates, one finds many saying they find no major differences among them, but at least as many questioning why one set is so different from another. This paper describes the first large scale comparison of private sector estimates, and an assessment of their similarities and differences.
A Unique Opportunity Claritas--one of the private data suppliers--is uniquely positioned to compare private sector estimates due to the acquisition activity of its corporate parent, VNU. Claritas had already merged with National Planning Data Corporation when VNU acquired Strategic Mapping Inc. (SMI) in May 1996. SMI (which had acquired DMIS) was completing its 1996/2001 estimates and projections at the time of the acquisition. VNU then acquired Urban Decision Systems (UDS) in January 1997, and National Decision Systems (NDS) in July 1997. Each firm had completed, and was providing its 1996/2001 estimates and projections when acquired by VNU.
1
Thus, Claritas has four sets of independently produced 1996/2001 estimates and projections--its own plus those of SMI, UDS and NDS. This paper reports a comparison of these four sets. It is not a hypothetical exercise, but a comparison of four actual sets of numbers produced by four then-independent firms. Each set was produced prior to VNU acquisition, and was in customer use over the normal shelf life of such data products. Like the 1986 and 1995 ICSC comparisons, the objective was not to determine which estimates were the most accurate, but the extent of variation among them. However, unlike all earlier efforts, this comparison examined current year estimates and five year projections for all 3,141 counties, 61,258 census tracts, and 266,399 block groups nationwide, as well as all residential ZIP Codes defined in 1996.
Methodology The methods used to produce the four sets of estimates and projections ranged from expedient efforts, requiring only a few weeks to establish county numbers and percentage them to block groups, to more ambitious efforts taking six months or more to incorporate small area input from numerous sources. A description of the methods used by each supplier is beyond the scope of this paper. However, methodologies and input data varied considerably, and specific differences are noted where potentially relevant to differences observed among the estimates.
The Comparison Design Data and Geographic Levels Comparisons focused on the estimates and projections of total population, population age 75 and above, and mean household income. Also compared were the suppliers' estimates of 1980 census population. It is not widely recognized that each supplier provides its own version of 1980 census counts reconfigured to 1990 census geography. Data were compared for all counties, census tracts (including block numbering areas), and block groups nationwide. Comparisons also were made for ZIP Codes, which are a non-standard geography, and merit a separate analysis. Measures of Difference The differences among four sets of population estimates can be measured many ways. This paper takes the perspective of the prospective data user who might be choosing between any two suppliers or from all four. The six pairs of the four suppliers are indicated below. Claritas vs. SMI Claritas vs. UDS Claritas vs. NDS SMI vs. UDS SMI vs. NDS UDS vs. NDS
2
For each supplier pair, the absolute percent difference (APD) between estimates was computed. Mean absolute percent differences (MAPD) were examined across all geographies, and in some cases, the distributions of APD were examined. Again, error rates could not be computed, but where comparable error rates have been reported for a census year, they provide a useful frame of reference for the MAPD values. To minimize the impact of extreme percent differences resulting from very small population numbers, differences were computed only for geographies where 1990 census population was 10 or more. When computing percent error, the difference between the estimate and the census count is expressed as a percent of the census count--the target against which error is being measured. However, when computing the percent difference between estimates, there is no basis for declaring one to be more accurate--or the logical denominator. For this reason, the denominators used in computing APD consisted of the mean of the two estimates being compared. The differences between supplier pairs suggest the uncertainty confronting a user choosing between two sources of data, and identifies the differences between specific suppliers. However, a user could choose from among all four suppliers, so it is important to measure the degree of uncertainty presented by the suppliers as a group. Differences for the suppliers as a group were based on the range between the highest and lowest estimate for each area. This measure was chosen because it is easily compared with the differences among supplier pairs, and because of its relevance to the user perspective. For example, if four population estimates were 950, 815, 809 and 804, the range between the high and low estimates is 146. Expressed as a percent of the mean of the high and low value (again, there is no basis for declaring either one the target), the APD is 16.6 percent. Discarding the middle values might seem questionable. However, in the world of mass produced small area estimates, the fact that the three lower estimates are similar, and the high estimate is an outlyer does not reduce uncertainty. The supplier estimating 950 might be the only one with a problem, but it could be the only one doing something right. As a data user, all one knows is that the suppliers suggest populations as high as 950 and as low as 804. The "range difference" of 16.6 percent reflects the uncertainty confronted by the user. Just what constitutes a large or small difference between estimates varies by application and user. This paper establishes no standards, but describes differences relative to all those observed. For example, mean percent differences in the low single digits were among the lowest observed, while those exceeding 20 percent were relatively high.
3
Results County Population County population estimates were similar for all suppliers. As indicated in Table 1, mean absolute percent difference (MAPD) ranged from a low of 0.6 (SMI/UDS) to a high of 1.7 (NDS/UDS). Such similarity was expected since all four suppliers based their county population estimates on the Census Bureau's annual county population estimates. Even the mean "range difference" (between the high and low estimates) was only 2.2 percent. The frequency distributions presented in Table 2 confirm that differences greater than five percent were rare--although the total within one percent ranged from 90.4 percent (SMI/UDS) to only 42.7 percent (SMI/NDS) and only 24.0 percent for the range. Also as expected, the mean differences were higher for the 2001 projections than for the 1996 estimates. MAPDs ranged from 1.9 percent for SMI/UDS to 4.0 percent for NDS/UDS (see Table 1). The mean range difference was 5.4 percent. The three largest mean differences involved pairings with NDS--perhaps because the NDS county projections were the only ones adjusted for conformity with metropolitan area population forecasts produced with regional economic models.7 Again, what constitutes a large difference depends on the user, but it is clear that, even for areas as large as counties, larger differences started to emerge in the five year projections--especially when the suppliers were considered as a group. Given the stability of county boundaries, one would expect few differences in the 1980 census populations presented by the four suppliers. Table 1 confirms that this is the case, as most pairs showed very small MAPDs (the 0.2 values actually round up from 0.16). However, mean differences involving UDS were conspicuously high at 2.1 percent--a greater difference than that typically observed in the 1996 estimates. The differences were not huge, but since there should have been exact agreement on most counties, the UDS 1980 county populations are unexplained outlyers.
Tract Population Just as estimation error typically increases for smaller geographies, estimation differences were greater for census tracts than for counties. Table 3 indicates that, for supplier pairs, MAPD ranged from 6.2 percent (NDS/SMI) to 7.8 percent (UDS/SMI). For areas as small as tracts, such differences were not large (by comparison, tract level mean absolute errors in 1990 were in the 11 to 16 percent range). The frequency distribution in Table 4 confirms that, for supplier pairs, only about 10 percent of tract population estimates differed by more than 15 percent (plus or minus), and roughly half were within five percent of each other. And just as estimation error decreases as tracts are aggregated, estimation differences would likely diminish with aggregation. Consistency was less impressive for the suppliers as a group. Mean absolute difference increased to 12.4 percent, 26 percent of estimates differed by 15 percent or more, and only 16 percent were within five percent of each other.
4
Again, the 2001 projections exhibited greater percent differences than the 1996 estimates. MAPD ranged from 10.4 percent (Claritas/NDS) to 13.4 percent (SMI/UDS), and the range MAPD was 20.9 percent (Table 3). Differences in 1980 census population converted to 1990 tracts were expected given tract boundary changes, and the creation of 1990 tracts in previously untracted areas. Details are sketchy, but the suppliers took different approaches in converting 1980 data to 1990 tracts. In fact, the suppliers agreed less on tract level 1980 census population than on 1996 estimated population. As reported in Table 3, mean differences ranged from 6.5 percent (Claritas/SMI) to 13.9 percent (UDS/NDS), and the range difference among all four was 18.2 percent. Such differences might be a surprise to many data users, since the suppliers do not label their 1980 data as "estimates." And with some suppliers relying on the projection of intercensal trends for their current year estimates, it is remarkable that such differences in the 1980 data (and 19801990 trends) did not translate to greater differences in the 1996 tract estimates. The similarity of 1996 tract population estimates is worth closer examination. Again, the MAPDs were in single digits for supplier pairs, and 12.4 percent for the range. In many tracts, the 1996 population estimates were quite similar, and it is easy to see how some users could get the impression that they are all pretty much the same. However, the tract estimates were not without important differences. For each supplier, Table 5 reports the distribution of estimated 1990-1996 population change. All suppliers estimated moderate change more frequently than rapid change (anything else would lack credibility). However, tracts estimated to have lost 10 percent or more of their population ranged from a high of 7.4 percent (UDS) to a low of 2.7 percent (NDS), and those growing 25 percent or more ranged from 7.1 percent (UDS) to only 2.9 percent (NDS). For very rapid growth (50 percent or more), Claritas was high with 1.1 percent of all tracts and NDS was low with 0.1 percent. Such differences do not translate to large mean differences, but they suggest that some suppliers were more aggressive than others in estimating rapid change. The suppliers may have estimated tract rates of change with similar frequencies, but crosstabulations identify the extent to which they agreed on which tracts were changing at specific rates. Appendix 1 presents crosstabluations of estimated 1990-1996 population change for the six supplier pairs and the range difference. Tracts on the main diagonal and those within two cells of the main diagonal reflect agreement or modest difference between suppliers. The shaded cells three or more cells away from the main diagonal reflect the larger differences. The crosstabulations merit individual attention, but Table 6 presents a summary of tracts "on," "within 2 cells," and "3+ cells off" the main diagonal. The summary provides further evidence of the similarity among tract estimates, as for all supplier pairs, most tracts were either on or within two cells of the main diagonal. Tracts three or more cells from the diagonal ranged from just 11.8 percent (NDS/UDS) to 18.5 percent (SMI/UDS) of total. However, these modest percentages translate to 7,060 and 11,082 tracts respectively. Thus, tracts where two suppliers disagreed on population change were a plentiful minority. And such disagreement was amplified when maximum and minimum rates of change were compared (Appendix 1G). Tracts on the main diagonal dropped from about 30 percent for supplier pairs to only 5.2
5
percent for the range, while those three or more cells from the diagonal jumped from 10+ percent to 35.9 percent--a total of 21,477 tracts (see Table 6). A data user choosing among the four sets of 1996 tract population estimates would confront considerable variation with respect to estimated change since 1990.
Block Group Population After producing tract level estimates throughout the 1980s, the private suppliers started producing block group estimates following the 1990 census--primarily because their enhanced desktop products required block group input. The logical expectation is that block group estimation error would be considerably higher than that for tracts, and one might expect block group differences to be much larger as well. Therefore, it is striking that the differences among the suppliers' 1996 block group population estimates were only moderately higher than those for census tracts (see Table 7). For the Claritas/SMI pair, the block group MAPD of 6.8 percent was only slightly higher than the 6.6 percent observed for tracts, and the range MAPD for suppliers as a group increased from 12.4 percent to only 16.5 percent. The increases in MAPD for the 2001 projections were similarly modest. The similarity of the tract and block group mean differences could be due to the fact that block groups are not that much smaller than tracts. Tracts have an average of only 3.7 block groups and a maximum of nine. Also, in 1996 the suppliers were doing little more than distributing tract estimates to block group based on 1990 census percentages. In the absence of unique block group input data, one might expect relatively small mean differences.8 The 1980 census populations for 1990 block groups showed relatively large variation (Table 7)-and these differences were much greater than those for tracts. MAPDs ranged from 20.6 percent (Claritas/SMI) to 38.2 percent (UDS/NDS), and the range difference was 49.3 percent. Contributing to these differences was the incomplete and imperfect block group level 19801990 geographic correspondence provided by the 1990 census TIGER files. Left to their own devices, the suppliers came up with rather different results in converting 1980 counts to 1990 block groups.
ZIP Code Population ZIP Code estimates are popular, but fraught with complications that are not widely understood. Defined by the U.S. Postal Service for the delivery of mail--not for the presentation of data-ZIP Codes are not just another geographic level. They are collections of delivery points rather than clearly defined areas. And unlike census geographies, which are stable throughout the decade, ZIP Codes change frequently at the discretion of local postal officials. A description of the complications of ZIP Code data would not fit in this paper, but it must be understood that such complications exist, and leave a heavy mark on the comparison of ZIP Code estimates.
6
The number and definition of residential ZIP Codes is always changing, and the suppliers seek to capture these elusive definitions each time they produce a set of estimates. In contrast to counties, tracts and block groups, where published 1990 census counts provide a common starting point, there are no "official" 1990 census counts for current ZIP Codes.9 For estimation purposes, ZIP Codes are typically defined in terms of correspondence with block groups. For example, a ZIP Code might be defined as the sum of: 100 percent of block groups 1 and 2, 47.8 percent of block group 4, and 6.2 percent of block group 5. Usually based on estimated ZIP Code boundaries from geographic data suppliers, block group-to-ZIP correspondence is identified for current ZIP Codes, and applied to existing block group data to produce estimates and census data for ZIP Codes. Due to such complications, the comparison of ZIP Code estimates began with the estimation of 1990 census population for residential ZIP Codes defined in 1996. There were 36,089 ZIP Codes in 1996 where at least one supplier estimated a 1990 population greater than zero. However, the suppliers differed widely in the number of residential ZIP Codes they recognized in 1996. Claritas recognized the largest number at 35,272 and NDS the smallest at 29,482. Such differences (summarized in Table 8) trace to the different geographic data sources used by the suppliers, and the handling of "P.O. Box" ZIPs, whose boundaries are especially ill-defined. Because of the differences in ZIP Code rosters, comparisons were limited to residential ZIPs recognized by all four suppliers in 1996, and where 1990 population was estimated to be at least 10 persons. The objective was to eliminate ZIP Codes whose existence was in dispute, as well as percent differences inflated by very small denominators. If anything, the ZIP Code differences reported below are understated. As indicated in Table 9, even where all four suppliers agreed on the existence of a ZIP Code, they varied widely in the estimation of 1990 census population. Most MAPDs for the supplier pairs were in the 20 to 30 percent range, and the mean difference between the high and low estimates was 41.3 percent. Notably consistent were the SMI and NDS estimates--with a MAPD of only 5.4 percent. This consistency likely reflects SMI's and NDS's common use of ZIP Code boundaries from Geographic Data Technology (GDT). Claritas also used GDT boundaries, but modified the resulting block group correspondence to produce ZIP Code data consistent with residential delivery counts. UDS used a different supplier for its ZIP boundary and correspondence files. Thus, one would expect larger differences in comparisons involving Claritas and/or UDS. Differences among the 1996 population estimates and 2001 projections for ZIP Codes also were quite large, but not dramatically greater than those for 1990 (Table 9). At least in terms of MAPD, much of the variation in the ZIP Code estimates and projections is explained by differences in estimating 1990. In other words, ZIP Code definition is the critical difference. Also, because the 1990 differences were so wide, 1996 differences could actually be lower in some ZIPs, thus dampening any increase in the MAPD. Again, however, the differences reported are understated by the elimination of ZIPs not recognized by one or more suppliers. Clearly, users of ZIP Code data (for census year as well as estimates and projections) would face wide differences in choosing among the suppliers compared.
7
Population by Age A detailed comparison of estimates of population by age is a subject for further research, but a limited comparison illustrates the types of differences that can be experienced. Estimates for the age "75 and above" category were compared because of its interest to users, including health care providers and the developers of assisted living facilities. Estimates for population segments were expected to differ more than those for total population because of the increased difficulty, and because the smaller totals would yield larger percent differences. Table 10 confirms that this is the case, as the 1996 MAPDs for the supplier pairs ranged from 3.1 to 6.9 for counties; 9.8 to 15.5 for tracts and from 12.9 to 19.9 for block groups. As usual, county differences were relatively small, while those for tracts and block groups were larger. Again, it was not until all four suppliers were considered that the extent of potential uncertainty was identified. The 1996 range MAPDs were 10.0 for counties, 24.5 for tracts and 31.1 for block groups. Also as usual, differences were consistently higher for 2001-with block group level pair differences as high as 31.5 percent (SMI/NDS) and a range MAPD of 48.0. Table 10 also reveals a consistently smaller difference between Claritas and SMI relative to other supplier pairs. This consistency may be due to the similarity of the cohort survival methods used by the two suppliers, as well as the fact that, unlike UDS and NDS, the SMI and Claritas age estimates began with age distributions from the 1990 census Modified Age/Race/Sex (MARS) files.10 The difference introduced by MARS is suggested by the MAPDs between the original and MARS totals for persons 75+. These differences were 1.0 percent for counties, 2.2 percent for tracts and 3.8 percent for block groups. While the typical user is most interested in the number of persons age 75+, and thinks in terms of differences as described above, part of this variation traces to differences in the estimation of total population. To control for this effect, the suppliers' estimates and projections were adjusted to conform with 1990 census total population (percent 75+ was applied to the common 1990 population total). The MAPDs in Table 11 confirm that, when differences in total population were eliminated, the data for population 75+ were somewhat more consistent. The reductions were not dramatic, but suggest that--especially for small areas--users could reduce uncertainty somewhat if they can use the age estimates in percent form.
Mean Household Income A comparison of average household income suggests the extent of variation that can be observed among income estimates and projections. The mean absolute differences in Table 12 do not follow the typical pattern. Differences tended to be larger for the smaller areas, but not always. For example, the Claritas/SMI 1996 MAPD was 7.5 for counties, but only 6.7 for tracts and block groups. As a summary statistic, mean income is not reduced in size (as are counts) for smaller areas. Therefore, larger estimation differences in small areas truly reflect the added difficulty and uncertainty associated with these levels, as opposed to the effect of
8
small denominators. Variation in estimates of the number of households by income likely would have fit the pattern observed for count based estimates. Even at the county level, there was substantial variation on mean income. MAPDs ranged from 4.4 percent to 14.1 percent in 1996, and as high as 14.2 percent in 2001 (Table 12). County range MAPDs were 15.9 for 1996 and 21.2 for 2001. At the tract level, the range MAPDs were 20.1 (1996) and 31.9 (2001), and for block groups the mean differences were 23.5 percent and 36.8 percent respectively. Definitional differences might explain part of the difference in estimated income. All four suppliers started with 1990 census household income, all four estimated "money income" in current dollar values, but not all "1996" income estimates reflected the same year. NDS and SMI estimated 1996 households by income earned in 1995 (just as the 1990 census asked income earned in 1989) while Claritas estimated income earned in 1996. UDS documents do not specify the target year for their income estimates. In any case, several of the income comparisons involved a built-in one year difference. Because Claritas targeted 1996 (and 2001), one would expect its estimates and projections to be higher than those targeted at 1995 (and 2000). Table 13 presents the mean algebraic percent differences (MALPD) among the mean income estimates. MALPD distinguishes positive and negative percent differences between estimates to provide a measure of bias--or the extent to which one set of estimates is higher than another. The Table 13 column headers indicate the direction of difference--a negative sign indicating that the estimates for the first supplier in the header were lower than those of the second. As expected, the Claritas 1996 income estimates were (on average) higher than SMI's targeted at 1995. All Claritas/SMI MALPEs were positive by modest amounts. Had SMI targeted 1996 (or Claritas targeted 1995), bias and mean absolute difference might have been lower. The Claritas/NDS comparison was more complex. The Claritas 1996 incomes were higher than the NDS 1995s at the county level, but lower for tracts and block groups (as well as national level). Such paradoxical reversals (also observed in a few other pairs) are possible when comparing means or other ratios, and merit further investigation. Nevertheless, it appears that, had NDS and Claritas targeted the same year, bias and mean absolute differences might have been greater than those actually observed for tracts and block groups. Thus, the one year difference in the income estimation date seems to explain some, but certainly not all of the differences observed in estimated mean household income.
Conclusions While far from exhaustive, the comparisons described above are the most extensive investigation of the differences among private sector demographic estimates. What constitutes a large versus small difference is to some degree in the eye of the user, but the results illustrate a mix of similarities and differences that make it apparent how some users might find little difference among the suppliers' numbers, while others report sizable differences.
9
As expected, differences were smaller for larger geographic units, such as counties, where population estimates rarely differed by more than a few percentage points. Tract and block group population estimates often were very close, but larger differences were not hard to find. Also as expected, there was greater variation on characteristics, such as age and income, but here too, the results were a mix of similarities and differences. In all cases, greater uncertainty was encountered when comparing the suppliers' maximum versus minimum estimates. Even if trivially obvious, this result is important from the data user perspective. Specifically, the relative similarity of estimates from two suppliers can create an illusion of consistency or certainty that is broken when one considers the range of alternatives. Perhaps surprising to some users would be the disparities in the conversion of 1980 census data to 1990 geography. At the tract and especially block group levels, there were major differences in the “estimation” of 1980 census population. Such differences suggest that small area 19801990 census trends, and estimates relying on the projection of such trends, should be used with caution. Also notable was the variation among ZIP Code estimates. The suppliers disagreed on the number of residential ZIP Codes in 1996, and by almost any standard, the differences among ZIP Code estimates would be considered large. The wide variation in 1990 census counts confirms that much of the disparity in ZIP Code estimates traces to differences in their definition by the four suppliers. One could summarize by observing that the four sets of estimates were striking in their similarity except where they were very different. Observed, similarities were impressive, but one could not summarily assert that there were no important differences among the suppliers' estimates. Methods were known to differ, and differences in the data often were consistent with differences in methods. Of course, the results do not identify which estimates were the most accurate–or whether any supplier's estimates were consistently more accurate. Estimates could vary widely, but share similar error rates. The observed differences--even where modest--leave room for substantial differences in accuracy, but the answers to the accuracy question must await evaluations against the 2000 census. Until then, users must scrutinize the alternative methods to gauge the suitability of the resulting data for their applications. The corporate acquisitions that made this paper possible have rendered it historical in that none of the four estimation programs function now as they did in 1996. SMI and UDS no longer exist as data suppliers. NDS and Claritas maintain separate identities within VNU, and Claritas continues to produce estimates combining elements from the original Claritas/NPDC, DMIS and NDS methods. Considerable competition remains, with as many as five firms currently producing small area estimates and projections. Methods still vary, and questions concerning data similarity are as relevant as ever. The present comparison does not encompass the current set of practitioners, but it provides rare and still relevant insights into the similarities and differences that can be expected among alternative sets of small area estimates and projections.
10
Table 1 MEAN ABSOLUTE PERCENT DIFFERENCE: COUNTY POPULATION:
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range1
1996 County Population 2001 County Population
1.3 2.3
1.3 2.2
1.1 3.3
0.6 1.9
1.6 3.9
1.7 4.0
2.2 5.4
1980 County Population
0.2
2.1
0.2
2.1
0.2
2.1
2.2
Data Item
1 MAPD based on difference between the highest and lowest estimate among the four suppliers.
Table 2 FREQUENCY DISTRIBUTION: ABSOLUTE PERCENT DIFFERENCE BETWEEN COUNTY LEVEL 1996 POPULATION ESTIMATES Absolute Pct Difference
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range1
0 to 1.0 1.0 to 4.9 5.0 to 9.9 10.0 to 14.9 15.0 to 19.9 20.0 to 24.9 25.0 to 49.9 50.0 or More
55.2 42.5 1.9 0.3 0.1 0.0 0.0 0.0
55.3 41.8 2.0 0.5 0.2 0.0 0.1 0.0
62.6 36.0 1.1 0.1 0.1 0.0 0.0 0.0
90.4 8.4 0.8 0.3 0.1 0.0 0.1 0.0
42.7 54.0 3.0 0.2 0.1 0.0 0.0 0.0
41.8 54.3 3.2 0.4 0.1 0.0 0.1 0.0
24.0 69.9 4.8 0.8 0.3 0.0 0.1 0.1
1.3
1.3
1.1
0.6
1.6
1.7
2.2
MAPD2
1 MAPD based on difference between the highest and lowest estimate among the four suppliers. 2 Mean Absolute Percent Difference.
Table 3
11
MEAN ABSOLUTE PERCENT DIFFERENCE: CENSUS TRACT POPULATION1
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1996 Tract Population 2001 Tract Population
6.6 10.7
7.2 11.9
6.4 10.4
6.2 13.4
6.4 10.8
7.8 11.5
12.4 20.9
1980 Tract Population
6.5
11.0
9.5
11.9
8.4
13.9
18.2
Data Item
1 Based on all census tracts with a 1990 census population of 10 or more. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
Table 4 FREQUENCY DISTRIBUTION: ABSOLUTE PERCENT DIFFERENCE BETWEEN TRACT LEVEL 1996 POPULATION ESTIMATES 1
Absolute Pct Difference
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
0 to 1.0 1.0 to 4.9 5.0 to 9.9 10.0 to 14.9 15.0 to 19.9 20.0 to 24.9 25.0 to 49.9 50.0 or More
13.1 40.4 26.8 10.8 4.3 2.1 2.2 0.4
14.0 40.3 23.6 10.6 5.0 2.6 3.3 0.6
13.0 42.1 26.5 10.3 4.1 1.8 1.9 0.4
10.5 35.3 27.7 13.5 6.3 3.0 3.3 0.4
12.2 41.0 28.2 11.2 4.3 1.6 1.4 0.1
13.0 41.8 26.1 10.5 4.3 2.0 2.0 0.2
0.3 15.6 35.0 23.1 11.8 6.1 7.1 1.0
6.6
7.2
6.4
6.2
6.4
7.8
12.4
MAPD3
1 Table includes all census tracts with a 1990 census population count of 10 or more. N = 59,366. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers. 3 Mean Absolute Percent Difference
Table 5
12
ESTIMATED POPULATION CHANGE 1990-1996 Pct Change -50 or more -25.0 to -49.9 -20.0 to -24.9 -15.0 to -19.9 -10.0 to -14.9 -5.0 to - 9.9 -0 to -4.9 0 to 4.9 5.0 to 9.9 10.0 to 14.9 15.0 to 19.9 20.0 to 24.9 25.0 to 49.9 50.0 or More
Claritas
SMI
UDS
NDS
0.1 0.5 0.5 1.2 3.5 10.3 19.9 24.0 15.7 9.3 5.5 3.2 5.2 1.1
0.0 0.0 0.1 0.4 2.9 10.9 20.4 23.0 15.6 10.1 6.2 3.9 5.7 0.8
0.1 0.9 0.7 1.7 4.0 11.4 19.9 20.6 15.0 9.3 5.7 3.7 6.4 0.7
0.0 0.1 0.3 0.8 1.5 4.7 18.2 30.1 21.1 11.2 5.9 3.2 2.8 0.1
N
59813
59812
59830
59815
5.8 16.1 6.3 43.9
4.3 15.2 6.5 43.4
7.4 18.8 7.1 40.5
2.7 7.4 2.9 48.3
10 pct loss 5 pct loss 25 pct growth -5 pct to +5 pct
13
Table 6
CORRESPONDENCE BETWEEN ESTIMATED POPULATION CHANGE 1990-1996 1 Percent of Tracts
On Main Diagonal2 Within 2 Cells3 3+ Cells Off Diagonal4 Opposite Direction5 N of Tracts
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
NDS vs. UDS
Range
30.4 56.5 13.1
31.6 53.0 15.4
29.5 58.5 11.9
25.4 56.1 18.5
28.7 59.4 11.9
30.0 58.2 11.8
5.2 58.9 35.9
3.1
2.6
2.6
4.9
2.6
6.5
11.4
59,826
59,830
59,813
59,830
59,812
59,815
59,804
1 Percentages from crosstabulations presented in Appendix 1. Table includes all census tracts with a 1990 census population count of 10 or more. 2 Estimated rates of change in same category. 3 Estimated rates not on main diagonal, but within two cells of it. 4 Estimated rates 3 or more cells from main diagonal. 5 One supplier estimates loss of 5 percent or more. Other supplier estimates growth of 5 percent or more. Minimum difference of 10 percentage points.
14
Table 7
MEAN ABSOLUTE PERCENT DIFFERENCE: BLOCK GROUP POPULATION1
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1996 Block Group Pop. 2001 Block Group Pop.
6.8 11.1
10.5 17.8
7.8 12.9
10.5 18.1
7.8 13.6
10.4 18.4
16.5 28.1
1980 Block Group Pop.
20.6
20.9
33.8
27.3
27.1
38.2
49.3
Data Item
1 Based on all block groups with a 1990 census population of 10 or more. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
15
Table 8
RESIDENTIAL ZIP CODES RECOGNIZED IN 1996 Total1 Claritas SMI UDS NDS All Suppliers2
36,089 35,272 34,672 31,634 29,482 29,161
1 1996 residential ZIP Codes where at least one supplier estimated a 1990 census population greater than zero. 2 1996 residential ZIP Codes where all four suppliers estimated a 1990 census population greater than zero.
Table 9 MEAN ABSOLUTE PERCENT DIFFERENCE: ZIP CODE POPULATION1
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1990 ZIP Code Pop.
25.3
32.4
27.0
22.8
5.4
23.2
41.3
1996 ZIP Code Pop. 2001 ZIP Code Pop.
25.5 26.3
33.0 34.2
27.3 28.4
25.1 27.4
8.8 12.1
25.2 27.5
43.7 46.7
Data Item
1 Residential ZIP Codes recognized by all four suppliers, and with estimated 1990 population greater than 10. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
16
Table 10 MEAN ABSOLUTE PERCENT DIFFERENCE: POPULATION AGE 75 AND ABOVE 1 Comparison
Data Item 1996 County Pop 75+ 2001 County Pop 75+ 1996 Tract Pop 75+ 2001 Tract Pop 75+ 1996 BG Pop 75+ 2001 BG Pop 75+
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
3.1 4.7
5.0 7.5
6.9 11.3
5.7 9.0
6.9 11.3
5.6 9.6
10.0 16.1
9.8 15.0
13.6 21.4
14.5 23.4
15.5 24.2
14.9 24.7
13.1 21.5
24.5 38.9
12.9 18.6
17.3 26.6
17.9 28.1
19.4 29.6
19.9 31.5
16.1 26.9
31.1 48.0
1 Table includes all areas where both the STF 1 and MARS 1990 population counts were 10 or more. 2 Difference based on maximum vs. minimum estimate among all suppliers.
Table 11 MEAN ABSOLUTE PERCENT DIFFERENCE: POPULATION AGE 75 AND ABOVE ADJUSTED FOR CONFORMITY WITH 1990 CENSUS POPULATION1 Comparison
Data Item 1996 County Pop 75+ 2001 County Pop 75+ 1996 Tract Pop 75+ 2001 Tract Pop 75+ 1996 BG Pop 75+ 2001 BG Pop 75+
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
3.2 5.0
5.1 7.3
6.8 10.3
5.7 9.1
6.6 10.3
5.4 8.3
9.8 15.1
7.2 10.7
11.1 17.0
13.0 20.7
12.7 19.2
13.7 22.3
10.3 16.6
20.3 31.6
11.1 15.4
13.5 19.5
16.2 24.9
16.0 23.5
18.3 28.5
11.3 18.6
25.8 38.7
1 Table includes all areas where both the STF 1 and MARS 1990 population counts were 10 or more. 2 Difference based on maximum vs. minimum estimate among all suppliers.
17
Table 12 MEAN ABSOLUTE PERCENT DIFFERENCE MEAN HOUSEHOLD INCOME1 Comparison
Data Item 1996 County Avg HH Income 2001 County Avg HH Income 1996 Tract Avg HH Income 2001 Tract Avg HH Income 1996 BG Avg HH Income 2001 BG Avg HH Income
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
7.5 12.3
14.1 13.9
8.0 14.2
8.0 10.7
4.4 10.3
9.1 8.6
15.9 21.2
6.7 11.0
11.8 16.2
12.0 22.7
8.0 15.9
10.5 21.3
15.8 18.5
20.1 31.9
6.7 10.9
14.0 19.4
13.3 24.7
10.4 18.1
12.2 23.0
19.1 25.3
23.5 36.8
1 Table includes all areas with a 1990 census household count of 10 or more.
Table 13 MEAN ALGEBRAIC PERCENT DIFFERENCE: MEAN HOUSEHOLD INCOME1 Comparison Data Item 1996 County Avg HH Income 2001 County Avg HH Income 1996 Tract Avg HH Income 2001 Tract Avg HH Income 1996 BG Avg HH Income 2001 BG Avg HH Income
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
5.8 8.7
13.6 -0.7
4.9 2.6
7.9 10.7
-0.9 -6.1
-8.7 3.4
3.6 4.4
10.1 -7.3
-3.6 -13.8
6.5 -11.7
-7.2 -18.2
-13.6 -6.5
3.5 4.1
11.6 -2.8
-4.2 -13.4
8.2 -6.9
-7.7 -17.5
-15.7 -10.2
1 Computation based on order of supplier in column title. A negative value indicates the tendency for the first supplier in the title to have lower estimates than the second supplier in the title. Table includes all areas with a 1990 census household count of 10 or more.
APPENDIX 1A
18
TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: CLARITAS VS. SMI SMI Claritas
-50 or more
-25 to -50
-20 to -25
-50 or more
1
3
-25 to -50
2
8
-20 to -25 -15 to -20
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
1
4
3
8
14
2
1
17
24
26
52
62
68
16
7
8
6
7
3
3
10
21
38
80
56
35
20
14
5
1
6
1
4
12
32
134
178
143
116
54
23
11
8
17
8
71
363
614
471
299
141
68
30
14
19
1
-10 to -15 -5 to -10
1
3
54
531
1861
1975
1016
405
164
89
31
36
6
-0 to -5
1
2
22
396
2101
4264
3094
1154
462
198
98
84
5
0 to 5
1
2
14
167
1085
3372
5114
2597
1160
434
200
196
10
4
37
334
1191
2457
2628
1471
660
325
267
10
2
15
101
383
923
1324
1318
800
372
329
24
2
2
53
125
295
538
703
697
466
381
25
1
5
13
60
142
227
332
344
315
445
31
25 to 50
5
26
76
164
203
297
363
419
1393
177
50 or more
4
7
18
41
32
35
45
51
236
186
5 to 10 10 to 15
1
15 to 20 20 to 25
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
1
N of Tracts
Pct
18,190 33,819 7,187 59,826
30.4 56.5 13.1 100.0
19
APPENDIX 1B TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: CLARITAS VS. UDS UDS Claritas
-50 or more
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
5
1
7
2
4
8
3
2
15 to 20
20 to 25
25 to 50
50 or more
2
1
1
1
-25 to -50
16
104
20
20
19
40
22
27
19
7
4
1
7
-20 to -25
4
42
64
37
25
29
22
25
19
8
4
5
5
1
-15 to -20
6
38
68
191
139
84
68
60
37
15
12
6
7
1
-10 to -15
8
46
59
229
580
474
310
180
102
43
23
20
23
2
-5 to -10
4
45
42
155
628
2203
1661
768
357
153
69
37
46
5
-0 to -5
4
88
58
154
442
2102
4468
2609
1067
423
207
98
150
12
0 to 5
12
88
50
118
307
1108
3481
4804
2462
1005
421
219
254
22
5 to 10
4
35
29
47
147
426
1108
2305
2654
1359
619
280
346
26
10 to 15
5
25
13
24
46
159
405
798
1188
1359
778
356
414
23
15 to 20
1
15
9
19
29
78
154
349
517
603
639
383
460
33
20 to 25
3
10
4
14
13
36
67
144
233
249
271
348
493
30
25 to 50
4
14
8
9
32
40
92
190
248
262
301
385
1375
163
50 or more
4
7
3
4
4
11
18
41
52
53
42
53
244
119
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
18,908 31,722 9,200 59,830
31.6 53.0 15.4 100.0
20
APPENDIX 1C TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: CLARITAS VS. NDS NDS Claritas
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
2
1
7
3
9
4
6
1
-25 to -50
10
22
65
62
34
30
44
17
14
3
-20 to -25
4
9
29
68
69
49
28
14
12
4
-15 to -20
2
9
33
84
233
184
104
42
24
11
2
4
-10 to -15
6
20
49
145
484
750
435
134
49
18
4
6
-5 to -10
10
31
44
132
891
2628
1713
518
148
24
19
15
-0 to -5
8
30
76
149
565
3951
4763
1675
459
130
49
27
0 to 5
8
25
94
121
319
2313
6220
3650
1093
350
102
52
5 to 10
3
8
39
55
105
606
2954
3274
1538
528
184
87
10 to 15
1
3
19
35
51
179
1024
1808
1415
699
239
117
1
15 to 20
2
5
23
24
70
367
767
883
637
327
182
1
20 to 25
1
1
7
8
14
41
164
325
444
395
306
207
2
2
8
18
22
52
144
360
534
639
575
757
11
2
7
16
8
13
26
47
78
119
114
200
22
-50 or more
25 to 50 50 or more
1
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
17,670 35,008 7,127 59,813
29.5 58.5 11.9 99.9
21
2 1
3 3
APPENDIX 1D TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: SMI VS. UDS UDS SMI
-50 or more
-25 to -50
-20 to -25
-15 to -20
-50 or more
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
1
1
1
2
4
5
1
2
2
15 to 20
20 to 25
25 to 50
50 or more
-25 to -50
2
3
-20 to -25
2
3
5
11
10
10
8
3
1
-15 to -20
1
22
17
29
43
65
39
17
9
1
4
1
-10 to -15
4
52
47
122
276
432
416
214
88
41
18
7
8
2
-5 to -10
11
96
102
251
604
1655
1921
1064
482
176
56
36
46
6
-0 to -5
8
111
105
257
603
2131
3754
2834
1306
563
240
114
166
11
0 to 5
23
119
71
183
442
1383
3265
3854
2413
1024
470
228
285
21
5 to 10
7
56
40
85
205
627
1428
2308
2137
1225
612
300
296
15
10 to 15
4
34
21
35
108
249
559
1076
1264
1134
696
407
439
32
15 to 20
4
26
8
28
55
123
237
466
626
660
578
347
501
25
20 to 25
2
16
3
14
29
45
131
215
315
324
346
308
517
41
25 to 50
4
18
8
10
28
64
115
232
293
360
350
411
1351
173
50 or more
4
6
1
4
3
7
18
19
35
23
33
215
111
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
15,195 33,553 11,082 59,830
25.4 56.1 18.5 100.0
22
1 1
1
APPENDIX 1E TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: SMI VS. NDS NDS SMI
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-50 or more
-0 to -5
0 to 5
1
2
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
-25 to -50
3
2
1
2
4
5
3
2
-20 to -25
6
2
10
5
7
14
8
1
2
-15 to -20
10
11
10
38
55
88
28
4
1
2
1
-10 to -15
7
20
39
128
340
693
366
100
22
7
3
1
-5 to -10
6
34
81
182
853
2559
2030
583
138
25
9
6
-0 to -5
5
34
110
191
746
3702
4843
1904
490
129
39
10
0 to 5
11
34
126
195
469
2363
5733
3379
998
311
115
37
5 to 10
3
5
41
73
178
837
2842
3235
1473
458
144
52
10 to 15
3
5
21
45
81
320
1196
1768
1531
718
263
104
1
15 to 20
1
7
20
28
43
130
475
853
934
717
324
150
2
20 to 25
2
9
16
21
64
254
384
511
468
363
211
1
25 to 50
2
5
10
13
30
83
194
398
544
672
581
877
5
50 or more
1
3
4
6
1
7
18
20
48
49
82
212
28
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
17,182 35,534 7,096 59,812
28.7 59.4 11.9 100.0
23
APPENDIX 1F TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: NDS VS. UDS UDS NDS
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
-50 or more -25 to -50
4
14
5
4
3
1
3
7
13
4
1
1
-20 to -25
13
24
2
9
9
26
11
21
31
7
3
4
3
-15 to -20
17
126
17
21
20
25
52
78
66
28
16
6
9
2
-10 to -15
12
107
113
117
102
95
88
106
81
40
29
13
17
3
-5 to -10
8
55
118
407
596
750
357
245
134
74
34
24
24
-0 to -5
3
60
76
253
916
3134
3964
1541
482
198
96
61
76
9
0 to 5
7
89
52
149
546
2042
5321
5942
2538
752
275
101
167
9
5 to 10
5
50
25
41
136
562
1656
3174
3587
2045
738
318
281
14
10 to 15
2
18
14
20
62
108
314
855
1443
1526
1067
572
667
24
15 to 20
1
14
3
4
12
39
77
217
385
598
743
561
848
54
20 to 25
1
3
2
3
6
9
29
75
127
189
280
335
790
75
25 to 50
1
1
4
5
11
40
64
77
111
192
935
219
7
29
1
50 or more
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
17,950 34,805 7,060 59,815
30.0 58.2 11.8 100.0
24
APPENDIX 1G TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: MAXIMUM VS. MINIMUM MIN MAX
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
-50 or more 2
-25 to -50 -20 to -25
3
2
-15 to -20
2
15
5
-10 to -15
7
40
42
61
17
-5 to -10
11
95
119
291
422
198
-0 to -5
16
125
160
413
1414
3024
772
0 to 5
30
179
133
436
1353
4385
5856
1062
5 to 10
13
125
106
247
694
2304
4863
4136
416
10 to 15
5
70
63
122
337
960
2135
3290
1807
168
15 to 20
6
46
30
85
162
418
940
1638
1657
750
75
20 to 25
6
32
14
55
91
201
487
818
975
776
291
23
25 to 50
7
43
27
55
126
246
556
953
1163
1276
1018
619
370
50 or more
9
14
10
10
25
27
49
104
113
168
192
183
298
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
3,119 35,208 21,477 59,804
5.2 58.9 35.9 100.0
25
16
APPENDIX 2 SUMMARY TABLE OF MEAN ABSOLUTE PERCENT DIFFERENCE BETWEEN POPULATION ESTIMATES 1 Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1.3 2.3
1.3 2.2
1.1 3.3
0.6 1.9
1.6 3.9
1.7 4.0
2.2 5.4
1996 Tract Population 2001 Tract Population
6.6 10.7
7.2 11.9
6.4 10.4
6.2 13.4
6.4 10.8
7.8 11.5
12.4 20.9
1996 Block Group Pop. 2001 Block Group Pop.
6.8 11.1
10.5 17.8
7.8 12.9
10.5 18.1
7.8 13.6
10.4 18.4
16.5 28.1
1980 County Population 1980 Tract Population 1980 Block Group Pop.
0.2 6.5 20.6
2.1 11.0 20.9
0.2 9.5 33.8
2.1 11.9 27.3
0.2 8.4 27.1
2.1 13.9 38.2
2.2 18.2 49.3
1990 ZIP Population 1996 ZIP Population 2001 ZIP Population
25.3 25.5 26.3
32.4 33.0 34.2
27.0 27.3 28.4
22.8 25.1 27.4
5.4 8.8 12.1
23.2 25.2 27.5
41.3 43.7 46.7
Data Item 1996 County Population 2001 County Population
1 Table includes all areas with a 1990 census population count of 10 or more. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
26
APPENDIX 3 SUMMARY TABLE OF MEAN ABSOLUTE PERCENT DIFFERENCE BETWEEN AGE AND INCOME ESTIMATES Data Item 1996 County Pop 75+ 2001 County Pop 75+
1996 Tract Pop 75+ 2001 Tract Pop 75+
1996 BG Pop 75+ 2001 BG Pop 75+
1996 County Pct Pop 75+ 2001 County Pct Pop 75+
1996 Tract Pct Pop 75+ 2001 Tract Pct Pop 75+
1996 BG Pct Pop 75+ 2001 BG Pct Pop 75+
1996 County Avg HH Income 2001 County Avg HH Income
1996 Tract Avg HH Income 2001 Tract Avg HH Income
1996 BG Avg HH Income 2001 BG Avg HH Income
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range1
3.1 4.7
5.0 7.5
6.9 11.3
5.7 9.0
6.9 11.3
5.6 9.6
10.0 16.1
9.8 15.0
13.6 21.4
14.5 23.4
15.5 24.2
14.9 24.7
13.1 21.5
24.5 38.9
12.9 18.6
17.3 26.6
17.9 28.1
19.4 29.6
19.9 31.5
16.1 26.9
31.1 48.0
3.2 5.0
5.1 7.3
6.8 10.3
5.7 9.1
6.6 10.3
5.4 8.3
9.8 15.1
7.2 10.7
11.1 17.0
13.0 20.7
12.7 19.2
13.7 22.3
10.3 16.6
20.3 31.6
11.1 15.4
13.5 19.5
16.2 24.9
16.0 23.5
18.3 28.5
11.3 18.6
25.8 38.7
7.5 12.3
14.1 13.9
8.0 14.2
8.0 10.7
4.4 10.3
9.1 8.6
15.9 21.2
6.7 11.0
11.8 16.2
12.0 22.7
8.0 15.9
10.5 21.3
15.8 18.5
20.1 31.9
6.7 10.9
14.0 19.4
13.3 24.7
10.4 18.1
12.2 23.0
19.1 25.3
23.5 36.8
1MAPD based on difference between the highest and lowest estimate among the four suppliers.
27
NOTES 1 The 1982 DMIS evaluation compared 1980 population and household estimates for census tracts and minor civil divisions against population and household counts from the 1980 census. Results from the evaluation were reported in several sources including: Engels, Richard. A Review of the Donnelley Marketing Estimates of Population and Households. Unpublished Donnelley Marketing paper. March 1982. Hodges, Ken and Mary Kay Healy. A Micro Application of a Modified Housing Unit Model for Tract Level Population and Household Estimates. Presented at the Annual Meeting of the Population Association of America. May 1984. 2 Hodges, Ken. Evaluation of DMIS Estimates of 1990 Population and Households for Census Tracts and Minor Civil Divisions. Unpublished DMIS paper. April 1993. Hodges, Ken. Evaluation of the DMIS 1990 Population and Household Estimates at the National, State and County Levels. Unpublished DMIS paper. April 1993. 3 Hodges, Ken. Evaluation of Claritas 1990 Population and Household Estimates. Unpublished Claritas paper. December 1993. The CACI 1990 evaluation is reported in documents distributed by CACI. 4 The suppliers' individual evaluations suggest relative accuracy levels, but are difficult to compare because the geographies evaluated are often different. For example, before 1990 census counts are used to evaluate 1990 estimates, they must be converted to the 1980 census tract and MCD geographies for which the 1990 estimates were produced. Suppliers accomplish this conversion in different ways, and the roster of geographies evaluated can vary widely. 5 The International Council of Shopping Centers (ICSC) conducted a vendor data comparison in 1986, a comparative evaluation in 1991, and another mid-decade comparison in 1995. The results were summarized in unpublished documents distributed to attendees at the annual ICSC conferences where the findings were presented. Some results from the 1986 comparison were described in: Chapman, John. 1987. "Cast a Critical Eye." American Demographics. 19 (2): pp. 31-33. 6 Pitkin, John. 1992. "A Comparison of Vendor Estimates of Population and Households with 1990 Census Counts in California." Applied Demography 7:1: pp. 5-8.
28
7 The NDS methodology is described in a September 1993 NDS document, "Demographic Updates and Forecasts: Our Approach Providing Demographics." 8 In 1997, Claritas and NDS introduced block group data from consumer databases as input to their block group population estimates. The set currently produced by Claritas continues the use of block group input, so differences between block group population estimates may be larger now than in 1996. 9 The Census Bureau did report 1990 census data for ZIP Codes in Summary Tape File 3B. However, these data were based on ZIP Code correspondence information from Geographic Data Technology--similar to that used by some of the private suppliers. Moreover, the STF 3B ZIP Codes reflect definitions current in 1992, and do not include "P.O. Box" ZIP Codes. Consequently, the 1990 census STF 3B data are of little relevance in the estimation of population for current ZIP Codes. 10 Because the Census Bureau released 1990 MARS data only at the tract level and above, SMI and Claritas contracted with the Census Bureau for special tabulations of the 1990 MARS data for all block groups nationwide. The Claritas tabulation was produced as Special Tabulation Product #171.
29
Ken Hodges
Claritas Inc.
Presented at the Annual Meeting of the Population Association of America, Chicago, Illinois, April 2 - 4, 1998.
Background The small area demographic estimates produced by private data firms are among the most widely used in the history of applied demography, but to many the data remain an enigma. Methodology statements range from informative to vague to nonexistent, and questions are often raised about how accurate such mass-produced numbers can be. Amid all the questions, considerable light has been shed on the private sector estimates. In 1982, Donnelley Marketing Information Services (DMIS) was the first to report an evaluation of tract level population and household estimates against census results.1 DMIS evaluated again after the 1990 census,2 and was joined by rivals Claritas and CACI.3 Although noteworthy, these nationwide evaluations made no direct comparisons between the various suppliers.4 Several independent investigations have compared the estimates of different suppliers. Most notable are the comparisons by the International Council of Shopping Centers (ICSC) in 1986, 1991 and 1995,5 and a 1992 study completed by John Pitkin.6 The 1991 ICSC and 1992 Pitkin studies were evaluations--computing error against 1990 census results, while the 1986 and 1995 ICSC studies were mid-decade comparisons investigating how different the suppliers' estimates were at various geographic levels. These investigations were useful, but limited to small numbers of rather large areas. The ICSC studies examined data for a small sample of metropolitan areas and counties, and for large census tract aggregations with populations usually in excess of 40,000. The Pitkin project made some comparisons at the census tract level, but also focused on larger areas, and was confined to the state of California. In the absence of a full comparison of private sector small area estimates, questions persist about how similar they are. One view is that methods and input data differ enough to produce significantly different results. Another view is that small area estimation capabilities are so limited that there is little difference between the estimates offered by the suppliers. Among the users of these estimates, one finds many saying they find no major differences among them, but at least as many questioning why one set is so different from another. This paper describes the first large scale comparison of private sector estimates, and an assessment of their similarities and differences.
A Unique Opportunity Claritas--one of the private data suppliers--is uniquely positioned to compare private sector estimates due to the acquisition activity of its corporate parent, VNU. Claritas had already merged with National Planning Data Corporation when VNU acquired Strategic Mapping Inc. (SMI) in May 1996. SMI (which had acquired DMIS) was completing its 1996/2001 estimates and projections at the time of the acquisition. VNU then acquired Urban Decision Systems (UDS) in January 1997, and National Decision Systems (NDS) in July 1997. Each firm had completed, and was providing its 1996/2001 estimates and projections when acquired by VNU.
1
Thus, Claritas has four sets of independently produced 1996/2001 estimates and projections--its own plus those of SMI, UDS and NDS. This paper reports a comparison of these four sets. It is not a hypothetical exercise, but a comparison of four actual sets of numbers produced by four then-independent firms. Each set was produced prior to VNU acquisition, and was in customer use over the normal shelf life of such data products. Like the 1986 and 1995 ICSC comparisons, the objective was not to determine which estimates were the most accurate, but the extent of variation among them. However, unlike all earlier efforts, this comparison examined current year estimates and five year projections for all 3,141 counties, 61,258 census tracts, and 266,399 block groups nationwide, as well as all residential ZIP Codes defined in 1996.
Methodology The methods used to produce the four sets of estimates and projections ranged from expedient efforts, requiring only a few weeks to establish county numbers and percentage them to block groups, to more ambitious efforts taking six months or more to incorporate small area input from numerous sources. A description of the methods used by each supplier is beyond the scope of this paper. However, methodologies and input data varied considerably, and specific differences are noted where potentially relevant to differences observed among the estimates.
The Comparison Design Data and Geographic Levels Comparisons focused on the estimates and projections of total population, population age 75 and above, and mean household income. Also compared were the suppliers' estimates of 1980 census population. It is not widely recognized that each supplier provides its own version of 1980 census counts reconfigured to 1990 census geography. Data were compared for all counties, census tracts (including block numbering areas), and block groups nationwide. Comparisons also were made for ZIP Codes, which are a non-standard geography, and merit a separate analysis. Measures of Difference The differences among four sets of population estimates can be measured many ways. This paper takes the perspective of the prospective data user who might be choosing between any two suppliers or from all four. The six pairs of the four suppliers are indicated below. Claritas vs. SMI Claritas vs. UDS Claritas vs. NDS SMI vs. UDS SMI vs. NDS UDS vs. NDS
2
For each supplier pair, the absolute percent difference (APD) between estimates was computed. Mean absolute percent differences (MAPD) were examined across all geographies, and in some cases, the distributions of APD were examined. Again, error rates could not be computed, but where comparable error rates have been reported for a census year, they provide a useful frame of reference for the MAPD values. To minimize the impact of extreme percent differences resulting from very small population numbers, differences were computed only for geographies where 1990 census population was 10 or more. When computing percent error, the difference between the estimate and the census count is expressed as a percent of the census count--the target against which error is being measured. However, when computing the percent difference between estimates, there is no basis for declaring one to be more accurate--or the logical denominator. For this reason, the denominators used in computing APD consisted of the mean of the two estimates being compared. The differences between supplier pairs suggest the uncertainty confronting a user choosing between two sources of data, and identifies the differences between specific suppliers. However, a user could choose from among all four suppliers, so it is important to measure the degree of uncertainty presented by the suppliers as a group. Differences for the suppliers as a group were based on the range between the highest and lowest estimate for each area. This measure was chosen because it is easily compared with the differences among supplier pairs, and because of its relevance to the user perspective. For example, if four population estimates were 950, 815, 809 and 804, the range between the high and low estimates is 146. Expressed as a percent of the mean of the high and low value (again, there is no basis for declaring either one the target), the APD is 16.6 percent. Discarding the middle values might seem questionable. However, in the world of mass produced small area estimates, the fact that the three lower estimates are similar, and the high estimate is an outlyer does not reduce uncertainty. The supplier estimating 950 might be the only one with a problem, but it could be the only one doing something right. As a data user, all one knows is that the suppliers suggest populations as high as 950 and as low as 804. The "range difference" of 16.6 percent reflects the uncertainty confronted by the user. Just what constitutes a large or small difference between estimates varies by application and user. This paper establishes no standards, but describes differences relative to all those observed. For example, mean percent differences in the low single digits were among the lowest observed, while those exceeding 20 percent were relatively high.
3
Results County Population County population estimates were similar for all suppliers. As indicated in Table 1, mean absolute percent difference (MAPD) ranged from a low of 0.6 (SMI/UDS) to a high of 1.7 (NDS/UDS). Such similarity was expected since all four suppliers based their county population estimates on the Census Bureau's annual county population estimates. Even the mean "range difference" (between the high and low estimates) was only 2.2 percent. The frequency distributions presented in Table 2 confirm that differences greater than five percent were rare--although the total within one percent ranged from 90.4 percent (SMI/UDS) to only 42.7 percent (SMI/NDS) and only 24.0 percent for the range. Also as expected, the mean differences were higher for the 2001 projections than for the 1996 estimates. MAPDs ranged from 1.9 percent for SMI/UDS to 4.0 percent for NDS/UDS (see Table 1). The mean range difference was 5.4 percent. The three largest mean differences involved pairings with NDS--perhaps because the NDS county projections were the only ones adjusted for conformity with metropolitan area population forecasts produced with regional economic models.7 Again, what constitutes a large difference depends on the user, but it is clear that, even for areas as large as counties, larger differences started to emerge in the five year projections--especially when the suppliers were considered as a group. Given the stability of county boundaries, one would expect few differences in the 1980 census populations presented by the four suppliers. Table 1 confirms that this is the case, as most pairs showed very small MAPDs (the 0.2 values actually round up from 0.16). However, mean differences involving UDS were conspicuously high at 2.1 percent--a greater difference than that typically observed in the 1996 estimates. The differences were not huge, but since there should have been exact agreement on most counties, the UDS 1980 county populations are unexplained outlyers.
Tract Population Just as estimation error typically increases for smaller geographies, estimation differences were greater for census tracts than for counties. Table 3 indicates that, for supplier pairs, MAPD ranged from 6.2 percent (NDS/SMI) to 7.8 percent (UDS/SMI). For areas as small as tracts, such differences were not large (by comparison, tract level mean absolute errors in 1990 were in the 11 to 16 percent range). The frequency distribution in Table 4 confirms that, for supplier pairs, only about 10 percent of tract population estimates differed by more than 15 percent (plus or minus), and roughly half were within five percent of each other. And just as estimation error decreases as tracts are aggregated, estimation differences would likely diminish with aggregation. Consistency was less impressive for the suppliers as a group. Mean absolute difference increased to 12.4 percent, 26 percent of estimates differed by 15 percent or more, and only 16 percent were within five percent of each other.
4
Again, the 2001 projections exhibited greater percent differences than the 1996 estimates. MAPD ranged from 10.4 percent (Claritas/NDS) to 13.4 percent (SMI/UDS), and the range MAPD was 20.9 percent (Table 3). Differences in 1980 census population converted to 1990 tracts were expected given tract boundary changes, and the creation of 1990 tracts in previously untracted areas. Details are sketchy, but the suppliers took different approaches in converting 1980 data to 1990 tracts. In fact, the suppliers agreed less on tract level 1980 census population than on 1996 estimated population. As reported in Table 3, mean differences ranged from 6.5 percent (Claritas/SMI) to 13.9 percent (UDS/NDS), and the range difference among all four was 18.2 percent. Such differences might be a surprise to many data users, since the suppliers do not label their 1980 data as "estimates." And with some suppliers relying on the projection of intercensal trends for their current year estimates, it is remarkable that such differences in the 1980 data (and 19801990 trends) did not translate to greater differences in the 1996 tract estimates. The similarity of 1996 tract population estimates is worth closer examination. Again, the MAPDs were in single digits for supplier pairs, and 12.4 percent for the range. In many tracts, the 1996 population estimates were quite similar, and it is easy to see how some users could get the impression that they are all pretty much the same. However, the tract estimates were not without important differences. For each supplier, Table 5 reports the distribution of estimated 1990-1996 population change. All suppliers estimated moderate change more frequently than rapid change (anything else would lack credibility). However, tracts estimated to have lost 10 percent or more of their population ranged from a high of 7.4 percent (UDS) to a low of 2.7 percent (NDS), and those growing 25 percent or more ranged from 7.1 percent (UDS) to only 2.9 percent (NDS). For very rapid growth (50 percent or more), Claritas was high with 1.1 percent of all tracts and NDS was low with 0.1 percent. Such differences do not translate to large mean differences, but they suggest that some suppliers were more aggressive than others in estimating rapid change. The suppliers may have estimated tract rates of change with similar frequencies, but crosstabulations identify the extent to which they agreed on which tracts were changing at specific rates. Appendix 1 presents crosstabluations of estimated 1990-1996 population change for the six supplier pairs and the range difference. Tracts on the main diagonal and those within two cells of the main diagonal reflect agreement or modest difference between suppliers. The shaded cells three or more cells away from the main diagonal reflect the larger differences. The crosstabulations merit individual attention, but Table 6 presents a summary of tracts "on," "within 2 cells," and "3+ cells off" the main diagonal. The summary provides further evidence of the similarity among tract estimates, as for all supplier pairs, most tracts were either on or within two cells of the main diagonal. Tracts three or more cells from the diagonal ranged from just 11.8 percent (NDS/UDS) to 18.5 percent (SMI/UDS) of total. However, these modest percentages translate to 7,060 and 11,082 tracts respectively. Thus, tracts where two suppliers disagreed on population change were a plentiful minority. And such disagreement was amplified when maximum and minimum rates of change were compared (Appendix 1G). Tracts on the main diagonal dropped from about 30 percent for supplier pairs to only 5.2
5
percent for the range, while those three or more cells from the diagonal jumped from 10+ percent to 35.9 percent--a total of 21,477 tracts (see Table 6). A data user choosing among the four sets of 1996 tract population estimates would confront considerable variation with respect to estimated change since 1990.
Block Group Population After producing tract level estimates throughout the 1980s, the private suppliers started producing block group estimates following the 1990 census--primarily because their enhanced desktop products required block group input. The logical expectation is that block group estimation error would be considerably higher than that for tracts, and one might expect block group differences to be much larger as well. Therefore, it is striking that the differences among the suppliers' 1996 block group population estimates were only moderately higher than those for census tracts (see Table 7). For the Claritas/SMI pair, the block group MAPD of 6.8 percent was only slightly higher than the 6.6 percent observed for tracts, and the range MAPD for suppliers as a group increased from 12.4 percent to only 16.5 percent. The increases in MAPD for the 2001 projections were similarly modest. The similarity of the tract and block group mean differences could be due to the fact that block groups are not that much smaller than tracts. Tracts have an average of only 3.7 block groups and a maximum of nine. Also, in 1996 the suppliers were doing little more than distributing tract estimates to block group based on 1990 census percentages. In the absence of unique block group input data, one might expect relatively small mean differences.8 The 1980 census populations for 1990 block groups showed relatively large variation (Table 7)-and these differences were much greater than those for tracts. MAPDs ranged from 20.6 percent (Claritas/SMI) to 38.2 percent (UDS/NDS), and the range difference was 49.3 percent. Contributing to these differences was the incomplete and imperfect block group level 19801990 geographic correspondence provided by the 1990 census TIGER files. Left to their own devices, the suppliers came up with rather different results in converting 1980 counts to 1990 block groups.
ZIP Code Population ZIP Code estimates are popular, but fraught with complications that are not widely understood. Defined by the U.S. Postal Service for the delivery of mail--not for the presentation of data-ZIP Codes are not just another geographic level. They are collections of delivery points rather than clearly defined areas. And unlike census geographies, which are stable throughout the decade, ZIP Codes change frequently at the discretion of local postal officials. A description of the complications of ZIP Code data would not fit in this paper, but it must be understood that such complications exist, and leave a heavy mark on the comparison of ZIP Code estimates.
6
The number and definition of residential ZIP Codes is always changing, and the suppliers seek to capture these elusive definitions each time they produce a set of estimates. In contrast to counties, tracts and block groups, where published 1990 census counts provide a common starting point, there are no "official" 1990 census counts for current ZIP Codes.9 For estimation purposes, ZIP Codes are typically defined in terms of correspondence with block groups. For example, a ZIP Code might be defined as the sum of: 100 percent of block groups 1 and 2, 47.8 percent of block group 4, and 6.2 percent of block group 5. Usually based on estimated ZIP Code boundaries from geographic data suppliers, block group-to-ZIP correspondence is identified for current ZIP Codes, and applied to existing block group data to produce estimates and census data for ZIP Codes. Due to such complications, the comparison of ZIP Code estimates began with the estimation of 1990 census population for residential ZIP Codes defined in 1996. There were 36,089 ZIP Codes in 1996 where at least one supplier estimated a 1990 population greater than zero. However, the suppliers differed widely in the number of residential ZIP Codes they recognized in 1996. Claritas recognized the largest number at 35,272 and NDS the smallest at 29,482. Such differences (summarized in Table 8) trace to the different geographic data sources used by the suppliers, and the handling of "P.O. Box" ZIPs, whose boundaries are especially ill-defined. Because of the differences in ZIP Code rosters, comparisons were limited to residential ZIPs recognized by all four suppliers in 1996, and where 1990 population was estimated to be at least 10 persons. The objective was to eliminate ZIP Codes whose existence was in dispute, as well as percent differences inflated by very small denominators. If anything, the ZIP Code differences reported below are understated. As indicated in Table 9, even where all four suppliers agreed on the existence of a ZIP Code, they varied widely in the estimation of 1990 census population. Most MAPDs for the supplier pairs were in the 20 to 30 percent range, and the mean difference between the high and low estimates was 41.3 percent. Notably consistent were the SMI and NDS estimates--with a MAPD of only 5.4 percent. This consistency likely reflects SMI's and NDS's common use of ZIP Code boundaries from Geographic Data Technology (GDT). Claritas also used GDT boundaries, but modified the resulting block group correspondence to produce ZIP Code data consistent with residential delivery counts. UDS used a different supplier for its ZIP boundary and correspondence files. Thus, one would expect larger differences in comparisons involving Claritas and/or UDS. Differences among the 1996 population estimates and 2001 projections for ZIP Codes also were quite large, but not dramatically greater than those for 1990 (Table 9). At least in terms of MAPD, much of the variation in the ZIP Code estimates and projections is explained by differences in estimating 1990. In other words, ZIP Code definition is the critical difference. Also, because the 1990 differences were so wide, 1996 differences could actually be lower in some ZIPs, thus dampening any increase in the MAPD. Again, however, the differences reported are understated by the elimination of ZIPs not recognized by one or more suppliers. Clearly, users of ZIP Code data (for census year as well as estimates and projections) would face wide differences in choosing among the suppliers compared.
7
Population by Age A detailed comparison of estimates of population by age is a subject for further research, but a limited comparison illustrates the types of differences that can be experienced. Estimates for the age "75 and above" category were compared because of its interest to users, including health care providers and the developers of assisted living facilities. Estimates for population segments were expected to differ more than those for total population because of the increased difficulty, and because the smaller totals would yield larger percent differences. Table 10 confirms that this is the case, as the 1996 MAPDs for the supplier pairs ranged from 3.1 to 6.9 for counties; 9.8 to 15.5 for tracts and from 12.9 to 19.9 for block groups. As usual, county differences were relatively small, while those for tracts and block groups were larger. Again, it was not until all four suppliers were considered that the extent of potential uncertainty was identified. The 1996 range MAPDs were 10.0 for counties, 24.5 for tracts and 31.1 for block groups. Also as usual, differences were consistently higher for 2001-with block group level pair differences as high as 31.5 percent (SMI/NDS) and a range MAPD of 48.0. Table 10 also reveals a consistently smaller difference between Claritas and SMI relative to other supplier pairs. This consistency may be due to the similarity of the cohort survival methods used by the two suppliers, as well as the fact that, unlike UDS and NDS, the SMI and Claritas age estimates began with age distributions from the 1990 census Modified Age/Race/Sex (MARS) files.10 The difference introduced by MARS is suggested by the MAPDs between the original and MARS totals for persons 75+. These differences were 1.0 percent for counties, 2.2 percent for tracts and 3.8 percent for block groups. While the typical user is most interested in the number of persons age 75+, and thinks in terms of differences as described above, part of this variation traces to differences in the estimation of total population. To control for this effect, the suppliers' estimates and projections were adjusted to conform with 1990 census total population (percent 75+ was applied to the common 1990 population total). The MAPDs in Table 11 confirm that, when differences in total population were eliminated, the data for population 75+ were somewhat more consistent. The reductions were not dramatic, but suggest that--especially for small areas--users could reduce uncertainty somewhat if they can use the age estimates in percent form.
Mean Household Income A comparison of average household income suggests the extent of variation that can be observed among income estimates and projections. The mean absolute differences in Table 12 do not follow the typical pattern. Differences tended to be larger for the smaller areas, but not always. For example, the Claritas/SMI 1996 MAPD was 7.5 for counties, but only 6.7 for tracts and block groups. As a summary statistic, mean income is not reduced in size (as are counts) for smaller areas. Therefore, larger estimation differences in small areas truly reflect the added difficulty and uncertainty associated with these levels, as opposed to the effect of
8
small denominators. Variation in estimates of the number of households by income likely would have fit the pattern observed for count based estimates. Even at the county level, there was substantial variation on mean income. MAPDs ranged from 4.4 percent to 14.1 percent in 1996, and as high as 14.2 percent in 2001 (Table 12). County range MAPDs were 15.9 for 1996 and 21.2 for 2001. At the tract level, the range MAPDs were 20.1 (1996) and 31.9 (2001), and for block groups the mean differences were 23.5 percent and 36.8 percent respectively. Definitional differences might explain part of the difference in estimated income. All four suppliers started with 1990 census household income, all four estimated "money income" in current dollar values, but not all "1996" income estimates reflected the same year. NDS and SMI estimated 1996 households by income earned in 1995 (just as the 1990 census asked income earned in 1989) while Claritas estimated income earned in 1996. UDS documents do not specify the target year for their income estimates. In any case, several of the income comparisons involved a built-in one year difference. Because Claritas targeted 1996 (and 2001), one would expect its estimates and projections to be higher than those targeted at 1995 (and 2000). Table 13 presents the mean algebraic percent differences (MALPD) among the mean income estimates. MALPD distinguishes positive and negative percent differences between estimates to provide a measure of bias--or the extent to which one set of estimates is higher than another. The Table 13 column headers indicate the direction of difference--a negative sign indicating that the estimates for the first supplier in the header were lower than those of the second. As expected, the Claritas 1996 income estimates were (on average) higher than SMI's targeted at 1995. All Claritas/SMI MALPEs were positive by modest amounts. Had SMI targeted 1996 (or Claritas targeted 1995), bias and mean absolute difference might have been lower. The Claritas/NDS comparison was more complex. The Claritas 1996 incomes were higher than the NDS 1995s at the county level, but lower for tracts and block groups (as well as national level). Such paradoxical reversals (also observed in a few other pairs) are possible when comparing means or other ratios, and merit further investigation. Nevertheless, it appears that, had NDS and Claritas targeted the same year, bias and mean absolute differences might have been greater than those actually observed for tracts and block groups. Thus, the one year difference in the income estimation date seems to explain some, but certainly not all of the differences observed in estimated mean household income.
Conclusions While far from exhaustive, the comparisons described above are the most extensive investigation of the differences among private sector demographic estimates. What constitutes a large versus small difference is to some degree in the eye of the user, but the results illustrate a mix of similarities and differences that make it apparent how some users might find little difference among the suppliers' numbers, while others report sizable differences.
9
As expected, differences were smaller for larger geographic units, such as counties, where population estimates rarely differed by more than a few percentage points. Tract and block group population estimates often were very close, but larger differences were not hard to find. Also as expected, there was greater variation on characteristics, such as age and income, but here too, the results were a mix of similarities and differences. In all cases, greater uncertainty was encountered when comparing the suppliers' maximum versus minimum estimates. Even if trivially obvious, this result is important from the data user perspective. Specifically, the relative similarity of estimates from two suppliers can create an illusion of consistency or certainty that is broken when one considers the range of alternatives. Perhaps surprising to some users would be the disparities in the conversion of 1980 census data to 1990 geography. At the tract and especially block group levels, there were major differences in the “estimation” of 1980 census population. Such differences suggest that small area 19801990 census trends, and estimates relying on the projection of such trends, should be used with caution. Also notable was the variation among ZIP Code estimates. The suppliers disagreed on the number of residential ZIP Codes in 1996, and by almost any standard, the differences among ZIP Code estimates would be considered large. The wide variation in 1990 census counts confirms that much of the disparity in ZIP Code estimates traces to differences in their definition by the four suppliers. One could summarize by observing that the four sets of estimates were striking in their similarity except where they were very different. Observed, similarities were impressive, but one could not summarily assert that there were no important differences among the suppliers' estimates. Methods were known to differ, and differences in the data often were consistent with differences in methods. Of course, the results do not identify which estimates were the most accurate–or whether any supplier's estimates were consistently more accurate. Estimates could vary widely, but share similar error rates. The observed differences--even where modest--leave room for substantial differences in accuracy, but the answers to the accuracy question must await evaluations against the 2000 census. Until then, users must scrutinize the alternative methods to gauge the suitability of the resulting data for their applications. The corporate acquisitions that made this paper possible have rendered it historical in that none of the four estimation programs function now as they did in 1996. SMI and UDS no longer exist as data suppliers. NDS and Claritas maintain separate identities within VNU, and Claritas continues to produce estimates combining elements from the original Claritas/NPDC, DMIS and NDS methods. Considerable competition remains, with as many as five firms currently producing small area estimates and projections. Methods still vary, and questions concerning data similarity are as relevant as ever. The present comparison does not encompass the current set of practitioners, but it provides rare and still relevant insights into the similarities and differences that can be expected among alternative sets of small area estimates and projections.
10
Table 1 MEAN ABSOLUTE PERCENT DIFFERENCE: COUNTY POPULATION:
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range1
1996 County Population 2001 County Population
1.3 2.3
1.3 2.2
1.1 3.3
0.6 1.9
1.6 3.9
1.7 4.0
2.2 5.4
1980 County Population
0.2
2.1
0.2
2.1
0.2
2.1
2.2
Data Item
1 MAPD based on difference between the highest and lowest estimate among the four suppliers.
Table 2 FREQUENCY DISTRIBUTION: ABSOLUTE PERCENT DIFFERENCE BETWEEN COUNTY LEVEL 1996 POPULATION ESTIMATES Absolute Pct Difference
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range1
0 to 1.0 1.0 to 4.9 5.0 to 9.9 10.0 to 14.9 15.0 to 19.9 20.0 to 24.9 25.0 to 49.9 50.0 or More
55.2 42.5 1.9 0.3 0.1 0.0 0.0 0.0
55.3 41.8 2.0 0.5 0.2 0.0 0.1 0.0
62.6 36.0 1.1 0.1 0.1 0.0 0.0 0.0
90.4 8.4 0.8 0.3 0.1 0.0 0.1 0.0
42.7 54.0 3.0 0.2 0.1 0.0 0.0 0.0
41.8 54.3 3.2 0.4 0.1 0.0 0.1 0.0
24.0 69.9 4.8 0.8 0.3 0.0 0.1 0.1
1.3
1.3
1.1
0.6
1.6
1.7
2.2
MAPD2
1 MAPD based on difference between the highest and lowest estimate among the four suppliers. 2 Mean Absolute Percent Difference.
Table 3
11
MEAN ABSOLUTE PERCENT DIFFERENCE: CENSUS TRACT POPULATION1
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1996 Tract Population 2001 Tract Population
6.6 10.7
7.2 11.9
6.4 10.4
6.2 13.4
6.4 10.8
7.8 11.5
12.4 20.9
1980 Tract Population
6.5
11.0
9.5
11.9
8.4
13.9
18.2
Data Item
1 Based on all census tracts with a 1990 census population of 10 or more. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
Table 4 FREQUENCY DISTRIBUTION: ABSOLUTE PERCENT DIFFERENCE BETWEEN TRACT LEVEL 1996 POPULATION ESTIMATES 1
Absolute Pct Difference
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
0 to 1.0 1.0 to 4.9 5.0 to 9.9 10.0 to 14.9 15.0 to 19.9 20.0 to 24.9 25.0 to 49.9 50.0 or More
13.1 40.4 26.8 10.8 4.3 2.1 2.2 0.4
14.0 40.3 23.6 10.6 5.0 2.6 3.3 0.6
13.0 42.1 26.5 10.3 4.1 1.8 1.9 0.4
10.5 35.3 27.7 13.5 6.3 3.0 3.3 0.4
12.2 41.0 28.2 11.2 4.3 1.6 1.4 0.1
13.0 41.8 26.1 10.5 4.3 2.0 2.0 0.2
0.3 15.6 35.0 23.1 11.8 6.1 7.1 1.0
6.6
7.2
6.4
6.2
6.4
7.8
12.4
MAPD3
1 Table includes all census tracts with a 1990 census population count of 10 or more. N = 59,366. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers. 3 Mean Absolute Percent Difference
Table 5
12
ESTIMATED POPULATION CHANGE 1990-1996 Pct Change -50 or more -25.0 to -49.9 -20.0 to -24.9 -15.0 to -19.9 -10.0 to -14.9 -5.0 to - 9.9 -0 to -4.9 0 to 4.9 5.0 to 9.9 10.0 to 14.9 15.0 to 19.9 20.0 to 24.9 25.0 to 49.9 50.0 or More
Claritas
SMI
UDS
NDS
0.1 0.5 0.5 1.2 3.5 10.3 19.9 24.0 15.7 9.3 5.5 3.2 5.2 1.1
0.0 0.0 0.1 0.4 2.9 10.9 20.4 23.0 15.6 10.1 6.2 3.9 5.7 0.8
0.1 0.9 0.7 1.7 4.0 11.4 19.9 20.6 15.0 9.3 5.7 3.7 6.4 0.7
0.0 0.1 0.3 0.8 1.5 4.7 18.2 30.1 21.1 11.2 5.9 3.2 2.8 0.1
N
59813
59812
59830
59815
5.8 16.1 6.3 43.9
4.3 15.2 6.5 43.4
7.4 18.8 7.1 40.5
2.7 7.4 2.9 48.3
10 pct loss 5 pct loss 25 pct growth -5 pct to +5 pct
13
Table 6
CORRESPONDENCE BETWEEN ESTIMATED POPULATION CHANGE 1990-1996 1 Percent of Tracts
On Main Diagonal2 Within 2 Cells3 3+ Cells Off Diagonal4 Opposite Direction5 N of Tracts
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
NDS vs. UDS
Range
30.4 56.5 13.1
31.6 53.0 15.4
29.5 58.5 11.9
25.4 56.1 18.5
28.7 59.4 11.9
30.0 58.2 11.8
5.2 58.9 35.9
3.1
2.6
2.6
4.9
2.6
6.5
11.4
59,826
59,830
59,813
59,830
59,812
59,815
59,804
1 Percentages from crosstabulations presented in Appendix 1. Table includes all census tracts with a 1990 census population count of 10 or more. 2 Estimated rates of change in same category. 3 Estimated rates not on main diagonal, but within two cells of it. 4 Estimated rates 3 or more cells from main diagonal. 5 One supplier estimates loss of 5 percent or more. Other supplier estimates growth of 5 percent or more. Minimum difference of 10 percentage points.
14
Table 7
MEAN ABSOLUTE PERCENT DIFFERENCE: BLOCK GROUP POPULATION1
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1996 Block Group Pop. 2001 Block Group Pop.
6.8 11.1
10.5 17.8
7.8 12.9
10.5 18.1
7.8 13.6
10.4 18.4
16.5 28.1
1980 Block Group Pop.
20.6
20.9
33.8
27.3
27.1
38.2
49.3
Data Item
1 Based on all block groups with a 1990 census population of 10 or more. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
15
Table 8
RESIDENTIAL ZIP CODES RECOGNIZED IN 1996 Total1 Claritas SMI UDS NDS All Suppliers2
36,089 35,272 34,672 31,634 29,482 29,161
1 1996 residential ZIP Codes where at least one supplier estimated a 1990 census population greater than zero. 2 1996 residential ZIP Codes where all four suppliers estimated a 1990 census population greater than zero.
Table 9 MEAN ABSOLUTE PERCENT DIFFERENCE: ZIP CODE POPULATION1
Comparison Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1990 ZIP Code Pop.
25.3
32.4
27.0
22.8
5.4
23.2
41.3
1996 ZIP Code Pop. 2001 ZIP Code Pop.
25.5 26.3
33.0 34.2
27.3 28.4
25.1 27.4
8.8 12.1
25.2 27.5
43.7 46.7
Data Item
1 Residential ZIP Codes recognized by all four suppliers, and with estimated 1990 population greater than 10. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
16
Table 10 MEAN ABSOLUTE PERCENT DIFFERENCE: POPULATION AGE 75 AND ABOVE 1 Comparison
Data Item 1996 County Pop 75+ 2001 County Pop 75+ 1996 Tract Pop 75+ 2001 Tract Pop 75+ 1996 BG Pop 75+ 2001 BG Pop 75+
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
3.1 4.7
5.0 7.5
6.9 11.3
5.7 9.0
6.9 11.3
5.6 9.6
10.0 16.1
9.8 15.0
13.6 21.4
14.5 23.4
15.5 24.2
14.9 24.7
13.1 21.5
24.5 38.9
12.9 18.6
17.3 26.6
17.9 28.1
19.4 29.6
19.9 31.5
16.1 26.9
31.1 48.0
1 Table includes all areas where both the STF 1 and MARS 1990 population counts were 10 or more. 2 Difference based on maximum vs. minimum estimate among all suppliers.
Table 11 MEAN ABSOLUTE PERCENT DIFFERENCE: POPULATION AGE 75 AND ABOVE ADJUSTED FOR CONFORMITY WITH 1990 CENSUS POPULATION1 Comparison
Data Item 1996 County Pop 75+ 2001 County Pop 75+ 1996 Tract Pop 75+ 2001 Tract Pop 75+ 1996 BG Pop 75+ 2001 BG Pop 75+
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
3.2 5.0
5.1 7.3
6.8 10.3
5.7 9.1
6.6 10.3
5.4 8.3
9.8 15.1
7.2 10.7
11.1 17.0
13.0 20.7
12.7 19.2
13.7 22.3
10.3 16.6
20.3 31.6
11.1 15.4
13.5 19.5
16.2 24.9
16.0 23.5
18.3 28.5
11.3 18.6
25.8 38.7
1 Table includes all areas where both the STF 1 and MARS 1990 population counts were 10 or more. 2 Difference based on maximum vs. minimum estimate among all suppliers.
17
Table 12 MEAN ABSOLUTE PERCENT DIFFERENCE MEAN HOUSEHOLD INCOME1 Comparison
Data Item 1996 County Avg HH Income 2001 County Avg HH Income 1996 Tract Avg HH Income 2001 Tract Avg HH Income 1996 BG Avg HH Income 2001 BG Avg HH Income
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
7.5 12.3
14.1 13.9
8.0 14.2
8.0 10.7
4.4 10.3
9.1 8.6
15.9 21.2
6.7 11.0
11.8 16.2
12.0 22.7
8.0 15.9
10.5 21.3
15.8 18.5
20.1 31.9
6.7 10.9
14.0 19.4
13.3 24.7
10.4 18.1
12.2 23.0
19.1 25.3
23.5 36.8
1 Table includes all areas with a 1990 census household count of 10 or more.
Table 13 MEAN ALGEBRAIC PERCENT DIFFERENCE: MEAN HOUSEHOLD INCOME1 Comparison Data Item 1996 County Avg HH Income 2001 County Avg HH Income 1996 Tract Avg HH Income 2001 Tract Avg HH Income 1996 BG Avg HH Income 2001 BG Avg HH Income
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
5.8 8.7
13.6 -0.7
4.9 2.6
7.9 10.7
-0.9 -6.1
-8.7 3.4
3.6 4.4
10.1 -7.3
-3.6 -13.8
6.5 -11.7
-7.2 -18.2
-13.6 -6.5
3.5 4.1
11.6 -2.8
-4.2 -13.4
8.2 -6.9
-7.7 -17.5
-15.7 -10.2
1 Computation based on order of supplier in column title. A negative value indicates the tendency for the first supplier in the title to have lower estimates than the second supplier in the title. Table includes all areas with a 1990 census household count of 10 or more.
APPENDIX 1A
18
TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: CLARITAS VS. SMI SMI Claritas
-50 or more
-25 to -50
-20 to -25
-50 or more
1
3
-25 to -50
2
8
-20 to -25 -15 to -20
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
1
4
3
8
14
2
1
17
24
26
52
62
68
16
7
8
6
7
3
3
10
21
38
80
56
35
20
14
5
1
6
1
4
12
32
134
178
143
116
54
23
11
8
17
8
71
363
614
471
299
141
68
30
14
19
1
-10 to -15 -5 to -10
1
3
54
531
1861
1975
1016
405
164
89
31
36
6
-0 to -5
1
2
22
396
2101
4264
3094
1154
462
198
98
84
5
0 to 5
1
2
14
167
1085
3372
5114
2597
1160
434
200
196
10
4
37
334
1191
2457
2628
1471
660
325
267
10
2
15
101
383
923
1324
1318
800
372
329
24
2
2
53
125
295
538
703
697
466
381
25
1
5
13
60
142
227
332
344
315
445
31
25 to 50
5
26
76
164
203
297
363
419
1393
177
50 or more
4
7
18
41
32
35
45
51
236
186
5 to 10 10 to 15
1
15 to 20 20 to 25
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
1
N of Tracts
Pct
18,190 33,819 7,187 59,826
30.4 56.5 13.1 100.0
19
APPENDIX 1B TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: CLARITAS VS. UDS UDS Claritas
-50 or more
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
5
1
7
2
4
8
3
2
15 to 20
20 to 25
25 to 50
50 or more
2
1
1
1
-25 to -50
16
104
20
20
19
40
22
27
19
7
4
1
7
-20 to -25
4
42
64
37
25
29
22
25
19
8
4
5
5
1
-15 to -20
6
38
68
191
139
84
68
60
37
15
12
6
7
1
-10 to -15
8
46
59
229
580
474
310
180
102
43
23
20
23
2
-5 to -10
4
45
42
155
628
2203
1661
768
357
153
69
37
46
5
-0 to -5
4
88
58
154
442
2102
4468
2609
1067
423
207
98
150
12
0 to 5
12
88
50
118
307
1108
3481
4804
2462
1005
421
219
254
22
5 to 10
4
35
29
47
147
426
1108
2305
2654
1359
619
280
346
26
10 to 15
5
25
13
24
46
159
405
798
1188
1359
778
356
414
23
15 to 20
1
15
9
19
29
78
154
349
517
603
639
383
460
33
20 to 25
3
10
4
14
13
36
67
144
233
249
271
348
493
30
25 to 50
4
14
8
9
32
40
92
190
248
262
301
385
1375
163
50 or more
4
7
3
4
4
11
18
41
52
53
42
53
244
119
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
18,908 31,722 9,200 59,830
31.6 53.0 15.4 100.0
20
APPENDIX 1C TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: CLARITAS VS. NDS NDS Claritas
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
2
1
7
3
9
4
6
1
-25 to -50
10
22
65
62
34
30
44
17
14
3
-20 to -25
4
9
29
68
69
49
28
14
12
4
-15 to -20
2
9
33
84
233
184
104
42
24
11
2
4
-10 to -15
6
20
49
145
484
750
435
134
49
18
4
6
-5 to -10
10
31
44
132
891
2628
1713
518
148
24
19
15
-0 to -5
8
30
76
149
565
3951
4763
1675
459
130
49
27
0 to 5
8
25
94
121
319
2313
6220
3650
1093
350
102
52
5 to 10
3
8
39
55
105
606
2954
3274
1538
528
184
87
10 to 15
1
3
19
35
51
179
1024
1808
1415
699
239
117
1
15 to 20
2
5
23
24
70
367
767
883
637
327
182
1
20 to 25
1
1
7
8
14
41
164
325
444
395
306
207
2
2
8
18
22
52
144
360
534
639
575
757
11
2
7
16
8
13
26
47
78
119
114
200
22
-50 or more
25 to 50 50 or more
1
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
17,670 35,008 7,127 59,813
29.5 58.5 11.9 99.9
21
2 1
3 3
APPENDIX 1D TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: SMI VS. UDS UDS SMI
-50 or more
-25 to -50
-20 to -25
-15 to -20
-50 or more
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
1
1
1
2
4
5
1
2
2
15 to 20
20 to 25
25 to 50
50 or more
-25 to -50
2
3
-20 to -25
2
3
5
11
10
10
8
3
1
-15 to -20
1
22
17
29
43
65
39
17
9
1
4
1
-10 to -15
4
52
47
122
276
432
416
214
88
41
18
7
8
2
-5 to -10
11
96
102
251
604
1655
1921
1064
482
176
56
36
46
6
-0 to -5
8
111
105
257
603
2131
3754
2834
1306
563
240
114
166
11
0 to 5
23
119
71
183
442
1383
3265
3854
2413
1024
470
228
285
21
5 to 10
7
56
40
85
205
627
1428
2308
2137
1225
612
300
296
15
10 to 15
4
34
21
35
108
249
559
1076
1264
1134
696
407
439
32
15 to 20
4
26
8
28
55
123
237
466
626
660
578
347
501
25
20 to 25
2
16
3
14
29
45
131
215
315
324
346
308
517
41
25 to 50
4
18
8
10
28
64
115
232
293
360
350
411
1351
173
50 or more
4
6
1
4
3
7
18
19
35
23
33
215
111
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
15,195 33,553 11,082 59,830
25.4 56.1 18.5 100.0
22
1 1
1
APPENDIX 1E TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: SMI VS. NDS NDS SMI
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-50 or more
-0 to -5
0 to 5
1
2
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
-25 to -50
3
2
1
2
4
5
3
2
-20 to -25
6
2
10
5
7
14
8
1
2
-15 to -20
10
11
10
38
55
88
28
4
1
2
1
-10 to -15
7
20
39
128
340
693
366
100
22
7
3
1
-5 to -10
6
34
81
182
853
2559
2030
583
138
25
9
6
-0 to -5
5
34
110
191
746
3702
4843
1904
490
129
39
10
0 to 5
11
34
126
195
469
2363
5733
3379
998
311
115
37
5 to 10
3
5
41
73
178
837
2842
3235
1473
458
144
52
10 to 15
3
5
21
45
81
320
1196
1768
1531
718
263
104
1
15 to 20
1
7
20
28
43
130
475
853
934
717
324
150
2
20 to 25
2
9
16
21
64
254
384
511
468
363
211
1
25 to 50
2
5
10
13
30
83
194
398
544
672
581
877
5
50 or more
1
3
4
6
1
7
18
20
48
49
82
212
28
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
17,182 35,534 7,096 59,812
28.7 59.4 11.9 100.0
23
APPENDIX 1F TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: NDS VS. UDS UDS NDS
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
-50 or more -25 to -50
4
14
5
4
3
1
3
7
13
4
1
1
-20 to -25
13
24
2
9
9
26
11
21
31
7
3
4
3
-15 to -20
17
126
17
21
20
25
52
78
66
28
16
6
9
2
-10 to -15
12
107
113
117
102
95
88
106
81
40
29
13
17
3
-5 to -10
8
55
118
407
596
750
357
245
134
74
34
24
24
-0 to -5
3
60
76
253
916
3134
3964
1541
482
198
96
61
76
9
0 to 5
7
89
52
149
546
2042
5321
5942
2538
752
275
101
167
9
5 to 10
5
50
25
41
136
562
1656
3174
3587
2045
738
318
281
14
10 to 15
2
18
14
20
62
108
314
855
1443
1526
1067
572
667
24
15 to 20
1
14
3
4
12
39
77
217
385
598
743
561
848
54
20 to 25
1
3
2
3
6
9
29
75
127
189
280
335
790
75
25 to 50
1
1
4
5
11
40
64
77
111
192
935
219
7
29
1
50 or more
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
17,950 34,805 7,060 59,815
30.0 58.2 11.8 100.0
24
APPENDIX 1G TRACT LEVEL ESTIMATED PERCENT CHANGE IN POPULATION, 1990 - 1996: MAXIMUM VS. MINIMUM MIN MAX
-50 or more
-25 to -50
-20 to -25
-15 to -20
-10 to -15
-5 to -10
-0 to -5
0 to 5
5 to 10
10 to 15
15 to 20
20 to 25
25 to 50
50 or more
-50 or more 2
-25 to -50 -20 to -25
3
2
-15 to -20
2
15
5
-10 to -15
7
40
42
61
17
-5 to -10
11
95
119
291
422
198
-0 to -5
16
125
160
413
1414
3024
772
0 to 5
30
179
133
436
1353
4385
5856
1062
5 to 10
13
125
106
247
694
2304
4863
4136
416
10 to 15
5
70
63
122
337
960
2135
3290
1807
168
15 to 20
6
46
30
85
162
418
940
1638
1657
750
75
20 to 25
6
32
14
55
91
201
487
818
975
776
291
23
25 to 50
7
43
27
55
126
246
556
953
1163
1276
1018
619
370
50 or more
9
14
10
10
25
27
49
104
113
168
192
183
298
On Main Diagonal Within 2 Cells 3+ Cells Off Diagonal Total
N of Tracts
Pct
3,119 35,208 21,477 59,804
5.2 58.9 35.9 100.0
25
16
APPENDIX 2 SUMMARY TABLE OF MEAN ABSOLUTE PERCENT DIFFERENCE BETWEEN POPULATION ESTIMATES 1 Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range2
1.3 2.3
1.3 2.2
1.1 3.3
0.6 1.9
1.6 3.9
1.7 4.0
2.2 5.4
1996 Tract Population 2001 Tract Population
6.6 10.7
7.2 11.9
6.4 10.4
6.2 13.4
6.4 10.8
7.8 11.5
12.4 20.9
1996 Block Group Pop. 2001 Block Group Pop.
6.8 11.1
10.5 17.8
7.8 12.9
10.5 18.1
7.8 13.6
10.4 18.4
16.5 28.1
1980 County Population 1980 Tract Population 1980 Block Group Pop.
0.2 6.5 20.6
2.1 11.0 20.9
0.2 9.5 33.8
2.1 11.9 27.3
0.2 8.4 27.1
2.1 13.9 38.2
2.2 18.2 49.3
1990 ZIP Population 1996 ZIP Population 2001 ZIP Population
25.3 25.5 26.3
32.4 33.0 34.2
27.0 27.3 28.4
22.8 25.1 27.4
5.4 8.8 12.1
23.2 25.2 27.5
41.3 43.7 46.7
Data Item 1996 County Population 2001 County Population
1 Table includes all areas with a 1990 census population count of 10 or more. 2 MAPD based on difference between the highest and lowest estimate among the four suppliers.
26
APPENDIX 3 SUMMARY TABLE OF MEAN ABSOLUTE PERCENT DIFFERENCE BETWEEN AGE AND INCOME ESTIMATES Data Item 1996 County Pop 75+ 2001 County Pop 75+
1996 Tract Pop 75+ 2001 Tract Pop 75+
1996 BG Pop 75+ 2001 BG Pop 75+
1996 County Pct Pop 75+ 2001 County Pct Pop 75+
1996 Tract Pct Pop 75+ 2001 Tract Pct Pop 75+
1996 BG Pct Pop 75+ 2001 BG Pct Pop 75+
1996 County Avg HH Income 2001 County Avg HH Income
1996 Tract Avg HH Income 2001 Tract Avg HH Income
1996 BG Avg HH Income 2001 BG Avg HH Income
Claritas vs. SMI
Claritas vs. UDS
Claritas vs. NDS
SMI vs. UDS
SMI vs. NDS
UDS vs. NDS
Range1
3.1 4.7
5.0 7.5
6.9 11.3
5.7 9.0
6.9 11.3
5.6 9.6
10.0 16.1
9.8 15.0
13.6 21.4
14.5 23.4
15.5 24.2
14.9 24.7
13.1 21.5
24.5 38.9
12.9 18.6
17.3 26.6
17.9 28.1
19.4 29.6
19.9 31.5
16.1 26.9
31.1 48.0
3.2 5.0
5.1 7.3
6.8 10.3
5.7 9.1
6.6 10.3
5.4 8.3
9.8 15.1
7.2 10.7
11.1 17.0
13.0 20.7
12.7 19.2
13.7 22.3
10.3 16.6
20.3 31.6
11.1 15.4
13.5 19.5
16.2 24.9
16.0 23.5
18.3 28.5
11.3 18.6
25.8 38.7
7.5 12.3
14.1 13.9
8.0 14.2
8.0 10.7
4.4 10.3
9.1 8.6
15.9 21.2
6.7 11.0
11.8 16.2
12.0 22.7
8.0 15.9
10.5 21.3
15.8 18.5
20.1 31.9
6.7 10.9
14.0 19.4
13.3 24.7
10.4 18.1
12.2 23.0
19.1 25.3
23.5 36.8
1MAPD based on difference between the highest and lowest estimate among the four suppliers.
27
NOTES 1 The 1982 DMIS evaluation compared 1980 population and household estimates for census tracts and minor civil divisions against population and household counts from the 1980 census. Results from the evaluation were reported in several sources including: Engels, Richard. A Review of the Donnelley Marketing Estimates of Population and Households. Unpublished Donnelley Marketing paper. March 1982. Hodges, Ken and Mary Kay Healy. A Micro Application of a Modified Housing Unit Model for Tract Level Population and Household Estimates. Presented at the Annual Meeting of the Population Association of America. May 1984. 2 Hodges, Ken. Evaluation of DMIS Estimates of 1990 Population and Households for Census Tracts and Minor Civil Divisions. Unpublished DMIS paper. April 1993. Hodges, Ken. Evaluation of the DMIS 1990 Population and Household Estimates at the National, State and County Levels. Unpublished DMIS paper. April 1993. 3 Hodges, Ken. Evaluation of Claritas 1990 Population and Household Estimates. Unpublished Claritas paper. December 1993. The CACI 1990 evaluation is reported in documents distributed by CACI. 4 The suppliers' individual evaluations suggest relative accuracy levels, but are difficult to compare because the geographies evaluated are often different. For example, before 1990 census counts are used to evaluate 1990 estimates, they must be converted to the 1980 census tract and MCD geographies for which the 1990 estimates were produced. Suppliers accomplish this conversion in different ways, and the roster of geographies evaluated can vary widely. 5 The International Council of Shopping Centers (ICSC) conducted a vendor data comparison in 1986, a comparative evaluation in 1991, and another mid-decade comparison in 1995. The results were summarized in unpublished documents distributed to attendees at the annual ICSC conferences where the findings were presented. Some results from the 1986 comparison were described in: Chapman, John. 1987. "Cast a Critical Eye." American Demographics. 19 (2): pp. 31-33. 6 Pitkin, John. 1992. "A Comparison of Vendor Estimates of Population and Households with 1990 Census Counts in California." Applied Demography 7:1: pp. 5-8.
28
7 The NDS methodology is described in a September 1993 NDS document, "Demographic Updates and Forecasts: Our Approach Providing Demographics." 8 In 1997, Claritas and NDS introduced block group data from consumer databases as input to their block group population estimates. The set currently produced by Claritas continues the use of block group input, so differences between block group population estimates may be larger now than in 1996. 9 The Census Bureau did report 1990 census data for ZIP Codes in Summary Tape File 3B. However, these data were based on ZIP Code correspondence information from Geographic Data Technology--similar to that used by some of the private suppliers. Moreover, the STF 3B ZIP Codes reflect definitions current in 1992, and do not include "P.O. Box" ZIP Codes. Consequently, the 1990 census STF 3B data are of little relevance in the estimation of population for current ZIP Codes. 10 Because the Census Bureau released 1990 MARS data only at the tract level and above, SMI and Claritas contracted with the Census Bureau for special tabulations of the 1990 MARS data for all block groups nationwide. The Claritas tabulation was produced as Special Tabulation Product #171.
29
