Public Opinion Quarterly, Vol. 69, No. 2, Summer 2005, pp. 232–245
THE COST-EFFECTIVENESS OF ALTERNATIVE ADVANCE MAILINGS IN A TELEPHONE SURVEY
LARRY A. HEMBROFF DEBRA RUSZ Michigan State University ANN RAFFERTY HARRY MCGEE Michigan Department of Community Health NATHANIEL EHRLICH Michigan State University
Abstract Advance letters are being used increasingly to combat the decline in response rates, but their effectiveness depends partly on their being read. In the face of the post-9/11 anthrax mailing scare, the Office for Survey Research (OSR) converted to using advance postcards rather than advance letters. Subsequently, the OSR conducted an experiment to test the effectiveness of sending letters, postcards, or nothing to directory-listed numbers selected in a large random digit dial (RDD) telephone survey. In terms of response rates, the results clearly indicate that sending advance letters is more effective than sending postcards, which, in turn, is more effective than sending nothing. A detailed cost-effectiveness evaluation indicates that sending letters actually saves money, despite the increased costs associated with the mailings, compared to sending postcards; sending postcards is also more cost-effective than not sending anything.
Introduction
Response rates in surveys have been declining for several decades (Groves and Couper 1998; Steeh 1981; Steeh et al. 2001), and the rate of decline increased after the mid-1990s (Curtin, Presser, and Singer 2005; Smith 2003). Groves and Couper (1998) and Curtin, Presser, and Singer (2003) have speculated that the decline in response rates might have been even
Address correspondence to Larry Hembroff; e-mail: hembroff@msu.edu. doi:10.1093 / poq / nfi021 © The Author 2005. Published by Oxford University Press on behalf of the American Association for Public Opinion Research. All rights reserved. For permissions, please e-mail: journals.permissions@oupjournals.org.
Cost Effectiveness of Advance Mailings
233
greater had it not been for heroic efforts on the part of survey research organizations to counteract the trend. Such efforts as increasing the numbers of call and refusal conversion attempts, sending advance letters, and providing preor postinterview incentives and rewards generally increase interviewer effort, as well as total survey costs. This article focuses on the effectiveness and costeffectiveness of one of these strategies, sending advance notification of the survey participation request. A number of previous studies have found that advance notification about a telephone survey for which a household will soon be contacted can improve cooperation in random digit dial (RDD) surveys (Camburn et al. 1996; Hembroff 1998; Link et al. 2003). The magnitude of the improvement has varied considerably, from 4.4 percent (Dillman, Gallegos, and Frey 1976) to 13.4 percent (Traugott, Groves, and Lepkowski 1987). However, a few studies have reported either smaller effects or no apparent effect at all (Brick et al. 1997; Hembroff 1998; Pennell 1990; Singer, Van Hoewyk, and Maher 2000). Other studies have indicated that the effectiveness of advance letters varies with such factors as the length of the letter (Dillman, Gallegos, and Frey 1976; White and Freeth 1996), the organization on the letterhead (Brunner and Carroll 1969), the time lag between mailing and telephone contact (Pennell 1990), and the person to whom the letter is addressed (Groves and Couper 1998). Particularly germane to the last point, Couper, Mathiowetz, and Singer (1995) indicate that, in about half of households, all the mail is sorted by a single individual, and that 60 percent discard some mail without opening it, but that this rarely happens to letters addressed to specific household members. In summary, advance letters tend to be less effective if their length dissuades people from reading them, if they are not opened and read, if they are read too long prior to contact to recall, and if their sponsorship discounts the value of what is read. Additionally, Camburn et al. (1996) found that letters that clearly indicate that the purpose of the survey is to serve the interests of a valued group (e.g., the immunization of children) increased cooperation and reduced refusals, compared to less explicit letters or sending no prenotification letters. The Office for Survey Research (OSR) at Michigan State University, in cooperation with the Michigan Department of Community Health, had experimented with the use of advance letters for directory-listed phone numbers in an RDD sample for Michigan’s Behavioral Risk Factor Survey (Hembroff 1998). Hembroff reported that advance letters improved the response rate by 10.6 percentage points, reduced the refusal rate by 10.1 percentage points, and reduced the number of call attempts needed to reach a final call disposition roughly 0.5 attempts, compared to listed numbers to which no mailing was sent. But, as has already been noted, it is only possible for advance mailings to effect a difference in the outcome of calling attempts if the letters are received and read by those to whom they are sent. The last point—that prenotice letters must be received and read—was, at least temporarily, made problematic by the anthrax-laced mailing incidents
234
Hembroff et al.
that occurred on the heels of the September 11, 2001, terrorist attacks on the World Trade Center and Pentagon. Several people died from the anthrax exposure, several others were sickened, and hundreds of individuals were temporarily dislocated while office buildings and postal centers were decontaminated. Moreover, there were widespread reports that large numbers of householders were discarding without opening mailings they received from sources they did not recognize out of fear that the letters might be contaminated with anthrax. Consequently, for several RDD surveys, the OSR elected to send a postcard rather than a letter. Much, but not all, of the key information about the phone survey could be included on the postcard. Since the information would be visible without having to open anything that could contain anthrax, we hoped this would decrease the likelihood it would be discarded unread. Postcards are considerably less expensive to produce than letters. They also appear less formal and “official” than a letter might; they are more difficult to personalize (Dillman 2000); and they can include less information about the survey than might be included in a letter.1 In the context of mail surveys, Dillman (2000) advocates letter formats rather than postcards for prenotification mailings. He argues that it takes only a few seconds to look at a postcard, flip it over, and lay it aside—too short a time to get it registered in long-term memory. Setting aside the possibility of enriching a letter over a postcard with more visual and trust-inducing elements, a letter’s envelope has to be opened, the letter extracted and opened up, reviewed, and then posted, stored, or disposed of, thus increasing the likelihood of the householder’s registering it in long-term memory. However, no previous studies have examined the impact of postcards on response rates or survey costs in telephone surveys. An experimental test of the effectiveness of postcards rather than letters had not been practical in the immediate post-9/11 time period, but a year later nearly all the anthrax-related anxieties had been greatly reduced. There was no reason to continue using postcards to allay fears, but there was a possible incentive to use them if their response rate effects were comparable to those of letters. The lower costs to produce, process, and mail postcards led us to design an experiment to determine: • • • The effect of advance mailings on response rates; Whether letters or postcards are equally effective; and The cost-effectiveness of both letters and postcards.
Our operating hypotheses were that advance mailings would increase response rates, would decrease refusals, and would reduce the calling effort needed to complete the survey (Camburn et al. 1996). Based partly on Dillman (2000),
1. Obviously, it would also be difficult to include an incentive with a prenotification postcard, whereas this can be done with a letter. In this study no incentives or rewards were offered for participation due to budget limitations and to keep the experimental manipulations as clean as possible.
Cost Effectiveness of Advance Mailings
235
we also expected that advance letters would be more effective in these objectives than advance postcards.
Methods
We conducted the experiment as a part of a list-assisted RDD telephone survey. This survey was conducted in 2002 and focused on nutrition and physical activity. The targeted sample size was 3,500 interviews with randomly selected English-speaking adults, following protocols established by the Centers for Disease Control and Prevention for the Behavioral Risk Factor Surveillance System (BRFSS).2 The protocol required up to 15 call attempts for each selected telephone number or 15 call attempts after first contact unless calling the number resulted in a completed interview, a hard refusal or a second refusal, or was determined to be nonworking or ineligible. The interview was designed to last approximately 25 minutes. The sampling frame was stratified into listed numbers and nonlisted numbers in 100-blocks containing one or more listed numbers (i.e., 1+ blocks).3 Listed numbers were sampled at a much higher rate than were nonlisted numbers. The final samples included 13,849 listed numbers and 8,401 nonlisted numbers. The RDD samples were purchased from Survey Sampling, Inc., in Fairfield, Connecticut (SSI). Once the sample of listed numbers was selected, SSI extracted the name and address of the household subscriber for all phone numbers listed in their residential telephone directory database and forwarded that information to OSR separately so that we could send advance mailings to the householder. A total of 13,849 listed numbers were used for the survey. SSI divided the samples into replicates of 100 numbers each. The replicates of listed numbers were then also divided randomly into one of three mailing treatment groups:
1. 2. 3. The Control Group: those households with listed numbers that received no advance mailing (N = 4,800); The Letter Group: those households with listed numbers that were sent an advance letter (N = 4,530), and The Postcard Group: those households with listed numbers that were sent an advance postcard (N= 4,519).
For purposes of this experiment, the letter and postcard contained the same text (a copy of each appears in figures 1 and 2). There were, however, some other differences. Both contained an organization logo; on the postcard it
2. BRFSS is the largest continuously conducted telephone health survey in the world. For detailed information about BRFSS, see http://www.cdc.gov/brfss/#about_BRFSS (accessed April 2005). 3. A 100-block is the set of 100 possible telephone numbers that can be formed by the first eight digits of the area code, exchange, and first two digits of the final four-digit suffix (e.g., the block formed by area code 906, exchange 632, suffix 20_ contains the 100 numbers from 906-632-2000 through 906-632-2099).
236
Hembroff et al.
Figure 1. Text of the postcard.
Cost Effectiveness of Advance Mailings
237
Figure 2. Text of the advance letter. appeared on the side where the recipient’s name and address were printed, but the logo was beside the text in the letter. The postcard carried a banner heading naming the survey in place of a salutation, whereas the letter was addressed “Dear Resident.” The postcard named the sender, while the letter not only named the sender but also contained a scanned signature. The name of the organization sending the postcard was also printed in boldface in the postcard
238
Hembroff et al.
but not in the letter. Some of these differences appeared necessary because of the space constraint differences (e.g., placement of the logo, space for a printed signature) or to meet expectations regarding what is conventional (e.g., signature on the letter). To minimize the degree of personalization in an otherwise conventional letter, it was addressed to “Resident” rather than a specific householder, and the signature was scanned (black) rather than obviously handwritten in blue ballpoint (Dillman 2000). We think the result is that the letter was obviously a form letter and, as a result, differed in its essential characteristics from the postcard only in form. Depersonalizing the letter in this way would lessen its comparative advantage, an advantage that is usually attributed to personalization. The total sample also included phone numbers for which there were no directory listings (N = 8,401). Obviously, no advance mailing was possible for households among this portion of the sample, and less than a quarter of these phone numbers were expected to be working household numbers. We will not report results for this group, since the relevant comparisons for assessing the effectiveness of advance mailings are among the control, letter, and postcard groups.
Results
IMPACT ON RESPONSE AND REFUSALS
As of 2002, BRFSS used a list of 32 final disposition codes that are assigned to individual phone number cases based on the outcomes of calling efforts to each number. Using the BRFSS computational methods for calculating the BRFSS CASRO response rate, the overall response rate was 35.5 percent (BRFSS CASRO).4 We have also mapped these disposition codes to those specified by the American Association for Public Opinion Research in Standard Definitions (AAPOR 2004). The AAPOR response rate for the overall survey was 33.4 percent (AAPOR RR3).5 Table 1 shows the response rates for the three mailing treatment groups. Table 1 indicates that the response rate (BRFSS CASRO) was 2.8 percentage points higher for the letter group compared to the postcard group and 5.4 percentage points higher than the control group. Furthermore, the table
4. The BRFSS CASRO response rate includes in the numerator completed interviews, partial interviews, and a percentage of breakoffs that do not satisfy the requirements to be included as partials. The denominator includes all eligible household numbers, as well as a portion of the unknown or ambiguous numbers, where the proportion is derived from the eligibility rate among numbers that are not ambiguous. The denominator with this calculation of e is very similar to the denominator in AAPOR RR4 (AAPOR 2004). The numerator differs by including the percentage of breakoffs. 5. AAPOR RR3 involves the use of an adjustment factor e to estimate the proportion of ambiguous outcome cases that are likely to be eligible households based on the distribution of outcomes that were not ambiguous. Eligible phone numbers were those of private, primary residences within the state of Michigan in which there is at least one English-speaking, noninstitutionalized
Cost Effectiveness of Advance Mailings Table 1. Response Rates Overall and by Mailing Treatment Group
239
Listed Numbers (Mailing Treatment Group) Total Sample BRFSS CASRO AAPOR RR3 AAPOR REF1 35.5% 33.4% 25.1% Letter 34.8% 35.8% 24.8% Postcard 32.0% 32.8% 27.7% Control 29.4% 30.0% 27.8%
NOTE.—The total sample includes not listed numbers.
indicates that the letter group had 2.9 percentage points fewer refusals than the postcard group and 3.0 percentage points fewer refusals than the control group. For purposes of simplicity, we have collapsed the 32 BRFSS disposition categories down to four groupings. These are “Completes,” “Noninterviews in apparently eligible households,” “Numbers of unknown eligibility,” and “Ineligible numbers.” Table 2 summarizes the outcomes for the various segments of the sample using these four outcome categories. Among the four segments of the sample, the percent complete was greatest for the letter group, followed by the postcard group. The letter group produced the lowest percent of unknown eligibility among the listed sample segments, followed by the postcard group. Overall, 22 percent of the listed sample numbers resulted in interviews. The table indicates that the distributions of these outcomes differed significantly between the letter group and the control group, between the postcard group and the control group, and nearly so between the letter group and the postcard group. Using logistic regression, with being sent a letter or not and being sent a postcard or not as dummy predictor variables, and a binary dependent variable (a completed interview or not), both predictor variables were statistically significant (Letter, B = .249, p < .001, odds ratio = 1.28; Postcard, B = .139, p < .01, odds ratio = 1.15). This indicates that sending an advance letter increased the odds of producing a completed interview by 28 percent, while sending an advance postcard increased the odds of producing a completed interview by 15 percent, compared to not sending anything. Thus, the advance mailings were effective at increasing responses and reducing refusals. The improvement in response in this particular case, however, was relatively small. With such a small improvement, is the effort worth the cost? or incapacitated adult (i.e., 18+ years old). The adjustment factor is used in the calculation of RR3 and REF2. For this survey, e was computed to equal 0.572 for the letter group, 0.578 for the postcard group, and 0.567 for the control group. Neither RR1 nor REF1 use e as an adjustor in calculating these rates. The corresponding values of RR1 for the mailing treatment groups were 30.7 percent for the letter group, 28.2 percent for the postcard group, and 25.2 percent for the control group. The CON3 rates were 86.4 percent, 86.5 percent, and 85.9 percent, respectively, and COOP4 rates were 56.7 percent, 51.9 percent, and 49.0 percent (AAPOR 2004).
240 Table 2.
Hembroff et al. Outcome Dispositions by Mailing Treatment Group
A Letter B Postcard
% N %
C Control (No Mail)
N % χ2
Outcome
N
Interviews Complete Noninterviews in Apparently Eligible Households Unknown Eligibility Ineligible Total
1,091
24.1
1,000
22.1
952
19.8
31.11 p < .001, df = 6
726 1,942 771 4,530
16.0 42.9 17.0 100.0
766 2,001 752 4,519
17.0 44.3 16.6 100.0
786 2,267 795 4,800
16.4 47.2 16.6 100.0
NOTE.—χ2 (A vs. B) 6.1, p =.052, one-tailed, df = 3; χ2 (A vs. C) 29.5, p < .001, one-tailed, df = 3; χ2 (B vs. C) 10.8, p < .001, one-tailed, df = 3.
COST-EFFECTIVENESS
Cost-effectiveness can be examined by comparing the costs of the alternative efforts required to accomplish the same results. What will be clear in the discussion that follows is that relatively small differences in effects can have substantial impacts on costs. In the case of this experiment, the principal cost elements are interviewer and supervisor labor, telephone charges, and mailing labor, supplies, and postage. A major cost component is the number of call attempts interviewers must make to obtain a completed interview or to obtain a final disposition of any type. The average number of call attempts required to reach a final disposition (i.e., mean call attempts) was significantly lower among those sent advance letters (8.2) compared to those sent postcards (8.5) by nearly a third of a call attempt (F1,9047 = 4.14; p < .05, two-tailed). The average number of attempts required was lower among those sent advance postcards than among those sent nothing (8.7) (F1,9317 = 3.34; p < .05, one-tailed). And, the average number of attempts required was about a half a call attempt lower among those sent advance letters than among those sent nothing (F1,9328 = 15.23; p < .001). Similarly, among cases that resulted in completed interviews, the average number of calls required was less in the letter group (5.9) than in the postcard group (6.0), which was also lower than in the control group (6.5). The difference in the means between the letter group and the postcard group was not statistically significant (F1,2,098 = 0.62; n.s.), but it was between the postcard
Cost Effectiveness of Advance Mailings
241
group and the control group (F1,1,950 = 4.17; p < .05) and between the letter group and the control group (F1,2,041 = 7.89; p < .01). The mailing treatment groups also differed with respect to the percentage of all calls that led to a completed interview, with 17.3 percent of calls leading to completed interviews in the letter group compared to 15.8 percent in the postcard group and 14.9 percent in the control group. That is, the overall efficiency of the calling was greater with the use of letters than postcards, and the calling was more efficient using either letters or postcards than using no mailing at all. Previous attempts to assess the cost-effectiveness of advance mailings have been only suggestive because of incomplete accounting information related to the various cost components (Traugott, Groves, and Lepkowski 1987). That is not the case in this study. Table 3 below provides much of the calculation as to the cost-effectiveness of the three different strategies regarding advance mailings: sending letters, sending postcards, sending no advance mailing. In line A, we report the average number of sampled phone numbers that were used in order to produce each completed interview for each experimental mailing group of listed phone numbers. The table indicates that it took fewer sample elements to produce a completed interview when letters were sent (4.152) than when postcards were sent (4.519), which, in turn, took fewer sample elements than when nothing was sent (5.042). A total of 3,042 interviews was completed from the listed phone sample. Line B shows the extrapolated number of telephone numbers that would have been required to complete all 3,042 interviews using each of the three mailing strategies. Line C reports the actual average number of call attempts needed to reach final dispositions for each of the experimental groups. Line D reports the extrapolated total number of calls that would have been required to complete all interviews using each of the mailing strategies. This line indicates that not sending anything would have required more than 30,000 additional phone calls to complete the interviews than if letters had been sent to all listed numbers, and sending postcards would have required more than 13,000 more phone calls than if letters were sent. Line E shows the calculated number of hours of interviewer labor that would have been required to make the phone calls under each strategy, based on interviewers making 15.75 calls per hour—the average for this survey. Line F shows the number of additional interviewer hours required if postcards were sent or nothing was sent, compared to if letters were sent. Line G shows the calculated cost of the wages for the excess interviewer hours and the accompanying additional supervisor hours at the OSR interviewer-to-supervisor ratio and current average wages for each ($7.52 for interviewers, $9.50 for supervisors). Having to purchase larger samples in order to produce the same number of completed interviews would increase the sample expenses. Line H shows the extrapolated additional sample that would be required to complete the 3,042 interviews for sending postcards or sending nothing, compared to sending
242 Table 3. Strategies
Hembroff et al. Cost-Effectiveness Calculations of Alternative Advance Mailing
Total Completes from Listed Sample = 3,042 A. Sample used per complete interview B. Sample needed to complete 3,042 interviews C. Avg. number of call attempts to finalize (actual) D. Total calls needed to complete 3,042 interviews E. Interviewer hours needed @ 15.75 calls per hour F. Extra interviewer hours needed vs. letter G. Extra interviewer and supervisor wages vs. letter H. Extra sample needed vs. letter I. Extra sample cost vs. letter ($0.08 per listed) J. RDD directory listing extraction ($0.05 per listed) K. Mail labor costs L. Mail supply costs M. Mail postage costs N. Extra phone connection charges vs. letter
Letter 4.152 12,630 8.17 103,190 6,552 0 $0 0 $0 $632 $501 $2,021 $4,673 $0
Postcard 4.519 13,747 8.46 116,298 7,384 832 $7,318 1,116 $89 $687 $363 $1,100 $3,024 $634
Control 5.042 15,338 8.73 133,899 8,502 1,950 $17,145 2,707 $217 $0 $0 $0 $0 $1,485
letters as reflected in Line B. Line I shows the calculated cost of purchasing the additional sample. However, sending advance mailings also incurs expenses with respect to the cost of matching RDD numbers to directory listings and extracting names and addresses, as well as mailing assembly labor, supplies, and postage. Line J shows the calculated additional cost of extracting the directory listings for the sample required if letters were sent to all or postcards were sent to all, compared to sending nothing. Line K represents the staff wage rate multiplied by the numbers of hours that would be required to assemble all the advance letters, based on a production rate of 200 letters per hour and 300 postcards per hour—as measured for a different project contemporaneous to this project. Line L shows the cost of the mailing supplies. In the case of letters, letterhead, envelopes, and printer costs totaled $0.16 per letter. In the case of postcards, the cost was $0.08 per postcard. Line M shows the postage costs if the mailing strategy indicated in the column were used for all listed numbers needed to complete the 3,042 interviews, following only that strategy. Line N shows the estimated additional phone charges that would result compared to the strategy that involve the fewest phone calls, that is, sending advance letters.6
6. The OSR is charged $0.12 per minute for all long-distance calls. We have calculated the surplus cost assuming a single minute of connect time. Analysis of all calls made for this project indicated that among listed phone numbers, 40.3 percent of the calls resulted in a connection. Calls that resulted in a “no answer” or “busy” or telephone company messages about the number
Cost Effectiveness of Advance Mailings
243
Calculating the net differences in the costs of using the various mailing strategies, the table indicates that sending advance letters to all listed numbers for this project would have been $5,386 less expensive than sending postcards to all, and $11,018 less expensive than not sending anything. Sending postcards to all listed numbers for this project would have been $5,632 less expensive than sending nothing.
Conclusion
The results provide fairly clear evidence that sending advance letters in RDD surveys is useful in improving response rates and is cost-effective. The magnitude of the improvement in the response rate was not huge—roughly 5.4 percentage points greater than sending nothing—but given how low the response rate was, that was almost a 20 percent improvement in the response rate among listed numbers, compared to what it would have been with no mailing at all. The magnitude of the effect of the advance mailings was less than was noted in a number of studies but greater than was reported by others. The magnitude of the effect of the mailings on reducing the number of calls required to complete interviews was similar to that reported by others. The results also suggest that sending advance postcards is cost-effective compared to sending nothing, but less effective than advance letters with respect to improving response rates and less cost-effective as well. The cost-effectiveness assessment made no attempt to calculate the “benefit” of having a higher response rate. There is a rapidly evolving discussion regarding the extent to which response rates are at all related in a meaningfully predictive way to nonresponse bias, so trying to estimate the “benefit” of a higher response rate is clearly premature. Nevertheless, the spreadsheet format of the analysis does make it possible to explore how robust the cost-effectiveness of advance mailings might be if, for example, the targeted number of completed interviews were substantially fewer, the calling protocol were less rigorous, the interviewer wage rate were greater/lesser, the phone rates differed, and so forth. These are certainly factors that might vary from survey to survey or survey organization to survey organization that might affect the generalizability of these specific results. We cannot say why the postcard was less effective than the letter. We noted several, what we believe to be, very minor differences in the visual aspects of the letters and postcards. We think these are unlikely to account for the difference in effects. It may be as Dillman (2000) suggests—that there are greater long-term memory effects of letters over postcards—or that letters being disconnected or temporarily out of service do not result in connection charges, but others do, including reaching answering machines. Line N then shows the cost of the additional calls’ connection charges required under that mailing strategy.
244
Hembroff et al.
make the requested task seem more important than do postcards. Additionally, we suspect that the effectiveness of the letter we used, and therefore its cost-effectiveness, was limited by the necessity to make its content similar to what fit on the postcard. Consequently, the letter included less information about the purpose of the survey, was less personal, was less trust-inducing, and made less of an appeal to social responsibility than a typical advance letter would try to accomplish (Camburn et al. 1996; Dillman 2000; Groves, Cialdini, and Couper 1992).
References
American Association for Public Opinion Research (AAPOR). 2004. Standard Definitions: Final Disposition Case Codes and Outcome Rates for Surveys. Lenexa, KS: AAPOR. Brick, Michael, David Cantor, Jeffrey Kerwin, Pamela Giambo, and Eleanor Singer. 1997. “Review of Methods to Increase Coverage and Response Rates on the Study of Privacy Attitudes.” Report to the U.S. Bureau of the Census under Task Order no. 46-YABC-6-00016, September 12. Brunner, G. Allen, and Stephen J. Carroll, Jr. 1969. “The Effect of Prior Notification on the Refusal Rate in Fixed Address Surveys.” Journal of Advertising Research 9:42–44. Camburn, Donald, Paul J. Lavrakas, Michael P. Battaglia, James T. Massey, and Robert. A. Wright. 1996. “Using Advance Respondent Letters in Random-Digit-Dialing Telephone Surveys.” American Statistical Association Proceedings: Section on Survey Research Methods 1995:969–74. Couper, Mick P., Nancy A. Mathiowetz, and Eleanor Singer. 1995. “Related Households, Mail Handling, and Returns to the 1990 Census.” International Journal of Public Opinion Research 7(2):172–77. Curtin, Richard, Stanley Presser, and Eleanor Singer. 2003. “Recent Response Rate Changes on the Surveys of Consumers.” Paper presented at the annual meeting of the American Association for Public Opinion Research, Nashville, TN. ———. 2005.“Changes in Telephone Survey Nonresponse over the Past Quarter Century.” Public Opinion Quarterly 69:87–98. Dillman, Don A. 2000. Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley and Sons. Dillman, Don A., Jean G. Gallegos, and James H. Frey. 1976. “Reducing Refusal Rates for Telephone Interviews.” Public Opinion Quarterly 40:66–78. Groves, Robert M., Robert B. Cialdini, and Mick P. Couper. 1992. “Understanding the Decision to Participate in a Survey.” Public Opinion Quarterly 56:475–95. Groves, Robert M., and Mick P. Couper. 1998. Nonresponse in Household Interview Surveys. New York: John Wiley and Sons. Hembroff, Larry A. 1998. “Explorations of Strategies to Minimize Possible Survey Error and Inefficiency: Noncoverage, Nonresponse, and Vendor Screening.” Paper presented at the annual meeting of the Behavioral Risk Factor Surveillance System by the U.S. Centers for Disease Control and Prevention, Atlanta, GA. Link, Michael W., Ali Mokdad, Machell Town, Jodie Weiner, and David Roe. 2003. “Improving Response Rates for the BRFSS: Use of Lead Letters and Answering Machine Messages.” Paper presented at the annual meeting of the American Association for Public Opinion Research, Nashville, TN. Pennell, Steven G. 1990. Evaluation of Advance Letter Experiment in 1988 New York Reproductive Health Survey. Report to the Centers for Disease Control. Ann Arbor, MI: Survey Research Center. Singer, Eleanor, John Van Hoewyk, and Mary P. Maher. 2000. “Experiments with Incentives in Telephone Surveys.” Public Opinion Quarterly 64:171–88. Smith, Tom. 2003. “Response Rates to National RDD Surveys at NORC, 1996–2002.” Paper presented at the annual meeting of the American Association for Public Opinion Research, Nashville, TN.
Cost Effectiveness of Advance Mailings
245
Steeh, Charlotte G. 1981. “Trends in Nonresponse Rates.” Public Opinion Quarterly 45:40–57. Steeh, Charlotte G., Nicole Kirgis, Brian Cannon, and Jeff DeWitt. 2001. “Are They Really as Bad as They Seem? Nonresponse Rates at the End of the Twentieth Century.” Journal of Official Statistics 17:227–47. Traugott, Michael W., Robert M. Groves, and James M. Lepkowski. 1987. “Using Dual Frame Designs to Reduce Nonresponse in Telephone Surveys.” Public Opinion Quarterly 51:522–39. White, Amanda, and Stephanie Freeth. 1996. “Improving Advance Letters.” Paper presented at the Seventh International Workshop in Household Survey Nonresponse, Rome, Italy. 