As my colleagues and I prepared the agenda for our September 15 workshop, “Putting Disclosures to the Test,” we dug into the archives and reviewed some of the disclosure-related studies and evaluations conducted by or on behalf of the FTC over the past four decades. We read a number of studies that were conducted to develop and evaluate standard disclosure forms and labels, for example, for life insurance costs, mortgages, and financial privacy notices. We also read several studies conducted to evaluate consumer understanding of advertising claims and disclaimers that might be made about those claims.
Development and evaluation of standard disclosure forms and labels
One of the oldest studies we reviewed was included in a 1979 FTC report on life insurance cost disclosures. At the time, many low-income consumers were receiving interest payments from life insurance policies that were well below readily available alternatives in the marketplace. The FTC report recommended that consumers and agents be provided with a clear disclosure of the average annual rate of return and other information that would allow them to compare the benefits and costs of different life insurance policies as well as well as other savings media. The report includes (in its appendixes) the results of two research studies. The first study was a laboratory study in which participants were asked to make a decision between three life insurance policies. Participants were also told they could request any of 37 types of information to help them make their decision. Researchers observed which information participants requested, and which information they actually used as they made their decisions. In the second study, participants were divided into six groups, with each group provided a different disclosure package, reflecting a specific combination (or omission) of buyers guide and additional information about the policies. Participants had to choose the least expensive policy from among eight policies and were quizzed on their comprehension of the disclosure packages. Taken together, the two studies provided insights into what information was most useful to consumers in making informed decisions, and demonstrated that a well-designed disclosure could help consumers: one of the tested disclosure packages resulted in 95% of participants selecting the least expensive policy.
More recently, in 2004 and 2005, the FTC conducted consumer testing of consumer mortgage disclosures and found that the disclosures in use at the time were not effective at conveying mortgage costs to many consumers. Using a combination of long interviews with recent mortgage borrowers and quantitative controlled testing, these studies found that prototype disclosures developed for the study were significantly more effective. Later the Consumer Financial Protection Bureau commissioned further prototype development and testing, which we will hear about at our workshop. These studies resulted in new federal mortgage disclosure forms in 2015. Both the mortgage disclosure studies and the previous life insurance disclosure studies are good examples of how consumer testing can be used to understand how consumers use disclosures and to make disclosures more effective.
Similarly, in 2004 the FTC and other federal agencies commissioned the development of a paper-based financial privacy notice that would be easier for consumers to understand and use than the typical privacy notices provided by financial institutions. In 2006 the FTC published a report that describes the iterative design and qualitative testing process used in the development of a prototype financial privacy notice. After further testing and development, the agencies released a final model privacy notice form that is now in widespread use by financial institutions.
Evaluation of disclosures about advertising claims
In 1998 the FTC published the results of a large scale project to copy test food health claims in advertising. This research explored several issues related to the Commission’s 1994 Enforcement Policy Statement on Food Advertising. The FTC Statement provided more leeway in advertising for certain types of nutrient content and health claims not authorized by FDA labeling regulations, provided those claims were sufficiently qualified, truthful, and not misleading. Researchers interviewed approximately 1,700 consumers at shopping malls in 12 cities. Participants were shown ads for fictitious food products and interviewed about their perception of nutrition or health-related claims made in the ads. For example, some participants saw ads for a fictitious brand of packaged sliced turkey that contained half the sodium of other leading brands, but did not meet the FDA definition of a low sodium food. A sizeable minority of consumers who viewed ads with unqualified claims had the misconception that the packaged turkey was a low sodium food. However, even when quantitative information about the sodium content or verbal advisories were added, these misconceptions remained for many consumers. Overall, the researchers concluded that at the time of their testing, consumers had a poor understanding of the quantitative information provided by food labels. They found approaches to correcting misconceptions about some claims that showed promise, but could not find effective approaches to correct misconceptions about other types of claims, such as the sodium claim described above.
Nearly a decade later, the FTC published the results of a copy test that studied how consumers perceived heart-health claims in print advertisements for a fictitious cooking oil low in saturated fat and for a vegetable oil spread containing no trans fatty acids. The research investigated whether consumers were deceived by the “low in saturated fat” and “no trans fats” claims into believing that the products were entirely healthy, when they were actually high in total fat and calories. The results showed that consumers were not deceived by these claims. However, the researchers found that consumers in their study lacked understanding of the impact that additional consumption of such products could have on weight gain and heart health.
The FTC issued a 2003 report on the effect of consumer testimonials and disclosures on ad communication for a dietary supplement. In this study, 200 consumers at shopping malls were shown either a one-page letter or the letter and a booklet touting the benefits of a dietary supplement. The booklet contained 18 testimonials that extolled the virtues of the supplement. Participants who saw the booklet were divided into five groups and saw either no disclosure or one of four disclosures in large type at the bottom of each page of the booklet that were designed to convey that people should not expect to have similar results as described in the testimonials. Participants were asked several questions about the health claims made about the dietary supplement, whether they believed the claimed health benefits were typical, and whether they recalled seeing a disclosure. The results showed that the testimonials conveyed efficacy claims about the nutritional supplement effectively. However, the tested disclosures were not effective at conveying that efficacy claims were not typical. The FTC also issued a 2004 report on the effects of consumer testimonials in weight loss, dietary supplement, and business opportunity advertisements that reported on a similar study of 1624 consumers who were shown print ads for one of three different products. Some participants were shown disclosures that said “Results not typical” or “These testimonials are based on the experiences of a few people. You are not likely to have similar results.” Some participants who were shown weight loss ads saw a disclosure that stated that the average user “loses about 10 pounds in three months.” Once again, ads that included testimonials communicated that users of the advertised products would achieve results similar to those portrayed in the testimonials, while most of the disclosures failed to communicate effectively. Only the “average …10 pounds” disclosure significantly reduced the tendency of participants to believe that testimonial results were typical.
In 2012 the FTC reported the results of a study on “up to” claims in advertising for replacement windows. Three hundred sixty people were recruited in shopping malls and surveyed. Each respondent saw one of three versions of an advertisement and were then asked questions about the advertised product. In the “up to” version, the ad contained the claim “PROVEN TO SAVE UP TO 47% ON YOUR HEATING AND COOLING BILLS!” The “non-up to” version, included the same claim without the words “UP TO.” The “disclosure version” displayed the up-to claim with the disclosure: “The average Bristol Windows owner saves about 25% on heating and cooling bills.” The results demonstrated that regardless of which version respondents were exposed to, a significant proportion believed that users of these windows could typically expect to save 47% on their heating and cooling bills. Thus respondents interpreted the claim similarly, regardless of whether it included the “up-to” qualifier and regardless of whether it was accompanied by a disclosure. We will hear more about this study at our workshop.
These studies illustrate the importance of consumer testing to gain an understanding of the effectiveness of disclosures and whether consumers are deceived by claims. Through qualitative and quantitative studies, researchers have gained important insights into how consumers use information in disclosures to make decisions. Our “Putting Disclosures to the Test” workshop builds on the FTC’s prior work on disclosure evaluation and reflects the agency’s commitment to ongoing evaluation.