This blog post was written by Deputy Assistant Secretary for Energy Efficiency at the U.S. Department of Energy, Dr. Kathleen Hogan, and originally posted to the Office of Energy Efficiency and Renewable Energy website. 

A working paper released in June by academics with the E2e Project wrongly suggested that the federal Weatherization Assistance Program (WAP) and other energy efficiency programs are not good investments.[i]  This result contradicts many past studies and was so surprising that the media was set abuzz.

Two new major, peer reviewed, national evaluations of WAP from DOE’s Oak Ridge National Laboratory (ORNL) provide fresh evidence that WAP is indeed a good investment with energy savings exceeding the costs by a factor of 1.4. The evaluations also show that under the Recovery Act, while achieving economic stimulus goals, the benefit cost ratio for WAP energy savings measures is estimated at just below 1, substantially better results than found by E2e.[ii] 

But WAP achieves more than helping low-income households lower their energy bills. While WAP crews are in homes installing energy savings measures, they also address health and safety issues by fixing broken windows, replacing faulty water heaters, repairing holes in roofs as well as installing other protective measures.[iii]  With health and safety benefits and costs included, the benefit cost ratio rises to 4. 

Over the past 40 years, this program has saved Americans billions of dollars and has been repeatedly demonstrated as cost-effective.  However, its work on health and safety issues, and its practices of combining WAP funds with other low-income directed assistance dollars make it apples and oranges to compare WAP to other residential efficiency programs.  Programs run by utilities and state and local governments across the nation are different and have been repeatedly demonstrated as cost-effective. [iv]

Comparing the Studies

We looked at these studies, and found that with a few reasonable alternative assumptions in the E2e study, its energy efficiency results are similar to the ORNL results for WAP as operated under the Recovery Act. So where do the ORNL and E2e studies agree and where do they differ?

1. Savings estimates from the E2e and ORNL studies are similar in that both studies find 10 percent to 20 percent energy savings from weatherization, with the E2e study representing the higher end of that range.  However, the E2e study estimates lower annual energy savings per household because it examined homes that use less energy than those typically served by WAP.

• The E2e analysis uses a couple of approaches to estimate annual energy savings from WAP-assisted homes, which produce saving estimates ranging from 10 percent to 20 percent per home on average. Their preferred approach, which relies on a randomized control trial of encouragement for eligible WAP recipients, yields savings estimates of about 20 percent per household with annual savings in home energy costs of about $235 per home. A second approach that relies on a larger number of weatherized homes yields estimates of about 10 percent and annual savings of $155 per home, but the authors state that this estimate may be too low for a variety of reasons.[v]
• The ORNL Recovery Act evaluation (of a similar time period as the E2e study) has similar findings and is based on a larger and more diverse set of weatherized homes.[vi]  This evaluation finds that, on average, the single-family homes assisted by the WAP in this period saved 12 percent on average, or $223 annually, with the energy costs and value of the savings nearly twice as much in homes heated by fuel oil or propane than they were in homes heated by natural gas.[vii]
• However, these studies looked at very different sets of homes. The E2e study examined homes only in southern Michigan that were largely heated by natural gas which also had low annual energy costs relative to homes typically weatherized.  The average annual energy costs in the E2e study in 2013 are estimated at $1,175.[viii] The ORNL Recovery Act study evaluated weatherized homes across the country, representing a diverse sample of housing types with differing heating and cooling energy demand, energy costs, and heating fuels (e.g., gas, oil, propane, and electricity). This study reported average annual energy costs of $1,863 for weatherized single-family homes.[ix]  If the E2e study's 20 percent savings estimate held up on a sample of homes more typical of those weatherized across the county, savings would be about 60 percent higher.

2. Cost estimates for energy savings measures in the E2e and ORNL studies are about the same, after the costs of health and safety measures are removed from the E2e study.

• An analysis of energy efficiency within WAP should include only costs associated with energy efficiency. But in addition to expenditures on energy efficiency improvements, the E2e study includes in their total average cost expenditures for addressing health and safety measures such as asbestos abatement and fixing faulty wiring. These health and safety measures and repairs are not expected to be cost-effective from an energy efficiency perspective; they are executed to protect the health and safety of families served by the program and this work is not typical of broader energy efficiency programs.
• The ORNL national evaluations extensively assess and differentiate between spending for energy and non-energy benefits. Costs for health and safety were in addition to these costs, and estimated at about 15 percent of the average total costs per household.
• If E2e had removed the cost of health and safety measures from their average costs, the costs found in both studies would be about the same. Applying that 15 percent from the ORNL study to the average costs from the E2e study ($4,581 per home), the costs for energy efficiency in the E2e study would be $3,894 or very close (within 3 percent) to the costs for energy savings measures in the ORNL study. 

3. Both the E2e study and the ORNL Recovery Act study examined homes weatherized during the unprecedented Recovery Act period, when the program’s many-fold expansion reached more than 1 million low income households and achieved economic stimulus goals, but under conditions and requirements that are not representative of the long term WAP program.

• Spending under the Recovery Act was designed to jump start the economy, with higher spending levels, higher wage requirements for implementers, and new programs for rapidly training and fielding weatherization crews.  The specific goals of the weatherization program – which were achieved -- were to help more than 1 million low-income households save money on their energy bills, improve their health and safety, and provide tens of thousands of jobs during the Great Recession. 
• ORNL found higher average costs of the energy savings measures in single-family homes during the Recovery Act period ($3,777 per household), compared to the pre-Recovery Act period ($2846 per household), as well as lower energy use overall in homes that were weatherized, and lower annual energy savings.[x]

4. Benefit-cost results for energy efficiency during the Recovery Act in both the ORNL and E2e studies are about the same when energy prices are handled similarly.  Moreover, the longer-term WAP has a benefit cost ratio of about 1.4 for energy savings. And when energy, health, and safety benefits are incorporated the benefit cost ratio rises to 4.

• The E2e study assumes that energy prices remain constant for the duration of the long lives of many of the installed energy efficiency measures and calculate $2,949 in average household energy savings over the 16-year weighted average lifetime for installed measures with a 3 percent discount rate.  If the authors instead assumed increasing energy prices consistent with common energy price projections,[xi] these savings would be about 10 percent higher.
• The ORNL Recovery Act study shows that single-family homes overall, assisted by WAP save $3,803 over the life of the measures, or greater than the average cost of $3,777. These calculations take into account escalating energy prices, a weighted average lifetime for installed measures of about 20 years, and a discount rate of 2.7%, consistent with federal protocols for assessing cost-effectiveness.   
• Of greater relevance, the ORNL evaluation during the period just before the Recovery Act examined homes weatherized under typical spending levels and established practices for training weatherization crews and prioritizing the available funding – conditions most applicable to today.  It establishes an overall benefit-cost ratio of 1.4 for energy efficiency measures.  Further, the ORNL study shows that when you add in the costs and the benefits for the health and safety (or non-energy) measures, that the total benefits are on the order of $13,000, for a benefit cost ratio of about 4.[xii]

5. Both E2e and ORNL studies find no evidence of a rebound effect.

• Generally, the rebound effect is the extent to which the savings to homeowners enabled by energy efficiency improvements are used to purchase additional energy. For example, a household might decide to keep their home cooler in the summer or warmer during the winter post-weatherization because the costs to be comfortable are lower. Multiple studies have analyzed this topic, with the most widely accepted rebound effect estimates ranging from 0% to 12%.[xiii] Both the E2e project and the forthcoming ORNL national evaluation found essentially no rebound effect, meaning that the WAP-assisted households did not alter the temperatures of their homes post-weatherization. This is great news -- these households continue to see the full energy savings from the weatherization services.

Weatherization Works

WAP is designed to assist vulnerable populations, prioritizing service delivery to elderly persons, persons with disabilities, families with children in the home, high-energy users, and those with high-energy burdens.  Its investment in energy efficiency and health and safety helps families in energy cost crises across the country, and produces lower energy bills, more comfortable and safe homes, and affects economic impacts through job creation.  The ORNL national evaluations demonstrate that this is a program that works.[xiv]   

There is always room to grow and improve.  The Recovery Act expanded the weatherization program during a difficult economic period and delivered tremendous benefits across the country.  DOE is building on the program’s success and track record to further develop the program, including continuing to improve the tools used to prioritize the measures installed in low-income homes.[xv]  DOE will be using the results from the national evaluations -- and all available research -- to continue to strengthen the program and reap the greatest benefits for the federal investment.  Let’s keep working together.
 

***
 

[i] The paper is “Do Energy Efficiency Investments Deliver? Evidence from the Weatherization Assistance Program,” by Meredith Fowlie, Michael Greenstone, Catherine Wolfram, NBER Working Paper No. 21331, July 2015.

[ii] These values represent conservative estimates that include funding from sources (e.g., LIHEAP) not subject to DOE’s SIR requirement that is used to guide the measures that are installed. These funds are often used for higher cost energy measures that result in lower SIR for the combined funds.

[iii] WAP was created by Congress in 1976 to “increase the energy efficiency of dwellings owned or operated by low-income persons, reduce their total residential energy expenditures, and improve their health and safety, especially, low-income persons who are particularly vulnerable such as the elderly, the handicapped, and children.” To date, the program has assisted more than 7 million families and lowered their energy bills by hundreds of dollars a year, depending on the housing type, fuels, and location and addressed health and safety issues.  The U.S. Department of Energy (DOE) provides the WAP funds to grantees, which include all 50 states, U.S. territories, the District of Columbia, and a small number of tribes. The grantees then administer funding to a network of approximately 800 sub-grantee weatherization agencies that perform the weatherization work in homes across the United States. The sub-grantees also leverage additional funds from the federal Low-Income Home Energy Assistance Program (LIHEAP), states, utilities, and other sources.

[iv] WAP is not an example of energy efficiency programs broadly.  Unlike utility-run energy efficiency programs, WAP has responsibilities to improve the qualities of life of low income households in addition to improving energy efficiency. WAP also pays for the full cost of energy audits and the installations as part of meeting program goals. Utilities are investing in energy efficiency programs because it is less expensive to them and their customers than building new power plants to meet their customer's energy needs.  Utilities are currently investing around $7.2 billion annually in energy efficiency programs, with half of this investment targeting residential customers.  The estimated cost of the “saved energy” from these programs is less than $.05 / kWh (based on a recent analysis of programs in 20 states over a five-year period), or less than half what most of us pay for electricity across the country and much lower than the cost of new supply in most regions of the country.  Customers at the same time are saving tens of billions of dollars in lower energy bills, and carbon dioxide and other pollutants are avoided as additional benefits.  See Ian M. Hoffman, Gregory Rybka, Greg Leventis, Charles A. Goldman, Lisa Schwartz, Megan Billingsley, and Steven Schiller, “The Total Cost of Saving Electricity through Utility Customer-Funded Energy Efficiency Programs:  Estimates at the National, State, Sector and Program Level” LBNL, April 2015.

[v] There is a bias downward due to a focus on natural gas-heated homes (under representing other types of home heating such as oil and propane, which can offer greater savings) and due to a sample using less energy per home than average.

[vi] See here for the latest on the national retrospective and ARRA period evaluations.

[vii] These savings estimates are in 2010 dollars.  Propane and oil fueled homes are underrepresented in the E2e study.

[viii] This cost is calculated using the E2e savings estimates of 20% and the $235 savings per year number they associate with this savings percentage.

[ix]  These average energy costs are in 2010 dollars and would be $1,874 in 2013 dollars. Single family homes are described here as they seem to be the housing type most closely aligned with the E2e housing samples and represent about 65% of the homes weatherized in the 2010 Report.  Natural gas heated homes in the ORNL sample had average bills of $1,634 in 2010 dollars.

[x] 2010 dollars.  

[xi]  Such as energy price projections from the Energy Information Administration

[xii] Non-energy benefits included here include those related to asthma, cold thermal stress, heat thermal stress, carbon monoxide, reduced fires, increased prescription compliance, reduced missed days of work, and reduced need for food assistance for a PV over 10 years (except for CO, which is five years) of $9364 (PY 2010 dollars and a discount rate of 2.7%), and total overall benefits of $13,167.  The total costs are $4344, average energy measure costs of $3777 and an additional 15% for health and safety costs.

[xiii] See: Steven Nadel, “The Rebound Effect: Large or Small?” ACEEE, August 2012; Greening, L., Greene, D., and Difigilo, C. 2000. Energy Efficiency and Consumption – The Rebound Effect – A Survey. Energy Policy, Vol. 28, No. 6-7, 389-401; and Sorrell, S., and J. Dimitropoulos, M. Sommerville. 2009. “Empirical estimates of the direct rebound effect: A review.” Energy Policy 37: 1356-1371. 

[xiv] See here for the latest on the national retrospective and ARRA period evaluations.

[xv] E2e points to the projected savings from the NEAT audit tool being much more than the actual savings in the sample they examined.  This difference would be much less with projections for energy price increases. DOE and the WAP continue to work to improve and field NEAT and other tools so as to ensure a cost-effective WAP program.