Field Assessment of Energy Audit Tools for Retrofit Programs

This project focused on the use of home energy ratings as a tool to promote energy retrofits in 
existing homes. A home energy rating provides a quantitative appraisal of a home’s energy 

performance, usually compared to a benchmark such as the average energy use of similar homes in the same region. Home rating systems can help motivate homeowners in several ways. Rating systems based on energy performance models, the focus of this report, can establish a home’s achievable energy efficiency potential and provide a quantitative assessment of energy savings after retrofits are completed, although their accuracy needs to be verified by actual measurement or billing data. Ratings can also show homeowners where they stand compared to their neighbors, thus creating social pressure to conform to or surpass others. 

There are several potential applications for home ratings, and the important characteristics—e.g., speed, accuracy, and clarity—will depend on the type of transaction one is trying to influence. One important consideration for rating tools aimed at the retrofit market is how they will integrate with existing home energy service programs, where technicians perform in-home audits. For residential programs that target energy savings only, home visits should be short and focused on key efficiency measures for that home. In order to gain wide adoption, a rating tool must be easily integrated into the field process, demonstrate consistency and reasonable accuracy to earn the trust of home energy technicians, and have a low monetary cost and time hurdle for homeowners.  

This project field-tested three different building performance models of varying complexity, in order to assess their value as rating systems in the context of a residential retrofit program. The major focus was the Home Energy Score, which was under development by the U.S. Department of Energy (DOE) at the time of this project. It is designed to give a complete home performance assessment while simplifying the building measurements to 36–67 data inputs, depending on a home’s configuration. The Center for Energy and Environment (CEE) was one of nine national pilot sites in the spring of 2011, and tested the Home Energy Score on 154 Minnesota homes. The goal of these pilots was to provide feedback to the DOE about technician experience, homeowner reaction, and the pattern of scores in a given region of the country, so that DOE could make necessary changes before the national launch of the tool. Numerous changes have been made to the national version, launched in June 2012, including adjustments for several issues raised though this analysis. 

This pilot also evaluated the energy modeling performance of SIMPLE and REM/Rate. SIMPLE, developed by Michael Blasnik and Associates, is a spreadsheet-based home energy model that runs on fewer than 50 streamlined inputs. It uses broad classifications for certain home characteristics and allows field technicians to switch between estimations and diagnostic measurements, depending on the scenario. REM/Rate, based on an audit protocol developed by the Residential Energy Services Network (RESNET), requires more detailed building inputs than either SIMPLE or the Home Energy Score. In particular, REM/Rate uses detailed construction characteristics for individual wall, window, foundation, and attic areas to characterize the building shell in detail.  

The energy performance of the three models showed similar trends, and importantly, the more detailed characterization of a home did not provide a better estimate of a home’s asset energy use: 

  • Overall, there was systematic overprediction of gas use by the Home Energy Score, SIMPLE, and REM/Rate, although the SIMPLE model results were the closest to utility bill data. The SIMPLE model overpredicted gas usage by 18% on average, compared to 55% and 63% for the Home Energy Score and REM/Rate, respectively.  
  • The Home Energy Score, SIMPLE, and REM/Rate all overpredicted electricity consumption, and had a larger minimum baseline electricity load than actual usage. The Home Energy Score overpredicted electricity by 23% on average. SIMPLE’s average overprediction was 29% and 25% for two alternate versions of the model, and REM/Rate overpredicted by 27% on average. These results include a small number of homes with very high actual electricity use.  
  • These asset models have similar levels of correlation for predicting natural gas use, from a low R-squared of 0.42 for the Home Energy Score to a high of 0.52 for REM/Rate. The asset models all have a low R-squared values for predicting electricity use, from 0.05 for the Home Energy Score to 0.30 for REM/Rate. The validity of this level of correlation will depend on the application at hand, but it urges caution in interpreting and presenting model results.  
  • We estimate that, when used as a stand-alone tool, the length of time required for a trained technician to collect and enter data for one home would be approximately 1.5–2 hours for the Home Energy Score, 1 hour for SIMPLE, and 4–6 hours for REM/Rate. 

In CEE’s experience, the Home Energy Score added 30 minutes of field time to the current 1–1.5 hour residential home visit, as well as 30 minutes for data entry. CEE’s home visits are conducted by two field staff. The most time-consuming entry was measuring a home’s total window area. The biggest challenge was how to classify certain home features common in the Midwest, in particular, unfinished basements and story-and-a-half style homes. The results showed that homes could not significantly improve their scores if they made the recommended investments, which could prove a challenge for motivating homeowners. Homes scored an average of 5.4 on the Home Energy Score, and implementing recommended upgrades would increase their score by an average of 2 points. No homes had an initial score greater than 8, and none could score a 10 after upgrades. This point distribution is by design, and will be updated before the final version is released. 

In addition, the Home Energy Score generates a list of upgrade measures that have a 10-year payback or less for that particular home. CEE compared these automated recommendations to the in-person recommendations made by the field crew, and found that field crews identified cost-effective upgrades 33% more often than the Home Energy Score. Notably, the types of recommendations given by the Home Energy Score did not correlate well with the in-person recommendations. The field crew gave 309 unique recommendations for air sealing, attic and wall insulation, and the Home Energy Score recommended these measures 150 times, or half as frequently. Part of the reason is that field staff can diagnose when small portions of the home require upgrades; e.g., a single side attic, while the Home Energy Score is not as specific. 

The results of this pilot favor the streamlined, lower cost approach of SIMPLE and the Home Energy Score for energy retrofit applications. However, each tool must balance the streamlined data collection with a thorough upfront protocol for how to classify homes consistently across a given region. Importantly, it would be insufficient to rely on automated recommendations from a building performance model without the direct input of field staff to diagnose the specific condition of a home. CEE also concluded that the Home Energy Score point system did not provide enough motivation for homeowners to invest in major upgrades, since very few homes could achieve a high score.

Full report (PDF):
Field Assessment of Energy Audit Tools for Retrofit Programs