The QA Specification is going to address uncertainty and risk. While we do not want to provide an exhaustive description of this topic, we do want to address it in a clear, concise way. The following is an excerpt from the QA Specification addressing this topic. We are asking the QA Technical Group, and the community at large, to comment and provide suggestions for this section.

The following is the current form of this Uncertainty and Risk section. Please send your comments or suggestions to [email protected]. Thank you for your continued support with this effort!

Uncertainty and Risk
The estimated energy-cost savings associated with the ECMs and package of measures is a critical value for investors considering EE projects. Unfortunately, savings estimates are typically calculated as a single number and do not indicate a probable range or an estimated uncertainty. Failure to provide information about uncertainty leaves the financial analyst with no means to price the appropriate rate of return. This causes the financial analyst to increase the required rate of return or to de-rate the savings before applying the financial model. This practice undermines the viability of energy projects (Mills et al. 2003).

For most uncertainty analysis methods, the inputs (assumptions) need to be specified as ranges, and their distribution type specified (such as normal, lognormal, uniform, log-uniform, etc.).  A statistical sampling method is then applied to develop sets of parameter values for the assumptions, representative of all possible combinations. The calculations are then performed with these sample sets, and a probability distribution function can be developed and reported, indicating the uncertainty.

For projects utilizing spreadsheet calculation methods, automated calculation tools can be employed, such as the Monte Carlo simulation method.  During a Monte Carlo simulation, values are sampled at random from the input probability distributions.  Each set of samples is called an iteration, and the resulting outcome from that sample is recorded.  Monte Carlo simulation does this hundreds or thousands of times, and results in a probability distribution of possible outcomes. Various applications are commercially available that can be applied as an add-in to Excel, and used to automate this Monte Carlo simulation approach.

For projects utilizing an energy model, a similar approach can be applied. Values are sampled from the developed input probability distributions, and then multiple iterations of the energy model need to be performed. While automated or standardized approaches to this uncertainty analysis have yet to penetrate the energy modeling venue, work is currently underway to develop options for completing uncertainty analysis. In particular, the next release of OpenStudio, an open-source application suite and software development kit which supports whole-building modeling using EnergyPlus and daylight analysis using Radiance, will feature uncertainty analysis tools and methods that draws from the DAKOTA project, an engineering optimization and uncertainty analysis modeling library developed by Sandia National Laboratories. This functionality will allow the assessment of sensitivity of the output to the input parameters, those parameters that contribute the most variance, and how they interact with each other. 

In many cases, resources and time may not be available to determine the uncertainty associated with a project. A cost- effective alternative to quantifying uncertainty is to reduce risk. This is accomplished by:

• Reducing the number of assumptions used in the savings calculation and cost estimation efforts.
  
• Utilizing conservative assumptions when these inputs are necessary.
  
• Applying best practices to all components of EE project development.
  
• Properly applying design, delivery, and operational processes.
  
• Training facility staff adequately.
  
• Performing operational performance verification.
  
• Providing systems and methods to monitor and track performance on an ongoing basis, and providing an adequate managerial and recognition / response plan.
  
• Performing a comprehensive quality assurance process on all components of the EE project development, avoiding bias at all costs.
  
  

These practices will reduce risk, providing the necessary level of confidence to an investor-ready EE project.

A particular challenge to energy efficiency project development, especially some multifamily projects, involves access to utility data. Critical to the development of a project's baseline, these data can be difficult to obtain, especially considering tenant privacy laws. Even when tenant privacy is not an issue, access to these data in a usable format can be challenging.

Consider this example: I am currently a member of my hometown's energy and sustainability committee. As part of our efforts to revise our city's master plan, we have been working with our utility to obtain aggregate monthly electricity and natural gas consumption data for the residential and commercial/industrial sectors within our community. Not individual customer's usage data (which would butt heads with tenant privacy laws), but aggregate data for the entire city. We started working with the utility to obtain these data six months ago. And we're still waiting...

But the winds of change are blowing, and gaining speed. Presented for consideration are two examples of significant developments on the data access front.

Green Button
Green Button is an industry-led effort determined to provide electricity customers with easy access to their energy usage data in a consistent and easy-to-use format. This data access concept is based on a common technical standard developed in collaboration with a public-private partnership supported by the Commerce Department's National Institute of Standards and Technology. 

The idea behind Green Button is to provide a secure, consistent, and easy-to understand method for downloading data directly from the utility's website. At current count, there are 31 utilities and energy suppliers that have implemented Green Button, and nine additional utilities have expressed a commitment to implement Green Button.

Providing access to utility data in a consistent and easy manner not only provides these data to end-users, but also provides a platform for developers to create innovative web-based, smartphone, or other platform applications. These tools can deliver a host of potential services, things like rate plan selection assistance, renewable energy sizing and financing, virtual energy audits, measurement of energy efficiency investments, or energy usage visualization.

E-Access
Another exciting development in the utility data accessibility world recently came from Senators Ed Markey (D-Mass) and Mark Udall (D-Colo) who joined together to introduce the Access to Consumer Energy Information (E-Access) Act on March 27, 2014.

Like Green Button, this bill seeks to improve access to energy consumption data, providing incentives to utilities and states that offer utility information to commercial and residential customers. If the bill is enacted, state activities meant to improve customer access to and understanding of energy usage would qualify for government funding. 

Furthermore, this E-Access Act would require the DOE to create guidelines for improving access to electric energy information, and would address specific issues including time frame, consumer data, privacy protections, security, and how to present the information in a readily understandable format.

Building Momentum
While access to utility data is still challenging, the market is clearly building momentum to improve this situation and overcome this long-standing hurdle to energy efficiency project development. Green Button, complimented by the E-Access Bill, represent two powerful tools that will help continue to push the industry forward, improving data accessibility, and promoting innovative uses for data that were, until recently, difficult to access or receive in a consistent, usable, easy-to-understand format.