LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY

The document examines the failures of California's Risk-Reward Incentive Mechanism (RRIM), where disagreements over counterfactual assumptions for energy savings led to massive swings in utility penalties and bonuses. It argues that measure-by-measure estimation is administratively burdensome and contentious, recommending a shift toward outcome-oriented metrics or the use of approved ex ante values to reduce regulatory conflict.

• The California Risk-Reward Incentive Mechanism (RRIM), implemented in 2006, allowed utilities to earn a portion of net economic benefits based on meeting energy efficiency targets for kilowatt-hour (kWh), kilowatt (kW), and Therms reductions.
• Discrepancies between utility ex ante estimates and CPUC ex post analysis caused significant financial volatility; for example, while utilities claimed over 100 percent compliance, CPUC staff estimated compliance at 65 percent for SDG&E, 70 percent for PG&E, and 80 percent for SCE.
• The use of tiered incentive levels in the RRIM amplified the impact of small estimation differences, potentially turning a $180 million bonus for PG&E into a $75 million penalty based on a slight shift in the savings threshold.
• Counterfactuals in energy efficiency are prone to controversy because they rely on uncertain assumptions regarding equipment life, energy impact, and 'free-ridership' (whether installations would have occurred without the program).
• The administrative cost of measure-by-measure verification in California was exceptionally high, with EM&V funding for the 2006-2008 cycle reaching $163 million, or 7.6 percent of the state's efficiency portfolio spending, compared to a 3 percent U.S. average.
• The author recommends moving toward outcome-oriented metrics, such as kWh per customer, which are more transparent and less controversial than measure-by-measure estimations.
• If measure-by-measure approaches are used, the document suggests publishing 'approved' ex ante values via independent bodies like the California Technical Forum to streamline estimates and reduce disputes.
• When shifting from ex post verified savings to ex ante estimates, the incentive rate should be lowered to reflect the reduced risk to shareholders; California reduced its return from 9 or 12 percent to 7 percent.

APA

ORVIS, R. (2016). LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY. Energy Innovation. https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf

Chicago

ORVIS, ROBBIE. LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY. Energy Innovation, 2016. https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf.

Wikipedia

{{cite report |last1=ORVIS |first1=ROBBIE |title=LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY |publisher=Energy Innovation |date=April 2016 |url=https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf |access-date=17 August 2026 |via=Climate Insights Directory}}

BibTeX

@techreport{orvis2016lessons, author = {ORVIS, ROBBIE}, title = {{LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY}}, institution = {Energy Innovation}, year = {2016}, month = apr, url = {https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf}, urldate = {2026-08-17}, note = {Indexed by Climate Insights Directory} }