LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY

This case study examines the California Risk-Reward Incentive Mechanism (RRIM), an energy efficiency shareholder incentive program, to identify challenges associated with using counterfactuals to vet utility savings claims. The document highlights how measure-by-measure estimation of savings leads to regulatory conflict, high administrative costs, and volatile incentive payments, ultimately recommending a shift toward outcome-oriented metrics.

• The California Risk-Reward Incentive Mechanism (RRIM), implemented in 2006 for the 2006-2008 cycle, allowed utilities to earn a portion of net economic benefits based on meeting efficiency targets for kilowatt-hour (kWh), kilowatt (kW), and Therms reductions. Bonuses were tiered at nine percent or 12 percent of net economic benefits, split between shareholders and customers, while failure to meet targets could result in penalties.
• The use of counterfactuals to estimate energy savings absent utility programs led to significant discrepancies between utility claims and regulator assessments. While all three utilities claimed over 100 percent compliance with savings goals, CPUC staff estimates were much lower: 65 percent for SDG&E, 70 percent for PG&E, and 80 percent for SCE.
• Tiered incentive structures in the RRIM exacerbated the impact of estimation differences, causing massive swings in financial outcomes. For PG&E, a slight adjustment in savings estimates could mean the difference between a $180 million bonus and a $75 million penalty.
• Counterfactuals based on measure-by-measure estimation are administratively burdensome and costly. In the 2006-2008 cycle, the CPUC authorized $163 million for evaluation, measurement, and verification (EM&V), representing 7.6 percent of the state's efficiency portfolio spending, which is significantly higher than the U.S. average of 3 percent.
• The document recommends moving toward outcome-oriented metrics, such as kWh per customer, rather than measure-by-measure estimations. This approach ties incentives to real-world performance and uses transparent adjustments for weather or economic changes, which are less controversial than the assumptions required for counterfactuals.
• If measure-by-measure approaches must be used, the document suggests publishing 'approved' ex ante values through independent bodies like the California Technical Forum or the Regional Technical Forum to reduce disputes. Additionally, it suggests using a lower incentive rate when relying on ex ante values to reflect reduced shareholder risk; California reduced its rate from 9-12 percent to 7 percent when it abandoned ex post evaluation in the RRIM.

APA

ORVIS, R. (2016). LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY. Energy Innovation. https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf

Chicago

ORVIS, ROBBIE. LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY. Energy Innovation, 2016. https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf.

Wikipedia

{{cite report |last1=ORVIS |first1=ROBBIE |title=LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY |publisher=Energy Innovation |date=April 2016 |url=https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf |access-date=17 August 2026 |via=Climate Insights Directory}}

BibTeX

@techreport{orvis2016lessons, author = {ORVIS, ROBBIE}, title = {{LESSONS FOR DESIGNING COUNTERFACTUALS IN PERFORMANCE INCENTIVE MECHANISMS: CALIFORNIA AS A CASE STUDY}}, institution = {Energy Innovation}, year = {2016}, month = apr, url = {https://energyinnovation.org/wp-content/uploads/LessonsOnCounterfactuals-white-paper.pdf}, urldate = {2026-08-17}, note = {Indexed by Climate Insights Directory} }