App’y hour—German energy law put to test

In a ruling due this September, the European Court of Justice (ECJ) is expected to grant the German energy regulator, the Bundesnetzagentur, greater powers and independence in its decision-making—but what will this mean for German energy networks? In this article, we will review key elements of the Bundesnetzagentur’s efficiency analysis, and demonstrate how the powerful capabilities of Oxera’s innovative new EffizienzApp can help networks prepare for the anticipated changes to German energy law.

In Germany, gas and electricity distribution and transmission system operators are regulated using a system of ‘normative rules’. This means that, in general, legal statutes stipulate how the allowed revenue and its components should be calculated, as exemplified by the framework governing TOTEX benchmarking (see below).¹

However, the rules may be about to change. German energy law is currently under review by the European Commission, which believes that the German framework violates two statutes on the common market for electricity and natural gas, namely Directives 2009/72 and 2009/73. The Commission argues that the specific features of German energy law limit the independence of the country’s regulatory bodies, and it has therefore initiated infringement proceedings against the Federal Republic of Germany. If this lawsuit is upheld by the ECJ—which is likely to be the case, in the opinion of Advocate General Pitruzella²—German energy law can be expected to undergo significant changes.

But what do the normative rules at the heart of this legal determination look like in practice?

Normative rules in TOTEX benchmarking: what and why

TOTEX benchmarking, which is at the core of incentive regulation, is intended to simulate competitive dynamics among network operators and incentivise economic efficiency.³ To ensure that customers are not paying for inefficiently incurred costs, the network operators’ allowed costs are successively reduced to an efficient level over the duration of a regulatory period. Despite the economic significance of this process, exactly how the Bundesnetzagentur determines efficient costs and efficiency scores is barely traceable for network operators. This creates a ‘black box’ effect.

To date, the regulator has conducted three efficiency comparisons over three regulatory periods across the four energy networks,⁴ and is currently preparing its fourth assessment. These analyses have been bound by the guidelines on efficiency comparisons set out in §§ 12–16 ARegV and Annex 3 to § 12 ARegV. Explicit stipulations are currently made regarding:

• the analytical methods that should be employed (data envelopment analysis, DEA, and stochastic frontier analysis, SFA, Annex 3 to § 12);
• the relevant costs and how they are derived (§ 14);
• the final determination of efficiency targets (§ 12 and § 15–16);

as well as specific instructions concerning efficiency analysis, particularly:

• the returns to scale to be applied in the DEA (Annex 3 to § 12);
• the approach to the determination of outliers (Annex 3 to § 12).

The ARegV also sets out rules regarding the choice of cost drivers (§ 13).

The normative-rules set-up of the current framework was initially intended to improve the quality of efficiency comparisons in light of the diverseness of the network providers. It was also meant to ensure that cost-reduction targets would be robust, achievable and might even be exceeded.⁵ However, at the time of its introduction, the framework had not been designed based on empirical evidence, which was not yet available. Many issues have since come to light, in particular following the publication of data by the Bundesnetzagentur in the third regulatory period.⁶ In the event that the Bundesnetzagentur is extended new responsibilities and greater freedoms by the ECJ’s ruling, it will have a range of options regarding the choices it can make at different stages or on different elements of its analysis. This may result in uncertain outcomes for energy networks.

To support the German energy networks in preparing for these expected changes to the regulatory framework, Oxera has developed an innovative piece of technology to help unlock the ‘black box’.

In the remainder of this section, we review key elements of the Bundesnetzagentur’s benchmarking analysis and use EffizienzApp’s powerful applications to highlight some potential implications of the anticipated changes to German energy law for the German distribution network operators (DNOs).⁷

Selection of cost drivers

There are currently no formal criteria for selecting cost drivers specified in ARegV. However, the Bundesnetzagentur often limits its cost-driver selection with reference to §13 ARegV, and has expressed constraints on its determinations on several fronts.

• The Bundesnetzagentur is of the opinion that normative rules mandate it to use the same cost drivers for both analytical methods (DEA and SFA). This unnecessarily restricts both methods and can unfairly affect some networks. For instance, some cost drivers (or combinations of cost drivers) may result in convergence issues in SFA,⁸ which are not relevant in DEA. On the other hand, certain cost drivers (e.g. dummies, categorical variables and ratios) are not straightforward to handle in DEA, while they can be used in SFA without any issue. Looking ahead to September’s ruling, the Bundesnetzagentur may have the ability to vary cost drivers by method, and networks should explore the consequences of these.
• Normative rules currently prescribe that cost drivers should not (even partially) be repetitive in their effect. The Bundesnetzagentur often cites this rule to exclude certain cost drivers from its analysis. For example, it has argued that the ‘pipe volume’ cost driver could represent nearly all aspects of the gas DNOs’ tasks including ‘changes in population age structure’.⁹ As a result of this rule, cost drivers such as the share of properties connected have not been properly considered in previous assessments. In future, they may be more thoroughly explored to capture important residual effects.

The selection of cost drivers has a significant impact on the estimated efficiency scores of the DNOs (see figure below for an illustrative example).

Figure 1

This figure displays the ‘best-of four’ efficiency scores of gas DNOs for two models which differ by one cost driver.¹⁰ The x-axis presents efficiency scores using the Bundesnetzagentur’s benchmarking model from the third regulatory period. The efficiency scores on the y-axis include ‘potential peak load’, which captures the degree to which a DNO’s network area is connected, in addition to cost drivers in the Bundesnetzagentur’s model. The diagonal line represents the point at which this change makes no difference to the DNOs. Every point above (below) this diagonal represents one DNO that would have been better (worse) off had this change been made. The figure clearly shows that the majority of the DNOs would have benefited from this improved model specification had the additional driver been included in the model.¹¹

Following the removal of constraints from the cost-driver selection, the Bundesnetzagentur would have more freedom to select cost drivers, which would also enable DNOs to propose credible alternatives and improvements to the Bundesnetzagentur’s proposals.

Returns-to-scale assumption

The efficient cost frontier in DEA is determined by companies that have incurred the lowest TOTEX for a given mix and level of outputs (i.e. peer companies). Under the assumption of constant returns to scale (as specified in Annex 3 to § 12 ARegV), the peer companies for an inefficient company may be of any size. ‘Comparability’ can supposedly be achieved by scaling the peer companies up or down by any factor. In the TOTEX benchmarking of gas DNOs, this assumption leads to the cost frontier being largely set by either very small providers (focused on distribution without transmission) or regional long-distance DNOs (mainly focused on transmission without their own distribution networks). This high level of heterogeneity, caused by some companies focusing on specific activities in the supply chain, can lead to significant distortions. Under a variable returns-to-scale assumption, which accommodates different relationships between TOTEX and the outputs, the efficient cost frontier can be composed of more comparable companies undertaking similar activities.

Returns-to-scale assumptions can have a significant bearing on the final efficiency values. The figure below visualises this by mapping the individual DEA efficiency estimates under the assumption of constant returns to scale on the x-axis and variable returns to scale on the y-axis. DNOs improving their efficiency values under the assumption of variable returns to scale are represented by points above the 45-degree line.

Figure 2

It is not possible to make a general assumption concerning returns to scale as context and technical specifications need to be taken into account. Some regulators allow for varying scale assumptions in their assessment, while others restrict the scale assumptions. Currently, the Bundesnetzagentur only partially accounts for the heterogeneous tasks of the DNOs through a limited set of cost drivers. There is also an inconsistency in the relationship assumed in DEA and SFA. In the latter, the Bundesnetzagentur considers flexible functional forms that can allow for varying scale assumptions,¹² while in the former, the returns to scale is fixed to constant. To improve the reliability of the Bundesnetzagentur’s analysis, after the removal of normative rules, (even if limited to some networks), adapting a more appropriate assumption on returns to scale could be considered.

Outlier analysis

The methods applied by the Bundesnetzagentur to estimate efficiency scores can be disproportionally influenced by DNOs which, due to structural differences or data issues, are not comparable to others. Such outliers or ‘unusual’ companies have to be removed from the dataset before the final efficiency scores are determined as these can skew the analysis and make the results unreliable for many. The ARegV specifies the methods for determining outliers (Annex 3 to § 12).

Empirical evidence over the past three regulatory periods shows that the Bundesnetzagentur’s prescribed outlier analyses are inadequate.¹³ For instance:

• the methods have proven unable to detect network operators whose data contained significant errors;
• the dominance test (one of the tests used in DEA) has rarely detected obviously dominant DNOs, despite some operators being a peer to nearly all other DNOs;
• a single round of super-efficiency¹⁴ analysis has resulted in some DNOs exhibiting extremely high super-efficiencies, and therefore being able to raise costs substantially while supposedly staying efficient, in the final sample;¹⁵
• the SFA outlier analysis performed cannot detect network operators that substantially affect the efficiency scores of most network operators (i.e. are unusual and have undue influence).

Whenever the insufficiency of its methods have been questioned in the past, the Bundesnetzagentur has simply stated that the normative rules prescribe the procedure it must use and that it has no discretion to improve on it. However, alternative methods have been shown to be better able to detect unusual network operators. For example, Oxera proposed a case in which the dominance test could be altered to be more appropriate for detecting unusual network operators.¹⁶

Figure 3

As shown in the figure above, as a result of the more homogeneous sample following a more appropriate dominance test, the efficiency scores of some network operators improve considerably. When the normative rules no longer apply, the Bundesnetzagentur can use its increased freedoms to ensure that more appropriate mechanisms for detecting outliers are used.

Regulatory discretion: boon or bane?

The three key elements of the Bundesnetzagentur’s efficiency analysis that we have explored above show how some small, reasonable changes can make a big impact for many. But this is just a snapshot. There are several other choices that the Bundesnetzagentur could make within these elements and on other aspects of its analysis (e.g. determination of model size; second-stage validation of model).

On the one hand, more regulatory freedom would mean that the Bundesnetzagentur could improve on aspects of its efficiency analysis to make its assessment more robust. Alternatively, such freedom could make the ‘margin of discretion’ so wide that it would make the benchmarking outcomes highly uncertain and extremely difficult for networks to show that the Bundesnetzagentur has made an ‘error’.

Preparing for change with EffizienzApp

The Bundesnetzagentur is currently preparing its TOTEX benchmarking for the fourth regulatory period. To understand what discretion the regulator already has to determine efficiency scores, how this might change following the ECJ’s September ruling, and what the repercussions might be for German energy networks, it is advisable to start preparing early for this discussion. Why not explore how preparing for change with EffizienzApp could help you?

¹ This framework is largely set out by the ‘Anreizregulierungsverordnung’ (ARegV, Incentive regulation ordinance, §§ 12–16 ARegV).

² OPINION OF ADVOCATE GENERAL PITRUZZELLA, Delivered on 14 January 2021 (1), Case C‑718/18, European Commission v Federal Republic of Germany.

³ Clearly, economic efficiency should not be pursued by compromising network reliability, customer service or environmental obligations. Timely investments should also be encouraged and a proper balance must be struck to ensure intergenerational equity. In this article, we focus on how the approach that the Bundesnetzagentur has followed historically to ensure the economic efficiency of the energy networks could change following the ECJ’s ruling.

⁴ In other words, the German gas and electricity distribution and transmission companies.

⁵ Bundesrat, Drucksache 417/07 vom 15.06.07, S. 55.

⁶ See Bundesnetzagentur (2018), ‘Effizienzvergleich Verteilernetzbetreiber’, May, available at: https://www.bundesnetzagentur.de/DE/Sachgebiete/ElektrizitaetundGas/Unternehmen_Institutionen/Netzentgelte/Gas/EffizienzvergleichVerteilernetzbetreiber/effizienzvergleichverteilernetzbetreiber-node.html. Following a court order, the data was removed on 17 July 2018. Data issues were picked up by many DNOs, including ED Netze GmbH (2018), ‘Stellungnahme zur Konsultation Effizienzvergleich Strom 3. RP’, August.

⁷ While we use the German gas DNO dataset to present potential implications, the insights are not limited to the gas DNOs and also relevant for the electricity DNOs and the transmission system operators given similarity in the procedure prescribed in the ARegV. 

⁸ Convergence issues describe a situation in which the algorithm used to find the solution to the SFA problem cannot find a solution. There can be many reasons for this, but the result is that this precise model cannot then be used or may need to be altered to produce a solution.

⁹ Frontier Economics (2019), ‘Effizienzvergleich Verteilnetzbetreiber Gas 3. RP’, July, p. 105.

¹⁰ According to the ARegV, the efficiency score assigned to a network operator is the maximum score derived from the two methods (DEA and SFA) and two options for calculating CAPEX (depreciated historical costs and annuities). Therefore, each network has four scores of which the best is used to set revenues, with the best efficiency score not lower than 60%.

¹¹ The model including potential peak load as an additional cost driver meets all of the statistical and operational requirements set out by the Bundesnetzagentur and is a clear improvement over the model used as it captures population change much more explicitly.

¹² For example, the Bundesnetzagentur previously considered SFA models with squared and interaction terms. These allow for increasing and decreasing returns to scale within the same production function.

¹³ See Kumbhakar, S., Parthasarathy, S. and Thanassoulis, E. (2018), ‘Validity of Bundesnetzagentur’s dominance test for outlier analysis under Data Envelopment Analysis’, Expert Report. August.

¹⁴ That is, by how much an efficient DNO could raise costs while remaining efficient.

¹⁵ For a detailed discussion of the issues, see Oxera (2020), ‘A critical assessment of TCB18 electricity’, April, pp. 65–69.

¹⁶ The proposed test is the Bootstrap test for dominance, which complies with the requirements of the ARegV. This test was developed in Kumbhakar, S., Parthasarathy, S. and Thanassoulis, E. (2018), ‘Validity of Bundesnetzagentur’s dominance test for outlier analysis under Data Envelopment Analysis’, Expert Report. August, and further discussed in Oxera (2020), ‘A critical assessment of TCB18 electricity’, April, pp. 66–67.