How open-source tech is changing NZ's electricity grid

Transpower, the owner and operator of New Zealand’s national electricity grid, has implemented open-source technology as a key part of a 10-year plan to modernise its Market System.

The Market System is a critical application that is designed to ensure a secure and robust power system, along with the real-time provision of optimal electricity prices.

Crunch time for the NZ grid

Transpower’s infrastructure consists of 174 substations, 25,000 transmission towers and almost 11,000 km of lines that run the entirety and width of the country. It must deliver bulk electricity (up to 220,000 V) to power cities, towns and major industrial customers.

In 2013, the grid operator faced a looming problem. The solutions it used to support mission-critical applications were nearing the end of their life and needed replacing. However, the company only had a limited number of experts available to work with these complex systems.

Confronted with the risks of outdated and unsupported technology, as well as a lack of available expertise to support architecture that had become increasingly costly to maintain and suffered long outages, Transpower initiated some sweeping changes.

The roadmap for transformation

To overhaul its Market System, Transpower created a 10-year integrated roadmap. Its aim was to ensure the system remained fit for purpose, cost-effective, resilient, flexible and scalable in the face of large-scale industry change.

The roadmap set out three distinct areas for transformation: people capability, processes and technology changes. As part of a change in processes and tools, Transpower deployed DevOps and agile ways of working.

Further technology changes included adopting open-source solutions to replace end-of-life proprietary products and to simplify architecture while shifting to modern programming languages. This would allow Transpower to avoid a full system rewrite, resulting in considerable financial savings and minimising service disruptions.

A development-first approach

As part of its aim to deploy developer-first platforms (ie, those that prioritise developer experience), Transpower had already used a range of solutions from US software company Red Hat, including Red Hat Enterprise Linux (RHEL) to support early transformation projects. This approach was critical in attracting and retaining key staff to support the grid operator’s modernisation process.

Transpower made full use of Red Hat Fuse, fully automating the deployment of Karaf clusters across development, testing, staging, training, and real-time and operational support environments. Red Hat Satellite and RHEL were deployed to support automation across the Transpower fleet of servers.

Automation allowed Transpower to build and rebuild environments from source code. By porting the corporate ESB technology to Red Hat Fuse, it also achieved a two-year payback period.

As Transpower plans the next stages of its modernisation journey, Red Hat Application Foundations continue to provide core technologies for building, deploying and operating these applications.

Building a foundation for innovation

To maintain a dependable electricity grid, vital functions such as a secure power system, energy dispatch, and the operation of the wholesale electricity market required applications that were both reliable and ready to react to innovations in the market.

Starting with the dispatch system, Transpower began by updating key applications as Java microservices on Red Hat OpenShift as an application platform. The lessons learned from the dispatch project gave Transpower the confidence to begin modernising the architecture of the wider Market System, also built on Red Hat OpenShift.

This stable yet flexible platform allows Transpower to adapt and take on complex challenges, particularly as new power generation types and new technologies are connected to the grid. In 2022, Transpower launched real-time pricing, enabling generators, retailers and electricity consumers to make decisions in real time about energy consumption or generation.

A win-win solution

With the new tools in place, Transpower has grown its Market Systems Development Team from three staff to over 20, helping to reduce organisational risk. This has also enabled the team to scale delivery and take on large industry changes similar to real-time pricing.

“The modernised system enabled us to deliver at scale with the largest single change in the electricity market since 1996,” said Daniel Crawshay, Operations, Process and Technology Improvement Manager at Transpower.

“It’s enabled us to build our capabilities while delivering a quality output to customers.”

Image credit: iStock.com/Mlenny