A crucial ERCOT technical failure had engineers scrambling last month and led to widespread confusion among power traders — although grid officials say system reliability was never endangered.
ERCOT, or the Electric Reliability Council of Texas, oversees the state’s main power grid and administers a segment of the state’s wholesale power market. The more-than-hour-long outage of its Energy Management System resulted in the interruption of data for many of its most important operating systems.
According to ERCOT officials, its Energy Management System failed completely for 24 minutes beginning at 11:41 a.m. on July 7, and then operated in a “degraded” state for another 78 minutes.
“I know ERCOT was scrambling to recover the system … but from a market perspective, it was pure chaos,” said Bill Barnes, energy director of the Houston-based NRG generation company.
The Energy Management System is a key component of ERCOT operations. The complex EMS software evaluates real-time power generation availability, allows ERCOT technicians to maintain constant energy frequency across the grid and provides dispatch instructions for power plant operators. Wholesale power traders depend upon EMS to make pricing decisions.
The outage was apparently caused by operator error. According to a discussion of the incident during a recent meeting of ERCOT’s Technical Advisory Committee, an ERCOT technician mistakenly loaded computerized test data into the active EMS network, resulting in the system outage.
Although a serious matter, the EMS failure did not create a blackout risk or undermine grid stability, according to ERCOT officials. “We have procedures and take actions to prevent blackouts even if EMS fails — there are multiple levels of mitigation and backstops to prevent real grid issues,” said ERCOT spokeswoman Robbie Searcy.
According to ERCOT, power schedulers that depend upon the EMS software were immediately notified of the outage. The schedulers were instructed both to disregard the bad EMS data and to maintain generation levels.
ERCOT officials said the July 7 incident was unrelated to recent upgrades to its EMS software. ERCOT officials also said they will continue reviewing the outage — both with an eye toward avoiding them in the future and also to settle pricing claims among wholesale market participants who were impacted by the data loss.
R.A. Dyer is a policy analyst for TCAP, a coalition of more than 170 cities and other political subdivisions that purchase electricity in the deregulated market for their own governmental use. Because high energy costs can impact municipal budgets and the ability to fund essential services, TCAP, as part of its mission, actively promotes affordable energy policies. High energy prices also place a burden on local businesses and home consumers.