Episode 3 - Optimizing the Grid with Time Series Data

Romita Biswas
11 hours ago
9 min read

What does it mean to optimize the grid?

The grid as we know is responsible for delivering power to the people - reliably, safely, and instantaneously. Meeting that need comes with a price. People are increasingly concerned about the price of electricity. I believe when we talk about optimizing the grid - the problem we’re trying to solve is reducing the local price of electricity.

We’ve mentioned this before, how the grid is very local and varies not just street to street but even from home to home. How things like outage recovery or solar interconnection can be difficult depending on where you live. The concept of grid conditions being location specific also extends to the price of electricity.

The Price of Electricity

The price of electricity as in the Locational Marginal Price (LMP) changes every few neighborhoods to reflect the local needs for power at a point in time. The market operator which manages a large regional chunk of the grid measures it at various points for the bulk coordination of electricity acting like a marketplace to connect buyers to sellers.

The price is set based on your typical supply demand curve. When you have higher demand relative to supply you have a higher price, and when you have higher supply relative to demand, you have a lower price sometimes even negative just to up the demand.

As you can see even though it’s one market operator, PJM, prices have a pretty large range and vary across the region. Even when Chicago is getting hit with a snow storm, its electricity prices are significantly lower than DC’s - negative actually.

The areas around Chicago have so much power from nuclear generation that it's hard to ramp up and down. They usually end up with too much energy and have to sell their electricity to other places and at negative prices locally. And the people of Chicago benefit from that negative price too. This is ComEd’s website for their Hourly Pricing Program.

Daily Averages of Real Time Hourly Prices for Customers: Source

So if we are to optimize the grid by reducing the price of electricity at each local node, what can we optimize upon? Can the energy demands be different enough between two nodes two blocks apart for there to be a 14 dollar difference?

Washington DC Real Time LMP Nodes. Source

The price, LMP, is constructed not just from supply and demand but the congestion, transportation loss, and the price of the last unit of energy needed to cover the last bit of demand. It’s like ordering door dash from a few places around the city to meet the energy demands for your stomach.

Door Dash as LMP

It goes like this: it's 5:30 pm, you’ve just come back from the gym, and you need to protein load but have no food in the fridge so you decide to door dash rice bowls, smoothies, and dumplings. Except it’s 5:30 so there’s a lot of traffic on the road.

Getting your food delivered to you will take some effort. You’re not the only one who wants dumplings. So there’s a delivery fee and surcharge because of rush hour. The smoothie you wanted from the store that’s a little further away, the one that has a bunch of protein, is not in the delivery range anymore because of traffic, so you have to order a smoothie from the closer, lesser quality store.

To cheer yourself up and to meet your remaining protein needs, you order a nutella crepe at the last minute for no added delivery fee. Except the marginal price to meet your protein requirement is set by the last most expensive source of protein. And that nutella crepe unfortunately has some protein.

Whose order is this? I'm sure someone has had this. I'd get all of these except the smoothie.

In electricity markets, price is set with something called merit order. You got 62 grams of protein but that last nutella crepe sets the final price, so you pay $2 per gram for 62 grams with delivery fees plus traffic surcharge. You had planned to pay $0.50 per gram for a grand total of $31 based on getting the smoothie from your usual place. But that smoothie store was too far, traffic was bad, and you ended up needing the nutella crepe. Yummy but pricey.

Imagine if Door Dash was Merit Order based that would suck. But if Dashers got paid that way that could be great.

Most people would say you should cook at home, but if you don’t have an oven or just can’t cook, then door dash is the only option you have. Electricity markets are one of the few markets that use merit order to set the price but that’s what makes it so lucrative to meet energy needs and most people can't get energy from anywhere else.

Realtime LMP Profile for 2026: Source

One may say that most people aren’t exposed to LMP so reducing the costs to procure energy or generating on-grid power isn’t relevant for optimizing the grid, but that’s not true. Even if you pay a flat rate of electricity and at the most have a time of use tariff, that rate is still based on the expected price of electricity over a region. It’s a defined rate that takes into account the risk of LMPs changing every five minutes.

Optimizing the Price Node

A time of use tariff is implemented when price peaks become higher and occur more often. The tariff is supposed to lower the use of electricity when congestion tends to be higher to ensure the customer rate is enough to cover the varying price of electricity, but they’re usually pretty static and don’t account for local price spikes happening outside of the tariff times.

Optimizing the local price node is the most relevant way of improving the grid for the people. Optimizing the local price node means identifying where there tends to be congestion, locations with weaker operating limits, and where local on-grid power can be used.

The applicable idea from time of use tariffs is reducing the amount of power procured from the larger electricity marketplace when congestion tends to be high. Reducing the price of power at the node means minimizing how much bulk power is purchased from the marketplace and maximizing the available capacity under the local price node.

LMP node optimization: maximize capacity, minimize purchase. (ipad drawing)

Capacity here means - how much power under a single node can be played with at any given time. It includes power from rooftop solar that isn’t getting fully utilized, an EV sitting in a garage with nowhere to go, or someone like me who doesn’t care much about the temperature in their house. All this energy is essentially capacity that changes throughout the day at each node and should be used as much as possible to meet local demand without having to buy energy from the marketplace. (variable - maximize its usage)

Once it’s known how much capacity is available, then it needs to be known where that capacity can be safely applied to the grid. If there’s a lot of capacity on a street, but the connecting road can’t handle it, that capacity can’t be used to help minimize the price. So maybe the house two doors down can give power to the grid from their EV, but I may not be able to because that line my house connects to isn’t strong enough.

Modeling these local operating limits is difficult because modeling secondary grid constraints under a price node is nowhere near mature as that’s all secondary data, which we talked about last week - how utilities usually don’t have street to street mapping within neighborhoods.

Local operating limits vary street to street even home to home

The physical lengths and ratings of cables and equipment determine how much power can flow through without frying something, which is essentially what a grid constraint is. Taking a step to defining local operating limits through a rule based system could be a good starting point - based on observations like this street tends to consume too much power and it strains the system or this area doesn’t like excess power. Eventually as secondary models are developed and evolve, the operating limits can become much more location specific and reflective of grid conditions in real time.

Then we go back to that time of use tariff and ask it to be a stronger price signal that reflects the local need to address congestion and free up capacity.

The shortcoming of the typical time of use tariff is that it’s too static and applied broadly to a whole city which can include a lot of LMPs.

But the advantage of the time of use tariff is that it’s tech agnostic, it already exists, and regulators, utilities, and customers are all comfortable working with this mechanism to shift consumption patterns.

An honest strong price signal is a signal that will vary with time and location. It’s like each neighborhood, street, and home having different AC schedules based on how hot it actually is in their area - is there rain, are there a lot of trees and shade versus a whole city having the same summer time AC schedule. If the time of use tariff is meant to actually reduce the LMP at each node, it has to change with evolving grid conditions, available capacity, and of course the real time LMP. And again that’s because you can have really high LMPs outside of the tariff times. Static tariffs are not responsive or inclusive to real time conditions.

Real Time LMP in June. Rank 5 to 8 not tariffed as On-Peak. LMP Node under Dominion — Real Time LMP in **June**. Rank 5 to 8 not tariffed as On-Peak. LMP Node under Dominion

This tariff is unable to accommodate high LMPs in June after 6 pm.

So how would a responsive price signal be designed in order to ensure that customers respond and commit to their decisions to change their consumption? I think the static time of use tariff is a great starting point and can indicate if the price of the tariff is initially successful in securing customer response. That can serve as a baseline in understanding what price to apply when local operating limits are being approached and, of course, when there are higher market prices.

Baseline tariff with expected congestion and capacity. Scaled tariff with really bad congestion and no capacity. Applied to prevent future congestion.

Based on how bad conditions are, the price can then be scaled within justified limits to try and really reduce consumption. The on-peak time periods become dynamic, changing based on real time conditions. Then comes in incentivizing people who have capacity to share it with the rest of their neighbors to maximize the use of capacity at a node, where they’re directly exposed to the electricity marketplace prices or are paid via this tariff price signal mechanism.

Tariff is like a responsive signal changing with real time and forecasted grid congestion and capacity.

The other nice thing about making the price signal more responsive is that we’re only charged for how much energy we actually consume so we’re not penalized by the larger marketplace for saying oh actually I don’t want to buy electricity from you right now. So right after a LMP is settled upon by the marketplace it lasts for five minutes, I can say after one minute no I don’t want to participate in your funny business which is something that the big players of the marketplace can’t do.

So customers on the distribution grid have the opportunity to be a lot more dynamic than their counterparts in the larger marketplaces, but the right incentives and mechanisms to actively play and provide network support isn’t there yet. This idea of dynamic pricing is something that is being explored and tested out in, of course, Australia with Project Edith.

Ausgrid Knowledge Sharing Report for implementing Project Edith

It’s one of my favorite solutions because it's purely incentives based, not tied to a specific energy management platform, and is clean - everyone understands there’s a price to using electricity.

And of course there’s a bunch of details in implementation like should customers receive push notifications, would an aggregator be used as a middle man to automate customer participation, and most importantly would utilities be incentivized to pursue such a route - to keeping rates frozen or even lowering them. Utilities being incentivized completely depends on the jurisdiction and how the commissions have decided to pay them out - is it based on performance, from hardware upgrades, or do they get to keep a cut of the flat rate covering the volatility of market prices? How does your utility get paid out, are they incentivized to keep your rate down, and if not how can you push your local commission to get that to change?

I hope you guys found this interesting and helpful in understanding electricity prices and a solution to reducing them. In the Next Episode I want to explore Decentralization of the Grid as a Mechanism for Increasing Resilience and Security.

References: