The Voice of Distibuted Energy
Menu
  • Stories
    • Distributed Generation
    • Storage & EVs
    • Virtual Power
    • Energy Efficiency
    • Policy
    • Microgrids Now
    • Prosumers
    • Decentralized Grid
  • Podcast
  • Resources
    • White Papers
    • Case Studies
    • Content Services
  • About
  • Newsletter
  • Substack
Energy Changemakers
Subscribe
Decentralized Grid

Death by a Million PDFs: Confusing and Inaccessible Data Delays Virtual Power Plants

by Elisa Wood

virtual power plant
Natasa Adzic/Shutterstock.com
March 3, 2024
Share

People tend to think it is the big, sinister acts that slow the energy transition — oil empires disseminating misinformation about electric vehicles or utilities lobbying the government to thwart customer-owned energy.

But less dramatic, even banal acts can cause equal damage. Take for example what virtual power plant (VPP) developers face in trying to access customer data from electric utilities.

The details vary by state, and some utilities perform the task better than others. Still, stories are widespread about how hard it is to gain information about customer energy usage, rate tariffs, or other data necessary to determine cost savings achieved by aggregating customers into virtual power plants. 

The scenario plays out something like this. A virtual power plant provider requests data from a utility and weeks go by before it arrives. Then it comes in a poorly formatted electronic spreadsheet or a form that doesn’t allow data manipulation, such as a PDF document or even on paper via fax. Analysts struggle to make sense of what’s before them.

“It creates a lag in your deal development,” said Duncan Campbell, vice president for Scale Microgrid Solutions, which designs, builds, finances, and operates distributed energy assets.

Some utilities offer a solution: Ask customers to download the data since they can access their electricity usage via a utility portal. But that’s not as easy as it sounds. “The customer might have no idea it exists. You try to explain it to them. They’re confused,” Campbell said. Frustrated, they ask the VPP provider to handle the download and offer up their username and password, which can “create a legitimate security issue.”

Why is customer usage data so hard to get?

Kenneth Schisler, senior vice president of regulatory & government affairs for VPP provider CPower says the problem goes back to Obama-era funding of utility smart meters. The American Recovery and Reinvestment Act designated $4.3 billion for smart meters that served as a springboard for a plethora of data being produced today by 119 million smart meters in the US  – 72% of total electric meter installations

“The problem with the Obama era stimulus money —  so much of which was plowed into the smart grid — is that we really didn’t think about how to get that data to companies like CPower who could then monetize that flexibility in the wholesale market. So we still have an awful lot of manual data acquisition work that is just not scalable. The smart grid deployments were not designed to create this backhaul capability to transfer the data in bulk at scale to aggregators,” he said.

Michael Murray, president of Mission:data, which advocates for customer-friendly energy data access policies, said utilities fear releasing information will lead to data breaches, litigation and reputational harm.

“If the customers pay for the meters, they should be entitled to 100% of the benefits and the control over their information and energy usage that is reasonably attainable,” he said.

16th-century terminology

Customer electric usage isn’t the only kind of hard-to-access data that virtual power plant providers need. Providers also must uncover the complexities of the utility tariff that applies to the customer – information that can be difficult to access, according to Klaar De Schepper, CEO of Flux Tailor, a software company focused on utility data driven clean energy solutions.

First of all, tariffs are not simple and may have several permutations. For example, a customer may be on a specific rate plan but could also be participating in a pilot program that manages demand charges, with further delineations under that, she said. So the VPP provider sometimes has to dig deep to fully understand the customer’s bill.

“When you’re building a model to see how much you can save by implementing a virtual power plant, you need to go under the hood of the model of the rate plan and optimize for when power is least expensive,” De Schepper said. “That involves getting into special rules that apply for special conditions with special types of customers, and it gets quite complex.”  

Getting under the hood isn’t easy because rate plan information is not available in structured machine accessible format, and there isn’t a standard format for representing this information across states and utility service territories. 

“It is a complete black box, and honestly, nobody benefits,” De Schepper said.

Underscoring how archaic tariff presentations can be, De Schepper pointed out that New York doesn’t even use page numbers but segments its tariff pages according to “leaves,” a 16th-century term.

Who’s in what program?

According to Murray, problems can also arise if a customer signs up for a conflicting energy savings program that the VPP provider is unaware of.

For example, the customer may have enrolled in a peak-time pricing plan. The utility rejects the customer from the VPP because it has already calculated the retail rate responsiveness into its capacity obligation. Customers don’t warn virtual power plant providers about the conflict beforehand because they are unaware of it. 

“These are byzantine rules that we can’t expect them [customers] to understand,” Murray said.

And then there’s the wholesale market 

When the Federal Energy Regulatory Commission issued Order No. 2222 in 2020, it looked like distributed energy resources would finally have a clear path into competing in wholesale markets – and be paid accordingly. 

But obstructions persist, data collection among them. That’s something Mission:data is now studying. 

One stumbling point is access to information about transmission zones. Virtual power plant providers must identify the transmission zone for each customer participating in a virtual power plant.

“You can imagine that if I have 50,000 customers in New York, I don’t necessarily know what transmission zone they’re in,” Murray said. “Some of these big apartment buildings are right on the line between two transmission zones.”

The lucky virtual power plant provider finds a friendly person at an electric utility with a spreadsheet that links customer addresses to transmission zones. Not all providers are lucky.

“It’s a hell of a vulnerability for a business,” Murray said.

Also see: Technology Isn’t Causing the DER Industry’s Data Woes. It’s People.

Subscribe to our free weekly newsletter about all things distributed energy.
Subscribe

Top Podcast

Xendee University Week

Inside This Company’s Effort to Unite Microgrid Stakeholders

An inside look at what led to creation of Xendee's University Week, June 8-10, 2026.

Latest Resources

2026 Market Survey: Microgrid and Distributed Energy Projects Grow in Complexity

Reports of distributed energy’s demise have been greatly exaggerated, according to a 2025 survey of distributed energy and microgrid industry professionals. When the One Big Beautiful Bill Act became law in July 2025, many industry experts feared solar-heavy distributed energy development would dry up with the loss of federal incentives. While survey respondents (67%) say […]
Ameresco energy allies

Energy Allies: Communities and Utilities Building Resilience Together

This white paper examines how successful partnerships among communities, utilities, and energy service providers can improve the grid and achieve mutually beneficial outcomes.
microgrids as building blocks

Microgrids as a Building Block for Future Grids

This white paper is the fourth in a series of seven white papers in support of the DOE Microgrid R&D Program and presents a broad vision for future grids where microgrids serve as a building block along with technologies that would need to be developed, use case scenarios and the research targets. The DOE Microgrid […]
utility bills

Utility Bills are Rising: PowerLines

New polling by PowerLines, a nonpartisan consumer education nonprofit, reveals the toll of rising utility bills on American energy consumers.

RSS Microgrids Now

  • Montgomery County, Maryland, Notches Another Microgrid Win, This One for Affordable Housing
  • Against the Odds, Communities Can Build Local Energy. Here’s How
  • Microgrids Are a Goldmine for Grid Operators — They Just Don’t Know It
  • What Makes a Community Microgrid Actually Work? Start by Listening, Not Engineering
  • Is Distributed Wind Poised for a Second Act in Virtual Power Plants and Microgrids?
The Voice of Distributed Energy
Learn about grid edge opportunities
Subscribe
Energy Changemakers
Energy Changemakers
600 Twentyninth Place Ct #1055
Charlottesville, VA 22901
elisa@energychangemakers.com
© Wood Energy Writers LLC. All rights reserved.
  • Terms of Service
  • Privacy Policy