The State of Dark Data in the Energy Industry

When we last wrote about dark data and its energy implications, we discussed how dark data is rarely leveraged to its fullest potential, as well as the roadblocks hindering its collection across industries. Let’s take a deeper dive into the current state of how dark data is used in the energy world, how lessons from another market might help energy, and understand how dark data can be harnessed for greater efficiency within the energy industry. 

In case you need a quick refresh, dark data refers to the data that organizations gather or have available to them but don’t actually use to its full business potential to generate revenue. That can include everything from email correspondence to log files to security video footage to that old filing cabinet full of contracts. All of those data points could potentially be harnessed to generate key insights, but instead , they sit idle and unused. With all that locked potential, dark data could play a much larger role in the energy industry. We’ll look at 3 different areas of the industry for their opportunity.

How Dark Data Will Improve the Grid

Dark or not, utilities and ISOs make data-driven energy decisions every day. Energy’s complex system requires incredible amounts of data on demand and supply to be optimized in real-time for smooth operations. Increasing sensitivity to grid security, questions about reliability due to the influx of distributed resources and a focus on resilience in the face of climate events are increasing the need for investment in new monitoring tools at the utility and ISO level. 

The growth of IoT-connected devices is also driving a big data boom in the energy industry. Smart meters, cloud computing, and other emerging technologies are making data collection and analysis easier than ever, leading to greater efficiency and a reduction of maintenance and operation costs along the energy value chain. But tons of useful information from these devices go underutilized. 

Take the example of smart meters. According to a study from the American Council for an Energy-Efficient Economy (ACEEE), many utilities that have deployed smart meter technologies have yet to maximize the value of dark that they collect on these devices. This prevents these utilities from providing optimization services and better managing demand on their own grid. 

Dark data will have to play a central role in optimizing grid operations and resources in the future. Other industries have tapped into the potential of complex data analysis — why not energy?

Better Understanding Your Customer with Dark Data

We increasingly know more about residential energy customers with certain technologies becoming more widespread: smart meters, smart thermostats, smart home equipment, and increasingly EVs. Service providers like Tesla, Nest and Amazon help these customers monitor energy usage and display it in easy to understand interfaces. By collecting their real-time demand data and understanding more about consumption behaviors, these providers can sell services to utilities that manage load or even reduce demand to provide “grid services” as well as regular consumer value. 

But the dark non-energy data in this scenario is still not unlocked. Consider for a moment that each of these service providers don’t know whether they are servicing the same customer. Or take it even further and share that data with the energy supplier and utility servicing that customer as well. How might the ecosystem benefit from a richer understanding of the energy consumer? 

A holy grail for the industry is the ability to predict load. By sharing information about when customers get home from navigation apps or when they use smart devices based on smart-home applications, utilities and suppliers will be able to get more accurate pictures of needs. Something as simple as knowing whether a person got a new Nissan Leaf can be game changing for infrastructure planning.

Energy Retailers Should Keep Its Eye Towards Transformation

Today, data is leveraged across energy generation, trading, transmission, distribution, and consumption. There’s a general consensus on how this existing data can be further leveraged for future insights and giving customers greater services, as ongoing energy digitization and decarbonization creates vast opportunities for further data usage. But it’s hard to determine what data is missing or unavailable when most available energy data goes unused to begin with. 

Take the simple example of helping customers better understand the type of energy deals that they are signing up for with energy suppliers. Often this process is full of dark data! Customers might not fully understand or know key energy risks or components that might be exposed to market fluctuations. This isn’t merely a problem of transparency but also of wasted data.

Saavy customers and energy consultants will recognize that where there is risk there is opportunity. By sharing with customers details about the types of risks they might be exposed to in an energy contract, you have the ability to sell them on more advanced offerings like demand response, battery storage or on-site solar. This can build you credibility and you also serve as a one person army helping bring more data to light in the ecosystem. 

Innovative players in the energy space (like Dash Energy) are working to go beyond that, building tools that make extracting, aggregating, analyzing, and sharing data easier than ever before. The more data is created, the more important data digitization, analysis, efficiency, and modernization becomes. And the more that data can be utilized by the customer for their advantage.

The energy industry needs to leverage the tools at its disposal to make the most of the data it generates.

When data analysis is pushed to its fullest potential, it is a major driver for energy efficiency — we just have to keep pushing it to new levels of discovery and intelligence.