Big Data Flows: Water, Outsourcing, and the Flood of Data

Big Data is booming, and water utilities are beginning to be able to take advantage by leveraging the skills of those who already know how to collect and manage data.

This article is a part of the Water for Agriculture Theme. For more articles in this category, click here

With the help of big data, old water meters like the one underneath this cover are starting to get a makeover. Image Credit: Teddy Page

With the help of big data, old water meters like the one underneath this cover are starting to get a makeover. Image Credit: Teddy Page

Floods like those in Texas in May make it seem like water is something out of human control. However, for the water industry, whose very purpose is to control the flow of water to homes and businesses, even better control may come with another flood: one of vast amounts of data.

Water metrology (the science of measurement) has been developing for more than 100 years. Recently, that development has brought water into the era of “Big Data.” Even though utilities aren’t technology companies that are necessarily prepared to either collect or manage a large amount of data, recent innovations are allowing outside parties to perform these tasks for them.

Current Technologies

There are currently several technologies to record the amount of water flowing through a meter, including displacement meters, velocity meters, and electromagnetic meters.

There also have been global developments in how the data these meters produce is handled, and how much data is collected.

Traditional meters are read via a dial that keeps a running total of how much water a consumer uses, like a car odometer keeping track of miles driven. Typically, readings are taken manually each month by a utility worker who walks up to and inspects each meter, and monthly bills are calculated using the difference between the current total and last month’s total.

This data collection process evolved over time around the world towards automatic meter reading (AMR) devices. These smart meters have a data logger which sends a digital signal at a predetermined rate to either a utility worker driving by or directly to the utility. While this doesn’t necessarily change the rate at which data is taken, it decreases the amount of time workers have to spend collecting the metering data.

There are significant markets for smart meters in North America and Europe, particularly in France and Germany. North America received 70 percent of all advanced water meter shipments in 2010.

At the most advanced end of the spectrum of water metering technology is advanced metering infrastructure (AMI). AMI represents a full commitment to a smart grid: not only are the meters smart, but so is the entire system that uses the information they produce.

With AMI, utilities gain two-way communication with the meters, and thereby access to a flow of data from the meters that is real-time and available on demand. The meters deliver data to a data collector, or endpoint, which then sends the data to a receiver over a fixed network (a network which doesn’t require the movement of utility personnel to collect the data). Once the utilities have the data, they can use any number of services to provide some or all of that data directly to consumers.

However, there are two significant hurdles inherent to AMI: implementing infrastructure to produce and collect the data, and managing the data that’s collected.

Implementation: Use Pre-Existing Networks

One way of clearing the implementation hurdle is to use a pre-existing data collection infrastructure. An example of such an infrastructure is the cellular network.

Cellular water metering represents “the first type of fixed network reading where infrastructure isn’t required,” says Kristie Anderson, a product-marketing manager for Badger Meter. The technology allows utilities to simply plug their meter endpoints into the cellular network.

This takes some of the burden of adapting to new technologies off of the utilities’ shoulders: without the existing cellular infrastructure, the water utilities would have to set up their own data receivers. However, as Anderson notes, by using the cellular network, the cost of implementing AMI is significantly reduced.

The market for cellular water meters is already growing (Figure 1). By 2020, it is estimated that 600,000 cellular water meters will be distributed annually, with companies such as Badger Meter, Arad Group, Neptune Technology Group, and Master Meter introducing cellular metering technologies. Moreover, partnerships between companies providing cellular capabilities and metering companies, such as those between Novatel Wireless and Capstone Metering in the U.S., and Vodafone and Kamstrup in Europe, suggest that cellular water metering is booming.

Figure 1: Cellular smart meter shipments are expected to rise throughout the decade, as is the percentage of overall smart meter shipments that are cellular. Image Credit: IHS

Figure 1: Cellular smart meter shipments are expected to rise throughout the decade, as is the percentage of overall smart meter shipments that are cellular. Image Credit: IHS


While this might suggest that cellular metering is the next big step in metering data collection technology in the near future, Anderson is not ready to say so just yet.

“I think everybody’s keeping their eyes open for what’s next,” she said.

Management: Use Others’ Software

To overcome the data management hurdle, water utilities can simply let someone else take care of their data.

With all of the data an AMI system might provide the utility, there needs to be software that can analyze the data and give relevant information on incremental usage to a billing system. But utilities that don’t have to have their own software or expertise to understand the data from their meters can instead seek help from companies that provide software as a service (SaaS).

In a white paper on cloud software solutions for water utilities, Anderson explains that instead of setting up a data center on site, the utility can hire a solutions vendor that hosts the analytics software in its own data center. Because the vendor specializes in storing and manipulating the data, and has experts who can quickly address problems, the utility can have its data maintained in a more secure fashion. Whenever the utility wants to access the data, managers can simply log in to the application software over the web.

With this system, utilities can leverage the expertise of vendors who specialize in data management to create a powerful system of data analytics.

Just as there are partnerships developing between cellular companies and water metering companies, similar partnerships between analytics groups and metering companies are on the horizon. For example, IBM is working with the Arad Group, an international provider of water metering systems, to provide data analytics solutions for utilities.

Several large companies lead the smart water management market, including IBM, General Electric, Itron, and Sensus. The market is projected to grow from around $7 billion in 2015 to more than $18 billion by 2020.

Potential Tradeoffs

The gift of cellular metering and outsourced data solutions to utilities is, in short, lots of data, which can have several benefits. The most obvious benefit is that it allows water users to more efficiently manage their water.

The extra data provided by a smart metering grid also allows suppliers get a much more accurate picture of where their water is going. From Anderson’s own experience, what users of smart water systems most often say is the most important benefit is the ability to detect leaks.

According to an Oracle survey, 62 percent of surveyed U.S. and Canadian utility managers picked leak detection as the biggest benefit.

It’s easy to see why: Leaks are an important and discouragingly common issue. The average household loses more than 10,000 gallons a year, adding up to more than 1 trillion gallons nationwide. Leaks in the U.S. water system represent 14 to 18 percent of daily water use.

In the Oracle survey, 35 percent of the utility managers also mentioned that giving their customers tools to monitor and conserve their water was beneficial. Anderson notes that the added customer service plays a huge role in implementing the systems.

“I’ve been pleasantly surprised by the utilities that have adopted it not to conserve but from a customer service perspective,” says Anderson. The additional data provides transparency and adaptability for consumers seeking more control in how much water they use, and allows the utilities to “(answer) questions before they become complaints.”

For consumers who are trying to conserve, the new meters also can tell them if their conservation efforts are having an impact,  Anderson said.

However, there are barriers to using cellular meters, or smart meters in general. One is cost. Regardless of the implementation of data collection or analytics, the utility still has to install and maintain its own meters and endpoints, and smart meters can cost more than $300, two or three times more than a standard meter with a simple dial. Even though the cellular meter readers and analytics solutions are meant to decrease the initial financial burden for utilities, 42 percent of the utilities surveyed by Oracle cited upfront expenses as a major hurdle, and 46 percent were concerned with a “lack of cost recovery or measurable return on investment.”

According to a paper by Michele Mutchek of Arizona State University and Eric Williams of the Rochester Institute of Technology, the large upfront costs of smart meters tend to go against the principles of the U.S. water industry:  providing a service people take for granted, at a price that helps them do just that. There are “institutional and political structures that favor the current system.” These include an infrastructure that supports incremental fixes rather than sweeping changes, and a low cost of water, at least in part perpetuated by the lack of meters, which prevents the utilities from building up too much capital.

Despite all of this, according to industry experts, it is “critical” for water utilities to adopt advance metering. Once the high costs and other barriers are overcome, utilities may start leveraging growing smart grid capabilities to enter the era of Big Data.

Alec Drobac is a senior physics major at Middlebury College in Vermont. He hopes to pursue a career as a theoretical physicist, potentially in the field of astrophysics or cosmology. Originally from California, he is particularly concerned with water usage and conservation, as well as the advancement of technology in agriculture.