Smart Data Infrastructure for Wet Weather Control and Decision Support May 2017
1
August 2018
CONTENTS
Disclaimer ...................................................................................................................................................... iv
Acknowledgments .......................................................................................................................................... v
Acronyms and Abbreviations ........................................................................................................................ vi
Glossary ........................................................................................................................................................ vii
1. Introduction ..................................................................................................................................... 1
2. Smart Data Infrastructure ................................................................................................................ 2
3. Smart Data Infrastructure and Technologies: Information Inputs .................................................. 4
Continuous Monitoring ....................................................................................................... 4
Level Monitoring ................................................................................................................. 5
Flow Monitoring .................................................................................................................. 6
Physical Flow Monitoring ...................................................................................... 6
Alternative Flow Monitoring Technologies .......................................................... 6
Rainfall Monitoring .............................................................................................................. 7
4. Collection System Optimization ....................................................................................................... 8
Capacity Management Operation and Maintenance and I/I Control ............................... 10
5. Real-Time Control Systems ............................................................................................................ 10
Components of an RTC System ......................................................................................... 12
Supervisory Control and Data Acquisition Systems ............................................ 12
Real-Time Decision Support Systems ................................................................................ 13
Level of Control ................................................................................................................. 14
Guidelines for Applying RTC .............................................................................................. 16
Key Considerations for RTC Systems ................................................................................. 17
6. Data Management and Sharing ..................................................................................................... 18
Big Data Management ....................................................................................................... 18
Data Sharing ...................................................................................................................... 18
Real-Time Public Notification and Transparency .............................................................. 19
7. Data Analytics ................................................................................................................................. 19
Data Validation and Filtering ............................................................................................. 20
Key Performance Indicators .............................................................................................. 21
8. Data Visualization and Decision Support Systems ......................................................................... 22
9. The Future of Data Gathering Technology for Wet Weather Control and Decision-Making ......... 23
10. References ...................................................................................................................................... 25
ii
APPENDIX A: CASE STUDIES
Buffalo, New York: Real Time Control of Inline Storage ............................................................................. A-1
Falcon Heights, Minnesota: Predictive Flood Control System ................................................................... A-3
Hawthorne, California: Real-Time Monitoring to Prevent Sewer Overflows ............................................. A-5
Louisville, Kentucky: Real-Time Control for Integrated Overflow Abatement ........................................... A-6
Newburgh, New York: Real-Time Control to Monitor Discharges for Reporting/Public Notification ........ A-9
Philadelphia, Pennsylvania: Real-Time Control to Manage Retention Pond Discharge........................... A-10
San Antonio, Texas: Real-Time Control for Cleaning Optimization .......................................................... A-12
San Diego, California: Stormwater Harvesting Augmentation Analysis ................................................... A-14
South Bend, Indiana: Real-Time Control and Real-Time Decision Support .............................................. A-16
Washington, DC: Real Time Controls for Rainwater Harvesting and Combined Sewer Overflow
Control ...................................................................................................................................................... A-18
Wilmington, Delaware: Real-Time Control to Reduce Combined Sewer Overflow Discharges ............... A-20
iii
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Disclaimer
The material and case studies presented in this document are intended solely for informational
purposes. This document is not intended, nor can it be relied on, to create any rights enforceable by any
party in litigation with the United States. Case studies used in this document are unique and site-specific,
and they may not be as effective as demonstrated. This document may be revised or updated without
public notice to reflect changes in the technologies and to update and/or add case studies. The .
Environmental Protection Agency (EPA) and its employees do not endorse any products, services, or
enterprises.
Mention of trade names or commercial products in this document does not constitute an endorsement
or recommendation for use.
iv
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Acknowledgments
Many stakeholders and subject matter experts contributed to this document, including:
• Jeff Wennberg, City of Rutland, Vermont
• Missy Gatterdam, Metropolitan Sewer District of Greater Cincinnati
• Edward D. Speer, CDM Smith
• Marcus Quigley, Opti
• Hari Vasupuram, Opti
• David Drake, SmartCover
• Tim Braun, Emnet
The document was developed under EPA Contracts EP-C-11-009 and EP-C-16-003.
v
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Acronyms and Abbreviations
CFR Code of Federal Regulations
CMOM Capacity Management Operation and Maintenance
CPU Central Processing Unit
CSO Combined Sewer Overflow
DSS Decision Support System
EPA . Environmental Protection Agency
FOG Fats, Oils, And Grease
GUI Graphical User Interface
ICS Industrial Control System
IoT Internet of Things
I/I Inflow and Infiltration
IT Information Technology
KPI Key Performance Indicator
LTCP Long-Term Control Plan
MMSD Milwaukee Metropolitan Sewerage District
MSD Metropolitan Sewer District (Louisville)
MSDGC Metropolitan Sewer District of Greater Cincinnati
O&M Operation and Maintenance
PLC Programmable Logic Controller
PWD Philadelphia Water Department
RTC Real-Time Control
RTDSS Real-Time Decision Support System
SAWS San Antonio Water System
SCADA Supervisory Control and Data Acquisition
SSO Sanitary Sewer Overflow
WWTP Wastewater Treatment Plant
vi
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Glossary
Agent-Based Control: System with locally interacting components that achieve a coherent global
behavior. Through the simple interaction of buying and selling among individual agents, a desirable
global effect is achieved, such as fair allocation of resources.
Big Data: Data sets that are so large or complex that traditional data processing application software is
inadequate to deal with them.
Cloud: Large-scale, offsite data storage facilities.
EPA SUSTAIN: Framework for the placement of best management practices in urban watersheds.
Gray Infrastructure: Engineering projects that use concrete and steel.
Green Infrastructure: Projects that depend on plants and ecosystem services.
Internet of Things: Process in which hardware is connected to a network (the internet) so that it can
better communicate with other systems.
Long-Term Control Plan: Written strategy required by the Clean Water Act for communities with
combined sewer systems to reduce and/or eliminate combined sewer overflow discharges in the long
term.
Machine Learning: Data analytic method used to devise complex models and algorithms that lend
themselves to prediction. This is also known as predictive analytics. There are many algorithms available.
Model Predictive Control: Model-based control strategy that predicts the system response to establish a
proper control action. This strategy explicitly uses a mathematical model of the process to generate a
sequence of future actions within a finite prediction horizon that minimizes a given cost function.
Real-Time Control: The ability of water infrastructure (valves, weirs, pumps, etc.) to be self-adjusting or
remotely adjusted in response to current weather conditions.
Smart Water and Smart Data Infrastructure: The ecosystem of technology tools and solutions focused
on the collection, storage, and/or analysis of water-related data.
vii
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
1. Introduction
Rain and snowmelt (referred to as wet weather
conditions) can significantly increase flows at
wastewater treatment facilities, creating
operational challenges and potentially affecting
treatment efficiency, reliability, and control of
treatment units at these facilities.
Current approaches to wet weather control rely
primarily on gray or green infrastructure, or a
combination of the two. In recent years,
however, municipalities and utilities have been
considering how they can take advantage of
technological advances to improve their
operations and infrastructure. These advances
include:
• Faster computer processing and network
speeds, providing ready access to reliable
information for informed decisions.
• Smaller, more accurate, and less expensive
sensors.
• Low-cost storage of large quantities of data.
• The advent of the “internet of things” (IoT),
allowing sensors to be connected over large
geographic areas.
• Smaller, higher-capacity batteries and
photovoltaics, reducing dependence on
permanent hard-wired power sources.
• Wireless transmittal of acquired data,
reducing the need for continuous or dial-up
hard-wired communications systems.
This document focuses on how municipalities,
utilities, and related organizations can use
advances in technology to implement “smart
data infrastructure” for wet weather control—
that is, how they can use advanced monitoring
data to support wet weather control and
decision-making in real time or near real time.
Case studies about communities that have done
this across the country are included as
appendices and referenced where applicable
throughout the report.
What Is in This Document?
This document summarizes key aspects of utility
operations where smart data systems can provide
significant benefits. It is organized as follows:
Section 2 presents an overview of smart data
infrastructure, its relationship with green and gray
infrastructure, its benefits, and a general
“roadmap” for implementation.
Section 3 describes technologies applied
specifically to wastewater collection and
stormwater systems and key considerations for
selection, design, implementation, and operations
and maintenance requirements.
Section 4 describes the use of smart data
infrastructure to promote collection system
optimization, as well as long-term control plan
implementation, modification, and development.
Section 5 discusses the use of real-time control
systems to maintain and meet operational
objectives.
Section 6 discusses data management, data
sharing, and public notification when using smart
data systems.
Section 7 describes data analysis in smart data
systems, including data validation/filtering and
the use of key performance indicators.
Section 8 discusses data visualization and decision
support systems.
Section 9 discusses the future of data gathering
technology for wet weather control and decision-
making.
Appendix A includes 11 case studies about
communities across the country that have
implemented smart data infrastructure
technologies.
1
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
2. Smart Data Infrastructure
Smart data infrastructure is the integration of
emerging and advancing technology to enhance
the collection, storage, and/or analysis of water-
related data. These solutions can generally be
grouped into a framework that consists of
hardware, communications, and management
systems.
• Hardware includes the devices that
measure and collect water-related data,
such as level meters, flow monitors, valve
actuators, and pump-run monitors.
• Communications refers to networks,
including wireless communications, that
migrate data from the hardware to the
systems that perform analysis.
• Management refers to the software tools
and analytical solutions that perform
analysis and provide actionable information.
It also includes data visualization to give
managers real-time information for
decision-making and to communicate with
the public.
Smart data infrastructure leverages hardware,
communication, and management analytics to
provide real and tangible benefits to utilities,
including:
• Maximizing existing infrastructure and
optimizing operations and responses to be
proactive, not reactive.
• Providing savings in capital and operational
spending.
• Improving asset management and
understanding of collection and treatment
system performance.
• Improving long-term control plan (LTCP)
implementation, modification, and
development.
• Meeting regulatory requirements.
• Prioritizing critical assets and future capital
planning.
• Providing the ability to better optimize
collection system storage capacity to reduce
peak flows and the occurrence of overflows.
• Enabling effective customer service and
enhancing public notification.
Smart data infrastructure can be used to inform
operational decisions that ultimately improve
the efficiency,
reliability, and
lifespan of physical
assets (., pipes,
pumps, reservoirs,
valves). According to
Global Water
Intelligence
Magazine,
implementing digital
solutions by
consolidating
monitoring, data
analytics,
automation, and
control could
potentially generate
up to $320 billion in
cost savings from
the total expected
capital expenditures
and operating
expenses for
different water and
wastewater utilities over the five-year, 2016–
2020 period (GWI 2016).
The potential cost savings and other factors,
such as regulations related to water quality, will
likely stimulate the water industry to invest in
smart data infrastructure and increasingly adopt
the management of data-driven monitoring and
control systems in the operation of various
combined sewer, separate sewer, and municipal
separate storm sewer systems.
2
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
In the future, data feeds and cognitive
computing could significantly assist system
managers by providing near-instantaneous
support information for many of the routine and
immediate response decisions that must be
made in both the municipal and industrial
sectors. Transformation may help water and
wastewater utilities take advantage of
innovations and opportunities in future
operation and maintenance (O&M)
(see Figure 1).
Figure 1. Better information and data can lead to
more effective O&M
Roadmap for Implementing Smart Data Infrastructure
There are few, if any, insurmountable technological barriers to implementing the various technologies described
in this document. Real-time control technology (Section 5), for example, has been around for nearly 30 years.
While its implementation in collection systems remains relatively limited, the effectiveness of real-time control
technology has been proven in many successful applications in wastewater treatment plants (. EPA 2006).
When selecting technology and level of complexity, it is important to understand the utility’s priorities and needs
(., O&M, information technology, security, data usage requirements). It is also important to remember that
smart data infrastructure is scalable. Utilities can start small, applying technology that is compatible with the
utility’s existing capacity to ensure full acceptance and utilization of that technology, then move toward a more
comprehensive approach with higher degrees of performance.
Regardless of the size or age of their infrastructure, utilities can benefit from this general roadmap for
implementing smart data infrastructure:
1. Vision for a utility of the future: Imagine how data, assets, and technology could be leveraged to benefit the
utility.
2. Schedule: Understand the capacity and timeframe for staff to accept change.
3. Technology evaluation: Validate data, prove benefits, and understand delivery.
4. Detailed planning: Seek funding and develop an implementation plan.
5. Phased implementation: Deploy the technology and associated platform.
6. Continuous improvement and innovation: Evaluate phase 1 performance and adapt the planning if
necessary.
Key considerations for developing and implementing the roadmap include the following:
• Ensure organizational commitment for staffing and budget needs. There will be initial investment, as well as
annual costs associated with the adoption of a technology.
• Communicate to ensure buy-in and support from all levels of management and foster strategic partnerships.
• Establish clear authority, roles, responsibilities, and communication channels.
• Define performance expectations.
• Educate and integrate team members early in the project.
• Provide continuous training and technical support to build the existing workforce’s capacity and attract a new
generation of workers.
3
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
3. Smart Data Infrastructure and Technologies:
Information Inputs
Smart data infrastructure can generate highly
informative data sets to support wastewater
and stormwater collection system decision-
making. These data sets help to answer critical
questions that allow operators to maximize the
effectiveness and efficiency of system operation
(Figure 2); however, the usefulness of the data
generated relies on accurate and relevant
information inputs.
The following sections describe specific
strategies and technologies for generating
useful wastewater and stormwater collection
system data, including key considerations for
selection, design, implementation, and O&M.
These strategies and technologies include:
• Continuous monitoring (Section )
• Level monitoring (Section )
• Flow monitoring (Section )
• Rainfall monitoring (Section )
Figure 2. Operational process supported by information inputs
Continuous Monitoring
Continuous monitoring refers to permanent
monitoring systems that report data back to a
central system for use. The physical quantities
to be monitored in a wastewater and
stormwater collection system for proper
operation and control are relatively basic and
typically consist of flows, water levels, and
rainfall conditions for dry and wet weather
operations. In addition, equipment (such as
pumps, gates, and valves) status needs to be
monitored to ensure safe O&M.
Continuous monitoring when combined with
proper data analytics and effective visualization
can generate significant O&M savings by
providing real-time insight into system
conditions, which allows operators to prioritize
asset management with effective targeted
maintenance. Some examples include level
trend detections that trigger alarms for
equipment maintenance (., cleaning),
proactive inflow and infiltration (I/I) risk
assessment, and data-driven work scheduling
and asset management.
Continuous Monitoring in Practice
Milwaukee Metropolitan Sewerage District
(MMSD) is using continuous monitoring to
monitor the performance, value, and health
of green infrastructure throughout the city.
MMSD is monitoring 11 separate sites,
including installations in public rights of way,
allowing managers to see the combined and
individual performance of green roofs and
bioretention cells in real time. Every storm is
recorded, performance can be reported in
aggregate or by event, and the data can be
used to fine-tune maintenance intervals and
maximize performance.
4
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Key considerations for continuous monitoring of
wastewater collection systems include the
following:
• The nature of wastewater systems presents
a harsh and largely variable environment for
monitoring equipment.
• The selection and installation of equipment
needs to consider physical and hydraulic
conditions, humidity, grit, sedimentation,
debris, and corrosion, as well as confined
spaces and maintenance access. For
example, permanent monitoring equipment
should meet explosive zone classifications.
• The advertised measurement accuracy of
any sensor may not represent actual
performance; as such, it will need to be
calibrated/verified.
• Maintenance requirements, as well as
hydraulic and physical conditions around
the monitoring equipment, should be
considered to balance out the increase in
cost and complexity to provide accurate
measurements. For example, forgoing some
level of accuracy by selecting equipment
with easier maintenance needs can ensure
more reliable readings.
Level Monitoring
Multiple technologies are used to monitor water
level in wastewater infrastructures. The most
common types of sensors are pressure
transducers, ultrasonic level meters, microwave
meters, and capacitive probes. Other discrete
devices for specific level detection, such as
floating devices and vibrating level sensors,
could be used in some cases. The most
important criteria for selecting a specific
technology will depend on the environment and
infrastructure configuration where level must be
monitored. More precisely, conditions such as
the presence of turbulences and sedimentation
in the water or the presence of fats, oils, and
grease (FOG); foam; and obstacles in the air
space above the monitoring location must be
considered to select appropriate technologies.
Pressure transducers need to be submerged in
the water where the level must be monitored;
they are therefore convenient for applications
where sedimentation is not a significant issue.
They are typically used where water can be
turbulent at the location of measurement.
Stilling wells are usually recommended to install
pressure probes away from potential debris in
the water flow and for easier maintenance.
Ultrasonic level meters are also very common in
wastewater applications and consist of installing
a probe mounted above the water surface. They
are usually preferred whenever space is
available above the location where monitoring is
needed. Multiple makes and models are
available on the market. Ultrasonic sensors are
recommended where minimal obstacles, FOG,
or foam is present above the surface of the
water. The sensor must be mounted far enough
from sidewalls to avoid bad readings due to
ultrasonic soundwave reflections.
When monitoring space is small or when FOG
can be found in the air above the water surface,
Doppler radar microwave meters are
recommended because they use a narrower
signal beam that improves the reliability of the
measurement.
Capacitive probes are particularly suitable for
multi-point water level monitoring and are
preferred when a high spatial resolution (of a
few millimeters) is necessary (., for a reliable
evaluation of stored volumes in big and flat
storage facilities). The main advantages of these
probes are that the sensors are easy to clean
and can handle temperature and pressure
variations. However, these sensors can
significantly disturb the flow and should not be
used in small pipes.
In general, sensors located above the water
surface have less O&M, but are subject to
corrosion and may experience issues with ice in
cold environments.
5
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
For locations where monitoring the water level
is critical, redundant sensors based on different
technologies are recommended. This strategy
would consist of using, for example, an
ultrasonic meter and a pressure sensor in a
storage facility to ensure water level monitoring
in all conditions and to maximize the availability
of measurements for safe infrastructure
operation.
Flow Monitoring
Operators can use several technologies and
methods of flow monitoring to better
understand the characteristics of their collection
systems.
Physical Flow Monitoring
Typical commercial flow meters available on the
market include ultrasonic Doppler devices,
acoustic Doppler sensors, transit time effect
sensors, and newer technologies such as
Doppler radar sensors and laser Doppler meters.
Flow meter technology has been developed to
fit a variety of applications; submerged and
“non-contacting” devices (sensors located above
the water surface) are available. Transit time
effect technologies consist exclusively of
installing one or multiple pairs of probes (a pair
includes one transmitter and one receiver) in a
crossing path within the water stream. These
probes can measure water velocity at different
layers in the conduit to compute flow values
according to water level and pipe section.
Submerged technologies are generally
recognized as being more accurate because they
can measure the different velocities that can co-
exist within a water flow section at the same
time, while non-contacting technologies can
only measure the velocity from the surface of
the water stream.
Practical experiences of wastewater flow
monitoring within sewer pipes ranging from 24
inches to 120 inches in diameter and above
have shown that submerged flow meter
technologies will generally provide
measurements with an accuracy from ±10
percent to 20 percent. Non-submerged flow
meter technologies will provide flow
measurements with an accuracy typically
ranging from ±15 percent to 30 percent. The
cost for procurement, installation, and
maintenance of “non-contacting” devices is
lower than submerged technologies. A
permanent flow meter installation in sewers
typically ranges from $15,000 to $75,000, and
can be even higher if significant work is needed
for the infrastructures and the electrical utilities.
Regular maintenance for cleaning, inspection,
and calibration is recommended at least twice a
year to keep monitoring reliable and accurate.
Alternative Flow Monitoring
Technologies
In some cases, where installing a physical flow
meter becomes too complex or expensive,
indirect means of flow monitoring can be
developed depending on specific hydraulic
conditions.
Implementing Monitoring Technology to Improve
Operations
The San Antonio Water System (SAWS) recently
participated in a study on the use of monitoring to
inform cleaning maintenance programs. SAWS
equipped 10 high frequency cleanout sites with
remote field monitoring units and used analytical
software to monitor day-over-day level trend
changes and receive messages for trend anomalies.
This analysis of the real-time monitoring data
detected small but potentially important changes in
water levels. These data enabled users to consider
actions such as a site inspection or cleaning. Based
on the monitoring data, SAWS reduced cleaning
frequency by 94 percent in the study areas. Other
than a short period in May/June 2016 when nearly
16 inches of rain overwhelmed the SAWS system,
there were zero sanitary sewer overflows at the pilot
locations.
Level to flow relationship: When pipe flows
remain under “free surface flow” conditions,
Manning equations can be used to estimate
6
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
flow (based on water level sensor data) and
physical attributes (pipe shape and dimensions,
slope, pipe material for the roughness factor) at
the level sensor location. However, the flow
estimation is invalid when the flow experiences
surcharged conditions or backwater effects are
present.
Equations of flow under the gate: When
modulating gates are used for flow control, gate
position and water level data upstream and
downstream from the gate can be used to
efficiently compute the flow regulated through
the gate. The mathematical formula would also
consider the gate’s hydraulic conditions and
physical dimensions, the regulation chamber,
and connection pipes. Optimal gate position
(., amount of submergence) can vary
depending on gate size and flow velocity and
must be determined through hydraulic analysis.
Based on several facilities’ operations using this
method, the relative error is under 5 percent
during dry flow conditions and around 15
percent in wet weather conditions.
Weir relationship: A common mathematical
means of computing flow values uses level
monitoring data from a static weir upstream.
Specific formulas must be used depending on
the shape of the weir, the physical dimensions
of the weir (length, width), and the angle of the
flow stream according to the weir. This method
can provide fairly accurate flow values for weirs
under 6 feet in length; weir relationship
calculations involve significant uncertainties for
longer weirs.
Bending weir relationship: Bending weirs
consist of mechanical flap gate devices with pre-
determined weights that are designed to
maintain a specified water level on the
upstream side of the equipment. When inflows
cause the upstream level rise, the bending weir
reacts by opening to evacuate excess flow. An
inclinometer can be installed on the bending
weir’s flap gate to monitor the angular opening
of the mechanical device. Flow can then be
estimated using the corresponding flow and
weir angle relationship charts provided by the
manufacturer.
Flap gate equations: Similar to bending weir
relationships, mathematical functions can be
developed for computing flows through flap
gates. These relationships will require installing
an inclinometer on the flap gate and a level
meter upstream of the gate. A downstream
level meter will also be required for situations
where the flap gate can become submerged.
Typically, a temporary flow meter calibrates and
validates the equation.
Model-based flow computations: Most utilities
have developed a calibrated hydrological and
hydraulic model (., EPA SWMM 5) to
adequately represent their wastewater system.
These models are typically used to plan, design,
and produce engineering diagnostics. They can
be configured for real-time simulations, based
on real-time rainfall and level data or forecasted
radar rainfall, to provide flow values virtually
everywhere within the wastewater collection or
stormwater system. A well-calibrated hydraulic
model is recognized for providing flow values
within an accuracy range from -15 percent to
+25 percent (WEF 2011).
Rainfall Monitoring
A typical rainfall monitoring system deploys a
network of spatially located rain gauges that
allow for representative measurement of
rainfall quantities over a region. As a general
rule for guidance, on average, one rain gauge is
recommended for every 500 hectares (1,235
acres) of coverage (Campisano et al. 2013),
although coverage needs vary depending on
local climate and need for predictive accuracy.
Common rain gauges use tipping bucket
systems—either optical or mechanical—that
count the quantity of rain trapped in a
calibrated cylinder. Each bucket tip will count a
7
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
specific quantity of rain (., inch) over a
specific time increment.
Such rainfall monitoring can be made available
in real time and can be used as inputs to a
hydraulic model to compute flow predictions in
the sewer collection system. The flow
predictions can then be used to determine the
time of concentration of the area tributary to
the monitoring location. In addition, when
combined with radar reflectivity data and
rainfall predictions, flow forecasts can be
provided with a more accurate level over the
entire territory. Generally, rainfall forecasting
windows and grid sizes should be proportional
to the hydrologic element’s longest time of
concentration in the tributary collection system
where control is desired—., a large combined
sewer overflow (CSO). Rainfall forecasts should
cover at least two hours ahead.
4. Collection System Optimization
A key benefit of smart data infrastructure is its
application in system optimization to maximize
existing infrastructure investment and reduce
the need for future capital investment. It
provides the framework required to optimize
the design and O&M of wastewater and
stormwater systems by collecting and analyzing
large data sets.
There are two types of system optimization.
One refers to system improvements that are
applied offline (Muleta and Boulos 2007). Some
typical examples include raising weirs to reduce
overflow discharge, developing best efficiency
curves to minimize energy costs and reduce
equipment breakdowns, or optimizing the
placement of localized stormwater management
and green infrastructure control. For example,
the EPA SUSTAIN modeling framework uses an
optimization approach to identify the least cost
and highest benefit solutions to achieve user-
defined objectives (. EPA 2009).
The second type of system optimization is
applied online to actively manage the operation
of wastewater networks and facilities in real
time, a process often referred to as “real-time
control” (RTC). RTC systems are discussed in
greater detail in Section 5 of this document.
Table 1 presents the data used in a smart data
infrastructure approach, regardless of
optimization type.
Optimizing Collection System Capacity and
Performance
The Philadelphia Water Department (PWD) has
committed to reducing billion gallons of
overflows in the city by 2036 through better
stormwater runoff management. As part of this
effort, PWD, in collaboration with a private
corporation, implemented smart data
technology to monitor and maximize the
performance of an existing stormwater
retention basin. The existing basin was
retrofitted with technology to monitor basin
water level and precipitation, as well as to
provide real-time active control to selectively
discharge from the basin during optimal times,
effectively increasing the useful capacity of the
asset.
8
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Table 1. Data Required to Optimize the Design, Operation, and Maintenance of Wastewater and Stormwater
Systems
Objective Cause of Problem Potential Intervention
Data Required
for System Optimization
Eliminate
sanitary sewer
overflows
• Rainfall-derived I/I
• Undersized pipes
• Pipe replacement
• I/I mitigation measures
• Level and flow measurements
• Sewer and land characteristics
• Cost of potential interventions
• Grease, debris, and • Improved operating • Level, velocity, and flow
sedimentation procedures measurements
buildup • Pipe replacement
• Cleaning (pipes streets)
• Flushing systems
• Camera inspection
• Cost of potential interventions
• Pipe breaks • Repairs • Flow measurements
• Leaking manholes • Pipe replacement • Camera inspections
• Offset joints • Smoke testing
• Cost of potential interventions
Minimize • High electricity • Pump replacement • Time-of-use electricity tariffs
operating costs consumption for
pumps and gate
operation
• Use of variable frequency
drives
• Improved set points
• Level and flow measurements
• Critical elevation for basement
and street flooding
• Improved controller • Gate, pumps, and actuator
parameters characteristics
• Cost of potential interventions
Minimize
maintenance
costs
• High equipment and
sensor failure rate
• Repairs
• Replacement
• Re-localization
• Preventive and predictive
maintenance
• Best efficiency point
• Level and flow measurements
• Equipment and sensor history
• Equipment inventory and cost
• Detailed alarms
• Maintenance and calibration
history
• Cost of potential interventions
• Sedimentation issues • Improved operating level
• Sewer modification to
increase velocities
• Flushing devices
• Level and velocity
measurements
• Camera inspections
• Cost of potential interventions
Minimize CSOs • Rainfall-derived
inflow
• Undersized facilities
(conveyance, storage
treatment)
• Upgrade of existing
facilities
• Addition of green and
grey infrastructure
• RTC implementation
• Level and flow measurements
• Sewer and land characteristics
• Operational and physical
constraints
• Cost of potential interventions
Reduce flooding • Rainfall-derived • Upgrade of existing • Level and flow measurements
risks inflow
• Undersized facilities
(conveyance, storage)
facilities
• Addition of green and
grey infrastructure
• RTC implementation
• Sewer and land characteristics
• Operational and physical
constraints
• Critical elevation for basement
and street flooding
• Cost of potential interventions
9
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Capacity Management Operation
and Maintenance and I/I Control
Optimizing the performance of the collection
system is the key component in capacity
management operation and maintenance
(CMOM) programs. CMOM programs combine
standard O&M activities with an increased level
of data gathering and information management
to more effectively operate collection systems.
Smart data infrastructure, equipped with the
data input tools described in Section 3, can help
accomplish this. Successful CMOM programs are
used to identify and mediate capacity-related
issues in a system, reducing the risk of system
failures such as sanitary sewer overflows (SSOs).
CMOM includes I/I control, the process by which
unintended clearwater sources (.,
groundwater and excess stormwater) exceed
the design capacity of a collection system,
typically due to antiquated, deteriorating, or
inadequately maintained infrastructure. Long-
term flow and level metering data can be
analyzed to determine performance trends over
a long period of time. Historical trends of I/I
peak flow rates and volumes can be used to
identify areas with high rates of I/I, prioritize
removal efforts, and evaluate the costs/benefits
of those efforts.
Real-time flow rate and level data collection can
be used to identify localized capacity limitations,
blockages, and sediment accumulation. These
data can then inform more proactive
management approaches that can reduce
overflows in both dry and wet weather
conditions. Such approaches help ensure that
the collection system capacity is maximized for
wastewater conveyance, which is a critical
component of all CMOM programs. In addition
to direct monitoring, flow rate and level
metering data can be used along with asset
management data to predict the “unmetered”
portions of a collection system and determine
other areas at risk of capacity-related issues,
such as high I/I.
Facilities can use smart data infrastructure
tools—such as real-time metering and
information analysis—to understand the
different variables that impact collection system
capacity and performance. This knowledge
would allow utilities to better plan for necessary
capital expenditures and optimize system
performance for current and future needs.
Using Smart Data Infrastructure and RTC to
Reduce CSOs
Louisville Metropolitan Sewer District (MSD)
was an early adopter of RTC, applying inline
storage since the 1990s and pioneering the
application of global optimal and predictive RTC
that has been in operation since 2006. The RTC
system is key to maximizing the MSD’s
conveyance, storage, and treatment capacity to
reduce CSOs, with consistent operational results
capturing more than 1 billion gallons of CSO
volume annually. Incorporating RTC into MSD’s
LTCP has resulted in approximately $200 million
in savings compared to traditional methods.
5. Real-Time Control Systems
RTC can be broadly defined as a system that
dynamically adjusts facility operations in
response to online measurements in the field to
maintain and meet operational objectives
during both dry and wet weather conditions
(. EPA 2006).
Wastewater systems are often purposefully
oversized to provide a factor of safety. This
10
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
extra capacity can provide short-term storage in
the conveyance and treatment system when
rain falls unevenly across the collection system
and varying runoff lag times that introduce
stormwater into the system. RTC presents
opportunities to optimize full system capacity
for both existing and proposed facilities.
Potential benefits include receiving water
quality protection, energy savings (Tan et al.
1988), flow equalization, reduced flooding,
integrated operations, and better facility
planning (Gonwa et al. 1993). Real-time or near
real-time reporting can also help utilities meet
the public notification requirements for CSO and
SSO discharges.
A well-designed RTC system can address a
number of different operational goals at
different times. Examples of operational goals
include (. EPA 2006):
• Reducing or eliminating sewer backups and
street flooding.
• Reducing or eliminating SSOs.
• Reducing or eliminating CSOs.
• Managing/reducing energy consumption.
• Avoiding excessive sediment deposition in
the sewers.
• Managing flows during a planned
(anticipated) system disturbance (.,
major construction).
• Managing flows during an unplanned (not
anticipated) system disturbance, such as
major equipment failure or security-related
incidents.
• Managing the rate of flow arriving at the
wastewater treatment plant.
The application of RTC in a stormwater system is
similar to that of a wastewater system. It
requires continuous monitoring (., water
level, rainfall, weather forecast), control devices
(., valves, gates), and data communication to
actively manage flows and adapt to changing
Using RTC to Maximize Capacity and
Performance
In 2008, the city of South Bend, Indiana, installed
and commissioned a real-time monitoring system
of more than 120 sensor locations throughout the
city. In 2012, the city and its partners
commissioned and distributed a globally optimal
RTC system to maximize the capacity and
performance of the city’s collection system. Since
2012, the city has added additional sensor
locations and rain gauges, bringing the total
number to 152 sites. It also added automated
gates at several stormwater retention basins to
better control when and at what rate stormwater
is released downstream into the combined
system. In the period from 2008 through 2014,
South Bend eliminated illicit dry weather
overflows and reduced its total CSO volume by
roughly 70 percent, or about 1 billion gallons per
year.
conditions. If required, temperature, infiltration
rate, and water quality parameters (., total
suspended solids, nitrogen) can be monitored in
real time and integrated into the RTC
management strategy. Associated benefits of
RTC application in stormwater management
include:
• Optimizing the design and sizing of control
measures.
• Reducing the frequency of flooding.
• Improving water quality with extended
residence time.
• Increasing stormwater harvesting and reuse.
• Adapting to evolving conditions through
operation change rather than new
infrastructure.
• Providing auditable performance and
supporting data from the monitoring system
components without additional costs.
• Reducing O&M costs by issuing alerts in real
time.
11
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Components of an RTC System
Figure 3 presents a typical layout of the
components that might be included within an
RTC system. Some components are essential for
RTC (., sensors, meters), while others may be
optional depending on the desired level of
control. The components are represented with
boxes, and the arrows that connect them
indicate the communications and data that are
passed on between the components.
Figure 3. Components of an RTC system
An RTC system, at a minimum, includes sensors
that measure the process, control elements that
adjust the process, and data communication
between them (Schilling 1989). Typical control
elements for a wastewater system are
regulators, such as pumps (constant or variable
speed drives), gates (sluice, radial, sliding,
inflatable), and adjustable weirs (bending weir,
weir gates).
At each remote site, sensors are connected to
the inputs of the local RTC device—in most
cases, a programmable logic controller (PLC) or
remote terminal unit. The PLC provides outputs
(control set points and signals) to the control
elements (., gates, pumps) based on the rules
embedded (programmed) into the PLC. These
rules are feedback algorithms, where action is
based on the difference between a set point and
the measured variable. For example, a PLC may
be programmed to maintain a certain level in
the wet well and will reduce the flow through
the pump if the level is too low or increase it if
the level is too high. The PLC programs can
include set points that are defined locally and
receive “remote” set points from a central
server.
Supervisory Control and Data
Acquisition Systems
Supervisory control and data acquisition
(SCADA) systems have become more prevalent
12
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
in the wastewater industry for collecting and
managing monitoring data. SCADA is a control
system architecture that uses computers,
networked data communications, and graphic
user interfaces for high-level process
supervisory management. Large SCADA systems
have evolved to be increasingly similar in
function to distributed control systems, which
are widely used for process control at the
treatment plants. SCADA system designs have
taken full advantage of advances in information
technology (IT) to collect, archive, and process
large amounts of data.
A SCADA system’s fundamental purpose is to
communicate data and control commands from
a centrally located operator to geographically
dispersed remote locations in real time. The
communication technology options include
telephone-based transmission (used in early
SCADA systems due to low cost), fiber-optic
cable, radio system, cellular-based
communication, wireless internet access, and
satellite-based systems.
Designing a SCADA system depends on a wide
range of practical considerations, including but
not limited to equipment enclosures,
environmental conditioning, field interface
wiring, system documentation requirements,
system testing requirements, IT requirements,
and cybersecurity.
As utilities invest in continuous monitoring and
SCADA, the generated data must be regarded as
an important investment to extract maximum
values. According to the . Geological Survey,
“poor data quality, redundant data, and lost
data can cost organizations 15 percent to 25
percent of their operating budget” (USGS .).
Information captured in the field needs to be
communicated from the remote stations to the
computers and systems that will process, store,
and archive it. The SCADA system is considered
the backbone of an RTC system. It includes
standard graphical user interface (GUI) tools
that operators can access, and it allows them to
manually override any remote site control
actions at any time. As the needs for real-time
or near real-time public notifications rise,
centralized data management can facilitate data
sharing and enable greater transparency.
RTC and CSO Control
The Metropolitan Sewer District of Greater
Cincinnati (MSDGC) has one of the most challenging
collection systems in the country to manage during
wet weather, as it contains more than 200 CSO
points. Together, these overflows discharge over 11
billion gallons of sewage into the Ohio River and its
tributaries annually. In 2014, MSDGC began
installing sensors throughout its largest watershed.
By early 2016, MSDGC had gained both real-time
visibility and control of its wastewater system in this
watershed and transformed the wastewater
collection system into a “smart sewers” network. To
date, MSDGC’s smart sewer system covers over 150
square miles (approximately half) of its service area,
incorporating two major treatment plants, six wet
weather storage and treatment facilities, four major
interceptor sewers, 164 overflow points, and 32 rain
gauges and river level sites. Remote monitoring has
improved the maintenance of wet weather facilities
and enabled upstream facilities to account for
downstream interceptor conditions, increasing
overflow capture basin-wide during wet weather.
Real-Time Decision Support
Systems
A real-time decision support system (RTDSS)
generally overlays the SCADA system. It is
connected to the SCADA database to retrieve
system status information. An RTDSS can use a
SCADA historian and GUI to program and display
system status and trends (., abnormal flow,
critical water level alarm) or provide additional
dashboards involving data analytics to support
O&M decision-making. In an RTC system, an
RTDSS performs complex calculations based on
information inputs to inform operational
decisions and help determine optimal system
set points (., flow to be pumped, water level
to be maintained in a wet well or pipe length).
13
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Typically, decision support uses advanced
computing algorithms that are interactive and
multi-objective and often involve using an
online model for weather forecasting.
Level of Control
The RTC system can be automated with a
centralized or distributed control technology.
The main difference is the control and the
input/output subsystems. In distributed control
architectures, the number and quality of central
processing units (CPUs) is determined by the
number of modules. Each module has a
controller, and the system usually features a
central master PLC. The module PLCs automate
their respective areas and usually do not include
visualization features.
A central architecture usually features a
computer, which deals with all tasks such as
input/output connections, PLC, and control.
Computing capacity, therefore, must be
significantly higher than that of a distributed
control technology system. There is only one
CPU, which means that only one such spare part
is needed. RTC system design criteria drive the
selection of a control system platform based on
the physical and logical components of the
system.
Regardless of the control platform, RTC can be
implemented using different levels of control,
including local, regional, and global. The levels
of control are classified according to progressive
increases in complexity, performance, and
benefits (Schütze et al. 2004).
These set points can be displayed to the
operator for manual control or be sent back to
the SCADA system in real time for automated
control of remote sites. The algorithms used to
determine control logics and set points vary in
complexity from simple operating rules to
complex mathematical optimization techniques
(Garcia-Gutierrez et al. 2014).
Local control, or a local reactive control system,
is the simplest form of automatic control. Local
control is used to solve specific issues that only
require information collected near a regulator
and is usually implemented as single-input,
single-output feedback loop designed to
maintain prescribed set points (., flow or
level set points). It is a good solution only if the
control objectives pursued can be reached
without transferring any information between
other remote sites.
Regional control is similar to local control
except that a telemetry system is required to
exchange data with other remote sites. Regional
control can be implemented as a distributed or
centralized system built on a SCADA system.
Some municipalities design their own decision
support system to control the collection system
based on the specific constraints and
opportunities of each control site. However, the
control remains reactive, not predictive. Based
on a reactive process, there are limitations in
the distances between the control structures
and measurements; as such, the operation must
remain conservative and suboptimal.
Global control is necessary when the control
objectives require strong coordination of the
control actions at numerous remote sites on a
system-wide level. The set points are usually
computed and refreshed periodically (., every
five to 15 minutes). The global strategy used to
determine the set points includes rule-based
and optimization-based techniques (Figure 4).
Rule-based control considers possible scenarios
that can occur during wastewater system
operation and determines appropriate control
actions based on experience. The rules are
generally easy for operators to implement and
understand. However, the quality and the
performance of those rules highly depend on
the available expert knowledge. For large and
complex wastewater systems, the strategy may
demand many rules.
14
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Figure 4. Control strategies for wastewater utilities
Optimization-based strategies involve an
optimization problem that represents the
desired behavior of the wastewater system.
Various algorithms can be used to solve the
optimization problem (., model predictive
control, agent-based optimization). More
detailed descriptions of optimization strategies
and mathematical models can be found in
Papageorgiou (1988) and Garcia-Gutierrez et al.
(2014).
In the last 20 years, model predictive control has
been the most extensively used optimization-
based strategy. This approach uses a
mathematical model of the wastewater system
to generate a sequence of future actions—
within a finite prediction horizon—that
minimizes a cost function (Gelormino and Ricker
1994). Interest in model predictive control is
justified by its ability to explicitly express
constraints in the system, anticipate future
system behavior, and consider non-ideal
elements such as delays and disturbances.
Optimizing the collection system requires
continuous and strategic adjustment of control
devices, as well as predictions of upcoming
inflows and their spatial distribution (Cartensen
et al. 1998). With proper conditions being
monitored, acknowledged, and controlled, a
global RTC system considers the distribution of
flow in the entire system, both in current
conditions and in the future. By using a global
RTC, a utility has the ability to control flow by
opening and closing gates or pumps allows for
transfer flow and storage capacity between
sites, thus providing the temporary storage and
controlled release of significant volumes of
wastewater.
Table 2 summarizes which components of the
overall system must work properly to support
different control modes/levels (. EPA 2006).
Notably, forecasting may be part of a rule-based
system, but it is not mandatory. A global RTC
system often involves a mixture of lower levels
of RTC and static controls.
15
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Table 2. Components Required for Different Control Modes
Control Mode
In
st
ru
m
en
ts
PL
Cs
SC
A
D
A
/C
om
m
un
ic
at
io
ns
Ce
nt
ra
l S
CA
D
A
s
er
ve
r
A
ct
iv
e
O
pe
ra
to
r
In
pu
t,
M
on
it
or
in
g
Ce
nt
ra
l R
TC
S
er
ve
r
Ra
in
fa
ll
Fo
re
ca
st
in
g
O
nl
in
e
M
od
el
Local manual control X X
Local automatic control X X
Regional automatic control X X X X
Supervisory remote control X X X X
Global automatic control—rule-based X X X X X
Global automatic control—optimization X X X X X X X
Guidelines for Applying RTC
In most cases, RTC implementation can offer
benefits and improve the performance of urban
wastewater or stormwater systems. The costs
and extent of these benefits may differ from
one system to the next.
The first step in evaluating if RTC is a suitable
and viable solution for a utility is to develop
criteria for a macroscopic evaluation of RTC
potential using a scoring system (Erbe et al.
2007, Schütze et al. 2004). Criteria may include
environmental and financial objectives, the
topology of the catchment area, collection
system characteristics and conditions,
operational system behaviors, etc.
The utility may, however, skip the first step if it
has already invested in a hydrological and
hydraulic model that adequately represents its
system and operation and/or has substantial
monitoring coverage (which provides good
system understanding and condition
assessment). The utility can use these existing
tools and data in the second step, which
involves a preliminary analysis of RTC potential
and costs/benefits. The analysis should include a
simulation study of a full range of RTC control
levels to determine which is the most
appropriate; staff interviews with operators,
engineers, and other stakeholders; and
equipment surveys.
If the various scenarios demonstrate the
feasibility and benefits of RTC, the third step
involves detailed planning of the RTC system
and its implementation, including:
• Detailed planning of control infrastructures.
• Detailed design of control algorithms.
• Risk and failure analysis.
• Detailed design of data infrastructure (or
gap analysis if data infrastructure exists).
• Staff training and other organizational
planning (., new roles and
responsibilities).
• Preparations for obtaining consent by the
regulatory authorities.
It is critical to involve operator input from the
beginning of the design process. The operators
are ultimately responsible for the system
operation and performance. Early involvement
will ensure that operators’ O&M concerns are
16
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
addressed in the system design and that
operators buy in and accept the RTC system.
Key Considerations for RTC
Systems
An RTC system should have robust operation,
adequate communication, supervisory manual
override, operational confidence, and
adaptability (Gonwa et al. 1993, Colas et al.
2004). The system must be designed and
configured to ensure a high level of
performance under normal conditions and safe
operation under downgraded conditions. Its
performance should be better than or equal to
the system that existed before RTC
implementation.
Under all conditions, there are critical
constraints, such as operating safely, avoiding
equipment damage, and avoiding flooding. A
well-designed RTC system must effectively
manage different operational objectives and
transition between different operational modes
to operate reliably and efficiently; at a
minimum, it must address externally caused
equipment failures and emergency conditions.
The fail-safe procedures must be configured so
that they are triggered when the requirements
for the system’s current operational mode
cannot be met. These procedures should
automatically place the system into the next
(lower) mode/level of operation that can be
fully supported. For example, if the system is
operating in local automatic control mode and
the PLCs malfunction or lose power, it would
need to revert to local manual control.
RTC system risk management procedures must
include the ability to deal with emergency
conditions detected using field measurements.
Special rules can be defined to react to
conditions such as rapidly rising levels within the
system. The emergency response can be either
to adjust the automatic control strategy or
change operational mode by giving the operator
a standard operating procedure.
Using Smart Data Infrastructure to Promote
Resiliency
In response to the historic drought conditions
recently experienced in California, the city of San
Diego has decided to quantify the potential nexus
between stormwater capture and its ongoing effort
to reclaim wastewater as a drinking water resource
(San Diego currently imports more than 80 percent of
its water supply). The city equipped its stormwater
control measures with RTCs and assessed them to
optimize the management of stormwater storage and
release to the reclaimed water system. The
simulations suggested that stormwater harvesting
could substantially augment local water supplies
while complying with stormwater quality regulations.
The reliability of all RTC system components is
key to successful implementation. In addition to
fail-safe and risk management procedures,
system effectiveness can be obtained through
the following:
• Proper selection, location, and number of
sensors to ensure accurate and adequate
measurements.
• Installation of redundant equipment at key
locations using different technologies.
• Real-time validation of monitoring data to
minimize the amount of low quality data
entering the decision-making process.
• Design of safety features, including
emergency isolation gates, power supplies,
generators, and equipment interlocks
specifically designed for safe operation
when a critical alarm is activated.
• Preventive and targeted maintenance to
ensure equipment availability.
• Stock of replacement pieces for critical
infrastructure.
17
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
6. Data Management and Sharing
Good data management and sharing can allow
operators and control systems to integrate data
faster and more effectively. Organized and
carefully designed data management systems
readily obtain and act on data from various
sources, reducing redundancy and the cost of
collection system operation.
Big Data Management
More monitoring requires more data
management and storage. To address the
challenges of storing, processing, recovering,
sharing, and updating large data sets,
organizations are finding smarter data
management approaches that enable them to
effectively corral and optimize their data use.
Some of the best practices for big data
management are to reduce the data amount
(because the vast majority of big data is either
duplicated or synthesized), to virtualize the
reuse and storage of the data, and to centralize
management of the data set to transform big
data into small data (Ashutosh and Savitz 2012).
A smarter data management approach not only
allows big data to be backed up far more
effectively, but also makes it more easily
recoverable and accessible at significantly lower
cost. Other benefits include the following:
• Applications require less to process data.
• Data can be better secured because
management is centralized, even though
access is distributed.
• Data analysis results are more accurate
because all copies of data are visible.
Data Sharing
In addition to the needs of public notification
and regulatory reporting (., post-construction
performance monitoring, permit compliance),
there is a rising need for data sharing among
various departments within an organization to
improve efficiency and interoperability.
Organizations must also be able to securely
exchange data with outside administrative
domains for transparency and for integrated
solutions on city-wide or region-wide scales.
As more data have moved to cloud-based
storage, the protection and encryption of off-
site data has become more important. While
there are still cybersecurity risks, significant
improvements have made it much more difficult
for outside parties to access critical data and
information.
Cybersecurity
The interconnectivity of hardware and data
management has increased the need for utilities to
plan and manage cybersecurity. Although
networking multiple systems provides operational
value, it can also expose systems to new data
security risks. As utilities move to advanced data
storage solutions, addressing cybersecurity will be
an essential aspect of master planning activities.
Cybersecurity provides insurance to protect utility
assets against attacks, outages, and threats, and it
reduces the costs of downtime.
Key considerations for data infrastructure and
data sharing include the following:
• As organizations become more dependent
on cloud-based systems and other internet-
based solutions, the importance of a robust,
maintainable, and secure network
infrastructure becomes critical. Nothing
works when the network goes down.
Secure, redundant, and scalable internet
connections are now required for day-to-
day business as essential processing is
moved off site.
• Network architecture is increasingly
important, and robust, secure solutions
must be designed into systems to manage
18
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
devices potentially numbering in the
thousands, each with multiple data points.
Simply using a “firewall” to secure a
network is no longer feasible.
• Formerly isolated SCADA/industrial control
systems (ICS) are now required to
communicate over the internet. To securely
realize the vast benefits of cloud computing
and the IoT, secure data interconnectivity is
essential. Standards have been produced to
ensure a high degree of interoperability and
security for evolving SCADA/ICS solutions.
Emerging Technologies for Big Data
Management
For big data management, all types of data
analytics will be more widespread and
incorporate more artificial intelligence. Already,
machine learning has been applied in predictive
analytics for I/I characterization, based on
analysis of long-term data trends.
Real-Time Public Notification and
Transparency
Implementation of a smart data infrastructure
allows utilities to disseminate relevant and
current information to ratepayers and
stakeholders. Public notification is becoming the
norm for informing interested parties of current
utility conditions. While some data must be kept
private due to security issues related to
7. Data Analytics
Most utilities already generate a substantial
amount of process and monitoring data for
various purposes. As the amount of data
generated each year increases at an exponential
rate, it is increasingly critical to convert those
data into useful information (Greiner 2011).
Technical advancements in complex
multidimensional data analysis and data mining
can help utilities analyze incredible amounts of
Real-Time Public Notification with SmartCoverTM
Systems
The city of Newburgh, New York, replaced its
combined sewer telemetry system with a wireless
SmartCoverTM System. The prior telemetry system
used pressure sensors that had to be located
beneath the influent channel, in direct contact with
the flow and in the combined sewer regulator
environment where they would be regularly
impacted and damaged or displaced by debris. The
new SmartCoverTM System’s sensors hang from the
manhole cover above and do not contact the water,
avoiding damage. The new system’s wireless satellite
connectivity is more reliable than land phone lines at
a lower cost. Any computer, tablet, or smartphone
with internet access can communicate with the
telemetry system, allowing for real-time staff and
public notification of CSO events.
protecting treatment processes, some data can
be shared to better inform the end user. A
common example includes the public
notification for current/recent overflow activity
to local receiving waters. The real-time
notification of overflow activity informs the
public that recreational uses may be temporarily
compromised, potentially reducing public health
issues. Public notification can also include
automated notification to the regulating
agencies as part of permit requirements.
data to detect common patterns or learn new
things. This can lead to significant operational
improvements and dollar savings for
wastewater systems.
Big data analytics, a well-established concept,
involves analyzing the data collected to discover
trends and correlations, uncover hidden
patterns and other insights to understand why
certain behavior or incidents happened, and
19
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
then use that insight to predict what will
happen. Today’s technology and advancements
in big data analytics bring speed and efficiency,
which enable utilities to analyze large quantities
of data and identify insights for immediate
decisions (Figure 5).
Figure 5. Big data analytics support enhanced
decision-making and more effective and less
costly operations
Utilities that have already invested heavily in
continuous monitoring could use data analytics
to get significant value from the data they
collect.
There are many data analysis and data mining
solutions, which also incorporate data
warehousing, database management systems,
and online analytical processing.
Data Validation and Filtering
Data validation is an important consideration for
wastewater utilities, particularly for monitoring
data within the harsh environment of a
wastewater collection system. Raw monitoring
data can contain erroneous readings, which
could be due to one or a combination of the
following:
• Noise (high frequency fluctuations)
• Missing values
• Values out of range
• Outliers (sudden peaks)
• Constant (or frozen) values
• Drifting values (changes in values over a
longer period of time)
As the quality of the insights gained from data
analytics or the control system’s performance
will be directly linked to the quality of the data
used, raw data collected from the sensors needs
to be validated and possibly filtered before
being used for further analysis or control
purposes. This is an important step to improving
the data’s reliability.
Emerging Technologies for Data Analytics
The IoT industry trend is to provide more
accessibility through cloud computing platforms
and open source technologies. The digital platform
will streamline the integration of data from
various legacy systems and eliminate data
duplication and bad data for more effective and
powerful data analytics and insight. Cloud-based
computing has already been implemented for
SCADA system applications and RTC applications.
Data validation can be carried on a single
variable (single data validation methods) or by
comparing two variables when two or more
measures are correlated (cross-validation) (.
EPA 2006, Sun et al. 2011).
Single data validation methods include the
following:
• Range validation: The values that are
outside an expected range are flagged as
invalid. The expected range is based on the
working range of the sensor itself and on
the process monitored. For example, a
water level in a collection system cannot be
lower than the bottom of the chamber
where the sensor is located and can seldom
exceed ground level.
• Gap filling: When data are missing (due to
communication failure, sensor automatic
20
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
calibration, etc.), it is possible to use an All RTC system data should be validated in real
estimate instead. In a real-time context, the time. Data validation can be implemented at the
last valid value can be used. If correlation local PLC and at the central control station.
exists with other measurements, cross-
validation techniques can also be used to
produce better estimates (see below). In a
post-event analysis, a simple linear
interpolation between the values before
and after the gap can often be used.
• Rate of change validation: If values change
at a greater rate than a probable change in
measured conditions and sensor noise, then
the value is marked as invalid.
• Running variance validation: A value is
flagged as invalid if the variation over a past
value is too small. A frozen value is often
due to a sensor failure.
• Long-term drift: Expected mean check and
acceptable trend check are two methods to
detect long-term drift. Once detected, the
source of the bias or drift then needs to be
identified as it could be caused by sensor
drift, as well as a long-term trend of the
measured value.
Cross-validation methods are used when it is
possible to develop a model or relation between
two or more values. The simplest case is where
some sensors are redundant and measure the
same value or if software can be used to
produce another sensor’s estimate. A range or
rate of change validation can then be carried on
the difference between the two values. In more
complex cases, the redundancy can come from
combining sensor data with a model to produce
many estimates of a specific variable (soft
sensors or virtual sensors). The data
reconciliation technique can then be used to
better estimate the variable.
Filtering can be used to reduce the
measurement noise inherent to sensor data.
The result is smoother and easier to analyze and
usually produces better results with control
processes.
Whenever possible, data validation processes
should take advantage of the correlation
between the measurements (., cross-
validation methods). At minimum, the data
validation algorithms should use sensor alarms
and be able to detect missing data, out-of-range
values, outliers, and frozen measurements.
Key Performance Indicators
Developing key performance indicators (KPIs)
based on computations of validated data can
provide a quick and general understanding of
the system’s performance. Some of the
meaningful KPIs applied for wastewater and
stormwater systems include the following:
• Precipitation frequency: The average
recurrence of rainfall can be assessed using
rain gauge readings (NOAA .). Maximum
rainfall depth over various durations is
calculated and compared to precipitation
frequency estimates for the area and
precipitation data used for hydraulic model
development and calibration.
• Treated flow: Maximum flow conveyed to
the wastewater treatment plant (WWTP) is
compared to the WWTP’s treatment
capacity. If CSOs or significant retention
occur while the treatment capacity is not
met, it can signal a suboptimal system or
control.
• Untreated flow: Estimated or measured
overflows from the collection system prior
to treatment is compared to total flow
treated at the WWTP. This is typically
measured as number of overflows and/or
the volume of overflows. These values can
be compared to those projected or allowed
under an approved Long-term Control Plan
or NPDES permit to assess system
performance and compliance.
• Partially treated flow: Estimated or
measured volume of wastewater receiving
21
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
only partial treatments prior to discharge.
These values can be used to assess system
performance and compliance.
• Retention duration: Exceedingly long
durations can lead to odor problems in
wastewater storage systems.
• Retention volume: Maximum stored
volume can be presented relative to full
capacity. If CSOs occur while the full
retention capacity is not met, it can signal a
suboptimal system or control.
• CSO/SSO volume and duration: Overflow
discharges can be reported to the public in a
timely manner.
8. Data Visualization and Decision Support
Systems
Data visualization is the presentation of large
amounts of complex data using charts or
graphs—a quick, easy way to universally convey
concepts. It enables data users and decision-
makers to visually explore analytics, so they can
grasp difficult concepts or identify new patterns.
Interactive visualization allows the user to take
the concept a step further by using technology
to drill down into charts and graphs for more
detail, to interactively change the data displayed
and how it is processed (SAS .).
Data visualization is a key component of the
user interface for any decision support system
(DSS). A DSS (also known as decision-making
software or DMS) is a computer-based
information system that supports business or
organizational decision-making activities. DSS
has three main functions: information
management, data quantification, and model
manipulation.
• Information management refers to the
storage, retrieval, and reporting of
information in a structured format
convenient to the user.
• Data quantification is the process by which
large amounts of information are
condensed and analytically manipulated
into a few core indicators that extract the
essence of data.
• Model manipulation refers to the
construction and resolution of various
scenarios to answer, “what if” questions. It
includes the processes of model
formulation, alternatives generation and
solution of the proposed models, often
through the use of several operations
research/management science approaches
(Inc. .). Its main objective is to convert
data into usable and actionable knowledge.
There are two main types of DSS tools, one for
planning purposes and another for real-time
decision support (Hydrology Project .). For
wastewater and stormwater applications, DSS is
typically structured to allow users to access and
analyze monitoring data, run model simulations,
and assess the impact of potential decisions by
using “what if” scenarios. While the data can be
displayed and analyzed in real time to identify
areas that need attention or improvement, the
appropriate actions can be taken at a later time.
For example, DSS can display real-time level
data correlating to expected flow behavior.
Abnormally high-level data would indicate a
potential debris blockage, and the
corresponding response decision would be to
schedule a maintenance crew to perform a field
investigation. However, this action could be
optimized with other work orders to improve
maintenance efficiency.
22
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
An RTDSS allows decision-makers to respond to
short-term variations in wastewater and
stormwater systems where lead times for
decisions vary from a few hours to a few days at
most. Typical RTDSS examples include:
• Hydraulic flow diversions
• Storage basins to manage levels or volumes
• CSO or SSO discharge warnings
• Flood forecasting and warnings
See Section for additional details on the
RTDSS.
Before buying the various computer systems
and software needed to create a DSS, utilities
should consider (Inc. ., WERF 2005):
• Establishing business needs and value for
DSS, such as providing guidance for complex
operation.
• Evaluating the development of DSS
applications using available software, such
as spreadsheets, SCADA, or asset
management software.
• Integrating information spanning more than
just one functional domain into the DSS, as
well as support decisions from multiple
domains.
• Creating user-friendly DSS for easy viewing
and access, as well as allowing users to
create scenarios and to simulate and
analyze the impacts of different scenarios.
• Ensuring the investment in terms of time
and effort to incorporate DSS into daily
operations.
• Providing necessary training and knowledge
to use DSS effectively.
• Understanding how the DSS is used, such as
the limitations or assumptions of the
mathematical calculations or processing
model used within the DSS.
• Examining other factors, such as future
interest rates and new legislation, in the
decision-making process.
9. The Future of Data Gathering Technology for
Wet Weather Control and Decision-Making
Rapid advancements in data gathering
technologies have already led to substantial
improvements for real-time operational support
and decision-making systems. Future
advancements will continue to be made in the
following areas:
• Monitoring the frequency, volume, and
duration of overflows and discharges within
combined and separate sanitary sewer
systems.
• Water quality of flows within sewer
systems, discharges, and receiving streams;
specifically, real-time measurements of
bacteria, nutrients, suspended solids, and
possibly emerging pollutants.
• Operational data to inform asset
management systems and long-term
planning.
As these advancements continue, dischargers
and regulators will need to adapt to new ways
of thinking and embrace the increased role that
smart data infrastructure will play in wet
weather control and decision support.
Dischargers will need to overcome barriers in
educating personnel to operate and interact
with new technology and systems, as well as
adapt to a new culture of enhanced data
operations.
23
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
New technologies will only be able to maximize
end-user benefits if they can be implemented
within the framework of regulations.
The advancement and proliferation of new
technologies for gathering and analyzing wet
weather infrastructure data will lead to the
generation of more accurate information and
provide for lower-cost operations. With more
accurate data, operators will be able to make
more informed decisions, increasing efficiency
and reducing risks.
Technology advancements will continue to
improve our ability to quantify wet weather
events and monitor water quality in ways we
have never been able to before. In the future,
better technology will exist for generating data
related to the frequency, volume, and duration
of wet weather events. Operators will have
increasingly better information to determine the
occurrence of wet weather discharges and to
calculate the impact of wet weather events on
collection system capacity. Better understanding
these system characteristics will lead to
improved infrastructure design and
management, and ultimately the prevention of
failures and overflows.
Pollutant sensor technology will also continue to
improve, and operators will be able to monitor
pollutant impacts on water quality more often
and in real time. Operators will also be able to
more closely monitor pollutants (such as
bacteria) of particular concern to public and
environmental health.
Continued improvements in data gathering will
increase the effectiveness and reliability of data-
informed operations, and ultimately change the
pace at which operational decisions can be
made, moving increasingly toward real time.
Increasing the amount and frequency of reliable
data will also enhance asset management
programs and promote more informed capital
planning. Wet weather system O&M was at one
time conducted on a solely reactive basis. As
technology and operational strategies have
advanced, and more precise and accurate data
are more readily available, operators have now
shifted their approaches toward preventive and,
in some cases, predictive O&M practices.
24
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
10. References
Ashutosh, A., and E. Savitz. 2012. Best practices for managing big data. Forbes Magazine. Accessed Feb.
14, 2017. <
data/#58202fdeef02>
Campisano, A., J. Cabot, D. Muschalla, M. Pleau, and P.-A. Vanrolleghem. 2013. Potential and limitations of
modern equipment for real time control of urban wastewater systems. Urban Water Journal.
doi:
Cartensen, J., . Nielsen, and H. Strandbaek. 1998. Prediction of hydraulic load for urban storm control
of a municipal WWT plant. Water, Science and Technology 37(12): 363–370.
Colas, H., M. Pleau, J. Lamarre, G. Pelletier, and P. Lavallée. 2004. Practical perspective on real time
control. Water Quality Research Journal of Canada 39(4): 466–478.
Erbe, V., M. Schutze, and U. Haas. 2007. Application of a guideline document for sewer system real time
control. Novatech Conference, Lyon, France: 761–768.
Garcia-Gutierrez, L., E. Escobar, J. Barrero-Gomez, N. Quijano, C. Ocampo-Martinez, and D. Tellez. 2014.
On the modelling and real time control of urban drainage systems: A survey. 11th International
Conference on Hydroinformatics, HIC 2014, New-York City, USA.
Gelormino, ., and . Ricker. 1994. Model-predictive control of a combined sewer system/
International Journal of Control 59(3): 793–816.
Gonwa, W., . Capodaglio, and V. Novotny. 1993. New tools for implementing real time control in sewer
systems. Proceedings of 6th International Conference on Urban Storm Drainage, Niagara Falls,
Ontario Canada: 1374–1380. Reference No. I6226.
Greiner, L. 2011. What is data analysis and data mining? Accessed February 15, 2017.
<
>
GWI. 2016. Need to know. Global Water Intelligence Magazine 17(12): 4–5.
Hydrology Project. . Decision Support Systems—Decision Support System-Planning (DSS-P). Accessed
February 15, 2017. <http://hydrology-
Inc. . Decision support systems. Accessed February 14, 2017.
<
Muleta, M., and P. Boulos. 2007. Multiobjective optimization for optimal design of urban drainage
systems. World Environmental and Water Resources Congress: 1–10.
NOAA. . Precipitation frequency data server (PFDS). National Oceanic and Atmospheric Administration.
Accessed February 15, 2017. <
Papageorgiou, M. 1988. Certainty equivalent open-loop feedback control applied to multireservoir
networks. IEEE Transcripts on Automatic Control 33(4): 392–399.
SAS. . Data visualization: What it is and why it matters. Accessed February 10, 2017.
<
25
Smart Data Infrastructure for Wet Weather Control and Decision Support August 2018
Schilling, W. 1989. Real time control of urban drainage system. The State-of the-Art IAWPRC Task Group
on Real Time Control of Urban Drainage System, Pergamon Press, London.
Schütze, M., A. Campisano, H. Colas, W. Schilling, and P. Vanrolleghem. 2004. Real time control of urban
wastewater systems—Where do we stand today? Journal of hydrology 299: 335–348.
Sun, S., J.-L. Bertrand-krajewski, A. Lynggaard-Jensen, J. van den Broeke, F. Edthofer, M. do Céu Almeida,
À. Silva Ribeiro, and J. Menaia. 2011. Literature review of data validation methods. EU PREPARED
.
Tan, ., . Dabke, and . Mein. 1988. Modelling and control of sewer flow for reduced cost
operation of a sewage pumping station. IEE Transcripts on Systems, Man, and Cybernetics 18(5):
807–813.
. EPA. 2006. Real time control of urban drainage networks. . Environmental Protection Agency. EPA-
600-R-06-120.
. EPA. 2009. SUSTAIN—A framework for placement of best management practices in urban watersheds
to protect water quality. . Environmental Protection Agency. EPA-600-R-09-095.
<
USGS. . Value of data management. . Geological Survey. Accessed February 15, 2017.
<
WEF. 2011. Prevention and control of sewer system overflows. WEF Manual of Practice No. FD-17, Third
Edition. Water Environment Federation, Prevention and Control of Sewer System Overflows Task
Force.
WERF. 2005. Decision support systems for wastewater facilities management. Water Environment
Research Foundation. WERF Report 00-CTS-7.
26
Appendix A
Case Studies
Buffalo, New York:
Real Time Control of Inline Storage
KEY FEATURES
OWNER
Buffalo Sewer Authority
Buffalo, New York
LOCATION
INCEPTION
Commissioned winter 2016; study
period March–May 2016
REFERENCES/LINKS
BSA Awarded EPA Environmental
Quality Award
york
• Reduced combined sewer overflow (CSO) by million gallons at two initial RTC sites between
March 1 and May 31, 2016.
• Sixteen real-time control (RTC) sites to be established by 2020.
• Expected to reduce CSO by 15 to 20 percent at full capacity.
• $145 million negotiated out of long-term control plan and consent agreement based on modeled
outcome of inline storage.
PROJECT DESCRIPTION
Once the 8th largest city in the United States, Buffalo has lost half of its population and most of its
industrial base since the 1960s. Before its decline, the city built a massive sewer system to acco
