There was a time when the greatest vulnerability to a municipality’s water system was a teenager, drunk, climbing a water tower, opening the hatch and getting rid of his beer, Mark Pepper, executive director of Wyoming Association of Rural Water Systems, said.
Then this past January and March, a hacker infiltrated a city’s supervisory control and data acquisition (SCADA), which gave them all a wake-up call to the seriousness of cyber threats to the state’s fresh and waste water supplies.
A wake-up call that has been four years in the making as cyber threats started creeping in a few years ago, and systems have been trying to address these types of issues ever since. But now, we don’t have the luxury of waiting any longer to really dig in and get systems up to speed, Pepper said.
He won’t comment on which town was potentially hacked only to say that they were able to shut it down before any malfeasance was done to the water supply.
Water Systems Infiltrated
What might have happened? One needs only to turn to a water treatment plant in Oldsmar, Florida, whose alert plant manager diverted a potentially serious threat last February when he saw his cursor moving around on his computer screen, opening various software functions controlling the water treatment.
The manager witnessed several functions being manipulated, including watching the sodium hydroxide, commonly known as lye, getting boosted up to 100 times its normal levels, according to the Associated Press (AP) article shared by PEW. The compound, which is also the main ingredient in liquid drain cleaners, is used to control acidity and remove metals from drinking water.
Had the hacker been successful, the 15,000 or so residents of the town may very well have been victims of lye poisoning that causes burns, vomiting, severe pain and bleeding.
Luckily, the operator was able to reduce the inflated levels of sodium hydroxide back to normal level once the hacker left his computer. Even if the hacker had succeeded, there were other safeguards in place to keep the water balances in check, the article noted, adding that the public was not actually in risk.
But still.
For Pepper and others in the field, the hack was enough to get their attention to the potential vulnerabilities in Wyoming’s water and waste treatment facilities.
Experts Paying Attention
Pepper shared these concerns in a recent forum at the CyberWyoming Alliance virtual conference in early October during a conversation with Texas computer network intrusion and detection expert Dr. Gregory White.
In the conversation, the two discussed the inherent vulnerabilities of Wyoming’s water supply networks and the importance of both residents protecting themselves by keeping at least a two-week supply of water on hand as well as the importance of increased cyber and IT training for water plant employees and financial buy-in from the state legislature and government officials.
Wyoming’s rural, relatively unpopulated towns and cities present a unique challenge to the state’s water infrastructure, Pepper noted.
The chairman of the U.S. Senate Committee on Environment and Public Works, Tom Carper, D – Del, would agree. During his opening comments at a committee meeting on July 21, Carper addressed the growing threat of cyberattacks on the nation’s critical water infrastructure.
“Cyber vulnerabilities in our water systems represent unique national security challenges. A major breach in our water infrastructure system could jeopardize the safety of our drinking water and impair communities’ ability to safely dispose of harmful waste, threatening human health,” he said.
Pepper is well aware of the risks.
780 Public Water Systems in Wyo
Currently, there are 334 community water systems in the state, 96% of which are owned and operated by municipalities with the rest overseen by special districts. In addition to these systems, there are more than 450 non-community water systems that must comply with the Safe Drinking Water Act but do not employ licensed water operators that are mainly guest ranches, dude ranches, bed and breakfasts and camp grounds.
In total, that adds up to around 780 public water systems with only 33 serving populations of 3,300 or greater. The vast majority instead service towns with population of 3,300 or fewer residents with 92 percent serving populations under 500.
This presents an enormous challenge when it comes to getting employees trained and systems updated in keeping with the Drinking Water and Wastewater Infrastructure Act of 2021 passed by Congress at the end of April by nearly unanimous, bipartisan consent. In part, the new law reauthorizes programs supporting water infrastructure with the goal of providing safe drinking water as well as wastewater facilities.
In tandem is the Safe Drinking Water Act (SDWA) enforced by the Environmental Protection Agency (EPA) that dictates drinking water standards for more than 90 contaminants in the interest of public health.
Under SDWA, all public water systems regardless of size are required to have an emergency response plan that should be updated as needed continually.
Wyoming met the timeline for the assessments for systems over 3,300 and is on track to meet the updated ERP requirement, according to Pepper, which dully made it clear that some of the computer systems and level of cyber awareness and training was by modern standards woefully out of date, he said, indicating a need for investment in both the human and technological upgrades in water infrastructure and training.
There’s a significant cost to doing both, Pepper acknowledged, as he and his association continue to encourage that these upgrades are made in the protection of public health.
Five Attempted Hacks So Far
To date, Wyoming has experienced five attempted hacks to its rural water systems. Two of those were the aforementioned attempted infiltrations in the municipality’s SCADA system.
They were interrelated attacks, Pepper said, due to the IT employee’s inability to completely root out the ransomware (albeit, a very sophisticated attack) in the first attack which left a backdoor open to vulnerability which the hacker exploited.
It was an auxiliary computer tied to the main computer system and the FBI and other agencies conducted a forensic investigation, Pepper noted, to which the results are still pending.
The other three infiltrations came through emails to the city clerk and other employees in email phishing scams. Though the water systems were not in direct jeopardy as a result of the attacks, the computer system governing billing and other functions were essentially shut down for a week.
Pepper worries that supply chain phishing scams will be the next scam on a long list given that nefarious individuals will no doubt want to exploit national and international clogs in the supply chain by too-good-to-be-true discounts on chemicals and PVC pipe already in short supply.
The biggest vulnerabilities, however, from a chemical standpoint are that hackers will be able to get into these systems and manipulate the legal limits of chemicals for disinfection and a myriad of other chemicals used to treat source water into drinking water which could lead to serious public health outbreaks.
Many of these attacks, he believes aren’t even targeted.
“I think a lot of hackers don’t know the effect but are sitting around drinking beer in a foreign country or wherever they are and get a hit and just start playing around to see what they can do,” he said. “In some respects, they are seeing what they can manipulate and what control systems are hackable.”
He doesn’t think that they’ve had any of their systems specifically targeted or whether the attacks that have happened are malicious intent. He just doesn’t know.
Scary Stuff
What he does know, however, is the seriousness of what such a hack can do to Wyoming’s vital water and waste treatment operations.
“If someone wanted to overdose chlorine to the point that a person filled up a glass and drank it, it could kill someone,” he said, “or a small community could drop dead of chlorine gas if they all turned on their faucets at the same time or had a release from the plant.”
On a positive note, they have partnered with expert IT people to provide training to water treatment plant managers and operators which is a great start, he said.
“The key is to get people trained and aware,” he said. “Training people to recognize what is happening and how to prevent problems and fix them and to also be aware of phishing email scams and to have enough awareness to know who to call when an issue arises.”
The latter proves vital in small, rural communities where the city government is tied to everything in town and where one contaminated computer system can have a larger reaching impact both on IT systems and facilities.
More importantly, he said, is raising awareness and convincing local and state government to make the investments in keeping their water supply and treatment plants safe.
Paying For Security
There seems to be a big disconnect on the cost of water, which for whatever reasons, raise the hackles of tax payers. That and potholes, he noted. He hears every day from Wyoming residents who complain about the cost of their water bill, and even watched a local mayoral candidate campaign solely on the promise of cutting water costs to the town.
“If people can flush a toilet or turn on the faucet that they are happy,” Pepper said. “They don’t care how it got there or what it cost to treat it. People will say that water should be free but will think nothing about going down to the local 7-11 and paying $1.69 for a 12-ounce bottle of water, which is not as regulated as tap water by the way.”
Yes, water might be “free,” he noted, but it costs a lot to treat it and deliver it to residents.
In the meantime, he’s focused as he says to crying to whoever will listen about the potential vulnerabilities in the state’s water systems.
Apart from paying a hefty ransom to unlock a computer system, the worst-case scenario involves potential doses of chemicals that make water deadly or cause serious public health risks due to contaminants in the disinfection process that might affect water quantity when pumps are shut off or water flow is impacted, allowing deadly bacteria to build. Only a fraction of Wyoming municipalities is currently able to manually operate their systems in the case of power being shut off.
Worst case scenarios are these effects to water quality or quantity that create potential public health risks on a small or massive scale.
“That’s my number one fear,” he said. “Number two would be ransomware shutting down the system that prevent us from operating or delivering water.”
The most recent hack was an eye opener for him and others, he said.
“The criminals are always one step ahead of figuring out the next scam.”
