I am both heartened by the fact that there are volunteers willing to help society, but also very sad that this is even needed. Where and how has the American government - both sides i feel - so disastrously failed its people, that it is needed to rely on volunteers to help with something that i feel should have been managed by government.

Before anyone starts rambling on about politics too much on any side, i blame all sides. What i recall from my basic education so many decades ago is that government should help to provide at least some fundamental areas of infrastructure (e.g. roads and such, etc.), and then commerce (private enterprise, etc.) can take place above it, and than things proceed from there, yada yada.

I don't know what is more basic infrastructure than water and its associated management? Where is the NSA in all of this? What about Department of Homeland Security's Cybersecurity and Infrastructure Security Agency (CISA), where are they in all of this?

Clearly, this story ticked me off, and apologies for that...but, if we're at the stage in society where volunteers is a viable solution for a very fundamental element in life...then clearly lots of other things have severely failed.

> What about Department of Homeland Security's Cybersecurity and Infrastructure Security Agency (CISA), where are they in all of this?

The CISA does this type of thing all the time. Here is an example just from their recent news releases in the past week: https://www.cisa.gov/news-events/alerts/2024/11/21/cisa-rele...

The CISA offers to assist infrastructure providers in a number of ways: https://www.cisa.gov/resources-tools/services/assist-visits Presumably the NRWA could have asked the CISA for help with this initiative.

The CISA has 3,000 employees and a budget of $3B, which is a lot, but not enough where they can be involved in everything. For reference there are 12,500 utility scale power plants and 148,000 public water systems in the United States. The scale of the problem means they must be mostly an advisory organization, where most of the work gets done by people at the infrastructure organizations or, sure, volunteers.

Now, I’m not saying that the CISA is doing a good job, I genuinely have no idea. Determining that would take a lot of knowledge, probably insider knowledge, and weighing what they’re doing against what resources they have available. But them not being involved in some random project that showed up in your newsfeed doesn’t mean they have severely failed.

Most of our infrastructure was built a long time ago (except roads) and don’t actively see maintenance. No one from the current generations has had to pay for real impactful infrastructure for decades. Now that all the infrastructure is failing or has security holes nobody wants to pay for it to be fixed. If you look up major infrastructure repairs in the last few decades you will see much of what could have been long term fixes were reduced to short term fixes.

Properly fixing infrastructure has become expensive as construction costs have skyrocketed in the last few decades.

Look at the CHIPS bill, huge infrastructure gain but also was a part of the 9% inflation because building this infrastructure is expensive. The American people were warned about it causing inflation but wanted it anyways. When inflation hit they all conveniently forgot the cost of new infrastructure in the US and chose to cling to lies about how disastrous the last 4 years have been.

That is why today no one does anything about problems until a bridge collapses. You can make more money today by passing the buck to the next generation and passing blame to whoever came before you.

I'm a recently retired software engineer who has been on the board of directors of a small rural mutual water company (i.e., owned by the customers) for ~15 years. We have less than 1,000 customers / shareholders. I'm in my mid-50s and the only one on the board under 75-years old. Our community didn't have wired internet access until eight years ago, and half the community still doesn't have cell reception. You can imagine the level of technical literacy.

It's hard to imagine significant help the government could give us, short of $100K+ / year to hire a security engineer. Even then, how would small utility companies find / evaluate them? We already feel saddled with how many hours of mandatory training we have to go through each year (e.g., board members have to sit through training to remind us not to direct company funds into our own bank accounts - I doubt that's a training issue!). Looking at our two neighboring water companies, their setups have very little in common with ours or each others. So any training would be too generic to be of much use.

I have to give credit to our company's chairman / general manager (himself 84-years old). He works to have diverse expertise on the board. He's a retired wild-land firefighter, plus there's a retired bookkeeper, a retired state employee, a retired farmer, and me. As a group, we have a lot of experience in a lot of areas. I suspect most small, rural water companies haven't found a way to have that breadth of knowledge available. But, even then, there's only so much that two full-time plus two half-time employees can do. And it's hard to have more while keeping rates sane.

On the security side, I'm no expert, but I pay attention. I do my best to help the employees understand the dangers of phishing and of downloading things onto company computers. We have a consumer grade router and none of them know how to get into it to open ports, etc. Even if things were locked down hard, we couldn't afford to hire someone to maintain that state as threats evolve. Our total compensation budget is $150K / year for all employees, and 80% of that is needed for state-licensed water treatment operators.

For us, all of that means being resigned to the fact that hacks will happen. It's more about minimizing the damage and being able to recover. I've pushed for using cloud services for things like customer billing - these services are not the ultimate in security, but they're far better than what we could do with software running on our computers. I've also put append only / offsite / offline backups in place so we can recover from encrypting ransomware (which has hit us before, luckily we could just wipe and re-install / restore).

The biggest thing we've done is around SCADA (software / hardware involved in our water treatment and distribution). When we put SCADA in place ten years ago, I pushed for it to be read-only. That is, it can be used for monitoring and alerting, but not for controlling anything (there's literally no hardware in place to do so). So, hackers can see how much water, chemicals, etc. we're using, but they can't directly change or shutoff the water. They can make us think we need twice the chemicals, but that will be a red flag for the operators who have to manually implement it. Even then, we do manual daily testing of our water in our lab and have monthly testing done by state-licensed independent labs.

But, when we put SCADA in place, I had to stop and think a lot about this given that I wasn't a security expert. And, of course, the vendor was certain it was 100% secure (a red flag in itself). But at least I had experience thinking about software issues / impact. The operators certainly wanted automated control, as that would save them from having to drive to the plant in the middle of the night. And without automated control, we have to use more chemicals, as we can't optimize usage by reacting in real time to changes in pollutants in our source water. But we'd all rather deal with those downsides than find out someone has compromised our water. So it was easy to sway them.

It's far from perfect, but we've mostly limited potential damage to things that aren't deadly, literally or figuratively. Without someone like our chairman / general manager being dedicated to bringing in diverse expertise via the board, I don't see what chance we'd have. And in certain parts of the country, it would be hard to find security (or at least software) expertise to sit on the board.

Something our state is doing to attack the lack of economies of scale (in this and other areas) is trying to force rural water systems to merge. About half in our area have been folded into the water company for the "big" city (5K people), 20-miles away from us. That's too far to move treated water given our local terrain, so they have to keep running the systems from afar. But at least they can spread (e.g., IT) expertise across these systems. Where this has happened near us, water rates have roughly doubled. Maybe that trade-off is okay, at least for those who can afford it?

I wish we could stop looking for someone to blame or complain about and start looking for solutions. I think that’s the spirit of volunteering here.

While I see your point, over time, I’ve come to think that cybersecurity is a fundamentally different and indomitable beast. Consider the sheer number of software projects, devices, and products being developed, each inevitably introducing all kinds of bugs, versus the relatively small number of people who truly understand the craft of real offensive security.