And if you ingest patches in seconds, you now have an issue of checking that the patch is not poisonous. Many a modern outage comes from changes deployed too fast. Damned if you do, damned if you don't.

▲ olmo23 7 hours ago | parent [-]

if a CVE drops at 23pm, attackers have at least 8 hours before someone in the office has a possibility to take a look at it. nowadays, that seems to be plenty.

i don't have a solution.

▲ saalweachter 7 hours ago | parent | next [-]

I think "good security/engineering practices" still has a chance.

Things like--

  * Minimizing attack surfaces by not running lots of software on the same server and not bloating feature sets beyond the minimum 
  * Limiting access so that most things can't be done by most users, there are *no* super-users, and escalating access is temporary ans requires multiple accounts to authorize
  * Encrypting data at rest

All software has flaws and it's now easier than ever to find and exploit them, but this disproportionately hurts people who were only putting on a show of security and relying on a single layer or two of defense instead of developing a robust insider threat model (which helps with both insider threats and threats that get inside) and defense in depth.

▲

lokar 7 hours ago | parent [-]

Full zero trust internally. Both inbound and outbound proxies (different machines) to carefully limit access, and provide a place to mitigate attacks. Etc.

If your plan is to quickly apply upstream patches, your plan is bad.

▲

forgotaccount3 7 hours ago | parent [-]

> Full zero trust internally

I try so hard for this, but don't even get approval to change an internal site to use https instead of http.

I get we gotta keep the new features and workflows coming, but a single resource dedicated to low hanging fruit of security could fix a lot in a very short time.

▲

lokar 6 hours ago | parent [-]

It’s a hard transition, but possible. Obviously less work to do it from the start.

▲

saalweachter 6 hours ago | parent [-]

It requires management buy-in, both to allow some number of engineers to be allocated to such "non-production" work, and also to allow the friction security creates to continue to exist and "slow down" the "real work", both by vetoing things that would weaken security and by forcing everyone to use more cumbersome workflows.

That usually doesn't happen unless the company has suffered a large financial or reputational loss as a result of poor security practices. Sometimes not even then, if the company doesn't get scared they'll start losing money or get in some sort of legal trouble.

	▲	lokar 6 hours ago \| parent [-]
		I’ve worked places that did a lot of this (not everything, but a lot) from very early on.

▲ avidiax an hour ago | parent | prev | next [-]

There's a really simple solution that people will hate: server goes offline at night.

I've seen this a few times, and it's pretty annoying as a user, but ultimately it's not that bad usually. If you are ordering something at 3am, the physical shipment still occurs during daylight hours. Many banking operations are not actually instantaneous, they settle in a batch job overnight.

And this doesn't have to be binary. Providing a read-only interface provides lots of value while having a reduced attack surface.

▲ wongarsu 7 hours ago | parent | prev | next [-]

Have on-call sysadmins or have them work in a three-shift schedule. That way there's always someone there to fix stuff as it happens, including deploying security patches or worst case taking affected systems offline until they can be patched

This used to be common before devops and cloud took over. Looks like it might make a comeback. Maybe with a new fancier job title

▲ himata4113 7 hours ago | parent | prev | next [-]

low attack surface applications is the future.

▲ purplerabbit 7 hours ago | parent | prev | next [-]

People are gonna hate this but: we’ll need agents ready and waiting to autonomously patch and deploy instantly

	▲	zacmps 7 hours ago \| parent \| next [-]
		Then you just prompt-inject the agent to convince it to deploy a malicious patch. This isn't a solution.
	▲	Someone 7 hours ago \| parent \| prev [-]
		We need to deploy essential services at least twice, apply CVE ‘fixes’ to a subset of them, and have tooling to diff their outputs. That tooling should be shut off from the internet [1], and if it uses a LLM that should be at least a few years old. [1] technically impossible if it also has the ability to veto responses, but the surface can be very small.

▲ naillumanist 3 hours ago | parent | prev [-]

23 pm where? Simple answer is distributed teams.