> A syslog server is a foundational tool for centralized log management in modern IT environments

I'd very much recommend a more modern log stack than a traditional syslog server.

There are many articles covering the limitations of Syslog. Better to replace that component by utilities like OpenTelemetry and JSON structured logging.

You can run a single binary version of Loki https://grafana.com/docs/loki/latest/get-started/deployment-...

Or use something like the Otel collector to send your logs to a remote host.

I have done my fair share of rsyslog and syslog-ng.

I would not say a "Syslog" server belongs in a modern stack.

▲

skullone 4 hours ago | parent | next [-]

I disagree. I work in a large environment, and rsyslog is where 90% of data goes to first. It can keep up with millions of messages per second, route them to higher order services for indexing (bigquery, splunk, elastic etc etc). Has rules engines, encryption, supports multiple protocols and obviously has TLS too. You can surely augment with otel and such where you can, but syslog is uhhhh, deployed in so many places that it would make an average app developer's head spin when all they're used to is application logging in a controlled structured place in their silo.

▲

otterley 23 minutes ago | parent | next [-]

On the other hand, traditional syslog is UDP based, so as soon as the receiver experiences CPU or I/O starvation and its receive buffer overflows, it will begin dropping messages. That's not great for observability, and may well be impermissible at many sites that need end-to-end log integrity (e.g. audit logs).

	▲	lanstin 2 minutes ago \| parent [-]
		cheaply dropping log msgs you cannot handle is absolutely essential for an observability system - otherwise excess load can take down the logging infra which can (if msgs aren't dropped) take down the prod network/app trying to send reliable log msgs. Audit logs are a distinct feature.

▲

shmoe 2 hours ago | parent | prev [-]

Even SC4S, splunk's docker appliance for turnkey syslog uses rsyslogd.

Edit: being pedantic -- it's syslog-ng actually.

	▲	skullone 2 hours ago \| parent [-]
		And the number of k8s envs that log stdout through them into.... more rsyslog, it's truly everywhere. Plus all the sidecar containers deployed that shuffle app logs, lots of syslog there, its so lightweight and simple and reliable. I watch all the gyrations people go through to achieve the same result, and it's always changing, hurts my brain thinking how much time they waste

▲

aftbit an hour ago | parent | prev | next [-]

Disagree. JSON logging is fine, but make it line based and just log to a syslog server. Then I can route it wherever I want (or to multiple places) including to a simple file on a disk that I can actually inspect, rather than having to use an API.

▲

1970-01-01 3 hours ago | parent | prev | next [-]

The problem with modern stuff is it doesn't do the very basics. Sometimes I really do want UDP dumping out into a file on another part of the network. The modern setups forget how to do this.

	▲	edoceo 2 hours ago \| parent [-]
		I saw one place that had the logs going into a database. On the same connection as the app-data. So, when the transaction failed, the logs also didn't get written. LMAO. I made them do syslog in their code, which for some of the devs was a mind-blowing. They were amazed at that we could just barf text quick&lightweight over UDP.

▲

Exoristos 39 minutes ago | parent | prev | next [-]

I've been down this road many times over the years, and each new promised land of logging always fails to replace syslog for me. Nothing else is as widely-compatible and performant. Fortunately, there are some great tools out there to modernize the network-admin experience: syslog-ng is a favorite of mine.

▲

lokar 3 hours ago | parent | prev [-]

Yeah, for a modern large scale distributed system both the client api and implementations are pretty bad.