Remix.run Logo
mikepurvis 6 days ago

The article is focused on interactive use, but I would be interested to also hear from folks using tmux as a process manager / dashboard for server-type environments. I was pretty unenthused about this practice when I first encountered it, but it's grown on me as a way to quickly get a sense of state from onboard a running system without having to rely on off-board dashboard or log parsers / alert systems.

In particular, I'm curious about libtmux/tmuxp and how well it works to have a more declarative bringup than just a bash script launching things into the background and then attaching tmux windows to them.

And I guess the broader question of whether this approach is an evolutionary dead end— like does it just become a pile of hacks as one discovers the need for restart policies, backoffs, ordering, whatever other stuff that is built into a "normal" process manager like systemd or runit? If you do end up wanting to send process stdout/stderr elsewhere or do alerting, is that now annoying if those streams are "trapped" in tmux sessions vs being written into log files or the journal?

accrual 6 days ago | parent | next [-]

> I would be interested to also hear from folks using tmux as a process manager / dashboard for server-type environments

Sure! I use tmux on most of my OpenBSD systems and copy the same .tmux.conf file around, but rarely tweak it other than to set a hostname.

My status bar is at the top and looks approximately like this:

   --~ msg auth pf       hostname. Mon Aug 04 15:05
* The first two windows (--) are regular shells.

* The tilde window (~) is where I do stuff as root.

* The message window (msg) tails `/var/log/messages` and `/var/log/daemon`, or `journalctl -f` and `journalctl -k -f` on Linux.

* The auth window (auth) tails `/var/log/authlog` to monitor login attempts and usage of doas and sudo.

* The pf window (pf) runs a script to monitor the packet filter rules (pass, block, match).

* hostname is just the hostname. The period (.) means no mail, it turns to an exclaimation mark (!) when I have mail.

* Simple date/time.

This setup gives me quick access shells, important logs, mail status, date/time, and works the same on every server I manage. I add extra windows to tail other app logs as needed.

quesera 6 days ago | parent [-]

> but rarely tweak it other than to set a hostname

In case helpful: You can use `#h` in a status line string to get the hostname

accrual 5 days ago | parent [-]

Thanks! My .tmux.conf is kinda old, I created it before it was in the base OpenBSD sets. Might need some updates like '#h'. :)

dancek 5 days ago | parent | prev | next [-]

I wrote a quick and dirty bash script that launches a tmux dashboard for a legacy linux server. The team has been happy with it. It does seem like an evolutionary dead end to be honest, but it was very quick to set up.

Note that the dashboard is read-only and mostly just shows journalctl for different units (we use systemd for process management). And we assume it's only used by a single user at once and everything is stopped on detach.

Here's a simplified sample of setting up a 3x2 layout with some placeholder content:

    #!/bin/sh
    set -e

    SESSION=example
    WINDOW=dashboard

    tmux new-session -d -s $SESSION -n $WINDOW "journalctl -fk"
    tmux set-option -t $SESSION -g mouse on

    tmux split-window -h -t $SESSION:$WINDOW "journalctl -fu nginx"
    tmux split-window -h -t $SESSION:$WINDOW "journalctl -fu postgresql"
    tmux select-layout -E

    tmux split-window -v -t $SESSION:$WINDOW "echo '<Ctrl-b d> to detach' && read"
    tmux select-pane -t $SESSION:$WINDOW.0
    tmux split-window -v -t $SESSION:$WINDOW "watch uptime"
    tmux select-pane -t $SESSION:$WINDOW.2
    tmux split-window -v -t $SESSION:$WINDOW "watch df -h"

    tmux attach-session -t $SESSION
    # after detaching, close session
    tmux kill-session -t $SESSION 2>/dev/null || true
hnlmorg 6 days ago | parent | prev | next [-]

I’ve seen people use this approach (and screen too) but personally I hate it for anything other than services that are interactive but you want detachable. Eg long running installs.

All the other examples I’ve seen where people have used it as a shortcut for daemonizing a service, it’s felt like just as much effort as writing a systemd service file.

I guess I could forgive people more back in the days of sysv-style initd. Writing those shell scripts was seldom fun. But with systemd, it’s literally a 5 minute task. And I say that as someone who dislikes numerous decisions made regarding the approach systemd takes.

mikepurvis 5 days ago | parent [-]

For daemonizing a single service, I agree it's silly. I think the argument becomes more complex when you have a whole group of services. The "proper" way for that involves a systemd unit for each process, and then if you want the dashboard experience you still have to launch all the panes and get each one tailing the respective unit journals.

It's tougher to make the case there that systemd units are "worth it", particularly given the other ergonomic benefits of a pure tmux solution, such as being able to ctrl-c a pane and trivially re-launch it.

And once you're launching services in a container, that's another world again, since systemd in that case is quite a bit worse and probably wants CAP_ADMIN.

hnlmorg 5 days ago | parent [-]

> For daemonizing a single service, I agree it's silly. I think the argument becomes more complex when you have a whole group of service

I dont agree that a bunch of services makes things harder if it’s already easier as one service. The set up here is linear in effort. Plus, if anything, being able to restart the host and have all services automatically come up makes systemd much more convenient.

> and then if you want the dashboard experience you still have to launch all the panes and get each one tailing the respective unit journals.

logs are a literal one liner with systemd. Though if you care about dashboards then you should be exporting your logs to a proper observability platform instead of SSHing into your service.

> such as being able to ctrl-c a pane and trivially re-launch it.

  systemd restart …
Is less effort than

  tmux attach …
  [prefix]+w (select window)
  ctrl+c
  [up]
  [return]
> And once you're launching services in a container, that's another world again, since systemd in that case is quite a bit worse and probably wants CAP_ADMIN.

If you’re launching a service in a container then you don’t need either systemd nor tmux in that container because the container itself provides all that scaffolding for you.

Whether it’s Docker/podman, LXC, FreeBSD jails, Solaris Zones, etc, you just set your service to be the entry point of that container.

Docker makes this even easier still with a literal directive called “entrypoint”.

Trying to start services using tmux inside of a container is massively more effort than calling it directly from the entrypoint.

———

I do get the appeal of using tmux for these usages. I honestly do. And if your services are constant in flux, for example inference where you’re constantly tweaking parameters then tmux does make sense (a proper ML pipeline makes more sense, but that requires knowledgable MLOps to build). However for anything consistent, the effort in doing things “properly” is usually less than you think it is.

walterlw 5 days ago | parent | prev | next [-]

I've used tmux like that a bunch before discovering process-compose [1]. Still use tmux a lot with remote connections and booting up a few scripts in parallel is still easier with it, but for longer-serving setups p-c has helped a lot

[1] https://github.com/F1bonacc1/process-compose

kachapopopow 6 days ago | parent | prev [-]

yep, that's how roughly 99.99% of major minecraft servers are ran up to this day. (and I hate migrating them)

Filligree 6 days ago | parent [-]

Systemd just does a poor job giving you a TTY on demand.