That's a more elegant approach. I usually just plow through obstacles, and the end result is not always ideal -- I like your approach better than the sidecar, I guess that I was using sidecars for other things and it sort of influenced my approach.

I'll try it your suggestions out and update the article, and thank you for your comment, already made sharing this worth it.

What is the purpose of putting the pid file into /dev/shm ? I’ve never seen that before and am curious to learn more about the technique.

I've used Caddy for some of my projects (e.g. https://github.com/fsmunoz/parlamentodb/blob/54e0b252485905e... ), but not for this intercept approach you mentioned, I will give it a look!

I'm not bothered by restart times but that's mostly because that has never been a priority... but one thing I have half-done is a controller that gathers per-namespace configs, and with that reload times will become more of an issue.

Part of the reason I chose Squid here was precisely because I found it interesting to reuse something that was such a staple of web architecture patterns.

Thank you!

Depending on what want for "lock down", this or something like this could work: you are essentially defining a single outbound communication path. In a way, your scenario was one of the reasons behind this experiment.

I'll take a look a the overflow thing, although I'm not sure if I will be able to fix it: I do have an image at the start which is an alternative to the text-based drawing, so nothing is lost. I use my own blogging solution that is essentially Texinfo (https://interlaye.red/Texiblog.html) so these blocks are the result of using an @example block (which is then converted into a preformatted block). I'm not sure this can be improved, apart from (as you said) using alternative images.

I'm not sure I understand the issue.

Wouldn't the pattern be to use a reverse proxy for ingress and everything goes through there into the backends? Keep the pod ips range that is not directly reachable from outside the cluster?

Using an http proxy like squid (or apache/haproxy/caddy/envoy/trafficserver/freenginx) does sound like what you should do next.

If you need the pod to do outbound connections as well as receive incoming traffic, usually that would be two different proxies (forward and reverse, respectively). Unless you do some fancy p2p service mesh.

I had challenges with split-DNS in my homelab k3s cluster trying to do this. I ended up just putting the apps in docker-compose on a VM that has static routes for my local homelab networks. I looked at tailscale to solve this since it has a kubernetes operator, but tailscale doesn't fit my use cases or work well with all of my devices.

I guess you have just described what I was hinting at here:

>Linked with several of the above (mainly the centralised configuration) is that when using ACL rules to limit communication to external domains, these are cumulative: all namespaces will be able to communicate with all whitelisted domains, even if they only need to communicate with some of them. > These limitations point toward why more sophisticated solutions exist, after all; a follow-up article will explore using Squid’s include directive to enable per-namespace configuration, and in doing so, show why you’d eventually want a controller or operator to manage the complexity.

... which is actually a good thing. More than making something "new", it's great to hear that the overall approach is sound.

Depends on what you want, I touch upon that somewhat: to replicate this specific pattern, you can replace Squid with something that fills in the gap without any major changes - so, nginx or Caddy for example -- but you would have to make sure the feature set is adequate: I see Squid as being egress-first, where others are ingress-first (nginx being used a an ingress controller, recently discontinued but still...), so I do think that for this specific purpose it works quite well.

As for Envoy and others, I think this would fit in a different architecture that I sort of point to near the end, one that includes using a service mesh: Istio for example uses Envoy for Egress Gateway, Cillium also has an Egress Gateway, etc. This to me would be a separate pattern though.

You can certainly use the Squid ACLs to limit the egress for agents. One of the current shortcomings (I explicitly mentioned it near the end) is that there's no per-namespace granularity, so you wouldn't be able to determine it on a per-agent level -- but you would be able to generally establish that all agents would only have access to a global whitelist.

Yes! And this can be partially a limitation that helps, in the sense that it forces you to add that. In this example, I had to spent some time with the Common Lisp dexador approach to make it work. I've added a "PROXY: " UI hint in the page at https://horizons.interlaye.red/ , you will see that it says "-- PROXY: http://squid.egress-proxy.svc.cluster.local:3128 --". This was actually something from my debugging that I decided to keep.

A next article will likely address this limitation though, and look into transparent proxying. This will involve nftables, sidecars, etc, and the more we go into this direction, the more installing a CNI that comes with this by default starts to make sense.

The older versions of Istio uses an init container to redirect inbound and outbound traffic from the main container to the Envoy sidecar. You still have to have some kind of admissions hook to inject things if you want it automatic, but the apps don’t need to understand proxies.

You are most welcome, and that was precisely what I aimed at. Thank you.