Remix.run Logo
TIL: You can make HTTP requests without curl using Bash /dev/TCP(mareksuppa.com)
85 points by mrshu 2 hours ago | 48 comments
basilikum an hour ago | parent | next [-]

> As it turns out, bash can speak HTTP by itself.

No, it can not. Bash lets you open TCP sockets.

What you are doing here is trying to speak HTTP yourself, which is fine for testing and debugging, and hella cool for fun to do by hand, but you will shoot yourself in the foot if you try to use this pseudo http client unattended in reality. This toy code does not parse HTTP properly and will break.

You could of course write a full http/1.1 client in bash, you can even do a full http server in pure bash: https://github.com/bahamas10/bash-web-server

For less insane, non-bash shells there is always nc which is usually probably the wiser choice.

a-dub an hour ago | parent | next [-]

it's not that insane. i've been manually typing http requests in since before http/1.1 and the mandatory host header.

it is insane to use it for anything serious (also the opposite, implementing webservers in bash), but for quick testing it's pretty great!

bitmasher9 34 minutes ago | parent [-]

Why wouldn’t you use curl for the quick test?

hnav 24 minutes ago | parent | next [-]

Sometimes you want to do something that curl cannot express, e.g. timing, protocol oddities, etc. For example you may want to issue a CONNECT to an echo server through a proxy and observe the bytes flowing back and forth. You may want to see what happens when conflicting hop-by-hop headers are specified without worrying about the client's (curl's) interpretation of them. A simple nc -c (or openssl s_client -crlf) lets you do all of that.

a-dub 5 minutes ago | parent | prev [-]

because in those days there was no curl, or wget. and then when there was, there was no guarantee they'd be installed.

telnet was always there though. it also worked for speaking all the other plaintext internet protocols. (imap, pop, smtp, etc)

TZubiri 18 minutes ago | parent | prev | next [-]

>No, you can't write 10 lines of code, you have to import a 100k LOC dependency

Common misconception, if you want to replace a dependency on a swiss knife you don't need to implement a swiss knife, sometimes you can just implement the last helix of the corkscrew.

morpheuskafka an hour ago | parent | prev | next [-]

> No, it can not. Bash lets you open TCP sockets.

I thought you had to use a program called netcat for that--if not then what is the point of that binary? And for that matter, can't you also use telnet to manually send HTTP?

some_random 42 minutes ago | parent [-]

nc is basically just a nicer interface for the same thing, in the same way that curl is.

https://linux.die.net/man/1/nc

mrshu an hour ago | parent | prev [-]

> No, it can not. Bash lets you open TCP sockets.

Very fair pushback -- I did get carried away and will update the article to be more precise. Thanks for raising it!

> For less insane, non-bash shells there is always nc which is usually probably the wiser choice.

For completeness, `nc` or any netcat equvialent I could think of was not available in the image I was trying this with. It would certainly be a better option though.

bearjaws an hour ago | parent | next [-]

This is the most Claude pilled comment I've seen here.

mrshu 2 minutes ago | parent | next [-]

It's pretty rough to learn I sound like Claude. Will need to do something about it then.

(For what it's worth I did write the message above manually but I understand why no one would believe that now. At least I did not call netcat "load-bearing" [https://mareksuppa.com/til/load-bearing/] or something...)

thih9 an hour ago | parent | prev | next [-]

This worries me. Some AI writing styles became mainstream; at first it was the em-dashes, now it’s “A, not B” patterns and excessive acknowledging. There will be more.

Was grandparent comment written by an LLM?

Or is this a human who copies a style they saw in a blog post, unaware that they’re copying an AI?

Or is this a human who spent too much time talking to an AI and now they just talk like this?

Or is this an organic human response and we’re all paranoid by now?

I don’t know which would be worse.

nialv7 33 minutes ago | parent | prev [-]

what would be a non-pilled way of saying the same thing?

xeyownt 24 minutes ago | parent | next [-]

Yeah. The comments saying it's AI-pilled comments are more annoying and less informative than the comments themselves.

WD-42 21 minutes ago | parent | prev [-]

Good point however netcat wasn’t available either.

throwrioawfo 27 minutes ago | parent | prev [-]

Bro really replaced the em-dash with "--"

m3047 a minute ago | parent | prev | next [-]

At least on my systems there's also /dev/udp...

simonw an hour ago | parent | prev | next [-]

Neat, works against example.com

  exec 3<>/dev/tcp/example.com/80
  printf 'GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n' >&3
  cat <&3
Outputs:

  HTTP/1.1 200 OK
  Date: Tue, 16 Jun 2026 17:37:45 GMT
  Content-Type: text/html
  ...
I always end up on example.com for this kind of thing because there are so few domains these days that don't enforce https!
QuantumNomad_ an hour ago | parent | next [-]

example.com is also great for that reason when something fails about a captive portal on a public WiFi.

I open my web browser and go to http://example.com and get redirected to the captive portal page again and retry completing what they need from me to get internet access.

some_random 40 minutes ago | parent [-]

Fun fact, this is almost exactly how active portal detection is done in the OS/browser!

https://gist.github.com/skull-squadron/edb8c0122f902013304c0...

QuantumNomad_ 5 minutes ago | parent [-]

Yep :) I just find example.com easier to remember and quicker to type than any of the OS or browser makers own URLs like

- http://captive.apple.com/

- http://connectivitycheck.gstatic.com/generate_204

- http://detectportal.brave-http-only.com/

Plus, it feels nice to depend on the reserved domain name example.com instead of relying on a domain that any one specific corporation has to maintain :D

gabrielsroka 30 minutes ago | parent | prev [-]

This works too

  exec 3<>/dev/tcp/example.com/80
  printf 'GET / HTTP/1.1\r
  Host: example.com\r
  Connection: close\r
  \r
  ' >&3
  cat <&3
You can even take out the \r though they should be there
mrshu 2 hours ago | parent | prev | next [-]

I ran into this while checking connectivity between containers on an internal Docker network where the image had neither curl nor wget.

The main surprise was that Bash has /dev/tcp which lets you do the equivalent of an HTTP request with a bit of shell magic, for instance:

  exec 3<>/dev/tcp/service/8642
  printf 'GET /health HTTP/1.1\r\nHost: service\r\nConnection: close\r\n\r\n' >&3
  cat <&3

Where `service` is just the hostname of whatever you’re talking to and 8642 is the port you are trying to talk HTTP to.

Pretty cool!

sevenzero 2 hours ago | parent [-]

It seems pretty cool, but I am wondering if there's any drawback on just using images that support curl? I can't think of any and to me it's kinda a must have, even on production images

OptionOfT an hour ago | parent | next [-]

I always recommend to not have any dependencies outside of the code.

So we start at compiling the codebase (Rust) against MUSL. That way we can run it with FROM scratch images.

If we need more tooling available at runtime, then we look at alpine, but still using MUSL.

If MUSL itself is proving problematic, or if some of the libraries we use need glibc then we can look at using some locked down image.

The cool part about FROM scratch images is that you'll never have to update your base image to address CVEs. Only your software and its (compiled) dependencies.

xmodem an hour ago | parent [-]

> The cool part about FROM scratch images is that you'll never have to update your base image to address CVEs. Only your software and its (compiled) dependencies.

What's the benefit really, though? If you still need to be able to rapidly deploy a new image in response to a dependency CVE, what have you gained?

regularfry 28 minutes ago | parent [-]

You've gained that happening much less frequently. The tradeoff is making every other problem harder to diagnose.

xmodem an hour ago | parent | prev | next [-]

More than one ~500 employee company I've worked at has had security policies either encouraging or requiring the use of "distro-less" images - images with no OS components other than the absolute minimum required to run the application. For go binaries this meant literally nothing in the container apart from the executable.

In theory it has a couple of benefits. You don't have to re-deploy your image to patch CVE's in OS components if you don't have any OS components. And it provides some measure of defence-in-depth - one could certainly theory-craft a scenario where an attacker gains some limited control over your application and then uses some OS component to escalate.

These days if a security engineer is proposing my team adopt distro-less containers to receive these benefits, I would point out that we need to weigh them against the very real drawbacks of not having standard debugging tools available where and when weneed them. And also to consider the relative impact of other defence-in-depth measures they could be pursuing instead - such as any sort of network policy to limit network traffic.

mrshu an hour ago | parent | prev | next [-]

That is indeed a solid pushback! :)

For what its worth, this container used `python:3.12.2-slim-bookworm` and I really would not expect that sort of an image to bundle `curl` -- even if it is intended for production.

TZubiri 11 minutes ago | parent | next [-]

You can also use the sockets lib in that case, you depend on POSIX instead of Linux

sevenzero an hour ago | parent | prev [-]

Ah I see so it was basically a minimal image that bundles just python? I can see why it wouldn't bundle curl! Thought it was a custom Image for some reason, hence my original comment

mrshu an hour ago | parent [-]

Yes, a very minimal image indeed. Had it been a custom image, curl would be one of the first things I would make sure it contains :)

figmert an hour ago | parent | prev | next [-]

This of course only supports http, not https. It's great for health checks e.g. in a docker environment. To do https, you'd have to use something like socat, but of course that doesn't use bash only.

TZubiri 10 minutes ago | parent [-]

Https is almost always terminated separately from the application code.

giobox an hour ago | parent | prev [-]

It's also a two line Dockerfile to add wget or curl to almost any pre-existing container image. This is a fun idea though.

sam_lowry_ an hour ago | parent | prev | next [-]

A few years ago I had to do this for a SpringBoot health check from a Docker container:

FROM openjdk:11-jre-slim HEALTHCHECK --start-period=10s --timeout=3s --retries=5 \ CMD perl -e "use IO::Socket; $sock = IO::Socket::INET->new(Proto => 'tcp', PeerAddr => 'localhost', PeerPort => '8888') or die $@; $sock->autoflush(1); print $sock 'GET /actuator/health HTTP/1.1' . chr(0x0a) . chr(0x0d) . 'Host: localhost:8888' . chr(0x0a) . chr(0x0d) . 'Connection: close' . chr(0x0a) . chr(0x0d) . chr(0x0a) . chr(0x0d); while (my $line = $sock->getline ) { if ($line =~ /UP/) {exit;} }; close $sock; exit 1;"

hn92726819 42 minutes ago | parent [-]

Note that this is not what the article is about. Bash has a fake /dev/tcp path that opens sockets. What you have there is just perl opening a socket normally. Great solution, but the interesting bit is that fake path.

dchest 28 minutes ago | parent | prev | next [-]

It's interesting that most of the comments here are about using this feature to bypass security restrictions (whether valid or not). It says a lot about the attack surface of GNU utilities caused by featuritis.

AndrewStephens an hour ago | parent | prev | next [-]

This is pretty neat if all you need is to ping a local server but please use curl (or something equivalent) for contacting remote services. HTTP1.1 seems like such a simple protocol but in the real world you need to deal with proxies, different encodings, and redirects. Curl takes care of that (and a host of other annoying stuff) for you.

mrshu an hour ago | parent [-]

Totally!

I was really just trying to see if intra-container connectivity works, and this ended up being a very quick way of doing so. (The alternative being building and deploying a new image, which would likely take significantly longer.)

KomoD an hour ago | parent [-]

> The alternative being building and deploying a new image, which would likely take significantly longer

You said the image was Python, though? Using that is way easier and faster. https://news.ycombinator.com/item?id=48558763

If all you need to know is that it can connect:

python3 -c 'import socket as s;s.create_connection(("8.8.8.8",53))'

or http:

python3 -c 'from urllib.request import*;print(urlopen("http://example.com").status)'

geoctl an hour ago | parent | prev | next [-]

I discovered this bash trick by chance when I was once trying to healthCheck the Envoy's official OCI image container which didn't include curl or wget while forcing the envoy admin interface to listen on localhost which breaks the traditional k8s httpGet checks.

orthogonal_cube an hour ago | parent | prev | next [-]

It was fun exploring this to make a native-shell-only peer-to-peer file transfer utility at work for some automation scripts. At least, it was until trying to replicate it in Powershell was somehow triggering Crowdstrike and the corporate Cybersecurity team thought I was writing malware.

devsda an hour ago | parent | prev | next [-]

Yes, it used to be my goto few times when some devices tried to lockdown everything with bare minimum core utils and no network capable tools like curl etc.

alienbaby an hour ago | parent | prev | next [-]

Reminds me of telnetting to port 80 to make a get request years and years ago

sc68cal 2 hours ago | parent | prev | next [-]

That's pretty neat, thanks for sharing

Steeeve 32 minutes ago | parent | prev | next [-]

brb. recompiling bash in all my base images.

alienbaby 35 minutes ago | parent | prev [-]

Reminds me of using telnet to port 80 to make get requests aeons ago