Remix.run Logo
JohnLeitch 5 hours ago

The reliance on LLMs is unfortunate. I bet this mystery could gave been solved much quicker by simply looking at the packet capture in Wireshark. The Wireshark dissectors are quite mature, SSH is covered fairly well.

danudey 4 hours ago | parent | next [-]

I'm anti-LLM in most cases, but:

> I bet this mystery could gave been solved much quicker by simply looking at the packet capture in Wireshark.

For some people who are used to using Wireshark and who know what to look for, probably yes. For the vast majority of even technical people, probably not.

In my case, I did a packet capture of a single keystroke using tcpdump and imported it into Wireshark and I get just over 200 'Client: encrypted packet' and 'Server: encrypted packet' entries. Nothing useful there at all. If I tcpdump the entire SSH connection setup from scratch I get just as much useful information - nothing - but, oddly, fewer packets than my one keystroke triggered.

So yeah, I dislike LLMs entirely and dislike the reliance on LLMs that we see today, but in this case the author learned a lot of interesting stuff and shared it with us, whereas without LLMs he might have just shrugged and moved on.

mystraline 4 hours ago | parent [-]

And thats a huge downside when people howl about "Encryption everywhere! ".

Try debugging that shit. Thats right, debugging interfaces aren't safe, by some wellakshually security goon.

You want a real fun one to debug, is a SAML login to a webapp, with internal Oauth passthrough between multiple servers. Sure, I can decrypt client-server stuff with tools, but server-server is damn near impossible. The tools that work break SSL, and invalidate validation of the ssl.

Yes, Esri products suck. Bad.

jabwd an hour ago | parent | next [-]

Sounds like blaming a tool on a problem it did not cause. Either way, solvable and encryption is important. Badly designed systems and or lack of tooling isn't really an encryption problem.

Anyway, VMs should not have authentication, it makes access sooo much easier. Also drop your IPs while you're at it. Might be useful for debugging later.

reincarnate0x14 2 hours ago | parent | prev | next [-]

I used to share that opinion but after decades in industrial automation I find myself coming down much more on the "yeah, encryption everywhere" because while many vendors do not provide good tools for debugging, that's really the problem, and we've been covering for them by being able to snoop the traffic.

Having to MITM a connection to snoop it is annoying, but the alternative appears to be still using unencrypted protocols from the 1970s within the limitations of a 6502 to operate life-safety equipment.

supern0va 3 hours ago | parent | prev [-]

It seems like a leap to suggest we shouldn't have widely deployed encryption...rather than just fix the debugging tools.

Particularly in today's political climate, encryption has only become more necessary.

pbar 5 hours ago | parent | prev | next [-]

Unfortunately with SSH specifically, the dissectors aren't very mature - you only get valid parsing up to the KeX completion messages (NEWKEYS), and after that, even if the encryption is set to `none` via custom patches, the rest of the message flow is not parsed.

Seems because dumping the session keys is not at all a common thing. It's just a matter of effort though - if someone put in the time to improve the SSH story for dissectors, most of the groundwork is there.

JohnLeitch 4 hours ago | parent [-]

Interesting, I thought it was possible to decrypt SSH in Wireshark a la TLS, but it seems I'm mistaken. It still would have been my first goto, likely with encryption patched out as you stated. With well documented protocols, it's generally not too difficult deciphering the raw interior bits as needed with the orientation provided by the dissected pieces. So let me revise my statement: this probably would have been a fairly easy task with protocol analysis guided code review (or simply CR alone).

fragmede 4 hours ago | parent | prev | next [-]

Asking an LLM about SSH (hint: the two S-es stand for security) would tell you why only having packet capture in Wireshark isn't going to reveal shit.

JohnLeitch 4 hours ago | parent | next [-]

Not even remotely accurate. While the dissector is not as mature as I thought and there's no built-in decryption as there is for TLS, that doesn't matter much. Hint: every component of the system is attacker controlled in this scenario.

fragmede 2 hours ago | parent [-]

> Not even remotely accurate.

> there's no built-in decryption

Is that because wireshark can't do that just from packet captures?

sureglymop 4 hours ago | parent | prev | next [-]

Wireshark can decrypt it, so I don't understand what you mean?

fragmede 2 hours ago | parent [-]

Not from packet captures, it can't.

4 hours ago | parent | prev [-]
[deleted]
turtlebits 5 hours ago | parent | prev | next [-]

Way to gatekeep. God forbid people use tools to help them investigate instead of knowing the exact approach to take.

kkkqkqkqkqlqlql 4 hours ago | parent | next [-]

My thoughts exactly. The OP used AI to get a starting point to their investigation, then used their skills to improve their game, with actual (I guess according to the article itself) proof of that, as opposed to just approving changes from the LLM.

This looks like an actual productivity boost with AI.

JohnLeitch 4 hours ago | parent | prev | next [-]

What I suggested (mistakenly so, see my revised suggested approach in response to one of your siblings) is the exact opposite of gate keeping.

rjh29 3 hours ago | parent | prev [-]

ChatGPT gaslit the OP telling it there was no such thing as keystroke chafing. So yes, in this case it would have been better to do the work oneself.

tonymet 4 hours ago | parent | prev | next [-]

obviously OPs empirical and analytical rigor are top notch. He applied LLMs in the best way possible: fill gaps with clumsy command line flags or protocol implementations. Those aren't things one needs to keep in their head all the time.

MrDarcy 5 hours ago | parent | prev [-]

How much are you staking on that bet?

JohnLeitch 4 hours ago | parent | next [-]

Well, I spent a good part of my career reverse engineering network protocols for the purpose of developing exploits against closed source software, so I'm pretty sure I could do this quickly. Not that it matters unless you're going to pay me.

whatevaa 3 hours ago | parent [-]

So you are basically overqualified to tell other people how to do it, especially with the payment part.

JohnLeitch 2 hours ago | parent [-]

What are you even trying to say? I suppose I'll clarify for you: Yes, I'm confident I could have identified the cause of the mysterious packets quickly. No, I'm not going to go through the motions because I have no particular inclination toward the work outside of banter on the internet. And what's more, it would be contrived since the answer has already shared.

stackghost 2 hours ago | parent [-]

I think the point they're making is that "I, a seasoned network security and red-team-type person, could have done this in Wireshark without AI assistance" is neither surprising nor interesting.

That'd be like saying "I, an emergency room doctor, do not need AI assistance to interpret an EKG"

Consider that your expertise is atypical.

mystraline 5 hours ago | parent | prev [-]

Sigh.

I'm still waiting for a systems engineering tool that can log every layer, and handle SSL the whole pipe wide.

Im covering everything from strafe and ltrace on the machine, file reads, IO profiling, bandwidth profiling. Like, the whole thing, from beginning to end.

Theres no tool that does that.

Hell, I can't even see good network traces within a single Linux app. The closest you'll find is https://github.com/mozillazg/ptcpdump

But especially with Firefox, good luck.

fragmede 4 hours ago | parent [-]

Real talk though, how much would such a tool be worth to you? Would you pay, say, $3,000/license/year for it? Or, after someone puts in the work to develop it, would you wait for someone else to duct tape something together approximately similar enough using regexps that open source but 10% as good, and then not pay for the good proprietary tool because we're all a bunch of cheap bastards?

We have only ourselves to blame that there aren't better tools (publicly) available. If I hypothetically (really!) had such a tool, it would be an advantage over every other SRE out there that could use it. Trying to sell it directly comes with more headaches than money, selling it to corporations has different headaches, open-sourcing it don't pay the bills, nevermind the burnout (people don't donate for shit). So the way to do it is make a pitch deck, get VC funding so you're able to pay rent until it gets acquired by Oracle/RedHat/IBM (aka the greatest hits for Linux tool acquisition), or try and charge money for it when you run out of VC funding, leading to accusations of "rug pull" and development of alternatives (see also: docker) just to spite you.

In the base case you sell Hashimoto and your bank account has two (three!) commas, but worst case you don't make rent and go homeless when instead you could've gone to a FAANG and made $250k/yr instead of getting paid $50k/yr as the founder and burning VC cash and eating ramen that you have to make yourself.

I agree, that would be an awesome tool! Best case scenario, a company pays for that tool to be developed internally, the company goes under, it gets sold as an asset and whomever buys it forms a compnay and tries to sell it directly and then that company goes under but that whomever finally open sources it because they don't want it to slip into obscurity but if falls into obscurity anyway because it only works on Linux 5.x kernels and can't be ported to the 6.x series that we're on now easily.