What if the common intermediate encoding is text, not binary? Then grep/diff/sed all still work.

If we had a formatting tool that operated solely on AST, checked in code could be in a canonical form for a given AST. Editors could then parse the AST and display the source with a different formatting of the users choice, and convert to canonical form when writing the file to disk.

▲

sublinear 4 days ago | parent | next [-]

Nobody wants to have to run their own formatter rules in reverse in their head just to know what to grep for. That defeats the point of formatting at all.

▲

pwdisswordfishz 4 days ago | parent | next [-]

That's why you grep for a syntactic structure, not undifferentiated text.

▲

michaelmrose 4 days ago | parent [-]

Which grep doesn't do and you need to either use a new different tool or more likely several for little real benefit

▲

hnlmorg 4 days ago | parent | next [-]

grep is half a century old now.

If we can’t progress our ecosystem because we are reliant on one very specific 50+ year old line parser, then that says more about the inflexibility of the industry to move forward than it does about the “new” ideas being presented.

▲

account42 4 days ago | parent | next [-]

We still use grep because its useful. And it's useful precisely because it doesn't depend on syntax so will work on anything text based.

	▲	hnlmorg 3 days ago \| parent [-]
		grep is great. My point isn’t that we shouldn’t use it. My point is that we shouldn’t be held back by it.

▲

komali2 4 days ago | parent | prev [-]

The things all being described are way beyond non trivial to solve, and they'd need to be solved for every language.

Grep works great.

	▲	hnlmorg 3 days ago \| parent [-]
		> The things all being described are way beyond non trivial to solve, and they'd need to be solved for every language. Except it already is a solved problem. If languages compile to a common byte code then you just need one tool. You already see examples of this with things like the IR assembly produced by LLVM, various Microsoft languages that compile to CLR, and the different languages that target JVM. There are also already common ways to create reusable parsing rules like LSP for IDEs and treesitter. In fact there are already grep-like utilities that are based on treesitter. So it’s not only very possible to create language agnostic, reusable, tools; but these tools already exist and being used by a great many developers. The problem raised in the article is that we just don’t push these concepts hard enough these days. Instead relying on outdated concepts of what source code should look like. > Grep works great For LF-separated lists it does. But if it worked great for structured content then we wouldn’t be having this conversation to begin with.

▲

jitl 4 days ago | parent | prev [-]

comby is fantastic, give it a shot. It’s saved me huge amounts of time.

▲

theamk 3 days ago | parent | prev [-]

You'd need all-news tools for non-text world as well.

So the real choice is either:

- new tool: grep with caching reverse-formatter filter.

- new tool: ast-grep with understanding of AST serialization format for your specific language.

At least in the first case, you still have fall back.

▲

pmontra 4 days ago | parent | prev [-]

All mainstream editors that agree to work on a standard AST for any given language could be nice. I'm not expecting that to happen at any time in future.

About grep and diff working on a textual representation of the AST, it would be like grepping on Javascript source code when the actual source code is Typescript or some other more distant language that compiles to Javascript (does anybody remember Coffescript?) We want to see only the source code we typed in.

By the way, add git diff to the list of tools that should work on the AST but show us the real source code.