Remix.run Logo
▲ Regex visualized as 3D marble run(regexmarblerun.com)
65 points by seizethecheese 2 days ago | 23 comments
▲seizethecheese 2 days ago | parent | next [-]

My coworker used to teach non-technical people how to code. Students always found regex hard to understand.

Let's be honest, regex can be hard to understand even for technical people. (At least it's hard for me.)

So coworker built this to help anyone understand arbitrary regex commands.

AI disclosure: this was built using Agent Duel, a local coding agent arena system (https://github.com/bottomless/agent-duel). Promotion disclosure: our company is building agent duel. To test it and find bugs we are building passion projects through the system. I nonetheless thought HN would find this interesting.

▲crazygringo 3 hours ago | parent | next [-]

Regex is conceptually incredibly easy to understand.

But regex is also syntactically designed by Satan himself.

Most programming languages use commands that are semantically understandable, like "for" and "let" and "return" and mostly symbols we already know like "+" and "=". They use newlines and indentation for organization. And they put multiple commands on multiple lines and you can add comments and everything.

Regex is exclusively punctuation, with its own idiosyncratic meanings, all compressed into a single line, zero organization, zero comments, and impossible to remember what each symbol does if you don't use it daily. Not to mention the nightmare of escaping regexes when expressed as strings, on top of escaping regex symbols within the regex itself.

I swear it's high time we come up with a regex replacement that looks like normal Python code or HTML or something. So instead of:

  ^(\+\d{1,3}-?)?\d{3}-\d{3}-\d{4}$
We would write something like:

  <start>
    <optional>  # Optional country code
      "+"
      <digit>[1-3]
      "-"
    </optional>
    <digit>[3]  # Area code
    "-"
    <digit>[3]
    "-"
    <digit>[4]
  <end>
Same thing. But one is completely inscrutable to someone who hasn't learned regex, while with the other you can guess pretty easily how it works...
▲wongarsu 3 hours ago | parent | next [-]

If you want readability you can write

    ^(?P<country_code>\+[0-9]{1,3}-?)?[0-9]{3}-[0-9]{3}-[0-9]{4}$
Most regex engines even have a verbose mode (typically the x flag) where this is a valid regex:

  ^
  (   
      \+
      [0-9]{1,3}
      -?
  )?   # optional country code
  [0-9]{3}
  -
  [0-9]{3}
  -
  [0-9]{4}
  $
I am not sure if everyone writes unreadable regex because the reputation of regex makes it seem pointless to try, or if being witty one-liners is a critical part of the value proposition of regex
▲mejutoco 3 hours ago | parent | prev | next [-]

Regex in xml? Now you have 2 problems. And the comments are not xml comments even.

Ultimately it is a dsl. Should equations in math also use full words and comments?

No, they should not. Maybe a nice syntax for grammars like raku is a better approach.

▲crazygringo an hour ago | parent [-]

I couldn't care less if it looks more like XML or Python or C or whatever. It's just an example.

I just care that things have readable text names rather than inscrutable punctuation and single letters.

Yes, it is a DSL. Just like, you know, Python or C or HTML or CSS. Just because it's a DSL doesn't mean it can't actually use keywords instead of punctuation.

▲dmd 2 hours ago | parent | prev | next [-]

There are certainly alternatives, like https://pomsky-lang.org/

▲bazoom42 3 hours ago | parent | prev | next [-]

Most regex dialects allow whitespace and comments, so that already gets you halfway there.

▲dylan604 2 hours ago | parent | prev | next [-]

wouldn't it be easier to just add a comment block in the code above the regex?

▲sgarland 3 hours ago | parent | prev | next [-]

You can enable extended mode for most engines, which allows you to add white space and comments.

Personally, I like regexes, in the same way that I like shell string manipulation: it looks utterly inscrutable, and performs magic. If I write one in code meant for others, I usually use extended mode with each line commented.

▲ogogmad 3 hours ago | parent | prev [-]

I think some languages do have a library-style version of regex. But the function names aren't consistent between the different libraries. Maybe then we need to put on special goggles to make them look the same.

▲atoav 6 hours ago | parent | prev | next [-]

I am mostly an autodidact and learned bits regex on the fly when I needed it and I was always baffled why people have auch a hard time with it.

The problem IMO is that a lot of the traditional teaching material starts with the complexity (and impressive possibilities) of regex, when the best way to start learning it is probably just thinking about it as an extension to simple text matching.

Getting to know regex fully is in fact complicated, getting know enough regex to filter/search/replace 90% of everything you will always need is not too complicated.

▲ogogmad 6 hours ago | parent [-]

Autodidact here. Beyond just the five core quantifiers

  .
  |
  ?
  *
  +
regex only has capture groups

  ()
and lazy variants of the core quantifiers "?", "*" and "+", which are helpfully called:

  ??
  *?
  +?
The rest is just shorthands.

So while I think it's a worthwhile suggestion not to show people the whole thing the first time round, I doubt that the whole thing is really that complicated.

▲ryangittins 8 minutes ago | parent | next [-]

> The rest is just shorthands.

If you really want to get technical: ., ?, and + are also shorthands. The formal definition of a regular expression is very minimal on purpose.

`.` is just `(a|b|c|...)` - List of all literal characters

`x?` is just `(x|)` - "x or empty string"

`x+` is just `xx*` - "x followed by zero or more x characters"

Whether this is exactly how they are implemented in any particular regex engine is an implementation detail, but formally there are very few operations which aren't just syntactic sugar.

▲StilesCrisis 4 hours ago | parent | prev [-]

Regex has many more features than this, e.g. backreferences. I agree that it's easy to learn enough regex to be dangerous with that subset, but that's not everything regex offers.

▲ogogmad 3 hours ago | parent | next [-]

Backreferencing inside of a regex (which is a way to find repeat occurrences of a substring) is a very controversial feature that some regex engines intentionally omit, while most others support it only grudgingly. Supporting backreferences opens up a big can of worms: Most regex engines will fall back to a slow, poorly optimised algorithm if they ever encounter backreferences.

Also, it's important for people not to confuse backreferences with capture groups. They look a bit similar, but capture groups are actually fine.

In fact, backreferencing is such a footgun that I don't see it as a possible regex feature. It's a PCRE feature - perhaps even the core distinguishing feature. While all regexes are PCREs, not all PCREs are regexes. It's simply not possible to create a consistently performant implementation of PCRE.

▲teiferer 3 hours ago | parent | prev [-]

There are hundred different flavors of what "regex" actually means. The basics are often the same but the extended features differ.

The foundation is in theoretical CS with deterministic and nondeterministic finite automata (DFAs and NFAs), and I found it easiest to teach regexes using DFAs since you can then just draw circles and arrows. Regex notation is just a different notation. It's a bit likeif you know piano and then learn another instrument, your brain can always refer back to the piano and it's all much easier.

Backreferences are not really "controversial" but just an extension which does not map to regular languages (those recognized by finite automata). That is, they make regexes "non-regular", so they are more a hack. Regex engines that want to support them need a fundamentally different approach, that's why some don't support them and in principle will never be able to.

▲ogogmad 2 hours ago | parent [-]

They are controversial because supporting them raises matching time to O(2^n) in the worst case - which is dangerous - while not supporting them enables matching time to be O(n) in the worst case. And a user might not be aware of that danger.

Also, I think it's misleading to mention NFAs and DFAs alone, like there's no deeper reason. The deeper reason is that:

  Regexes (or NFAs, or DFAs) classify exactly the properties of strings which can be checked in constant space.
It turns out that any such string property can also be checked in O(n) time, and in one single left-to-right pass.

Briefly returning to backreferences, supporting them makes constant space unachievable. They also make linear time unachievable, since matching them becomes NP-COMPLETE.

Also, no regex implementation actually uses DFAs. NFAs yes, but DFAs no. The point of NFAs is that they're easy to execute in constant space and linear time, while also making concatenation and Kleene star easy to implement. It seems much harder to implement concatenation and Kleene star using most other constant-space automata - like DFAs - which is what actually forced people to invent NFAs.

▲teiferer 2 hours ago | parent [-]

I think we are actually on the same page, but it is funny that we express things in exactly the opposite way as well as reach opposite conclusions. :)

It's not about being pedantic, quite the opposite It's intuitive to use circles and arrows as the mental model, instead of the cryptic regex syntax. Both have the same semantics. The quite visual representaton of the finite automata makes the meaning more approachable, not unlike the marble run illustration.

In contrast, bringing "constant space" and "O(n) matching" and NP-completeness etc into the discussion is all formally correct (and I'm a big fan of formally correct) but that's where things become pedantic from the common folk's POV and is also where you lose people who just want to understand how something works practically.

Therefore I'm saying, start withthe easy circles-and-arrows intuition and only go to the theoretical background (aka be pedantic) once people want to know more.

(Backrefs can't be expressed with circles-and-arrows which illustrates why they are something special.)

▲ogogmad 6 hours ago | parent | prev [-]

Really cool. Do you teach people the mechanics, like what the app shows? Because I wonder if that actually makes it harder. I'm wondering about where the hard bits are, given that it all boils down to just:

  "|", which means "or"
  ".", which means "any symbol"
  "?", which means "at most once",
  "*", which means "however many times"
  "+", which means "at least once"
Maybe encountering it next to PEG scrambles your brain.
▲froh an hour ago | parent [-]

add to that

- using groups "(" .. ")" in replacement "\"i

- character ranges "[" .. "]"

- character classes "[:" .. ":]"

- multiline match

- ignore case

- beginning / end of line / word

▲monster_truck 6 hours ago | parent | prev | next [-]

This is actually pretty satisfying, I wish it had sound. Maybe a dramatic follow cam for the balls

▲fwlr 5 hours ago | parent | prev | next [-]

Seems the input text box is limited to 301 characters.

▲Sarkie 4 hours ago | parent | prev [-]

Now I have 3 problems