If you're on Python and your problem is too big for a single regex, but too small for a full-blown parser like in the article, I can recommend the built-in re.Scanner[0]. It's a regex-based tokenizer[1] (so no recursive parsing) in the `re` module, that for reasons is completely missing from any official documentation.

You give it a pattern for each token type, and a function to process each match, and you get back a list of processed tokens.

Importantly, it processes the list in one pass and ensures the matches are contiguous, where a naive `re.findall` with capture groups will ignore unmatched characters. You also get a reference to the running scanner, so you can record the location of the match for reporting errors.

    import re

    # Initialize the scanner.
    scanner = re.Scanner([
      (r"\d+",   lambda scanner, token: ('COUNT', int(token))),
      (r"\w+",   lambda scanner, token: ('ANIMAL', token.removesuffix('s'))),
      (r"[,. ]",     None), # None -> skip token.
    ])

    # Scan a sample string.
    results, remainder = scanner.scan("45 pigeons, 23 cows, 11 spiders.")

    # Ensure we parsed the whole thing.
    assert not remainder

    print(results)
    # [('COUNT', 45),
    #  ('ANIMAL', 'pigeon'),
    #  ('COUNT', 23),
    #  ('ANIMAL', 'cow'),
    #  ('COUNT', 11),
    #  ('ANIMAL', 'spider')]

[1]: https://en.wikipedia.org/wiki/Lexical_analysis#Tokenization

Wow! This is a nice tidbit of information.

It is a nice addition to a weird region of “more than re” but “less than parsimonious”.

Usually these topics are mentioned with no overlap (at least in the narrow rabbit hole I went down last time I was interested in making regexes simple to read for humans.)

https://github.com/mikaelho/python-human-regex

https://github.com/vinta/awesome-python#text-processing