This is pretty relelevant to a project I'm working on - a new web browser not based on Chromium or Firefox.

Web browsers are extremely complex, requiring millions of lines of code in order to deal with a huge variety of Internet standards (and not just the basic ones such as HTML, JavaScript and CSS).

A while ago I wanted to see how much of this AI could get done autonomously (or with a human in the loop), you can see a ten-minute demo I posted a couple of days ago:

https://www.youtube.com/watch?v=4xdIMmrLMLo&t=42s

The source code for this is available here right now:

http://taonexus.com/publicfiles/jan2026/160toy-browser.py.tx...

It's only around 2,000 LOC so it doesn't have a lot of functionality, but it is able to make POST requests and can read some Wikipedia articles, for example. Try it out. It's very slow, unfortunately.

Let me know if you have anything you'd like to improve about it. There's also a feature requests page here: https://pollunit.com/en/polls/ahysed74t8gaktvqno100g

Took a quick glance through the code, its a pretty decent basic go at it.

i can see a few reasons for slowness - you arent using multiprocessing or threading, you might have to rework your rendering for it though. You will need to have the renderer running in a loop, re-rendering when the stack changes, and the multiprocessing/thread loop adjusting the stack as their requests finish.

Second, id recommend taking a look at existing python dom processing modules, this will allow you to use existing code and extend it to fit with your browser, you wont have to deal with finding all the ridiculous parsing edgecases. This may also speed things up a bit.

Id also recommend trying to render broken sites (save a copy, break it, see what your browser does), for the sake of completion

If you're interested in Python web browsers, may I suggest you take a look at Grail?

https://grail.sourceforge.net/