Thanks for reposting! I'm the author of ATTN-11. Happy to answer any questions about the fixed-point arithmetic, the PDP-11 hardware, or the training process.

▲

functional_dev 7 hours ago | parent | next [-]

Incredible work! Fitting transformer into 32KB RAM is crazy

For those who read this project and do not know PDP-11 it could be hard to understand that working with these memory limits is difficult. Here is visual guide for PDP11 architecture - https://vectree.io/c/pdp-11-hardware-architecture

Thanks for this amazing project!

	▲	PaulHoule 7 hours ago \| parent [-]
		That PDP-11 was the most fun minicomputer of the late 1970s in my opinion. Growing up in NH about an hour north of Digital's HQ all sorts of schools from primary to secondary as well as museums had PDP-8, PDP-10, PDP-11 and later VAX machines. The PDP-11 had a timesharing OS called RSTS/E which could give maybe 10 people a BASIC programming experience a little bit better than an Apple ][. If you were messing with 8-bit microcomputers in 1981 you might think a 16-bit future would look like the PDP-11 but the 1970 design was long in the tooth by 1980 -- like 8-bit micros it was limited to a 64kb logical address space. Virtual memory let it offer 64k environments to more users, but not let a user have a bigger environment.

▲

dare944 6 hours ago | parent | prev | next [-]

Fun stuff! At one point I wondered about building something similar. But I lack the AI chops, and have too many other projects going on anyway.

I'm curious as to the type of memory in the 11/34. I also have a working PDP-11, an 11/05 with 32KW of actual core. I wonder what performance would be like with EIS emulation grafted in. Stunningly slow, I imagine.

Thanks for publishing this.

▲

McGlockenshire 4 hours ago | parent | prev [-]

Thank you for the inspiration, I now have a practical-impractical assembly project for my TI TMS99105A homebrew! The 64k barrier is a real pain.

	▲	rahen 3 hours ago \| parent [-]
		I also have a working design for a small Transformer on the original Game Boy. It has around 4000 parameters fitting in the 8 KB cartridge SRAM, where the "saved game" is the trained model. A TI-82 with its 32 KB of RAM would be even more comfortable.