The ENIAC seems to be the first general purpose electronic digital computer. It wasn't stored program, though - no good memory devices. Plugboards and lots of rotary switches. Took hours to load a new program. Unrelated to Colossus.

The first machine to have it all was the Manchester Baby.[1] Now this really was sort of a descendant of Colossus, with some of the same people involved. It was mostly a test rig for the Williams Tube memory device.

Once there was something that could do the job of RAM, things took off quickly. Within two years there were quite a number of stored program electronic digital computer projects. Electronic arithmetic worked fine, but everybody had been stuck on the memory problem.

[1] https://en.wikipedia.org/wiki/Manchester_Baby

Several earlier electro-mechanical computers were closer to a modern computer than ENIAC from the point of view of program storage, as they had a clearly defined instruction set and the programs were stored on punched tape (taken from teletype machines).

With ENIAC, reconfiguring the computer for solving a new problem was done essentially in the same way as for an analog computer (or nowadays for an FPGA), by rewiring the connections between the arithmetic units, the storage registers and the control sequencers, so that ENIAC will solve the new problem when powered on.

The resemblance of ENIAC to an analog computer is not an accident, but its architecture has been conceived as an electronic substitute of the electro-mechanical analog computers known as "differential analyzers", which had been in widespread use both before WWII and during WWII, for computing solutions of systems of differential equations, which were found in various engineering problems, including in many of military importance.

On the other hand, Harvard Mark I had been inspired by Babbage's proposal for a digital computer with stored program, hence its architecture much closer to modern digital computers.

While ENIAC had an architecture inspired by the mechanical differential analyzers, for the schematics of its electronic arithmetic and register circuits it used some information from the designers of the earlier Atanasoff-Berry Computer, which was a special-purpose electronic computer for solving systems of linear algebraic equations, and which included even the first DRAM memory (the second DRAM memory will be the British Williams CRT).

Any good books to recommend on computer history?