I also hate drag and drop.

It'd be good if when you picked something it automatically added the border too - otherwise I think this won't work on short pages?

I have some hesitations though - relying on scroll as the positioning method means that if you don't have a fine-grained scroll method e.g. on desktop, if you have a non-continuous wheel, you'll need to rely on "drag and dropping" the scroll bar, which doesn't really improve things (and has its own issues if your page is very long).

I think I'd prefer something other than scroll-to-position. Like what about making gaps between items with a "drop" button? Or adding up/down arrow buttons to the "picked" element so that fine grained adjustments could be done?

Take inspiration from ghetto sim racing setups (which are also implemented by some middle click scroll versions) and use the mouse cursor y position relative to the picked object as scroll velocity instead of destination position. Change cursor to up/down. Needs some vertical space above the list though.