I’ve been building a little Python tool to extract a MIDI drum track from an mp3 or other song file. Eventually I want to buy an e-drum kit and learn to play along with my favorite songs.

From what I can tell, we’re about even with the current state of the art in academic publications right now, at least on MDB. Runs faster than real time in pytorch on a few years old MacBook.

It even does a decent job creating easy/medium/hard versions from the original “expert” track. I compared ours to some charts from Rhythmverse and while we’re not as good as the official ones, we’re pretty darn close.

Spent some time trying to extract an instrument model for the drum sounds from the recording too, but I think that simply requires too much data for lots of the songs that I like where the mix is really dense with a ton of sound from distorted guitars and cymbals etc. Works a treat on super clean mixes like Back In Black though.

Next step is probably to create my own rhythm game, since my initial experience with Clone Hero and YARG left a lot to be desired.

That sounds cool. For my uni project 10 years ago I chose to use neural networks to detect guitar notes, but not by pitch, but to detect exactly which string/fret was played (because you can play same note on different strings). It worked okish with a very basic network, quite suprised. I made a small game you could play with your guitar.

Since then I wanted to develop further such a game, and now I'm almost ready to release https://ultimidi.com , check it out, it's not really a rhythm game, but my goal is to add more mini-games to it (including rhythm games) and more instruments (now has piano, but planning to add guitar, ukulele too, not sure if drums would work too).