I'm not sure if this is exactly what you're referring to, but apparently years ago there were exploits bundling JAR files into GIFs to sneakily have them executed by the Java browser plugin: https://en.wikipedia.org/wiki/Polyglot_(computing)#GIFAR_att...

Back in the day I wrote a PoC exploit for my employers app that abused an image upload api by embedding a jar file inside an svg as XXE which then got me RCE. Fun times.