Apples model to generate Gaussian splats from a single image. Takes about 30 seconds on an M1 Pro.

It falls apart once you move too much, but for a little side-wiggling or a second-eye view for VR, it's great. And looks a lot better than the old approach of depth map + vertex shaders that I use in https://github.com/combatwombat/tiefling. But ml-sharp has 2.6 GB weights, a bit too big to run in the browser.

ben7799 3 hours ago | parent | next [-]

Is this thing the basis of what Apple is doing in the Photos app in the most recent OS revisions?

It seems to work a lot quicker than 30s now on iDevices and Macs.

▲

RobotToaster 8 hours ago | parent | prev [-]

Any idea what kind of vram is needed to run this on Nvidia?

	▲	kridsdale1 8 hours ago \| parent \| next [-]
		It runs natively on the Photos app in Vision Pro, which shares 16GB of ram with GPU and CPU.
	▲	sovok 8 hours ago \| parent \| prev \| next [-]
		The Python process peaks at about 15 GB, so maybe a 24 GB GPU works. But others seem to have problems wih Nvidia: https://github.com/apple/ml-sharp/issues/84
	▲	mnorris 4 hours ago \| parent \| prev [-]
		The smallest Nvidia gpu I've run Sharp on is an Nvidia T4 which I think has 16 GB on the chip