Waypoint-1: Real-Time Interactive Video Diffusion from Overworld

roskelld 8 hours ago | parent | next [-]

The context seemed to last a few seconds. I went from a mock up screenshot of a fantasy video game, complete with first person weapon. Then as I moved forward the weapon became part of the scenery and the whole world blurred and blended until it became some sort of sci-fi abstract space. Spinning the camera completely changed look and style.

I ended up with a UI that closely resembled the Cyberpunk 2077 one complete with VO modal popup. I guess it must have featured a lot in the training data.

Really not sure what to make of this, seems to have no constraints on concept despite the prompt (I specifically used the word fantasy), no spatial memory, no collision, or understanding of landscape features in order to maintain a sense of place.

▲

avaer 7 hours ago | parent [-]

Accurate to my experience hacking on this model today, but I don't think anyone's blowing smoke about it.

Thinking back to where GPT-3 was 5 years ago, I can't help but be a little bit excited. And unlike GPT-3 this is Apache.

	▲	Grimblewald an hour ago \| parent [-]
		I'd put this closer to gpt2 tbh. GPT3 was already quite impressive and functional. We haven't come particularly far since imo. More small noticable steps, but no significant jumps.

▲

ecmulli 4 hours ago | parent | prev | next [-]

I dont have a big enough GPU but I was able to play around with the model using this plugin https://github.com/daydreamlive/scope-overworld via Runpod - very cool!

	▲	cmuir 3 hours ago \| parent [-]
		Wow, very cool. Starring now.

▲

lcastricato 6 hours ago | parent | prev | next [-]

BTW, there is a gradio space here:

https://huggingface.co/spaces/Overworld/waypoint-1-small

And our streamed version:

https://overworld.stream

▲

avaer 7 hours ago | parent | prev | next [-]

If you think this is cool you might also be interested in https://github.com/MineDojo/NitroGen which is kind of the opposite (and complimentary).

▲

Plankaluel 8 hours ago | parent | prev | next [-]

An RTX 5090 for 20-30fps for the small model: That is not as unreasonable as I had feared :D

▲

lcastricato 7 hours ago | parent | prev | next [-]

Hi,

Louis here. CEO of overworld. Happy to answer questions :)

▲

rcv 30 minutes ago | parent | next [-]

Looks like your login is busted. I get the following when trying to log in with Google or Github:

``` { "code": "REDIRECT_URL_NOT_WHITELISTED", "error": "Redirect URL not whitelisted. Did you forget to add this domain to the trusted domains list on the Stack Auth dashboard?" } ```

▲

anotheryou 5 hours ago | parent | prev | next [-]

Wouldn't a little google maps style navigation solve latency mostly?

Project on to a sphere, crop a little bit, do onset of motions by rotating or moving in the sphere

▲

dsrtslnd23 6 hours ago | parent | prev [-]

great work! Will the medium model be also open/apache-licensed?

	▲	lcastricato 6 hours ago \| parent [-]
		Medium is going to bc cc by sa nc 4.0. We may reevaluate in the future and make it more lenient. Small is meant to be the model for builders and hackers.

▲

dsrtslnd23 6 hours ago | parent | prev | next [-]

10,000 hours training data seems quite low for a world model?

▲

lcastricato 6 hours ago | parent [-]

60fps training data goes a long way ;)

	▲	echelon 5 hours ago \| parent [-]
		You guys have my support. I'll pay you when you open up payments. We need open source world models.

▲

khimaros 8 hours ago | parent | prev [-]

this is like an open weights version of DeepMind's Genie