Remix.run Logo
wronglebowski 5 days ago

The live demo of this is brutal. https://x.com/ns123abc/status/1968469616545452055

llmthrow0827 5 days ago | parent | next [-]

All the VR/AR/XR demos are so insanely trivial and yet still manage to be much more difficult than current methods of doing things. Like, really, cooking?

Normal method:

* Search for a recipe

* Leave my phone on a stand and glance at it if I forget a step

Meta glasses:

* Put glasses on (there's a reason I got lasek, it's because wearing glasses sucks)

* Talk into the void, trying to figure out how to describe my problem as well as the format that I want the LLM to structure the response

* Correct it when it misreads one of my ingredients

* Hope that the rng gods give me a decent recipe

Or basically any of the things shown off for Apple's headset. Strap on a giant headset just so I can... browse photos? or take a video call where the other person can't even see my face?

hdjrudni 5 days ago | parent | next [-]

I dunno, if these worked perfectly I don't think it'd be awful to be able to open my fridge and say "what can I make with this" and it could rattle of some suggestions based on my known preferences and even show me images in their new display.

Hands-free while cooking (not having to touch my phone with messy hands) is not a bad thing either.

vasco 5 days ago | parent | next [-]

I touch my phone with messy hands all the time. They are water resistant now, just wash it after

mewpmewp2 5 days ago | parent [-]

I think more so I feel like after touching the phone I should really wash my hands before touching the food or doing anything food related etc.

vasco 5 days ago | parent [-]

Yeah but I already live by the 5 second rule anyway so I'm more careless, you do have a point though, it's less hygienic for sure.

mewpmewp2 5 days ago | parent [-]

Yeah, I care less myself, and I'd probably believe I was training my immune system, but my partner would kill me though.

hattmall 5 days ago | parent | prev | next [-]

It sucks now, no idea why, but a few years ago, with the Google Home mini, I could just yell out all kinds of cooking related questions with "Hey Google" and it would always give me a good answer, was great for doing stuff hands free when cooking, like when I just don't want to get raw chicken or whatever on my phone.

But yeah, it doesn't give me good answeres any more, usually trys to start an unrelated YouTube video or email me something about Youtube plus or w/e

aembleton 5 days ago | parent | prev | next [-]

I suppose thats a bit easier than reading it out to ChatGPT.

croes 5 days ago | parent | prev [-]

But your $800 glasses are exposed to the cooking area with steam, grease fumes, heat etc.

jayd16 5 days ago | parent [-]

So wipe it. It's not like it's got an air intake.

croes 5 days ago | parent [-]

But microphones and speakers. And what about the cooling of the chips?

jackbrookes 5 days ago | parent | prev | next [-]

This reads a bit like like a pre-PC take: "Why use a computer when a cookbook works fine?"

Imagine it’s 1992:

Cookbook: Open book, follow steps.

PC: Turn on tower, wait for DOS, fiddle with floppies, pray the printer works, hope the shareware recipe isn’t weird.

Not saying you're wrong but its easy to miss the big picture

aembleton 5 days ago | parent | next [-]

> "Why use a computer when a cookbook works fine?"

I still feel that way. I have cookbooks because I find the UX better than searching for recipes.

wpm 5 days ago | parent [-]

So I can read the 20,000 story about how the author was told this recipe by their brothers husbands step-grandmother while vacationing at the lake house with their golden retriever named Max before I can get to the recipe.

atrus 5 days ago | parent [-]

While this joke is never mentioned and is hilarious every time, you'd be hard pressed to find a recipe site that didn't have either a "print" or "go to recipe" button at the top.

pavel_lishin 5 days ago | parent | prev | next [-]

Right, but we're in the 1992 of these glasses. Maybe they'll be good eventually. They aren't now.

And frankly, even the online recipe experience leaves much to be desired. Skip past the blog post. Skip past the list of ingredients. Skip past another blog post. Find the single statblock on the bottom that lists ingredients & amounts, & instructions - hoping that it exists.

Like other commenters, I've also started going back to paper cookbooks.

rs186 5 days ago | parent | prev | next [-]

Not the same.

Internet and recipe websites solve a real problem: accessing recipes was expensive and not easy

AR headsets don't solve any problems. If anything, they make up a nonexistent problem, attempts but fails to solve the problem, during which the experience becomes even worse.

aylmao 5 days ago | parent [-]

I mean, depends on how you describe it. One could easily say:

Phone method:

* Find phone

* Search for the right app, before finding the right recipe

* Leave my phone on counter, constantly having to move it as I move plates, pans etc.

* Wash and dry hands after each step, before unlocking phone

* Clean it every time gunk gets to it

Meta glasses:

* They're already on, just ask for recipe

* No need to ever wash/dry hands, move a device around, or clean it since one can easily unlock it without touching it

Right? Similarly with cookbooks, the best case is great and the worst case is terrible. There's a reason there's a market for recipe websites, cookbooks, etc.

48terry 3 days ago | parent | prev | next [-]

Okay. Now: Imagine it's 2025:

Cookbook: Open book, follow steps.

New gadget from mult-billion dollar company: showcases on a live demonstration that it's a broken piece of crap that doesn't work.

Like, are we forgetting that it didn't work? It sucked at the job! Let's not what-if or have some imaginary "okay, but pretend it's actually good," deal here. It was bad!

0x457 4 days ago | parent | prev | next [-]

No? Because traditional cookbook (paper or digital) is deterministic and LLMs are not.

endymion-light 5 days ago | parent | prev [-]

honestly cookbooks genuinely are better

i got the art of italian cooking recently and it's genuinely far easier to get a recipe than trying to scroll through a 50 page monologue about the intracicies of someones childhood before even listing the ingredients

rhetocj23 5 days ago | parent | next [-]

Indeed. There is an element of trust with an actual cookbook - it signals quality.

The internet over time has been riddled with junk, especially since the cost of production of information is just your opportunity cost of time. Even that is going away with the use of LLMs....

endymion-light 5 days ago | parent [-]

Core issue within the content age that I don't see being readily resolved. Unfortunately, I think the SEO marketing crowd are slowly catching up with LLMs, which is leading to poorer actual output when attempting to get information.

In the same way that google search used to be amazing before it was taken over by optimization, I think we're seeing a mass influx of content production to attempt to integrate itself into training corpus.

rhetocj23 5 days ago | parent [-]

TBH I for one am glad about this.

I have always believed there is a cost borne to get the best of something. This means a sacrifice is entailed. Theres something very important about this re. the culture - a culture in which everything is free is how you get crap stuff produced. And people settle for crap stuff just because its free.

People who can see the bigger picture when you have this, can see the dangers of it.

makeitdouble 5 days ago | parent | prev [-]

To note, you can buy the recipes and skip the dumpster internet or register to a site like cookpad. At this point even YouTube is a decent place for that.

I agree random recipes are hell on the internet, but it's also not something we're forced into if we care any bit about recipes in the first replace.

twalichiewicz 5 days ago | parent | prev | next [-]

Watching the announcement, every feature felt like something my phone already does—better.

With glasses, you have to aim your head at whatever you want the AI to see. With a phone, you just point the camera while your hands stay free. Even in Meta’s demo, the presenter had to look back down at the counter because the AI couldn’t see the ingredients.

It feels like the same dead end we saw with Rabbit and the Humane pin—clever hardware that solves nothing the phone doesn’t already do. Maybe there’s a niche if you already wear glasses every day, but beyond that it’s hard to see the case.

Gareth321 5 days ago | parent | next [-]

If executed well I think this could reduce a lot of friction in the process. I can definitely unlock my phone and hold it with one hand while I prepare and cook, but that's annoying. If my glasses could monitor progress and tell me what to do with what while I'm doing it, that's far more convenient. It's clearly not there yet, but in a few years I have no doubt it will be. And this is just the start. With the screens they'll be able to offer AR. Imagine working on electronics or a car and the instructions are overlaid on the screen while the AI is providing verbal instructions.

01100011 5 days ago | parent | prev [-]

I'm oldish, so maybe I'm biased, but this sort of product seems like something no one will want, outside a few technophiles, but that industry desperately needs you to want. It's like 3d TV, a solution in search of a problem because the mfgs need to make the next big thing with the associated high margins.

To me the phone is a pretty good form factor. Convenient enough(especially with voice control), unobtrusive, socially acceptable, and I need to own one anyway because it's a phone. I'm a geek so I think this tech is cool, but I see zero chance I would use one, even if it were a few steps better than it is.

5 days ago | parent | prev | next [-]
[deleted]
hombre_fatal 5 days ago | parent | prev | next [-]

On the other hand, having to constantly consult a recipe on my phone while I cook is the main quality of life aspect of home cooking that could be improved.

You're missing the part where I'm reminded that my phone autolocks so I have to go into the settings in the middle of cooking to make it never autolock (or be lazy and unlock it every time I need it). And then I have to find a clean knuckle to scroll the ingredient list and the recipe steps every time I'm trying to remember what step I'm at.

You could do some killer recipe UX with a HUD on some glasses.

SchemaLoad 5 days ago | parent | prev | next [-]

These companies are reaching really hard for use cases while ignoring the only ones VR actually works well for. If they just went all in on gaming it would be a much better product than trying to push AI slop cooking help.

bayarearefugee 5 days ago | parent | next [-]

As a gamer, in my experience people don't want to play VR games either.

Beat Saber as a social party experience with friends in the same room, sure, that's fun... but for day to day gaming the amount of people who want to play VR games on the regular is nearly zero.

If they really want to lean into the VR use case that people want, its porn, but I suspect they won't put that front and center.

swalsh 5 days ago | parent [-]

I LOVED VR gaming, but after playing the same 2 games for 10 years, it never really evolved. They stopped innovating and went all in on AR.

SchemaLoad 5 days ago | parent | next [-]

I had a HTC Vive and I really loved playing VR games, particularly a shooter called Pavlov. Felt pretty social with a ton of absurd custom maps where the actual game was almost secondary to experiencing the immersive and strange maps.

But since I moved I didn't want to screw the base stations in to the walls again and haven't played in a long time. I feel like I probably still would like VR gaming but haven't been tempted enough to buy any of the newer systems since it seems like Meta has fully captured the market and it all seems pretty distasteful now.

tsimionescu 5 days ago | parent | prev [-]

I think you're very much in the minority. Also, VR games didn't really evolve because it can't really evolve - the fundamental thing that makes it attractive (immersion in a digital space) can't work well because of motion sickness. So, the only way to make an immersive VR game is to have an extremely tiny game world or an on-rails experience, and that drastically reduces the appeal.

Of course, you could make all sorts of traditional top-down or isometric games work well without motion sickness - but no one is going to pay for VR to play Civilization or Star Craft or Baldur's Gate 3, since these would be fundamentally the exact same experience as playing on PC or console, but with a display strapped to your head.

swalsh 5 days ago | parent | next [-]

> can't work well because of motion sickness.

This is an overated problem. You play VR for a small amount of time then you adapt to it. You get your "VR Legs" as they say.

xdfgh1112 5 days ago | parent | prev [-]

This is such nonsense. The new Batman game on VR has full motion and smooth turning. It's not on rails at all. Games have got better at reducing motion sickness, and players also adapt over time.

dontlaugh 5 days ago | parent | next [-]

The many of us who get motion sickness have simply stopped bothering with VR. Since the market has shrunk anyway after the initial excitement, the few VR games left can afford to be less accessible.

isoprophlex 5 days ago | parent | next [-]

Indeed. I put on any kind of VR helmet for more than 2 minutes and I'll be queasy and/or throw up outright. My level of motion sickness is maybe extreme... but i guess that definitely messes with the total addressable market.

xdfgh1112 5 days ago | parent | prev [-]

Yeah I appreciate that. There are things like vignetting that can help and newer games do them. But some people will never be able to play them.

DonHopkins 5 days ago | parent | prev [-]

They adapt to the taste of their own vomit? Or mitigate it by drinking lots of chocolate milk before playing?

swalsh 5 days ago | parent [-]

Your brain just learns to understand it's in VR, and then it feels normal.

DonHopkins 3 days ago | parent [-]

It's the part about getting my brain to learn to enjoy the taste of puke that I'm having trouble with.

thepryz 5 days ago | parent | prev | next [-]

In my experience, the biggest obstacle to broader AR and VR adoption beyond reducing the price, size, and weigh of the hardware will always be the lack of good content creation tools.

I've been involved with two VR projects that were ultimately cancelled because, while we developed a sexy tech demo that showed the potential, building things out into something sustainable required too many resources and took too much time to maintain.

bee_rider 5 days ago | parent | prev | next [-]

VR gaming seems like it is a bit of a niche, though. I think they want to sell glasses in quantities more like cellphones than gaming peripherals.

I agree they are reaching (and not finding) for an application.

ubb_server 5 days ago | parent [-]

I agree that VR gaming is a niche, but I think it could be explosively improved if we had the kind of all-in idealism that the previous commenter referred to. I think because VR gaming IS niche, we haven't yet delved into what VR/AR could do in non-gaming.

An idea that I've had before is like 'augmented curated experiences' for all kinds of things--for example imagine playing a Magic the Gathering (or similar) card game, and watching your cards come to life on the board in hologram-esque 3D. Or while watching a sports match, being able to pull up the stats or numbers of any players, or flip through channels of POV camera from helmets. Car navigation that shows you what turns to make by augmenting lanes or signs with highlighting. Brick and mortar stores having a live wayfinding route to products in their store based on your grocery list, recognizing and highlighting items you like.

tsimionescu 5 days ago | parent | next [-]

> for example imagine playing a Magic the Gathering (or similar) card game, and watching your cards come to life on the board in hologram-esque 3D

This is the kind of thing that buries VR ideas. It's very cute in a demo, but as an actual product, the cost of coming up with 3D models and animations for all MTG cards currently being played is many orders of magnitude more than the total number of people who would pay for this. Ultimately this is completely unnecessary fluff for the game, like chess games where the pieces actually fight: irrelevant, and it actually detracts from the game because it interrupts the flow of what you're actually doing.

jdprgm 5 days ago | parent | prev | next [-]

I remain convinced VR gaming is niche because despite these companies being willing to drop boatloads of money on all kinds of things they for some reason never decided to just allocate a few billion to create a handful of true AAA games and jumpstart the industry. I think even just 3 proper games with several hundred mil budgets and VR gaming might be in an entirely different space than it is now.

xdfgh1112 5 days ago | parent | next [-]

Facebook made a very expensive new Batman game in VR, there's also Resident Evil, Assassin's Creed, a ton of other high budget games like Red Matter.

It just isn't taking off. In my experience even though VR is unique and amazing, it's not that much better than playing those games flat screen. I tend to spend most of my time in Beat Saber.

jdprgm 5 days ago | parent [-]

Expensive in the context of other VR games sure. I couldn't find any official numbers but i'm sure it pales in comparison to dozens of other games that came out this year.

Also i'm not sure what these single player relatively short playtime/runtime games accomplish as you buy it play it in less than a week and are done. What I would like to see is the large scale infinitely playable MMO type game done on VR with at least at 250M budget.

tsimionescu 5 days ago | parent | prev | next [-]

I think this is extremely doubtful. The reality remains that it's impossible to make a first person or even third person VR game with free movement, because of fundamental limitations in how human brains process movement. Having your eyes tell you are moving but your muscles and inner ear tell you that you are not makes you extremely sick very quickly, and technology can't actually fix this. The better and more immersive the visual illusion of movement, the worse the movement sickness you'll experience.

And without free movement, you can't build any of the mainstream game genres. You can't build and get people excited in a Call of Duty or Assassin's Creed or Fortnite or Elden Ring or Zelda where movement works like Riven, the sequel to Myst. Valve actually tried with the first Half-Life game in a decade, and even that didn't work.

Add to this massive gameplay limitation the second massive issue that you can't get a mass audience to pay hundreds of dollars extra for a peripheral without which they can't play your 70-80 dollar game.

WhiteNoiz3 5 days ago | parent | next [-]

> Valve actually tried with the first Half-Life game in a decade, and even that didn't work.

Half Life Alyx is still considered to be one of the best VR games ever made and one that is still consistently recommended to new users even years after release. IMO people buy hardware because of the exclusive content. If a standard game console came out and it only had one AAA game on it, I probably wouldn't bother buying it. But if there were 3-4 games that looked really interesting it starts to look more worth the investment. Playing VR games takes a lot of committment (time / physical space / $$$) so the payoff has to be worth it or you'll lose people. With the huge amount of money spent on R&D for new hardware I think it's a valid argument to say that maybe funding content would have been a better investment in terms of ensuring platform growth.

Also, side note but not every game requires free motion. Plenty of hits had no movement or teleport etc. A lot of these were completely new (sub-)genres that didn't exist or hit the same as they would in a traditional pancake game. Plus lots of kids seem unaffected by free movement (maybe as high as 50% of users by my rough estimate).

xdfgh1112 5 days ago | parent | prev [-]

Those games literally exist now. Almost all new VR games use free movement not teleportation. It is frustrating that you seem to be talking confidently when your knowledge is 5 years out of date.

swalsh 5 days ago | parent [-]

10 years out of date. Free motion has been the norm for indie games since HTC vive. The bigger studios kept using teleportation because that was the "best practice" gamers got their VR legs and preferred free motion.

bee_rider 5 days ago | parent | prev [-]

Maybe a really high budget VR shooter game could be successful, I don’t know.

I played some VR sword-fighting games and they were bad in a way that AAA budgets would not fix. Stuff like an attack animation being pre-scripted feels incredibly goofy in VR.

I think this is a general problem. VR worlds need to be more dynamic than typical games. AAA games tend to have higher quality assets, but arranged in a more restrictive and scripted configuration. More innovative indie work is needed to work out what the language of VR should be (it is a bit weird compared to the past because stuff like Quake was innovative, AAA-equivalent for the era, but also small and independent enough to be innovative).

Aeolun 5 days ago | parent | prev [-]

We should re-watch Dennou Coil every few years to be reminded of what we’re working towards :)

jayd16 5 days ago | parent | prev | next [-]

Well it's clearly a first gen product. They could ship Snake and Tetris on it, probably, but I'm certain they're thinking about how to get apps and games on it.

intrasight 5 days ago | parent | prev | next [-]

> the only ones VR actually works well for

I had really expected a different "only one"

numpad0 5 days ago | parent | prev | next [-]

No offense, but there it this chart, and what this tells me, maybe just me, is that gaming is a niche within VR, not even majority use case. Zuck is probably right about VR/AR being the next big social media, only he's wrong that it'll be like Facebook/Instagram type of social media; it's old Twitter type of social media.

[1]:

Most played VR games

  Rank Name          Curr   24h pk All-time
  1.   VRChat        33,032 46,652  66,824
  2.   War Thunder   26,388 65,589 121,318
  3.   PAYDAY 2      23,513 31,619 247,709
  4.   No Man's Sky  22,509 46,010 212,613
  5.   OBS Studio    11,434 22,388  27,334
  6.   Phasmophobia   7,716 22,789 112,717
  7.   Forza Hz 5     4,940 13,617  81,096
  8.   Assetto Corsa  3,885 13,598  19,796
  9.   OVR Adv. Sett. 3,030  4,299   6,418
  10.  Tabletop Sim.  2,902  7,755  37,198
1: https://steamdb.info/charts/?tagid=21978
croes 5 days ago | parent [-]

To me the chart shows that VR is mainly used for games. And the steam chart don't include the games played directly on the Quest headsets.

numpad0 5 days ago | parent [-]

That's certainly one useful spin, but the red flag here is that these don't correlate well with games known as best VR games to VR communities. What I believe to be a more accurate interpretation is, there's nothing but VRChat in VR, and gaming demand in VR can be ~10x smaller per title relative to it.

tempodox 5 days ago | parent | prev [-]

Games are not a prolific spy tentacle for hoovering up all kinds of data. They may have changed their name, but this is still the facebook company.

jayd16 5 days ago | parent | prev | next [-]

Voice input is just too annoying but with the display and wristband I think the dream is there. Your hands are deep in messy food prep, you have a recipe up, you can still pause your music or take a call with the wristband and without stopping to wash up or getting oil or batter on everything.

dyauspitr 5 days ago | parent | prev | next [-]

I wear my glasses all the time. If I could just talk to the void and get help with things I’m directly seeing reliably that would be a game changer. I’ve used Gemini’s video mode and we’re not all that far away.

rhetocj23 5 days ago | parent | prev [-]

People dont realise how amazingly efficient touch interfaces already are.

THere is no need for these stupid glasses. Some refuse to accept it - especially Zuckerberg who relies on folks like Apple to make his money. Thats really whats driving this project if you tear away all the BS.

zmmmmm 5 days ago | parent | prev | next [-]

If you watch it carefully, he preempts the AI with "What do I do first" before it even answered the first time. This strongly suggests it did this in rehearsal to me and hence was far more than just "bad luck" or bad connectivity. Perhaps the bad connectivity stopped the override from working and it just kept repeating the previous response. Either way it suggests some troubling early implications about how well Meta's AI work is going to me, that they got this stuck on the main live demo for their flagship product on such a simple thing.

daemonologist 5 days ago | parent | next [-]

I think preempting the AI the first time was meant to be a feature (it's not trivial to implement and is something people often ask for). Failing from there definitely wasn't great, although it's kind of what I'd expect from an(y) LLM.

WD-42 5 days ago | parent [-]

No, he preempted it because it was about to list all the ingredients necessary to make a steak sauce, despite having them in front of him. These are glasses, it should have skipped that part and went straight to what to do first.

exitb 5 days ago | parent | prev | next [-]

The way he clung to „what do I do first” makes me think that the whole conversation was scripted in the prompt and AI was asked to reply in specific way to specific sentences. Possibility not even actually connected to the camera?

whywhywhywhy 5 days ago | parent | next [-]

Yeah as a fully integrated system and the selling point I'd expect you'd say something like "Look again I think you're getting ahead of yourself".

Maybe the tech wasn't quite fool proof and they tried to fake it and then the fake version messed up.

klabb3 5 days ago | parent | prev [-]

I distrust meta (and hate these voice assistants) as much as the next guy but to me it’s obvious that you would prepare the prompt and use pretty much the exact phrasing. Also, repeating yourself is normal if there’s no response at all. If it was truly all fake why not just cheat outright and just prerecord all of it?

mrandish 5 days ago | parent | prev | next [-]

> Either way it suggests some troubling early implications about how well Meta's AI work is going

I fully expect the AI to suck initially and then over many months of updates evolve to mostly annoying and only occasionally mildly useful.

However, the live stage demo failing isn't necessarily supporting evidence. Live stage demos involving Wifi are just hard because in addition to the normal device functionality they're demoing, they need to simultaneously compress and transmit a screen share of the final output back over wifi so the audience can see it. And they have to do all that in a highly challenging RF environment that's basically impossible to simulate in advance. Frankly, I'd be okay with them using a special headset that has a hard-wired data link for the stage demo.

bauruine 5 days ago | parent | next [-]

I assume you couldn't watch the video because it's just a live stream of a guy standing in a kitchen and talking to his glasses. He's not on the stage with hundreds of people on the wifi and you can't see what the glasses are displaying at all.

mrandish 4 days ago | parent [-]

The link in this thread to the live glasses live demo is of Zuckerberg at FB Connect. The "fail" is when someone repeatedly tries to call the glasses he is wearing on stage. The person calling apparently has no trouble making the in-bound calls but the glasses Zuckerberg is wearing on stage fail in successfully answering the call. And the streamed video clearly shows the interface of Zuckerberg's glasses full-screen, as well as showing that the interface is being sent to the stage screen so the live audience can see it.

So, the failure was apparently with the glasses Zuckerberg's wearing on stage not establishing a two-way video call while simultaneously streaming it's own interface for the live stream and big-screen. He said it worked dozens of times in rehearsal and one notable difference was that for the real demo hundreds of other wifi devices present in the room.

I have quite a bit of experience producing live keynote demos at large tech events, so I don't think I've confused about this. As an aside, when we're being shown "Zuckerberg's POV" through the glasses I believe that's actually something custom put together for demos because the normal glasses don't even have a mode which shows the wearer's POV. Creating that view requires sending both the internal output of the glasses, which is the corner inset overlay AND the full screen output of the glasses live camera - which are then being composited together backstage to create the combined image we see representing what Zuckerberg sees. Sending all of that while establishing a two-way video call is a lot for a resource constrained mobile device.

hattmall 5 days ago | parent | prev [-]

I run multiple live streams from speakers to conference rooms and other bandwidth intensive offerings throughout the day in an incredibly crowded RF space. WiFi is certainly up to the task. Meta is a nearly 2 Trillion dollar company a failure of this order is ridiculous.

5 days ago | parent | prev [-]
[deleted]
explorigin 5 days ago | parent | prev | next [-]

I've done live demos of AI. Even with the same queries, I got a different answers than my 4 previous practice attempts. My demos keep me on my toes and I try to limit the scope much more now.

(I didn't have control over temperature settings.)

danjc 5 days ago | parent | next [-]

It looks like true 0-temperature (i.e. determinism) will happen. Here's some good context: https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

ayewo 5 days ago | parent | next [-]

HN discussion https://news.ycombinator.com/item?id=45200925

FergusArgyll 5 days ago | parent | prev [-]

But 0 temp is much less "Creative" and may not be conducive to showing off the AI's latest tricks

explorigin 5 days ago | parent [-]

True. It depends on the feature you're demoing...but determinism is a VERY DESIRABLE feature for giving demos.

hdjrudni 5 days ago | parent | prev [-]

> (I didn't have control over temperature settings.)

That's...interesting. You'd think they'd dial the temperature to 0 for you before the demo at least. Regardless, if the tech is good, I'd hope all the answers are at least decent and you could roll with it. If not....then maybe it needs to stay in R&D.

danpalmer 5 days ago | parent [-]

Reducing temperature to 0 doesn't make LLMs deterministic. There's still a bunch of other issues such as float math results depending on which order you perform mathematically commutative operations in.

riffraff 5 days ago | parent [-]

I keep reading this but I don't get it: for the same input shouldn't the order of resulting operations be deterministic too?

NitpickLawyer 5 days ago | parent | next [-]

It gets more complicated with things like batch processing. Depending on where in the stack your query gets placed, and how the underlying hardware works, and how the software stack was implemented, you might get small differences that get compounded over many token generations. (vLLM - a popular inference engine, has this problem as well).

danpalmer 5 days ago | parent | prev | next [-]

Not necessarily. This is a good blog post from a few days about it: https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

riffraff 4 days ago | parent [-]

Fantastic article, thanks!

bschwindHN 5 days ago | parent | prev | next [-]

Previous discussion:

https://news.ycombinator.com/item?id=19567011

And a quora link (sorry):

https://www.quora.com/If-floating-point-addition-isnt-associ...

HDThoreaun 5 days ago | parent | prev [-]

Associative property of multiplication breaks down with floating point math because of the error. If the engine is multithreaded then its pretty easy to see how ordering of multiplication can change which can change the output.

santiagobasulto 5 days ago | parent | prev | next [-]

This one was also pretty bad: https://x.com/jason/status/1968496622884495847?s=46&t=9d1Ha4...

I think there’s some respect to give cause they’re doing it live and non-scripted.

jansan 5 days ago | parent | next [-]

Non-scripted? You must be kidding.

_1 5 days ago | parent [-]

I take it they meant pre-recorded. It was definitely scripted and practiced.

whywhywhywhy 5 days ago | parent | prev [-]

Respect for trying it live now Apple just does pre-recorded with a ton of VFX.

TIPSIO 5 days ago | parent | prev | next [-]

If you’ve ever used the current Meta Ray Ban and AI, this almost exactly happens when the connection is bad. Pure confusion but the AI still tries to give you an answer.

I bet the device hardware is small/cheap and susceptible to interference

stavros 5 days ago | parent | next [-]

I have the Meta glasses and I've never noticed this, and don't even understand why it could be the connection's fault. The AI gets your audio and your image, if it gives the wrong answer, it's because the AI went wrong. How would the bad connection ever affect it?

vunderba 5 days ago | parent | next [-]

Exactly. Like... what are they even saying here - that if the connection drops then it falls back to a tiny "dropped on their head as a child" 4b parameter LLM embedded in the physical firmware and so that's why it is giving inane responses?

Mad props to the presenter for holding it together though.

dmbche 5 days ago | parent | prev [-]

The ai is in the cloud

Edit0: ie without internet access the ai is unable to produce an answer other than some prerecorded ones I guess

In the live showcase the presenter even mentions that the wifi must have been bad for the ai to repeat the answer

stavros 5 days ago | parent [-]

You're saying "you've already used the first two ingredients, so go ahead and add the sauce" is the prerecorded response when it doesn't have a connection?

dmbche 5 days ago | parent | next [-]

No, that's the last queried answer. There is no ai in the glasses without a connection, so all it (edit1:it here being the program being run on the glasses, client to the ai between other things)can do (seemingly) is loop around and re-read the last queried answer, which was the mistaken "you've already...".

In the glasses is just a client to the ai. Like there is no ai in your phone when you talk to chatgpt, you are querying it and it will not keep talking to you if you cut off the wifi

The prerecorded responses I speculated about would have been things like "i'm having some connectivity problems, I'm unable to chat at this time, I'll let you know when I'm back." - the same kind of prerecorded things your earbuds tell you when they're low on power.

tsimionescu 5 days ago | parent | next [-]

This can't possibly be the case, because the AI voice says slightly different things between the two attempts. The first time it says "you've already combined the base ingredients, so now grate a carrot to add to the sauce"; while the second time it says "you've already combined the base ingredients, so now grate the carrot* and gently combine it with the base sauce".

Unless you think they've added some inference logic on the device to slightly re-state the last answer they got from the cloud, it's clear that the glasses were connected and receiving the same useless answer from the cloud.

* side note, but it can also sound like "pear" to me this second time

dmbche 5 days ago | parent [-]

Oh could very well be the case I've only listened once!

stavros 5 days ago | parent | prev [-]

If you believe that they made the glasses repeat the last answer when they don't have connectivity, instead of saying "I don't have connectivity", I don't know what to tell you.

I own a pair of Meta glasses, and the response when they don't have connectivity is "this function is not available at this time".

dmbche 5 days ago | parent [-]

Isn't this a very odd discussion to keep going? I'm not sure why you're being so confrontational as well. I see you have a lot of points, is that a way to drive engagement?

WD-42 5 days ago | parent [-]

Are you a bot? Also "It must be the wifi" has got to be the lamest, unimaginative, predictable demo failure excuse I've ever heard, and you're trying to defend it.

dmbche 5 days ago | parent [-]

yes, i am a bot, and i'm paid by meta to convice you to buy their glasses by telling you they are shit? what are you on about?

Edit0: and what are you even doing? Where do you think this is going?

zmmmmm 5 days ago | parent | prev [-]

the thing is, if it loses the connection, why on earth would the correct behaviour be to just keep repeating the last response? It should just straight up say, "Sorry I'm having trouble connecting". Even the best case scenario here suggests terrible product design.

dmbche 5 days ago | parent [-]

Hard agree on terrible. I guess i'd have disabled the no connectivity message for the demo to give it a chance to reconnect gracefully/quickly if at all (by non stop querying even without wifi) but that's just guessing on my part. I think they're garbage and same for meta, if that needs saying

m3kw9 5 days ago | parent | prev | next [-]

next time they need 1 public and 1 private router and shut the public off right before the demo.

krustyburger 5 days ago | parent | prev [-]

Even if it’s small/cheap, if the item is scanned multiple times this will prevent any electrical infetterence.

chatmasta 5 days ago | parent [-]

I don’t even think that’s a word!

losvedir 5 days ago | parent | prev | next [-]

Ouch. Kudos for trying, though. I miss the days of live demos at Apple events, instead of all these polished videos of people standing in silly poses around the Apple campus.

303uru 5 days ago | parent | prev | next [-]

It’s the WiFi, ya sure.

klabb3 5 days ago | parent [-]

Yeah I was also cringing at that cop out. It doesn’t appear connectivity related. Plus even if it was, it beautifully highlights the connectivity requirement which sucks for so many reasons.

HaZeust 5 days ago | parent | prev | next [-]

I have mad respect to them for actually attempting this on the fly - especially a public company. Nothing really to gain versus a scripted demo, and absolutely everything to lose. Admirable.

xandrius 5 days ago | parent [-]

Obviously scripted, just the LLM didn't follow its part of the script.

anal_reactor 5 days ago | parent | prev | next [-]

Hearing this AI-generated voice awakens some primal aggression in me.

klik99 5 days ago | parent | prev | next [-]

This is why Jobs spent months prepping for each presentation.

But hey, at least it's not all faked

gretch 5 days ago | parent | next [-]

When I was at Meta (then facebook), people lived and died by the live demo creedo.

Pitches can be spun, data is cherry picked. But the proof is always in the pudding.

This is embarrassing for sure, but from the ashes of this failure we find the resolve to make the next version better.

Anon1096 5 days ago | parent | next [-]

Yep I hope that mindset never dies. Meta is one of the last engineering-first companies in big tech and willing to live demo something so obviously prone to mishaps is a great sign of it. It's not unlike SpaceX and being willing to iterate by crashing Starships for the world to see. You make mistakes and fix them, no big deal.

gcr 5 days ago | parent | prev [-]

why did they choose to air this live?

For an internal team sure absolutely, but for public-facing work, prerecorded is the way to go

com2kid 5 days ago | parent | next [-]

One of my internships was preparing Bill Gate's demo machines for CES. I setup custom machine images and ran through scripts to make sure everything went off w/o a hitch (I was doing just the demos for Tablet PC, each org presumably had their own team preparing the demos!)

Not doing it live would've been an embarrassment. I don't think the thought ever crossed anyone's mind, of course we'd do it live. Sure the machines were super customized, bare bones Windows installs stripped back to the minimum amount of software needed for just one demo, but at the end of the day it sure as hell was real software running up there on stage.

bee_rider 5 days ago | parent | prev | next [-]

If it was pre-recorded we’d know it was staged and that assume they didn’t have a working product.

Their actual result was pretty bad, but, ya know, work in progress I guess.

whywhywhywhy 5 days ago | parent | prev | next [-]

Watch their big "Metaverse" presentation where its all vaporware and faked, presumably this is a cultural shift from that era.

stonogo 5 days ago | parent | prev [-]

The same unwarranted sense of confidence that tells them this product is worth making tells them that they can easily pull off a live demo. This is called "culture fit"

SoftTalker 5 days ago | parent | prev | next [-]

I saw Jobs give a demo of some NeXT technology and the system crashed and rebooted right in the middle of it. He just said “oops” and talked around it until the system came back up.

postalcoder 5 days ago | parent | prev | next [-]

i love jobs but i do remember the “everybody please turn off your laptops” presentation.

live demonstrations are tough - i wish apple would go back to them.

paxys 5 days ago | parent [-]

Totally agree. Up until a few years ago failures during live demos on stage used to be a mark of authenticity, and companies playing recordings was always written off as exaggerated or fake. Now all of Apple's keynotes are prerecorded overproduced garbage.

5 days ago | parent [-]
[deleted]
neilv 5 days ago | parent | prev | next [-]

"At least it's not faked" was my main reaction, too. Some other big-tech AI-related demos the last couple years have been caught being faked.

Zuckerberg handling it reasonably well was nice.

(Though the tone at the end of "we'll go check out what he made later" sounded dismissive. The blame-free post-mortem will include each of the personnel involved in the failure, in a series of one-on-one MMA sparring rounds. "I'm up there, launching a milestone in a trillion-dollar strategic push, and you left me @#$*&^ my @#*$&^@#( like a #@&#^@! I'll show you post-mortem!")

garbawarb 5 days ago | parent | prev [-]

I appreciate the live demo but I'm suprised they didn't at least have a prerecorded backup. I wanted to see how video calls work!

paxys 5 days ago | parent [-]

Considering there's no camera pointing to your face they can't be all that interesting.

jjfoooo4 5 days ago | parent | prev | next [-]

It was painful even before it started malfunctioning

baby 5 days ago | parent | prev | next [-]

The demo gods were not present that day

5 days ago | parent | prev | next [-]
[deleted]
anonu 5 days ago | parent | prev | next [-]

It was the WiFi though

herval 5 days ago | parent | prev | next [-]

Typical Meta product. I used to believe and wasted money on multiple generations of Quest & Ray-bans. I expect this device to be unsupported at launch, just like Quest Pro was

hattmall 5 days ago | parent [-]

The portal was like their best product and they just abandoned it.

m3kw9 5 days ago | parent | prev | next [-]

so when I talk but not to it, it may response like i accidentally say siri? Except is every time?

joshdavham 5 days ago | parent | prev [-]

For those who didn't pick up on it, they were being sarcastic about the issue being wifi related haha

bigtones 5 days ago | parent | next [-]

That was not sarcasm. They were being serious.

dxxmxnd 5 days ago | parent | prev | next [-]

I’m surprised everyone is saying they weren’t sarcastic. They were even being MORE sarcastic about it being the Wi-Fi after the failed WhatsApp call.

stavros 5 days ago | parent | prev [-]

It didn't sound like sarcasm at all to me?