| ▲ | kilpikaarna 3 hours ago |
| > I think it is fundamentally a crisis of trust. I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over. [...] > I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer. I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world. That is totally on us, and I think it’s the criticism you should be making, instead of all this stuff about messaging and marketing. > We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that. Turns out "winning back trust" doesn't have anything to do with any actual concerns people may have re: employment, electricity prices, stock market bubble, intellectual property, scams, cybersecurity, environmental issues etc. Rather we'll just do all of that even harder and the miracles ("curing cancer", lol) we've so far failed to deliver are bound to arrive in short order! |
|
| ▲ | traceroute66 an hour ago | parent | next [-] |
| > The thing that will work is actually curing cancer. This is honestly hilarious. Dario must think people are stupid. First, cancer research charities are some of the most well funded on this planet. They all obtain donations on the basis of "together, we will cure cancer" messaging. They all fund the best science they can. Second, the human body is a complicated thing. You can feed your fancy LLM as many textbooks and academic papers as you like, but the reality on the hospital ward will always be different. Why do you think student doctors have to spend so many years "doing the rounds" Dario ? They are all academically smart, they are all capable of memorizing text books ... but there is no substitute for seeing and doing the reality. I barely trust Claude to write code, let alone find a cure to cancer. |
| |
| ▲ | eloisant an hour ago | parent | next [-] | | Besides cancer isn't really "a disease" but rather "a family of diseases" that can vary a lot. Even breast cancer can be one of many different cancers. It's like saying "we'll cure infectious diseases" | | |
| ▲ | alphazard 10 minutes ago | parent | next [-] | | "Cancer isn't a single disease" is often shouted from the Dunning-Kruger peak of cancer research. I think it was popularized by an explainer video on Youtube, and now it's being memed. The people who think this way want to legitimize funding a particular research niche that they benefit from. A more sophisticated view is that cancer (one idea) is an problem of rates (like most things in biology) and there is a tumor burden per unit time from whatever causes vs. the immune system's capacity to detect and destroy them per unit time. As humans age, the balance tips in one direction. If the balance is too far off for too long, tumors accumulate, doctors take notice, and assign whatever labels to the condition. Looking into particular causes of tumors (and calling each cause its own cancer) can create a vast research program that employs a lot of people, since there are so many mutations that lead to tumors. But interrupting the development of a particular mutation is not going to have the same society-changing effect that solving the fundamental issue of rates would. | |
| ▲ | traceroute66 39 minutes ago | parent | prev | next [-] | | > It's like saying "we'll cure infectious diseases" Don't worry, I'm sure that's his next blog post. Right after Claude has finished solving famine and poverty. ;) | |
| ▲ | fnord123 33 minutes ago | parent | prev [-] | | We will cure virus. | | |
| |
| ▲ | jacquesm 36 minutes ago | parent | prev [-] | | > Dario must think people are stupid. All of those guys think people are stupid. That theme has been repeating over and over again. |
|
|
| ▲ | pj_mukh 2 hours ago | parent | prev | next [-] |
| > The thing that will work is actually curing cancer. I think typical bubble behavior the leaders have set up the whole promise to fail.
Everyone is expecting some faux super intelligence to come and find a cancer solution everyone else missed. However, it is just as likely that vanilla current LLM's will create enough of a productivity boost for back office automations in research heavy hospitals to create the space for regular humans to create these breakthroughs, but LLM's won't be able to claim that for themselves and inevitably “fail”. |
| |
| ▲ | andy99 an hour ago | parent | next [-] | | Savings from back office automation go entirely into administrative bloat. I find it even less likely such efficiency would lead to breakthroughs than I do LLMs becoming super intelligent and doing it themselves, which I don’t find likely at all. | |
| ▲ | anon-3988 2 hours ago | parent | prev [-] | | I mean they did solve the protein folding problem did they? NOt too far fetch to think it can automate cancer research or problem finding in some way. | | |
| ▲ | mchinen 42 minutes ago | parent | next [-] | | The protein folding solutions like ESM/Alphafold were not due to this LLM/agentic coding or autoresearch type approaches though. They were designed by bio ML researchers. It's hard to keep track of the frontier on bio ML, but it seems that we're going slower than what Demis Hassabis said in 2024 with 5 years to full cell molecular simulation. We still aren't able to reliably model a tiny surface of the cell membrane. And of course there's Derek Lowe's takes on the drug discovery pipeline waiting for the proof in the pudding. To me the only reasonable bullish position is that there is a very non-linear AGI threshold for accelerating progress that we haven't hit yet. For me personally, I'm looking at other more tractable fields as a proxy to measure this kind of progress. The best modest evidence is from the agentic coding area, (modest because these kinds of gains may not translate to bio progress). Other soft-ish fields to like legal/law/tax are also interesting to watch, as a small amount of people are now trusting AI for these areas that were considered totally unusable a year ago. Another proxy is being able to generate generally entertaining media. | |
| ▲ | inigyou an hour ago | parent | prev | next [-] | | The field of AI did very well in protein folding, but it was completely different from LLMs. | | |
| ▲ | octocop 44 minutes ago | parent [-] | | AI made a large improvement in accuracy on a specific protein dataset. We still don't know how exactly protein folding works. So no, it's not solved. |
| |
| ▲ | disgruntledphd2 2 hours ago | parent | prev | next [-] | | > I mean they did solve the protein folding problem did they? They made huge progress, but I would say that the vast majority of work on this problem was designing the harness for the model. That's a lot of work for each and every domain. | |
| ▲ | RandomLensman 2 hours ago | parent | prev [-] | | There is already (partial) automation there, for example for finding new drug candidates. Isn't that so far only static folding? | | |
|
|
|
| ▲ | dgellow 3 hours ago | parent | prev | next [-] |
| The effective altruist and longtermist crowd are into it as a religion. They would happily sacrifice anything human to satisfy their dreams of AI. They care way more about what the AI needs than their what their fellow humans need. We will end up with an AGI having better living and working conditions than humans and the whole AI industry will be clapping |
| |
| ▲ | vrganj an hour ago | parent [-] | | It's an ideological thing. (We just had a conversation about ideology actually, remember?) Capitalism has always been a tension between capitalists, who want maximum return on capital and workers, who want pesky things like a living wage, sick leave or safe work conditions. For the longest time, the only power labor has had to get those things was the power of collective bargaining. No agreement with your workers meant no production happened. Now, there's finally a chance at salvation. The capitalist Messiah is AGI and it will finally deliver them from those annoying laborers. This ideological bent is why they're putting everything they have into AI. It's why VCs and their fellow capitalists are going so crazy. They see it as a way to finally solve the contradictions of their ideology, but in reality it'd only create a new one: If everyone's out of a job, who will buy their products? | | |
| ▲ | eloisius 30 minutes ago | parent | next [-] | | If they had AGI, why would they need consumers anymore? Just use your drone army to secure all natural resources and produce whatever you need for yourself. Other humans are just pets. | |
| ▲ | inigyou an hour ago | parent | prev | next [-] | | I wish I was smart enough to credibly promise capitalists they could replace all their workers. It doesn't matter that I can't do it. They'll pay me lots of money if they think I can. | |
| ▲ | thelastgallon 26 minutes ago | parent | prev | next [-] | | > Capitalism has always been a tension between capitalists, who want maximum return on capital and workers, who want pesky things like a living wage, sick leave or safe work conditions. The underlying purpose of AI is to allow wealth to access skill while removing from the skilled the ability to access wealth. (read on HN, not my words) | |
| ▲ | danqqqq an hour ago | parent | prev [-] | | [dead] |
|
|
|
| ▲ | mindwok 2 hours ago | parent | prev | next [-] |
| I think it's more reasonable than it sounds on the surface. People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own. Curing cancer sounds insane, but it's also a research problem, not a societal level coordination problem. And one AI has already proved to help with breakthroughs (alphafold). IMO it makes sense for them to shoot for something like that as proof of AI's beneficial sides. |
| |
| ▲ | willis936 an hour ago | parent | next [-] | | >People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own. This is not true. There are two companies at the center of AI direction: Anthropic and OpenAI. If there were anyone on the plant who has the ability to influence our direction then it would be Dario Amodei. Acknowledging the grievances is a good step, but it's not enough. There needs to be a clear explanation of actions to address them, a plan to enact those actions, and commitments with consequences in failure of those actions. Tell people how you're going to make them more employable and effective and needed. Tell people how your datacenters will be carbon neutral. Tell people how financial actions resulting in a frothy market will be coming to an end. He and Sam Altman alone have this power and their inaction says everything we need to know about their intent. | | |
| ▲ | jacquesm 14 minutes ago | parent [-] | | > If there were anyone on the plant who has the ability to influence our direction then it would be Dario Amodei. Possibly, but not necessarily in a better direction. And that's the whole problem here. They should all go watch Phantasia. |
| |
| ▲ | smoe an hour ago | parent | prev | next [-] | | I don't think the problem is that people don't believe AI can be beneficial. The problem is the perception that AI will affect people's lives more negatively than positively, while a few get even more obscenely rich and powerful. If you don't tackle the societal level problems, nobody will care about research breakthroughs. | |
| ▲ | fyredge an hour ago | parent | prev | next [-] | | Based on rhetoric from open ai and Anthropic, i don't think this is reasonable. If anything, their miracle of an AI should be able to solve the job market and economic bubbles. Deploy their agents on it, set up automated hedge funds, distribute the profits equitably to everyone on the planet and on and on. But something tells me either they can't or they won't, so no trust will be built. | |
| ▲ | disgruntledphd2 2 hours ago | parent | prev | next [-] | | > Curing cancer sounds insane, but it's also a research problem, I just don't buy the AI labs approach to this stuff. Like, unless we can basically simulate the entirety of human biology, I don't really see how LLMs can make progress here. Maths is different as it doesn't require a real-world interface, and programming already (by definition) can be simulated on a computer. Without that, I can't see much (if any) progress being made on domains like biology. | | |
| ▲ | mindwok 2 hours ago | parent | next [-] | | I'm a noob on this topic, but I think drug discovery is more amenable to this structurally than other problems. Simulating biology is what we were doing with protein folding before Alphafold, and the search space was far too large to find stuff in reasonable timeframes. Alphafold showed that you could take a physical process and make a neural net clever enough to learn just enough structure that it starts finding things we might care about, and still physically accurate, much faster. Drug discovery is similar AFAIK. The space of possibilities is even larger than protein folding, but it's structurally similar enough that I think AI will help to make progress on the discovery side. Actually getting the drug tested and approved is another matter though for sure. | | |
| ▲ | tsimionescu an hour ago | parent | next [-] | | There are two relevant problems here, I think. For one, the question for Anthropic is whether LLMs, specifically, not AI techniques more generally, can help significantly with cancer research. And here, all experience so far is that LLMs only really work when they can easily automatically verify their own outputs and self correct - such as in math (using automatic proof verifiers) or programming (using compilers and unit tests). The second problem is that biological research speed is highly dependent on slow biological processes, such as cultures and long term studies. In programming, if an LLM could provide excellent insights and research suggestions 100x faster than a human, it would speed up the work roughly 100x. But in biology, it would only speed up the total work by a small amount - as any insight, even if absolutely brilliant and spot on, would still require months and years of actual experimentation. | |
| ▲ | disgruntledphd2 17 minutes ago | parent | prev [-] | | > Drug discovery is similar AFAIK. The space of possibilities is even larger than protein folding, but it's structurally similar enough that I think AI will help to make progress on the discovery side. I do agree that this kind of targeted approach makes sense. However, discovery is not really the issue here. Running the clinical trials (1/2/3) is much much more difficult, and consumes basically all of the time in drug development, so even if LLMs perfectly automate this, the speedup will not be particularly large. |
| |
| ▲ | XorNot 2 hours ago | parent | prev [-] | | The real issue is that biology needs actual experiments done in the physical world, which isn't nice and orderly and well behaved and easily loadable onto a 19" rectangular box. | | |
| ▲ | blitzar an hour ago | parent | next [-] | | There is a lot of computational chemistry and biology done in the early stages of research - there are now multiple orders of magnitude of computation power available that is doing nothing but feeding forward on billions of random numbers. | |
| ▲ | bananaflag an hour ago | parent | prev [-] | | On the other hand, AI means actual experiments done in the physical world but coordinated by an entity that never sleeps and never gets depressed and can multiply itself manifold and always comes up with new ideas. | | |
|
| |
| ▲ | watwut an hour ago | parent | prev [-] | | > People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own. Anthropic is literally creating the bubble. It is not beyond their scope of influence, it is literally what they are consciously achieving. As for peoples jobs, same actually applies. Anthropic is selling itself on dream of replacing jobs, even or especially where they are well aware AI does not perform that well. They are actively trying to replace people quickly before management notices it does not work well. And also, they can influence how much their data centers contribute to global warming. |
|
|
| ▲ | ironqcold 6 minutes ago | parent | prev | next [-] |
| I really can't disagree. What's the point of all this talk if for most ordinary people these technologies do more harm than good? |
|
| ▲ | sigbottle 37 minutes ago | parent | prev | next [-] |
| Honestly, it kind of reminds me of my life. I need to start going to the gym, stop being a slob, start showing up to my job not like I'm a college freshman attending classes but as a respectable young man. But instead of doing that, I'm going to drop everything and become a linux kernel maintainer. Learning about the internals of how RCU concurrency is implemented across subsystems is surely what I need to do right now! (/j before I get crucified) |
|
| ▲ | amarcheschi 3 hours ago | parent | prev | next [-] |
| rapture but for tech people |
| |
| ▲ | spwa4 3 hours ago | parent [-] | | Not really. The problem is that the trust is violated here. What people fear is that government and large companies are going to use AI against them, and make the reverse impossible. Sadly, this is exactly what is happening. And that's exactly what EU's AI regulation does. Of course, AI is being blamed for this happening, despite that last time I checked every EU parlementarian, every EU commission member is flesh and blood, the fact that it is 100% intentional government policy, humans, that are doing this to you using very un-AI methods. You see, in this regulation, governments are allowed to use AI, and to approve uses of AI. So there is absolutely nothing in EU's AI directive at all that prevents government and large companies, with approval, from answering the phone, and all "support requests" with AI, and not with people. Also "if you agree" (just like you've agreed to selling your location data in your cell contract) The watermarking algorithm itself has a VERY specific property that should have set off everyone's alarm bells: it is NOT the case that you can "check text for AI watermark". What it technically mandates is that if you provide access to a model, those people should be able to check if text is generated by that model. Not by any other model. It is NOT a general "is this AI?" check. So let's analyze how this works in 2 specific cases: You get a mail from the government about taxes. You want to check if that text is generated by an AI and whether you even should ask to talk to a human. So you've got a piece of text, or an audio recording. Can you check if it's AI or not? NO YOU CAN'T. SynthID requires access to the model keys, and the legislation only mandates checks if you've got access to the model. The government or some large company, however, puts in it's regulations that you're not allowed to use AI to communicate with them (we all know this is coming), and they get a piece of ChatGPT text (or from any public model) from you. They would like to enforce that they won't talk to AI can they do this? YES, since they have access to the model too, SynthID allows for this usecase AND the AI regulation makes this mandatory. Is this in any way a problem with AI? No. Can anything be done about this by "fighting AI"? Well let's see ... will government institutions be forced to stop worsening services further if ChatGPT etc become inaccessible or obviously excluded by government and large company services? No. What is being outlawed in the EU, in other words, is exactly ONE thing: that you use AI to help you in your dealings with governments and large companies. To assert your rights, to resolve issues faster, to make them wait rather than you, to ... THAT is being outlawed, nothing else. Only that YOU get help from AI. This legislation will make it a big advantage to have access to "your own" model, because you'll have the SynthID keys, and the law behind you if you do that. And I don't mean local model, I mean your own model with your own SynthID keys, ie. not Google, not ChatGPT, not Claude, not Grok, and if the EU can make it happen, not Chinese models either. Finetuned models, however, will not have SynthID and so you'll never be able to prove text generated by them is AI, and you will not effectively have the power to refuse contracts or laws that specify they can use AI anyway. It's the EU government making it clear they are using AI to avoid even having to talk to you at all while disallowing the use of AI by mere citizens to assert their rights, against them, or anything they have interests in, like phone or electricity companies. | | |
| ▲ | RandomLensman 2 hours ago | parent | next [-] | | >The watermarking algorithm itself has a VERY specific property that should have set off everyone's alarm bells: it is NOT the case that you can "check text for AI watermark". What it technically mandates is that if you provide access to a model, those people should be able to check if text is generated by that model. Not by any other model. It is NOT a general "is this AI?" Where do I find that in the Act (or code of practice etc.)? IANAL, but Article 50 reads different to me but if there is a comment or guide how to read it - also fair enough. | | | |
| ▲ | trentor 2 hours ago | parent | prev | next [-] | | The only factual point you made is solved on February second. All Model providers have to provide offline tooling to check for invisible watermarking. | | |
| ▲ | disgruntledphd2 2 hours ago | parent | next [-] | | Yeah, it's particularly common when the EU does stuff. | |
| ▲ | spwa4 2 hours ago | parent | prev [-] | | There is nothing in the AI act that requires offline tooling, nor is there anything about requiring the passwords and/or API to work for every model, only for the people you are providing the model to. “The detection solution may be made available in the Union as one or more of the following: (i) a public, ideally standardised, specification allowing any third party to implement a detection mechanism; (ii) a piece of software (e.g., a standalone executable or library); (iii) a cloud-based service accessible to users in the Union through an API.” Option 3: "a cloud-based service". And accessible "to users". NOT to the public. Note: this is not the actual law, it is the code of practice, ie. guidance for model providers. The start of that document clearly states that you can comply with the law in other ways if you want, you'll just have to justify yourself. So you have a choice to not even do this. The actual law is here: https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-5... As to who decides the law clearly states who decides if someone is legal, like in most EU legislation. It's not the courts, it's "National market surveillance authorities designated by each EU Member State", and there is an EU office as well (and it is explicitly stated that they are not allowed to override each other). So every EU country has the right to provide exceptions to the law, just like they do for the GPDR. You do not have any rights under this legislation as an individual. Only these "National market surveillance authorities" get rights under this legislation. Secondary: article 7 additionally gives the EU commission the power to declare any code of conduct they want that declares what compliance with the AI act actually means. (and, of course, this is yet another attempt at declaring math illegal. The only way to actually enforce this legislation is for all models to comply with this, all over the internet. Obviously the EU does not remotely have the power to make that happen) |
| |
| ▲ | pjc50 an hour ago | parent | prev | next [-] | | While there is a large element of paranoia here, we're already seeing serious problems in e.g. planning consultations that people can just spam them with AI. AI makes a lot of traditional social processes stop working properly. > puts in it's regulations that you're not allowed to use AI to communicate with them (we all know this is coming), .. do we? > You get a mail from the government about taxes. You want to check if that text is generated by an AI No, actually, I want to know whether it's correct and whether it's legally binding. Using AI makes it less likely to be correct, sure, but ultimately whether it's human, spreadsheet, or AI the important thing is the legal right to correct process. | | |
| ▲ | cucumber3732842 42 minutes ago | parent [-] | | >AI makes a lot of traditional social processes stop working properly. Which is mostly a good thing because most of those processes were cooked up to be exclusionary but with plausible deniability God forbid some upstart get a big contract they "shouldn't" have gotten because AI lets them create a submission that's on par with a big guys. God forbid Joe Schmo be able to prepare the paperwork and dot his I's and cross his T's and be able to take advantage of some stupid zoning exemption that you spec'd out to essentially be only available to big moneyed interests. The ability to suffer one's way through beurocratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI. But people don't frame it like for obvious reasons. | | |
| |
| ▲ | watwut an hour ago | parent | prev [-] | | I trust EU massively more then I trust to Amodei. |
|
|
|
| ▲ | A_D_E_P_T 3 hours ago | parent | prev [-] |
| This is actually crazy, because biochemistry is where LLMs are weakest. It's one of science's most fuzzy and unpredictable domains, in general, and a lot of published works which an LLM might take at face value might be unreplicable or subtly flawed. (See e.g. the entire history of Alzheimer's.) It is also where real-world lab and clinical trial work is most important. So Anthropic are going to spin up a medicinal chemistry lab and start mouse experiments? I mean, it would be nice, but I don't think that the guys at Anthropic know what they're talking about here, or what they might be getting themselves into. (If indeed this is more than just PR.) |
| |
| ▲ | discreteevent 3 hours ago | parent | next [-] | | See also this article by a genomics PhD about how intelligence is not the bottleneck in medical research. https://www.writingruxandrabio.com/p/intelligence-is-not-the... | |
| ▲ | kilpikaarna 2 hours ago | parent | prev [-] | | AI bros consistently handwave away any notions of the material world imposing limits, because (spoiler!) it's a religion where you start from the miracle and work backwards. "Oh, we'll just set up fully automated robotic research labs everywhere!" Where will you get the raw materials? Oh we'll mine the asteroids! How will you find the energy to get there? Oh, the AI will invent new physics! (again without needing to run experiments). Et.c. Didn't and don't mean to disparage anyone's medical struggles, but "a cure for cancer" is a well-worn strawman. One that's been achieved for the low hanging fruit, the higher ones are seeing steady progress (already before LLM chatbots, even!), the bottleneck isn't "intelligence" and to the extent there are socioeconomic (access to screening, treatment) or environmental/lifestyle factors involved the AI boom is likely just making things worse! I'm sure Dario knows this, and it's anyway too pedestrian compared to the usual list of fruits of ASI. The text probably originally read "nanobots eating you alive and uploading to the cloud" or something, but they figured that wouldn't go over with the intended audience. "What do the peasants care about? Oh I know! Curing cancer!" | | |
| ▲ | bananaflag an hour ago | parent [-] | | > AI bros consistently handwave away any notions of the material world imposing limits, because (spoiler!) it's a religion where you start from the miracle and work backwards. How else are you going to work towards a dream? It's how Elon Musk managed to get reusable rockets when everyone said it's unfeasible. Granted, most ideas don't work, this is why you need testing, but I am a bit surprised to see this attitude on hacker news. | | |
| ▲ | A_D_E_P_T an hour ago | parent | next [-] | | If they said that their dream is to solve physics, I'd say "that's cool and I can't wait to see how it turns out." LLMs have gotten so good at mathematics, and are so adroit at analyzing vast quantities of data, that they might have a really good shot. But they said that they want to cure cancer. That's far outside the core competencies of any LLM, and it requires a lot of real-world wetwork with liquids, chemicals, cell line experiments, animal experiments, etc. They can't merely analyze existing data -- they'd need to generate vast amounts of new data, which isn't really the case in physics. And then regulatory approvals and so forth. "We're working on curing cancer" sounds more like a poor PR attempt than an actual effort, though I'd love to be wrong. | |
| ▲ | kilpikaarna an hour ago | parent | prev [-] | | Fair enough, but taking the history-ending outcome as given so externalities don't matter doesn't seem like a great approach. |
|
|
|