| ▲ | Mg6yDfjp5U 3 hours ago |
| I recently left Google having worked on a number of projects with various YouTube teams. I think I can explain why it's being handled this way by YouTube. This is a fairly nuanced/involved issue, so the task of classifying the bug likely made it's way to one of the engineers responsible for the implementation of this feature. That engineer has already launched this project, and filed it away under their GRAD (performance) artifacts for when promo/annual review talks roll around. There's no motivation for this engineer to waste time fixing this bug because it won't benefit their promo packet, and they are already being put under pressure to launch other projects which _will_ benefit their promo packet. So they do what they can to sweep it under the rug because that's what the promo/annual review framework (GRAD) incentivizes and rewards. |
|
| ▲ | NamTaf an hour ago | parent | next [-] |
| I design and build trains. If I ignored a safety issue that I discovered - not one I caused by design but even one I discovered in an existing design -
because of a performance review my engineering licence would be revoked and I would be kicked out of the industry. This is a prime example of why programmers are not seriously considered engineers. |
| |
| ▲ | brailsafe 31 minutes ago | parent | next [-] | | > This is a prime example of why programmers are not seriously considered engineers. Seems to me like your comment is simply an example of prejudice. You're just describing another standardized incentive structure that you're operating in, and using that as a basis to extrapolate that programmers of all kinds—whether they work on a video platform or on machinery that could cause catastrophe if it fails—are implicitly careless careerists who refuse responsibility by nature. | | |
| ▲ | j45 11 minutes ago | parent | next [-] | | I understand the direction of your comment, engineering doesn't guarantee security either. Hubris is the single biggest downfall, whether it's pegged on insecurity, or a false sense of knowledge, superiority or entitlement. The very best and most experienced people I know have deep expertise, and maintain a healthy mistrust of their own work to keep an eye on it and improving it. Real world experience and run history is a big thing, and people can re-learn the lessons of the past over and over with their egos, or also be open to learning from others to learn quicker. | |
| ▲ | 23 minutes ago | parent | prev [-] | | [deleted] |
| |
| ▲ | HelloMcFly 18 minutes ago | parent | prev | next [-] | | "The rat is always right." - B.F. Skinner. When the rat presses a lever, don't blame the rat. This is super reductionist of course, but I always keep it in mind. | | |
| ▲ | bagels 12 minutes ago | parent [-] | | It's worse than that. Google will get rid of you if you are just fixing bugs. Ergo, the people who are inclined to fix are forced out or forced not to fix. |
| |
| ▲ | fathermarz 37 minutes ago | parent | prev | next [-] | | I think there is a fine line. YouTube is not critical software and no one’s life depends on the safety (putting mental health aside) of the code running. Some software engineers do however write code that is critical, but to your point, I don’t think they are ever considered liable. I went through an acquisition as a Canadian software developer getting acquired by an American company. They wanted us to be called engineers like the rest of their SWEs but in Canada it’s a protected namespace. It’s illegal to call yourself an engineer without having the ring and the papers. Which personally I can appreciate. | | |
| ▲ | m00x 29 minutes ago | parent [-] | | Youtube should consider their engineers responsible for the software they write. Big companies these days are just bureaucracy tricks and politics. There's a small handful of real talent, but they're quickly moving to new startups. Also, I'm Canadian as well, and almost everyone calls themselves "software engineer" these days. You just can't say P.eng. in your title. You could be forced to remove it from linkedin/etc if you're called out, but it rarely happens. |
| |
| ▲ | beambot 31 minutes ago | parent | prev | next [-] | | The entire rail industry suffers from massive deferred maintenance issues that manifest as serious safety concerns. This shit happens in every industry: dieselgate, 737max, flint water crisis, PG&E camp fire, etc. Let's not pretend one engineering discipline is holier than thou -- especially when the consequences are derailments versus some leaked youtube videos. | |
| ▲ | mschuster91 34 minutes ago | parent | prev | next [-] | | > This is a prime example of why programmers are not seriously considered engineers. The problem isn't the programmers ffs. In your industry, if your superior orders you (or creates the incentive) to hide bad stuff under the rug, you have the ability to push back, at least to some degree. Programmers? We don't have that. Maybe the few of us who actually work on security critical stuff, but some generic AI BS? No chance. You're being treated as a cog. | | |
| ▲ | Arainach 31 minutes ago | parent [-] | | All sorts of employees are treated as disposable. The issue is absolutely that software engineers have no culture of responsibility or safety and no professional licensing group to enforce it for them. | | |
| ▲ | brailsafe 13 minutes ago | parent [-] | | > no culture of responsibility or safety and no professional licensing group to enforce it for them. Naturopaths and chiropractors are licensed to do various things too, physicians, etc.. a license does not imply that there would otherwise exist a culture of responsibility, foundation in evidence or anything of the sort. It's an incentive structure and regulatory practice. One may even keep their license while being a monster and abusing other incentive structures that don't have a bearing on that license. Software engineers are not typically licensed as engineers, that's all one can say without dipping into prejudice. |
|
| |
| ▲ | richardfey 41 minutes ago | parent | prev [-] | | I remember hearing this perspective when I first started in the software industry, and I agreed with it for quite some time. But frankly, we’ve never been further from it. |
|
|
| ▲ | throwrioawfo 3 hours ago | parent | prev | next [-] |
| I feel like things have become so much more cynical in the last 5 years, in this regard. I feel like part of it is the "over-systemization" of promos. I see the logic behind it to some extent - if there's a system, it's "fairer"/"more democratic". But, then we end up with ridiculous gamified promo systems. |
| |
| ▲ | campbel 2 hours ago | parent | next [-] | | objective systems become gamified subjective systems become politicized pick your poison | | |
| ▲ | ismailmaj an hour ago | parent | next [-] | | I'll pick small company, thank you. | | |
| ▲ | bartread an hour ago | parent | next [-] | | This isn’t a bad approach but it’s not a panacea: small companies can be pretty messed up too, albeit perhaps in different ways. | | |
| ▲ | manquer 20 minutes ago | parent [-] | | The impact is local though, it would be only a problem if the median small company is more messed up than the large co. It not likely to happen because being small there are more threats or market forces to deal with so they cannot do as they please. Monopolies or just economies of scale affords large co and the small number of executives that control them outsized influence - both good and bad. |
| |
| ▲ | an hour ago | parent | prev [-] | | [deleted] |
| |
| ▲ | anonymars an hour ago | parent | prev | next [-] | | This is great. I'd begun to conclude the pendulum swung too far towards "moneyball" and both approaches have trade-offs, but this is perfectly succinct | |
| ▲ | doctorpangloss 37 minutes ago | parent | prev | next [-] | | Yeah... there are no systems that are not political. Even if you agree objectivity is a thing, someone has to persuade others to buy into whatever that objectivity is, and that's still politics, and not cynical at all. | |
| ▲ | BadBadJellyBean an hour ago | parent | prev [-] | | Why not both? | | |
| ▲ | lacunary an hour ago | parent [-] | | it is both because the "objective" system is also rife with subjective judgements |
|
| |
| ▲ | 25 minutes ago | parent | prev | next [-] | | [deleted] | |
| ▲ | jambalaya8 2 hours ago | parent | prev | next [-] | | Eh, clearcut promo paths used to be a bigger thing in the 90s and they did work for a little while, they just didn't handle exceptions well, and then the whole developed world up and thought they were also exceptions. Certifications used to matter more, now they are so cheapened that you cannot do much without them. | |
| ▲ | ikiris an hour ago | parent | prev | next [-] | | 5 years ago they had the same incentives. | | |
| ▲ | tmoertel an hour ago | parent [-] | | But five years ago they had a stronger engineering culture. The old values were rapidly eroding, but some still held. |
| |
| ▲ | wahnfrieden 2 hours ago | parent | prev [-] | | It’s not about fairness or democracy (maybe you meant meritocracy?) at all although it’s sold that way to participants - it’s primarily about ownership’s ability to cascade management duties, including mitigating latent negotiation powers by individual workers and groups of workers |
|
|
| ▲ | ronbenton 3 hours ago | parent | prev | next [-] |
| Glad to hear this is a universal big tech experience. The promo process is entirely antithetical to shipping good products |
| |
| ▲ | gguncth 2 hours ago | parent | next [-] | | Shipping great products is about the details that almost nobody will notice A good promo process needs to notice the invisible Apple did it for decades | |
| ▲ | Aunche 2 hours ago | parent | prev | next [-] | | I don't think it's the promo process itself. If the bug was something that actually affects Google's bottom line, I guarantee that Google would find a way such that the engineer would be incentivized to fix it. | |
| ▲ | tiahura 2 hours ago | parent | prev | next [-] | | Sweep it under the rug is not limited to any paticular industry. | |
| ▲ | citizenpaul 3 hours ago | parent | prev [-] | | What do you mean? Youtube is unquestionably one of the most successful projects ever launched? Seems like the process works astoundingly well. | | |
| ▲ | strictnein 3 hours ago | parent | next [-] | | Youtube wasn't launched by Google, it was purchased. | | |
| ▲ | UnlockedSecrets 2 hours ago | parent [-] | | Youtube launched 1 year and 8 months before being acquired by google.... It's largely semantics to say that what Youtube is today, isn't a direct result of Google's ownership for nearly 20 years now.... | | |
| ▲ | ismailmaj an hour ago | parent | next [-] | | From talking to someone that worked at YouTube for 15 years, they still had a lot of core Python code in 2016 that was legacy from the OG company/team and that code needed to be transitioned to follow the Google way of doing things in C++/Go. I don't think it was distinct enough from the Google culture like Android was at the start of the acquisition but it seems they had leeway to do their own thing. | |
| ▲ | grg0 an hour ago | parent | prev | next [-] | | Google had Google Video and couldn't hold up, that's why they bought Youtube. | |
| ▲ | sdevonoes an hour ago | parent | prev | next [-] | | What YT is now: - ads every now and then - addictive shorts no one needs - suggested videos nobody asked for - geo ban of videos | | |
| ▲ | dizhn 13 minutes ago | parent [-] | | No concept of language in a user facing way. No filter by language no search by language. On the contrary searches are translated before running and return all languages, videos are dubbed even when you speak the original language, same but with titles being translated etc. Search being shit is kind of on par with being a Google product though. I wonder if they had any language preferences before Google bought them. I don't remember that far back. |
| |
| ▲ | strictnein an hour ago | parent | prev [-] | | Huh? It's not semantics to point out that a project wasn't launched by Google, when the point was about a successful project launch from Google. |
|
| |
| ▲ | mid-kid 3 hours ago | parent | prev | next [-] | | Youtube survives on google's massive repertoire of products being vastly more profitable, not because it's the best of its kind. | | | |
| ▲ | ghurtado 3 hours ago | parent | prev | next [-] | | And you honestly believe the main factor in YouTube success was the quality of the code? That's a thought that doesn't even deserve further comment. | |
| ▲ | dooglius 2 hours ago | parent | prev | next [-] | | Did the promo process exist at YouTube's creation? | |
| ▲ | OtomotO 3 hours ago | parent | prev | next [-] | | Good != Successful. I assume that's why they wrote good and not successful. It's an average software product with incredible scaling behind it and a lot of elbow grease to keep it chumming along, but it's not great software by the definition of "bugs actually get dealt with" | | |
| ▲ | jascha_eng 3 hours ago | parent [-] | | It's great software in the sense that it makes a shit ton of money though. In the end software that doesn't get used and doesn't make any money but has no bugs is not valuable either. Not saying that this is the trade off you have to make but if you have a working mode in place that achieves usage and money somewhat consistently i can understand being hesitant about changing it to optimize for less bugs instead. | | |
| ▲ | estaroc 3 hours ago | parent | next [-] | | The only people for whom it makes sense to define "great" as "makes money" are the people who produce and sell said product. Similarly, most people don't put much stock in the salesmen of a product describing their own product as great. Stop debasing all of quality to profitability. | |
| ▲ | ori_b 2 hours ago | parent | prev | next [-] | | Surely the Therac would have made more money if they had covered up the deaths instead of fixing the bugs and owning up to them. Why do you think they would compromise how good their software is merely to save lives? | |
| ▲ | OtomotO 2 hours ago | parent | prev [-] | | That's just two different scales. Weapons are a great product for weapon dealers and manufacturers as well, just not so much for the people killed by them (or their families, or survivors) So sure, if making a shitload of money is the metric, YouTube is a great product. That wasn't the point of the person you answered to though. |
|
| |
| ▲ | 3 hours ago | parent | prev [-] | | [deleted] |
|
|
|
| ▲ | mlmonkey 3 hours ago | parent | prev | next [-] |
| This is what you get when the MBAs are in charge. They just go with P&L, Spreadsheets, etc. and care only about the current quarter and meeting the goals. |
| |
| ▲ | wahnfrieden 2 hours ago | parent [-] | | Google leadership has been from research/engineering and product backgrounds. This is how hierarchical businesses operate | | |
| ▲ | foltik 6 minutes ago | parent | next [-] | | Not really, in such large companies there's enormous selection pressure favoring career politicians. Maybe the survivors did some engineering at some point, but expertise fades fast when you stop getting your hands dirty. | |
| ▲ | lesuorac 9 minutes ago | parent | prev [-] | | Except leadership is largely not from employees moving up the rank Sundar (CEO) is from Mcksinsley. Ruth (President) is from Morgan Stanley. TK (Cloud CEO) is from Oracle. Mohan (YouTube CEO) is from DoubleClick which is Google at this point (~15 years). --- Largely the story of the past several decades is that "doing your time" is a bad strategy. Always move to another company to go upwards. |
|
|
|
| ▲ | sscaryterry 2 hours ago | parent | prev | next [-] |
| The rot is deep. |
|
| ▲ | cdbdbspt 2 hours ago | parent | prev | next [-] |
| I also used to work at Google and what you have described is not the way the VRP works at all. 1. The engineers on the VRP teams set the severity of the bug based on impact. The engineering team responsible for the fix can argue the severity but only if they can show there is some other mitigating factor that the VRP team wasn't aware of. 2. Google has a great security culture and while it may be true that maintaining existing code may not be as sexy as building new features, fixing vulnerabilities does look good on GRAD (performance) because the impact is already well documented. 3. Believe it or not, the VRP team does like to give away rewards. However, to do this, they have to follow a rubric to keep all of the payouts consistent and fair. 4. Constructive and polite discourse is welcome and a researcher may reply to their bug asking for more details or to make their case in the event that they think the VRP team did not understand the severity. The team is made up of humans who are open to the idea that they missed something in the initial report. They, like all other bug bounty programs, are also struggling to keep up with the huge influx of AI generated slop so mistakes can happen. |
| |
| ▲ | jonahx 2 hours ago | parent [-] | | My first thought when reading the article was: "The generous interpretation here is that whoever is fielding reports gets so many false positives that they miss true positives (like this report), especially if there's any gray area." I'm not saying that excuses it, but it is one likely explanation for how it happened. When looking at just one report, the response seems negligent. When looking at a pile of 1000 nonsense reports, with a handful like this, I understand the difficulty. |
|
|
| ▲ | ghurtado 3 hours ago | parent | prev | next [-] |
| Of all the fucked up things in this comment, giving a single Engineer lifetime responsibility for all bugs in code they wrote is probably the dumbest. And it's slowly becoming the norm. The last place I worked at, a large and well known Tech company, didn't even roll with QA's. That just wasn't a role anywhere in the division. You are fully responsible for all the bugs in all the code you ever wrote Cute at first. Unsustainable in the long term |
| |
| ▲ | weitendorf 2 hours ago | parent | next [-] | | I disagree with this pretty strongly. If you’re not going to take responsibility for your bugs I don’t want to work with you. Don’t make other people QA your work; if you’re not able to figure out how to do that yourself while you work you’re legitimately bad at your job. Once you leave an employer obviously you have no obligation to fix bugs in IP you don’t own or anything. | | |
| ▲ | tredre3 2 hours ago | parent | next [-] | | I think it's reasonable to have a culture where you're encouraged to consult the IC who wrote the code even after they've moved on to other projects. But I don't think they should be responsible for fixing the bugs. And I don't mean this to excuse the bad code written by ICs. I just think it's not sustainable from the POV of the org itself to depend so heavily on individuals, especially ones who aren't familiar with the entire codebase anymore. The team currently in charge needs to have full ownership and be responsible for the code, even if they didn't write it. | | |
| ▲ | nomel an hour ago | parent [-] | | That works as long as there's a finish line. If you make a framework, or a set of libraries, it's easy to get pigeon holed into all new features/tangential work around those. |
| |
| ▲ | mk89 an hour ago | parent | prev [-] | | OP used the word "lifetime" which makes a key difference. I don't want to be responsible for a bug in my 8 years old code, which I probably even forgot how it worked etc. I probably don't even work anymore in the same team or on the same service. Why the hell should I be responsible and how is this sustainable? I am not even sure if your criticism makes any sense at all anymore nowadays. AI is writing 80% of the code, if not more. It's technically not even your code anymore, although there is your name on the commit. Why should I be responsible for that 3 years from now, when I have again moved team or service etc. Accountability ok, but you should not retire with your code. | | |
| ▲ | mschuster91 27 minutes ago | parent [-] | | > Why the hell should I be responsible and how is this sustainable? Well, it works for professional engineers, you know, the people designing bridges, tunnels, heavy machinery, aircraft, spacecraft or medical instruments. When something happens and they can't show that their work adhered to the generally accepted best standards at the time... they're held liable. And sometimes, that liability includes jail time, particularly when people are seriously injured or die. And how it is sustainable? Simple: legal requirements that force managers to allot enough time and tooling to their engineering teams, because engineers whose professional license is on the line will rather quit than be forced to sign off something that is unsafe. In the software world, this might result in AI not being used at all - simply put: no matter what, AI in its current form is always going to be vulnerable to in-band attacks, or to use an older term... phreaking [1]. It might result in software having to go through formal proof programs, fuzzers, whatever. It might result in entire programming languages just being outright banned in production code in favor of programming languages that eliminate entire classes of vulnerabilities. And before the usual "but China/India/... would outcompete us" complaints come... well, have you ever seen a Chinese widebody airliner in Western airspace? No. Because China is not able to pass over the engineering gates we have set in place. We could easily do the same with software. Requiring at least some sort of quality gates on software would not be bad for you as a programmer. Quite the contrary: it would hand you power over your incompetent beancounter boss. [1] https://en.wikipedia.org/wiki/Phreaking | | |
| ▲ | jerojero a few seconds ago | parent [-] | | I think the problem I see with your argument is that people simply do not value reliable and secure consumer software as much as they'd value reliable and secure airplanes. Of course, software that is in charge of things where people value security a lot, such as the software in airplanes, is much more scrutinized and adheres to better standards. This is the case precisely because when it goes bad people die in ways that attract a lot of attention. You can't enforce those same policies on most consumer software because people consume it the same way they do food. You can have Michelin starred restaurants with the best practices but most people can't afford to eat there so instead they will buy hot dogs on the street. The idea of "high quality hand crafted artisinal software" is closer to luxury products than it is to the engineering of planes, trains and bridges. |
|
|
| |
| ▲ | boredatoms 30 minutes ago | parent | prev | next [-] | | Lifetime is too much. One or two re-orgs at most. People only spend a couple of years at each company anyway | |
| ▲ | goosejuice 2 hours ago | parent | prev | next [-] | | It's not cute, it's a sensible way to build greater understanding by learning from mistakes. The thing is, it has to be engrained in the culture and that also means it may need to take priority over other work. Responsibility doesn't need to mean you have to write the code, just see it through. | |
| ▲ | vlovich123 3 hours ago | parent | prev | next [-] | | Ok. So QA finds a bug. Who’s responsible for fixing it? The only value of QA is to try to make sure you become aware of issues before customers find them | | | |
| ▲ | dfxm12 2 hours ago | parent | prev [-] | | It's even worse when you don't work at a tech. Even the simplest of Excel formulae, power automate flows simply go abandoned once the creator moves on, or maybe a very expensive consultant is onboard to maintain what amounts to a handful of lines of code. It's embarrassing how little initiative the average information worker has when it comes to stuff like this. |
|
|
| ▲ | dfxm12 2 hours ago | parent | prev | next [-] |
| It's ultimately Google's responsibility to ship bug free products. I don't care who implements a fix, but Google management should make sure someone fixes it. |
| |
| ▲ | carl_dr 2 hours ago | parent | next [-] | | No, it’s really not, it’s none of our jobs to do that. It’s our job to make our employer (even if you are your own employer) money. It’s incredibly rare you have the luxury of even trying to deliver bug free code, let alone achieve it. | | |
| ▲ | dfxm12 2 hours ago | parent [-] | | People eventually stop using, and paying for, buggy code. | | |
| ▲ | ZiiS an hour ago | parent [-] | | ROFL this has not been my experience. Many more people stop paying because of some featuritis request you snubed to keep the bugs under control. |
|
| |
| ▲ | wahnfrieden 2 hours ago | parent | prev [-] | | Spoken like a user and not an owner |
|
|
| ▲ | newtonianrules 2 hours ago | parent | prev | next [-] |
| [dead] |
|
| ▲ | varispeed 3 hours ago | parent | prev [-] |
| > This is a fairly nuanced/involved issue Is it though? |
| |
| ▲ | Mg6yDfjp5U 2 hours ago | parent [-] | | Definitely. The front line support agents handle only the most basic requests. Anything even remotely complicated, such as this, would be internally kicked around until they found someone familiar with the project to give input. Which most likely is someone who worked on the original implementation. |
|