There needs to be a word for this type of interaction because it’s so common in software engineering:
Q: I need help doing X
A: if you’re doing X, you’re doing it wrong.
I propose the word shamesplaining. What do you think?
Not saying your opinion isn’t valid. It just doesn’t answer the question and it’s disturbing that this is the top voted answer. It sounds more like a criticism than an answer.
For 1, effectiveness still depends on the size and complexity of the codebase.
From my experience, as complexity and size grow, each prompt takes longer, does less, and is prone to more mistakes and disruptions to other parts of the codebase.
The code base I work on is > 30 million lines (excluding comments and blank lines) and most (but not all) of the people on the team are finding it pretty effective. That doesn't mean it wouldn't be more effective on a smaller code base.
This is a cherry picked statistic. We still have lower life expectancy, high infant mortality rates, and one of the highest rate of preventable and treatable deaths among developed countries
The unfortunate truth is that if you break life expectancy out by ethnicity, it tells a different story. Generally Asians, Latinos, and Whites are above while Blacks and Native Americans are below and pull down the average: https://pmc.ncbi.nlm.nih.gov/articles/PMC9256789/#S12 If you break the groups out and individually compare against countries of similiar ethnicity, the US is often ahead.
For infant mortality specifically, in the US we count every baby with a sign of life, regardless of the age. In many developed countries, they simply don't count too premature (under 22 weeks or 500 grams, iirc) and therefore don't consider them in the metrics. It makes for an apples to oranges comparison.
You're right on "preventable and treatable deaths" with heart disease and diabetes being the biggest contributors. The question for that comes down to "is that a result of systemic issues or individual choices?" because we can do lots about one of those.
Statistical hustling is common, but the most usual form is that the US excludes uninsured individuals to make itself look like it belongs in the developed country cluster.
This means that the average life expectancy of white Americans is actually slightly below the average life expectancy of all Americans.
Incidentally, while a lot of developed countries do not track life expectancy by ethnicity, the UK seems to have a few some studies. (https://www.ons.gov.uk/peoplepopulationandcommunity/birthsde...) Though the data is very old (2011-2014), and the data is broken out by sex with no summaries, whites actually fare the worst. Black African females came out at the top, with a life expectancy of 88.9. I asked Google's search what the comparable black American female life expectancy was in 2011, and it gave me 78.2, along with a link. (https://www.cdc.gov/mmwr/preview/mmwrhtml/mm6244a8.htm)
Our overall ranking generally is below all "developed countries" that I can see, which range from Germany (~80) to the ~84 of "developed" countries like Japan, Switzerland, and Sweden (along with a few countries still classified as "developing" like Kuwait). It's not completely terrible, but considering how much the United States spends on health care, that's quite a poor value.
There are multiple identified factors for explaining life expectancy, but access to healthcare is identified as a very significant factor. In a system, like America, that is very expensive (likely due to highly inefficient, over-bureaucratic, overly complex, and over-quasi-monopolized systems) and without universal coverage, it follows that those who are poorer may not have the same access to healthcare and may succumb to entirely curable illnesses. So I don't think it's a coincidence that the racial breakdown above almost matches the median ethnic household income (the median white American household earns more than the median Hispanic household, but elsewise it aligns).
On infant mortality, at least one link I found -- https://www.healthsystemtracker.org/chart-collection/infant-... -- which adjusted data due to the reported difference, and still found significantly higher infant mortality in the United States. Though the data is a little old (2016).
Cite the source for "access to healthcare" as a "very significant factor" in US life expectancy? We have pretty good numbers on the mortality cause differences between the US and other countries and they don't align with this claim (unless you're doing some bank-shot argument about how access to health care makes our car crashes more lethal or something).
You'd want to be looking for a scholarly source that puts numbers on this. It's been done! As I noted elsewhere on the thread, the study we're commenting on is based on 1990s numbers about differing mortality of the uninsured. But here you're looking breakdowns of all mortality causes and tying them to insurance, a trickier proposition. Will be interested in whatever you come up with.
Our life expectancy and infant mortality rates are downstream of driving a lot of miles and unhealthy behaviors (mostly diet and exercise) of American mothers (i.e., obesity). Those are things it would be good to improve, but it isn't the fault of the ER or L&D departments or their billing models that people get in car accidents or mothers are obese.
Life expectancy differences between the US and other countries are dominated by just a few factors:
* Car accidents, because we drive much more and are much more spread out than other countries.
* Drug overdoses, though other countries are starting to catch up to us there.
* Homicide, because of our gun policy and the universality of firearms (which also bears on our suicide stats).
* CVD.
That last item sounds like an indictment of the US health care system, but it isn't. If you break CVD out by state, northeastern states like Massachusetts have outcomes resembling the Nordics, and Mississippi has outcomes like a developing country. But the structure of the health care system is the same in both places.
> Car accidents, because we drive much more and are much more spread out than other countries.
Somewhat off-topic, but I think this explanation of the car accidents misses the mark. It might be part of the cause, but car accident deaths were dropping in the US and Europe until around 2010 when they kept dropping in Europe, but flatlined and then rose back up in the US.
I don't think we were driving less and less and are now driving more and more. We changed policies. (I think it's mostly bigger cars, plus less focus on traffic infra/systems/rules that prevent fatalities.)
I'm sure there are qualifiers and rationalizations when comparing cancer survival statistics as well. As someone already pointed out, uninsured Americans aren't necessarily included. They cited sources, you rebutted without sources.
I believe a single payer system is a matter of when not if. It works just fine in other countries. In our country, the healthcare industry has evolved into extracting as many dollars as it can from the economy. This is because when someone is sick and needs treatment, healthcare providers hold all of the cards.
Uninsured Americans are in fact universally included in death statistics and in cancer survival outcomes; we have top-tier cancer outcomes.
Europe has several universal coverage systems without single payer. The original sin of the US system isn't private insurance, it's employment-based coverage; that's the thing nobody else has.
Again, you cite zero sources. Survival rates for lung cancer in the USA, the most common cancer in the world, are much lower than Japan and South Korea. South Korea is single payer and Japan has strict government regulation of healthcare costs.
The second most common cancer is Breast Cancer. The USA is "top-tier" but so is pretty much every other Western Country. Australia, also famously single-payer is a mere 0.4% behind the USA.
This is the only source of modern survival statistics by country:
Take note of two things: First, some states weren't even included in the study and second, there is no mention of insurance. The statistics are only tracking people who were diagnosed with cancer. It's reasonable to assume that some people who had cancer symptoms did not seek treatment because they didn't have insurance and died without being diagnosed.
I'm not saying that the US is better than every other country. I explicitly said somewhere else on this thread that there's a common critique of our outcomes that we just do detection better, and that our life expectancy outcomes aren't materially better.
What I am saying is that it's difficult to make a case that US life expectancy is materially altered by our health insurance system. You won't be able to use cancer to make that case, because the US has in fact quite good cancer outcomes. That's it: that's the whole argument.
Again, though: this repeated claim that "the uninsured aren't included in survival statistics" --- I don't know where that's coming from. It's not true.
I literally posted a link to the CONCORD-3 paper and made arguments using it and you rebutted it with "start with CONCORD-3".
I'll restate my argument so it's clear: CONCORD-3 does not mention insurance status anywhere in the paper. It does state that the statistics require diagnosis. It's reasonable to assume that if you can't afford healthcare, you're less likely to seek treatment or diagnosis.
You understand that CONCORD-3 is about tracking people who enter the healthcare system. Isn't it reasonable to assume that if healthcare is free or very affordable, there would be higher participation? And on the flipside, if it's outrageously expensive, there would be lower participation?
Sorry, I'm talking about US overall outcomes, which is what I thought you were arguing with all this breast cancer stuff. I am not taking seriously your axiomatic argument that the uninsured are systematically not captured by the US cancer statistics, and was not bothering to rebut that, nor will I.
His experience is completely plausible. He’s in a niche that requires highly performant code and most complex, highly performant games do nit have source available for models to train on. It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become.
Have you actually tried performance optimisation using an agent?
With any programming language/framework that has quality profiling tooling (which is a prerequisite for most projects) I have had huge success with automated hotspot profiling where the LLM can propose theories, test the impact of fixes, convince you of which to pursue, etc.
High performance algorithms are quite well documented so it isn't unreasonable to expect an LLM to apply them appropriately when given the ability to "see" where they need to be applied.
TFA's author's experience is the opposite of your claims. Your claims may be right in _your_ circumstances, but not theirs.
> High performance algorithms are quite well documented
That may be true for bloom filters or what have you. But the author states the obvious: all recent games are closed-source. So any algorithms or techniques for real-time 3D that an LLM was trained on are going to be a long way behind the state of the art. The author makes that point extremely clearly, and they have credibility.
> this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago
TFA? No. Your comments here? Yep. Try to imagine a world where different kinds of work have different applicability of tools.
> That may be true for bloom filters or what have you. But the author states the obvious: all recent games are closed-source. So any algorithms or techniques for real-time 3D that an LLM was trained on are going to be a long way behind the state of the art.
Neither you, nor the author of the article has to use LLMs for coding. But if you want to, there are some practices you should follow to get best results, and that includes setting up your environment to give the agent the best chance for success. If you have various rules that are non-obvious, you add them to your AGENTS.md file (as we have at my corp and in my little niche). The agent will then follow those rules, and learn from the surrounding code.
I'll just be blunt here - I don't believe the author would have done anything more than the bare minimum to test his pre-existing bias that coding agents are bad. I don't believe he would have put the effort into getting it writing good quality code to suit the project he was working in.
We write high performance, low latency java for trading systems. Our codebase is highly structured around this and the READMEs and AGENTs file contain the information for how to successfully write code like this, originally for human consumption, and now agents.
Yeah, but we already have rules-based coding agents, they are called compilers.
I have a suspicion that a lot of the stuff people use AI coding agents for would be served just as well by, say, the WYSIWYG editor from Visual Basic 6. But we threw away that kind of technology long ago and settled for Reactslop, and now you need an LLM to write the Reactslop.
I am trying to not sound offensive here, but given the question you're asking, I don't think we're going to join up here. I think one of us is missing something fairly fundamental, and my hunch is it is you.
The agent will already have seen enough super optimised code and read enough material about it. It will read the entire repo you're in, and understand how the code should be structured and written to work efficiently. And anything it doesn't get at first, you write about in the AGENTS.md file and tell it.
Then it works. If you can teach a human to write specialised 3d rendering code, you can distill the same teachings to an agent in text form, and it'll work.
I am supremely confident of this, and I bet that neither you nor the author of the article has bothered to try properly.
I suppose you wouldn't train it using a traditional LLM type of training and you might not have an LLM type of model either.
American Fuzzy Lop usually manages to generate valid files of any format you want, using a genetic algorithm that the author didn't even call ML. AlphaGo trained against itself.
Such things aren't impossible when you can automate the reward function, though you might have to come up with novel techniques.
LLMs work perfectly fine for me but I’m building web app equivalents of binder keepers, like most people. Essentially store data, display data. The novelty is in what/how we’re displaying. LLMs are owning this market.
Unless you yourself are doing commercial game development, calling the author disingenuous puts your own pro-AI bias on display. It’s based on speculation. I prefer to take his very specific examples and experiments at face value.
To be honest, the optimization the author talks about isn't really high level optimization. And Claude's pattern actually calls for a more extensible design. Strictly speaking, the author's instruction could be considered incorrect. I'm not trying to dismiss the author.
If performance were truly critical, they wouldn't have been using Unity in the first place. They would have used Unreal, as mentioned earlier. And if they were sticking with Unity, they would have tried ECS.
Unity is fundamentally based on the template method pattern. The idea of pulling Update out and handling it in a single manager class is really more of a small scale indie game approach. It's a technique that scales very poorly.
In practice, there are many better optimization techniques for GameUpdateable. So I'm not sure why this particular example was used to demonstrate performance optimization.
Typically, you could use GameUpdateable with object pooling, which would be a safer approach. There are also many batching techniques available.
In other words, this isn't about performance. It's a technique used for small indie game development. By handling it directly through a manager, registration and removal no longer depend on the Unity framework and become manually scheduled by the user. This, in turn, means you have to handle many more edge cases, which creates additional work. This is a common pattern, sacrificing future extensibility for immediate performance gains.
It's a technique used in small indie games. Converting per frame Update callbacks into a central loop that iterates over all objects is where GameUpdateable would actually be a better choice. So rather than viewing this as an optimization for performance, it should be understood as a design choice made to make small games easier to manage.[1]
I’ve specifically used Opus to diagnose and fix performance bottlenecks in parallel Rust code on multiple occasions (e.g improving NPS for a chess engine) and it works well.
I’ve done plenty of performance architecting in my day-job and rule #1 is generally “you can’t fix what you can’t see/measure”. I have a suspicion that many folks aren’t investing in letting AI actually introspect iterative execution via the appropriate harness, and are then acting surprised that it is no oracle.
I understand you're trying to draw a parallel (no pun intended) between what you did with Rust and the work the author performs as a professional game developer who optimizes game code for a living (it says this in his bio).
Since, I'm assuming you are not a professional game developer, the parallel is speculative and sort of reaching. Therefore, is it feasible to you the author of the blog knows better than you what tools work for his chosen field and that he came to the conclusions he did in good faith?
> It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become.
It is a common observation but I don't buy it. AI is clearly very good at Rust, but that is probably one of the least represented languages in its dataset. Anecdotally, I've also been having very good outcomes with a rather niche combination of technologies (opencv.js + JS in a browser extension) since early 2024. I would imagine there is way more C++ game code in the training set than that particular combination.
I think the more likely reason is that certain languages, projects or technologies tend to be organized in ways that are not ideal for LLMs. Specifically, I think Object Oriented approaches are not ideal for LLMs.
My theory is the key factor for effective LLM use is how effectively you can stuff the context with only the relevant data. OO tends to result in logic spread across inheritance hierarchies and templates (and even overloaded operators /shudder) which resides in a bunch of different files comingled with a whole lot of other logic. This just tends to confuse the LLM. On the other hand, I ended up using a lot more functional programming style which let me pinpoint the exact files or snippets of code relevant to a task, and the LLM pretty much never went wrong.
These days the models (and likely the harnesses) are much stronger and need much less curation of context, and hence can power through any kind of project organization. But I suspect they are still a bit sensitive to all the noise polluting their contexts and hence can produce very inconsistent results.
What I like is that this is a measurable productivity increase that can be directly linked to AI and they go over their methodology. AI-positive posts skew heavily anecdotal and hyperbolic.
This is a company selling picks and shovels for the AI bubble. Of course they're going to drip feed "AI is so awesome guys!" stories on their blog.
If you strip away the hyperbolic statement about compilers and the Empire State Building, it's an internal success anecdote.
Would be interested to hear how all of the self-directed AI decisions worked out after launch. This is where I run into problems with AI. Sometimes it makes product decisions that aren't immediately visible and make no sense and cause serious problems down the road. Also, an analysis of how much time was actually saved by using AI? since it sounds like there was a lot of guidance and testing to make sure what the AI did was correct.
Don't agree with the hyperbole in the article but I think the broad strokes are there. I'm the same as you, AI has excelled at small tasks and daily chores. Like writing a complex query in seconds that used to take an hour to work out.
Doing feature development seems to take just as long or productivity gains are modest (where I work at least) and I think the reason is touched on in the article. Most orgs are just terminally bad at building software.
Q: I need help doing X
A: if you’re doing X, you’re doing it wrong.
I propose the word shamesplaining. What do you think?
Not saying your opinion isn’t valid. It just doesn’t answer the question and it’s disturbing that this is the top voted answer. It sounds more like a criticism than an answer.
reply