By verifying that all of its possible behaviors conform with the "it works" spec, regardless of which of those behaviors it chooses.

Monitoring with a known-safe fallback is the easiest case.

USA-specific: At ~$38K/year (including SS) you probably need to own a home without substantial deferred maintenance to make it work. So you can retire on $250K but only if you're over 62 and only if you have substantial wealth outside of those liquid assets (so probably a net worth at least 2x your liquid). If you have a spouse also on SS it much easier.

scala's repl is decent. It has its annoyances, but so does python's (white space sensitivity + repl + terminal emulators stuck in the late mid century don't mix).

If you've ever read through FORTRAN code from a mathematics department or MATLAB/C/C++ from (non-software) engineering disciplines, then you probably understand why productionizing a jupyter notebook is definitely not the worst of all possible worlds.

Also worth mentioning 2026 - 4 = 2022. Not a great year to be stick building from a cost perspective.

Author is a senior staff engineer. A big part of his job is to help his org's engineers and non-technical decision makers understand basics. My assumption (because it's my experience) is that he is writing this piece because has seen this confusion in others > N times.

And to be fair to those juniors and less-technical folks: big groups of AI phds make the same mistake.

Papers in prominent ML conferences up to like 2024-2025 that were consistently presenting comparisons of proprietary model end-points to open weight models as if they were apples-apples comparisons. Even though there were obvious examples of prop model providers doing "stuff" behind the endpoint as early as Fall 2021.

Most of those papers did not put in the legwork to figure out if/when that "stuff" was happening, or work to figure out what the "stuff" probably was, or conjecture to what extent that "stuff" might overlap with the benchmarks being used, or try to add basic interventions to open-weight models to get a better comparison point, or even simply state the caveat that they don't actually know what happens when they hit a chat completions endpoint!

Not exactly the same thing as conflating agent with model, but 100% conflating "model+software" with "model", which is at the end of the day basically the same confusion. And those papers were being written by and for llm nerds. So if ICML/NeurIPS authors and reviewers aren't careful about "software+model" vs "model" in stating and evaluating their claims, it's unsurprising that some a SWE or non-CS SME also doesn't understand how the LLM software stack works.

It doesn't help that some model providers (OpenAI...) actively obfuscate how the stack works in their marketing.

I'm deeply suspicious. I do not yet have a concise statement for why, but a lot of literature on the sociology of knowledge work sort of points at my thoughts.

Section 5 of the Thurston article cited by Tao touches the elephant. Raduchel's article on the economics of software [2] also touches it.

I've tried to put words to this for a few years. I think I'm just going to start writing versions of it as see if that helps me shape the thought into something more concise.

So, in the spirit of this article's style, here are some postulates:

1. There is a sociological process happening in the production function during knowledge work.

2. That production function and the associated sociological process spans years or even decades, and must outlast many of the artifacts that are produced during the early years of the function.

3. You cannot get the right lines of code or the right theorems proved without running that sociological process alongside the artifact production process.

4. It is impossible to completely separate the sociological process from the artifact construction process. If you just iterate on artifacts then too much of the required hidden state is lost to make progress in the right direction. This is true even if you include distilled artifacts capturing pieces of the sociological process (eg meeting notes, documentation, commit logs, prompts).

5. So you need that sociological process, or something like it, to still happen.

6. For a lot of knowledge work that process plays out in extremely high-fidelity social interactions [3] that we have not yet captured in the datasets that would be required to reproduce those dynamics.

7. And even if we do collect that data, our current architectures and training algorithms and hardware would be useless given the size of the datasets.

So: the technology today gives us the ability to iterate on the production of artifacts. But it does not sufficiently simulate the social process which gives rise to the Right artifacts.

This isn't exactly what I actually think, but it's a version of the thing that I intuit when I watch heavy use of AI in both software projects and formalization projects. And simulating that process feels way harder than people are currently assuming.

[1] https://arxiv.org/pdf/math/9404236 Section 5.

[2] https://www.nationalacademies.org/read/11587/chapter/11 pp 166-168.

[3] there is a reason we still gather in-person around white boards, and why doing so is more crucial for some types of work than others.

I would use a different adjective: exceptional.

There are not not times where lone geniuses produce amazing output. But most of humanity's progress over the last few thousand years (or at least certainly the last few hundred) resulting from a different type of work.

Referring to my list of examples as exceptional in the sense of being rare is rather unfair, given how numerous these examples are relative to the body of mathematical work we would consider incredible and especially relative to the desire of the average human to socialize. The fact that such a large % of that body of work occurred while the individual was in relative isolation is something we should pay attention to.

If people don't like socializing then your argument not only falls apart, but actually leads us to something like the opposite of the conclusion you found.

And people definitely don't like socializing! Because we're not talking about "socializing" in the sense of parties or even water cooler conversations.

We're talking about socializing in the context of a group of professionals working together on under-specified problems/solutions.

The English word for that type of socializing is:

Meetings.

Saying that "people like socializing" in the context of professional knowledge work is like saying "people like woodworking" in the context of stick framing cookie cutter developments in Georgia in the summer.

I don't think I've ever worked with a productive scientist or mathematician who didn't loathe meetings. In fact, I think I can probably count on one hand the number of people I've met who seem to not loathe meetings.

Also: there are pretty powerful social/professional forces -- in Mathematics and related disciplines in particular -- which incentivize people to play into the perception that they are a "lone genius" or at least the "main character".

So the group of people we're talking about both hate meetings and have a strong incentive to be perceived as an independent genius/main contributor.

And yet the vast majority of good science and mathematics involves large groups of people (who mostly complain about the resulting necessity of meetings, lol).

I'll also submit that most of the "lone genius" instances are confounded by the strong social incentive to be perceived as either a lone genius or at least the main contributor. There are at least a few prominent historical examples of math/math-adjacent giants who "worked alone" but were in fact in deep conversation with both the literature and their peers.

But, again, there's a lot of gray area, and that's what I meant in my original post when I said "This isn't exactly what I actually think". Two examples of gray area:

1. I'll certainly concede that there are some types of work that do not require that sort of social process to happen. That's the sort of work you seem to be pointing toward here. But opportunity to do that type of work is rare in general, and those opportunities are far more scarce today than it was in 1690 or even 1990.

2. There are also some types of work where the corresponding sociological process is pretty simple and is probably sufficiently captured.

So I'm not bearish about what's possible, at least per se. But technology executives and analysts in particular have some really fundamental misunderstandings about how this type of labor works in most cases, and I see some of those same misunderstandings among lots of professional mathematicians.

(And I only reached for the LLM after working through all the flowcharts and tables in the shop manual, reading relevant parts of two engine repair books, and watching a bunch of youtube videos.)

It's especially awesome in domains where:

1. you personally don't have the time to learn,

2. you can trivially evaluate the output ("does engine go brrrrrr or brr-splaugh-crunch"), and

3. there are no bad outcomes; or, there are bad outcomes (destroy engine, hurt myself) but you can augment with ground-truth ("do X as suggested by nlp machine but follow shop manual for doing X").

It was a 3B or 8B model too, so I probably burned more energy on unnecessary oil changes than on running the llm inference.

Say what you will about LLMs, and dear god can the claude-style harness outputs be annoying in the workplace. But they're great tools for accelerating the sorts of ultra-narrow-path learning curves that commonly pop up in tinkering/hacking/similar sorts of activities.

(Also, people without critical thinking skills believing everything they read pre-dates LLMs by thousands of years. This was a widely discussed problem with popular media consumption in the USA during the several years preceding the invention and popularization of LLMs (say 2015 to 2021). There's a different, more fundamental educational diagnosis for that problem, which LLMs may exacerbate, but which they did not cause and cannot cure.)

So, to be fair: most philosophy majors wouldn't have much luck with Rudin.

> Dive into the classics after you have gained the maturity from modern texts.

Diving into old texts is a skill unto itself. That's why a lot of institutions do the great books thing as a core curriculum (so, maybe 2-3 courses taught in this style, as an alternative to more conventional phil 101/history 101 style distribution requirements). Then a more conventional education from there onward. The theory is that this is a mid-point precisely because it provides lots of transferable skills for diving into the classics in your chosen field, while avoiding the "let's learn analysis from descarte" excesses.

Of the lot, Elements feels misplaced.

Lots of people actually do read Elements as part of their course of study. It's niche but there's a whole cottage industry within academia for that sort of thing. There are probably over a dozen institutions that have either a degree program or a core curriculum that is organized around original texts, with Euclid usually serving as the math distribution of that sequence. So running into people who have read (big chunks of) Elements is not that uncommon. That's true even IRL outside of online discussions forums on thread topics that likely select for such people.

My impression is that this is not really true of the other examples. Except maybe Godel's proofs; I do think a sufficiently motivated instructor could pull a decent chunk of college students through the original text in a semester. Probably better ways to spend everyone's time, though.

... plus it's relatively approachable as such things go, and short enough that closely reading most or all of it isn't a crazy idea, and it was recently-enough widely used as an actual textbook that between that and ongoing modern interest in its use in that capacity, there are tons of study-oriented editions of it floating around and still being published. I mean hell "recreational mathematics" is a thing and lightly-annotated-and-updated Euclid's a pretty solid text for people with that kind of interest to noodle on, with bonus historical appeal since it's super-old and also is assumed background for all educated people into at least the early 20th century, so pops up all the time in historical writing and literature.

Now, Newton? That's more like it. Nobody reads a large amount of his mathematics unless they're some variety of mathematical historian.