That Aella girl just inserts herself into stuff with her very liberal about sex personality to stay relevant. She used to post edgy nerdy nudes back in 2010s on reddit. I wouldn't believe anything she says she does IRL tbh.
most people in ML have no idea what transformers actually are.
Traditional networks, at every layer, used to be output = [weights matrix][input], where input is a vector, and weights matrix is the weights, where each row corresponds to the set of weights for each neuron.
Transformers upscale the dimension of the data. Instead of the above, transformers do [output] = [input][weights_matrix]. When you multiply an input by a matrix, you get an output matrix back. Thats all that happens. Nothing fancy. You have weights matricies for K/Q/V, which when post multiplied with the input, give you the KQV vectors, and then you just simply multiply them together and apply a scaling factor.
There is nothing magical about K/Q/V. There is nothing about any one doing any querying or any one representing some keys. The naming is just a carry over from how they that selection process is used in pre llm data science fields where you manually define the key and query matricies to define relationships between components.
The reason of why it works is because is an extension of something called kernel tricks from pre LLM machine learning days - you map a lower dimensional space to an extra dimension based on some equation, and it lets you apply some classifier on the combination of existing values and new value. Thats what transformers are doing - they are mapping the individual token to the dk x n_heads latent space, which allows for a higher dimensional representation of the data, capturing complex relationships.
You can do Transformers with 5 matricies instead of 3, you can do this with 4-dimentional tensors, and so on. The thing is, there really isn't any way to tell if any of that gives you more advantage - it certainly would give you more granularity, but as of right now, in terms of training to generate a specific token given previous ones before it, it seems that you don't need any more dimentions than dk x n_heads. Interestingly enough, you also can mathematically represent any such transformer including the starting one with a sequence of linear layers like in traditional networks, the only thing is that it becomes computationally inefficient due to having duplicates of data.
The reason why RNNs and others and others didn't work is because RNN training is effectively trying to linearly regress on chaotic effects - i.e what set of starting conditions would evolve with a given process into what you want. This is an NP hard problem, and you can't really do it linearly.
Transformer models on the other hand, use breadth instead of compute to capture interactions. In those learned weight matrices, you have a latent space of a bunch of "knowledge" compressed, and an algorithm to search on that "knowledge".
But, its very possible that an RNN can be smarter than a frontier model while being much smaller in size - in the same way that its very possible that you can have the right set of prompts for an existing local inference smaller model that can basically be very close to AGI in terms of being able to solve any problem across any domain. Right now, the space is about exploring those prompts, which is the frameworks and harnesses, to get to there, as well as making the compute portion more efficient so you can explore that space faster.
And the thing that comes after harnesses/efficiency in terms of progress should be obvious if you understand all of the above.
Hey I like your exposition of attention in terms of the kernel trick, but the big huge difference is that kernel methods use the inner product which is a commutative operation -- it's bidirectional (and both tokens are projected into the space by the same function prior to being dot-producted).
This means that it can't capture unidirectional relationships, like "ball" is the object on which the verb "threw" acts in the sentence "I threw the ball". This relationship is true in only one direction; it isn't true to say "threw" is the object on which the verb "ball" acts.
I do think it would be fair to say that transformers generalize the kernel trick to noncommutative relations by applying a different projection function (W^Q and W^K) to the two tokens being considered. This makes the overall operation (project then dot product) a noncommutative operation.
And the thing that comes after harnesses/efficiency in terms of progress should be obvious if you understand all of the above.
Nirvana? Singularity? Paperclips? Vernor Vinge rising from the dead? I'm curious; please share!
I mean, given sentence construction, you don't really need to capture directionality, you just have a mapping of how sentences are constructed to the latent space of some representation.
>Nirvana? Singularity? Paperclips? Vernor Vinge rising from the dead? I'm curious; please share!
Simulated evolution. Thats how you "solve" highly nonlinear chaotic systems. And generally, if you think about it, you have to have some secondary system on top of the knowledge embedded in LLMs to drive them to select certain tokens, which then starts to eerily resemble what humans call emotions in themselves.
I mean, given sentence construction, you don't really need to capture directionality, you just have a mapping of how sentences are constructed to the latent space of some representation.
Uh, no, you absolutely do need directionality.
Without directionality you're simply grouping words into equivalence classes. That's nice, but graphs are strictly more powerful than equivalence classes.
Simulated evolution. Thats how you "solve" highly nonlinear chaotic systems.
Eh. I dunno. Evolution is a horribly inefficient way to do learning. Nature uses evolution because it's the only learning algorithm that you can implement using uncoordinated chemical reactions on DNA/RNA/proteins.
As soon as evolution hits a point where it can build neurons all the effort switches over to gradient descent. Just look at humans.
Even cellular/self-organizing systems can be driven more efficiently by gradient learning than evolution... if you want your mind blown, read the paper that this page summarizes:
Im saying you don't need to capture directionality if you capture all possible cases of sentence construction.
As for evolution, you can still go gradients, the problem is that you can't do gradients in a space with many false positives. You need some method of figuring out the true optimal point.
> You can do this with 5 matricies instead of 3, you can do this with 4-dimentional tensors, and so on
As a outsider I have many dumb quesion like these. I am trying to understand transformers in a Occam's razor way. It's a complicated machinary after all.
Imagine you have a soccer field, a ball with position x and y, a kick strength, and direction in an angle. Your job is to write a function that determines if the ball will end up in a goal. So that is 4 values. However the function itself will contain many intricacies, like trig functions, simulated drag, and so on.
In the contest of LLMs, you cant have these types of coded function. Your function has to be a mathematical equation that is smooth - i.e no discrete steps, no singularities. The reason for this is when any neural net is trained, you use backpropagation of the error to adjust weights, and how much you adjust them is directly proportional to the weights effect on the final output, and in order to compute this, you have to have smooth functions from start to finish.
So what you do instead is you add data to your 4 values, that capture different relationship between them. If your 4 values are x,y,k,and h, your first data point can be a1x + b1y + c1k + d1h. The second point can be a2x + b3y + c4k + d5h. And so on.
You can have as many of those values as you want. And then you can add, combine, and scale those values in any way you chose.
This basically gives you a map of 4 values into a binary decision whether the ball will end up in a goal or not, after sufficient training. However, the total number of extra values that you chose has to be large enough to capture all possibilities - if you don't have enough, you will start to make mistakes for some initial conditions.
Very interesting analogy, thank you! When I initially learned about linear regression, i learned that to capture non-linearities, instead of choosing a more complex, non-linear hypothesis function, I can just come up with "arbitrary" features for my data set. Basic example: house price calculation. Obvious features are square_meters, age, n_rooms, ... But I can make even this linear model learn complex connections by transforming or combining these input features and add them as additional inputs, such as n_rooms * age, or log(square_meters) or whatever.
What you're explaining sounds very similar. Is it, or am I understanding it wrong? (Idk why it's so hard for me to understand this attention thing...)
If you think anything Elon doing is groundbreaking, you have no idea how the world works. Recent Space X ipo showed that the launches aren't cheaper, they are just heavily subsidized. Tesla was a piece of crap until they got their model 3, the only reason Tesla succeeded with their S model is because Elon was the edgy hype dude who managed to generate enough hype to carry them through the bullshit with the car. Self driving was supposed to be solved last year, and tiny companies like Comma AI manage to build self driving systems that are in someways better than Teslas.
I bet you think Steve Jobs was a visionary as well lol.
>If not for Tesla there won't be EVs you see today.
Most manufacturers already were working on hybrids, which to this day are still suprerior to EVs. Chevy Volt, outside of being Chevy, was still one of the best cars ever made for utilitarian purpose. Nobody wanted to foot the bill to do electric conversions until this was necessary.
Tesla only opened up a market segment for high end electric cars, which I guess is cool, but far from revolutionary. The model 3 was a big success only because again, it was subsidized. Meanwhile BYD actually makes cheap affordable electric cars, and we both know why they are not sold in US.
>If not for SpaceX there wouldn't be gigabit internet connectivity in the middle of the ocean.
Plenty of companies were doing geostationary orbits with satellite connectivity. SES for one.
Any more Elon slop? You realize you are defending a dude that is literally a Nazi, right?
My response was to your statement “Give me one actual reason why a smart person would take an underpaid overworked position at any of his companies.”
I hated seeing him do the “Roman” salute and will not try to defend it. I said nothing in favor of his politics or personality, only that his companies are ambitious. Seems like you may benefit from a few deep breaths before responding with attacks.
Also, the alternative technologies you propose are pretty weak comparisons as they are of a different classes.
Your response was along the lines of "despite Musk's character, his companies do groundbreaking stuff, so in return for being overworked and underpaid, they get to work on things that no one else is working on"
My point is that none of the companies do groundbreaking stuff, and there are other options for people wanting to work at those.
I guess we’ve found the point of disagreement then. If you are disregarding the scale, then you might be able to make a case, but even then, I’m fairly certain that you would still be wrong.
> More than 4,400 current and former SpaceX employees are likely to become millionaires in the I.P.O., according to an analysis by Hill.com, a San Francisco-based investment platform. Of those, about 400 are expected to earn $100 million or more.[1]
Not to mention the whole launching reusable rockets thing which is pretty cool too.
I mean maybe the super smart person will work at GOOG/META/Netflix to do some absolutely ground breaking work stuffing ads and slop pixels into as many ocular nerves as possible.
However, I could see a smart person taking an "underpaid" (SpaceX SWE: ~$200-400k pre-IPO) to do the following:
Neuralink: to see the impact of increased independence and autonomy of a paraplegic one day after the operation
SpaceX: to quite literally approach the final frontier. Currently launches 80-90% of all orbital mass. Starlink is saving lives constantly.
Tesla: to kickstart the EV revolution and reduce fossil fuel dependence
Boring Company: to radically decrease tunneling costs applicable to all sorts of critical urban problems from transportation to utilities etc
The better solution for security is sandbox execution. Most agents used in practice, if given terminal access, can find ways to get around most of the MCP restrictions.
You can verify this yourself. Get an old laptop to act as a wifi hotspot and forward traffic over usb ethernet adapter to your actual router. Then run tcpdump on the computer. You will see the multitude of phone-home traffic.
Is there evidence of this anywhere on the internet? I couldn't find anything with ten minutes of searching, and I don't use little snitch, and I'm not going to put the effort in to remediate this just to figure out if this comment has weight.
> As it turned out, this behavior is on purpose. There’s an explicit whitelist that allows certain macOS services to bypass any third party firewalls and to communicate on the Internet without being even noticed by the user. A hole in the wall.
Was remediated later, but shows there is precedent.
What is "root" in macOS is more or less a power user role. Apple took away the true ability to operate as root a long time ago with System Integrity Protection which walls off the critical parts of the OS from the user entirely.
Unfortunately, SIP also restricts basic system functions that are trivial in other OSes. Apple made this very difficult, and MS would have loved to do this in Vista had they not received the backlash that they did.
You can disable SIP if you want. You can run whatever you want on your machine. Disabling SIP is indeed a security risk… then again, so is running something as root. Having a Mac w/o SIP is no more a security risk than having a Linux w/o a r/o kernel AFAIK.
The way apple talks about SIP its like a miracle we survived at all pre SIP. Pretty sure I've had SIP disabled for years now when I got nagged for something or whatever and had to disable it.
SIP is a good thing and prevents a lot of (usually PEBKAC) problems. It is also the direction where Linux is going (is now? or maybe they gave up? idk, it’s been a while since I heard about immutable distros).
For tech-aware people, it is probably not necessary indeed.
However, given that now “tech-aware” people are running completely random and unvalidated scripts w/o second thoughts (or even first thoughts actually) on their main machine, I’d say the “tech-aware” line is very very high…
Well, I block all connections to iCloud for every app (user and system) completely with Little Snitch, and the fact that it does indeed break some little features I can absolutely live without, indicates that it has at least some effect.
But still, I fully share the sentiment that creators of an OS are perfectly capable of bypassing whatever there is running on top of it.
When you are doing matrix math, compute is compute. Apple cant be more efficient due to physics. The only reason Macs are more efficient in general is that they have tightly bundled hw and sw for specific tasks.
Ive essentially followed that paradigm with Python and C. I start out writing Python code. If I need something to run fast, I build a standalone C application that either reads from a file or listens on a socket, and just invoke it from Python. No need to write the entire thing in Rust and deal with all its semantics when it will be at best like 2% faster.
Write python code, ask any llm to translate it to C, then compile the C code - if it produces errors or fails to run, ask LLM to fix it. Then take it a step further and ask it produce machine code, and repeat the procedure.
Then RL the llm on the above, and you basically have a Python -> Machine code compiler. If you cover every single possible python syntax, every single possible C syntax, every possible standard library call, and all the compiler optimization examples (all of which is a final set), you should get something that is extremely accurate.
Your analogy is poor, and you are missing a very important fact.
Most of the human written code, in places where that code needs to make money, is decidable either entirely or in large parts. I.e without running the code, you can take a domain of inputs and build a complete range of outputs solely by looking at the code.
The way that works in your head is that you are effectively doing a compilation to a logical like structure, which then you can use to infer what the output will be from what the input is, and its a direct mapping that is invertible and separable, so if you know what the output should be, you know what the input is, you can pinpoint the exact location where it breaks. Thats how humans write code.
If thats not clear, imagine a piece of code that splits strings by spaces, deletes the empty strings, and returns the number of words in a string. The fact that you can say that if you want 3 words, there should be maximum 2 sequences of continous spaces between words, is you effectively transpiling that program into a latent space inside your brain neurons and inverting it.
LLMs essentially do this, with the added advantage of having been trained on a HUGE number of codebases, so they can recognize patterns that a human cant.
Where LLMs struggle is complex behavior - they can't simulate things like a human can and choose the best course of action. Even harnesses for agentic loops that can auto run and debug code can't match what a human can do in this regard (hence why self driving still sucks rn).
So moving forward, being a good coder isn't going to be about writing code, or even about prompting LLMs. Its going to be all about whether or not you can design good custom agentic loops, which necessarily involves knowledge of the model at hand (i.e what words you have to use to get it to do the right thing). This will be especially true as investment into "private" inference grows where companies will be using smaller models that have less detailed RL and thus will need much more guidance to do the right thing.
I disagree, to keep it short, what you're describing is not understanding, it's superstition. And I think it's a wrong direction of engineering, relying on some sort of irreproducible expert intuition, one that has been successfully replaced by enlightenment and scientific method.
There are 3 major obstacles in understanding LLMs:
1. They use inscrutable internal language of embeddings
2. They communicate in natural language which is itself ambiguous
3. The weights and training inputs are being hidden as a "trade secret"
"with the added advantage of having been trained on a HUGE number of codebases"
This doesn't really mean much unless we understand what is the quality and relevance of these sources for the problem at hand. Without this understanding it's just a superstition.
>They use inscrutable internal language of embeddings
No different than the electrical signals in the intermediate neurons in your brain that comprises the latent space where all the processing happens
>They communicate in natural language which is itself ambiguous
They lack one-shot precision, sure, but it doesn't matter. They are precise enough with refinement over multiple prompts.
>the weights and training inputs are being hidden as a "trade secret"
For frontier models that make the company money through api pricing, sure. There are plenty of open source models that can be used for the same tasks, which have open weights.
>This doesn't really mean much unless we understand what is the quality and relevance of these sources for the problem at hand.
All of the modern models are RL trained on specific tasks when it comes to coding. I.e the initial training run learns to predict the next token based on context, from all the available texts, but then the RL runs specifically train the model in a harness where it produces code and RLed to produce correct code with specific formatting.
reply