I'm confused. They already "work with leading AI companies", manifestly failed to maintain containment ("highlighting the problem Irregular is trying to solve"), and somehow they're playing it as a win?
The whole point is the quantified output. If you just ask an LLM to type out its confidence "manually", it'll make up some nonsense. The logprob numbers are more reliable.
I got this technique to work extremely reliably last year. However there were a bunch of caveats:
1) Firstly, you must institute a check that the multiple choice tokens dominate the output distribution. They should sum to 95% or more, ideally 99%, or the LLM is not following instructions properly. This is also the problem with constrained decoding - if the LLM really doesn't want to output a valid answer, the one you extract will not be high quality.
2) You need to ask it multiple times, permuting which option corresponds to which letter, and average the results. LLMs are surprisingly biased towards picking "A", especially if they're otherwise not sure.
3) For the same reason, performance improves if you frame the prompt as if it were the middle of a quiz. "Question 1" carries baggage that "Question 12" doesn't.
4) You must be exceedingly careful with tokenization.
But when all was said and done, I got a general purpose A/B classifier that gave high resolution quantitative output for the cost of a couple dozen tokens ingested and a couple inference passes.
>The whole point is the quantified output. If you just ask an LLM to type out its confidence "manually", it'll make up some nonsense. The logprob numbers are more reliable.
The whole point of my argument is that neither is good, but from a technical perspective logprobs is probably the worst unless you train a model on specific outputs. In which case you'd throw out the generality again, so when I think about it more, it's actually the worst overall. In my experiments, having the model simply assign "high" or "low" probability in a structured output generally performs best. You can try numbers, but you will never get anything close to what you could expect from traditional ML. And most certainly not from logprobs.
Yes, for vision classifiers we have in prod, I've seen a huge difference between A, B, C, 1, 2, 3 style answers and emitting a string. Even from just base model behavior pre-sft/rl. It was one of those obvious in retrospect moments.
> LLMs are surprisingly biased towards picking "A"
GP pointed at a causal explanation for this: almost every sentence in English that's a statement will start with "A" or "An", so "biased towards picking ''A''" will include most attempts at saying anything long-form for any reason.
I don't think that's the source of the bias I saw. I am confident that my prompting strategy eliminated attempts to generate long form content - specifically, I took care to wrap (A) and (B) in parentheses, so the completion looked like "Answer: (" - with this scheme an LLM is very unlikely to want to write "Answer: (A sentence goes here...". I know this, quantitatively, because I reliably got 99% distribution coverage with only A+B - that is, no inclination to write "The" or other common sentence starter. That's the beauty of the scheme - you can pretty directly and quantitatively validate how well the LLM understood the instructions. You expect it to only output A or B - so does it?
Meanwhile, the bias could be as much as 70% in favor of A in ambiguous cases - a signal completely drowning the <1% inclination to violate the format.
I would also note that models aren't people and don't think like people, so it's also possible that (at least for autoregressive ones) it could just be more likely to say "A" than "B" at that point, not necessarily because of "want" or "reason" but simply because that's what it was trained to do (such as in English writing).
What about switching to numbers or just some random Unicode character like smiley faces. Could be interesting if someone tested what LLMs like to say on a "cold start" lol.
Sure, it was high resolution (precise), how was accuracy compared to Jev (or existing open source implementations of the same concept, like laya)?
Also, Jev/laya do it in one forward pass, for multiple questions about the same state, rather than multiple passes for one question about that state. Well, for the usual multilingual configuration, two forward passes through different small models for laya, but that's because one is the router which chooses which model should do the real work, but still.
I never claimed that what I did was comparable to these modern options - I didn't validate it in more than an ad-hoc way anyway, and it used an off-the-shelf LLM rather than something specially trained. I didn't consider it worth releasing or making a big fuss of.
I contribute my experience here only because I've seen a lot of chatter lately about doing exactly this sort of thing, and I thought I'd share how I made it work for me. There are a lot of ways it can silently fail and give bad numbers if you aren't careful, and I wouldn't want people to think it doesn't work just because they used a vibe coded GitHub project from the last 48 hours that doesn't take these things into account.
Ah yes, he "even talked to them". You don't think that perhaps therefore Edward Snowden might've had a better conception of the viability of "the proper channels" than you do?
And, before you answer that too hastily - have you ever heard of William Binney? How about Thomas Drake? Do you know what happened when they used "the proper channels"? If so, your failure to mention them when discussing Snowden's motivation is in bad faith. If not, you're simply not informed enough to be coming into threads speaking authoritatively about this.
“Snowden talked to the IG” only applies to his conversation with the CIA IG about management and IT. There was no NSA IG complaint about surveillance which contrasts with the others (Binney and Drake that you mentioned). So no he did not even try. Was the Binney and Drake incidents ugly? Yes. But neither are in jail and both still live in the US.
They said that Snowden caused a shift in the public debate not that nobody listened to them beforehand (you should know they were raising concerns going back to 2002).
Baltimore Sun was reporting on Drake in 2006. Binney was publicly talking about it to Wired News and Democracy Now in 2012. Snowden happened in 2013.
So that makes my point even plainer than it possibly could have been made. The people who did the right thing by your definition got nothing done (and this was what caused Snowden to do it the way he did) and since Snowden, those previous cases were taken seriously by the government
Well their complaint triggered a formal DoD IG investigation and produced a highly critical audit plus fraud investigations. Trailblazer program was killed in 2006. That all happened well before Snowden. Snowden skipped all of those channels entirely. Binney and Drake were taken seriously but you are glossing over all do the other stuff that Snowden leaked that had nothing to do with the privacy concerns deemed that went beyond what FISA courts authorized (and no court found violation of 4th amendment).
Snowden did it completely wrong and you can’t convince me otherwise. I don’t see this conversation going further as you move the goalposts.
I'm "glossing over it" because you are, I explicitly asked you in another thread what he leaked that wasn't relevant for the public interest.
I'm not sure what your angle is, but the NSA proved that they were beyond contempt when they lied to the Senate and there were people in the room that knew the situation who didn't pull them up on it.
Regarding Binney specifically; the IG's report largely supported his claims, but it was classified and changed little. Thomas Drake, a serving NSA official who raised similar concerns internally, also got nowhere.
The government's serious attention came only after the New York Times revealed warrantless wiretapping in December 2005, and it came as a leak investigation. In 2007 the FBI raided the homes of Binney, Wiebe, Loomis and Roark; Binney has said agents drew guns on him while he was in the shower.
Drake was indicted under the Espionage Act in 2010. The case collapsed in 2011, he pleaded guilty to a minor misdemeanour, and the judge sharply criticised the prosecution's conduct. So the substance of Binney's warnings was mostly ignored or denied.
Binney went fully public around 2012, and NSA officials publicly denied that they kept files on ordinary Americans.
The core contrast is that Binney went through proper channels, took no documents and was not believed. He was investigated anyway and achieved almost no reform. Snowden bypassed every channel, took vast amounts of classified material and was pursued far more aggressively, but he forced a public debate and concrete legal changes.
You can pretend that's not true, but you'd be ignoring reality.
That GCHQ was involved definitely interests me, that GCHQ was being used by NSA to spy on US citizens (and vice-versa) to bypass citizen protections should be of interest to civilians of both countries.
I don’t trust civil servants to not abuse access into everyones communications, the damage a stalker or pedophile can do is immeasurable, and we have some statements that was happening.
As loud as possible? That's naive. They would have black bagged him instantly, no trial. In fact a public trial is his only stipulation for voluntary return - that this has been denied should tell you everything.
That's my current setup. Only 2 pcie lanes on the nvme though. CPU is feeling a bit long in the tooth for some compiling tasks.
The current t14 with the 16:10 screen and improvements to repairability look good. I'm trying to patiently wait for them to start hitting the used market, but I also recently snapped off one of the two usbc ports on the 480.
Hey, me too :) I made myself one of those T25/T480 Frankenpads. But I'll probably use it until I spill coffee on the keyboard, because the mod cost the earth and everything except the keyboard is replaceable (and indeed, everything except the motherboard and bottom case has been replaced).
Bad luck about your USB port. At least replacement T480 motherboards are not scarce!
Note to lazy skimmers: this is not ZuckOff (which is a cash grab slop fest) but the app whose idea it stole - a hand coded, opensource equivalent with no trackers or in-app purchases.
That argument applies to all software. Yet we still run software locally - not just commodity software either, but even complicated, heavy, niche software like Ansys Workbench. Sovereignty has quite a lot of value, it seems.
reply