Hacker Newsnew | past | comments | ask | show | jobs | submit | largbae's commentslogin

Seems off-topic for HN, but let's look at it technologically: if Ukraine and Iran have demonstrated anything, it is that drones are like guerilla warfare squared: even the smallest player can land hits and deny areas to the largest forces.

It might be tending towards impossible to achieve regime change militarily at this point. Maybe we all should just figure out how to get along.


Alright AI maximalists, what's the estimated token budget to remove the Google dependency?

GrapheneOS has the bootable AOSP and will have Google-alternative device support.

We probably need an equivalent to Play Services, app signing/porting/publishing tools.

With these in hand could we talk Valve into providing the scalable alternative to the play store?


GrapheneOS accepts donations and, to my knowledge, they spend it in hiring full time engineers.

They have replacement portions for Play Services already (attestation, an app store, and location), but it'd be interesting if they also offered something for push notifications.

If you have it to give you can spend your budget on a donation and fund the effort directly.


I believe GrapheneOS talked about hosting their own unifiedpush server and baking it into the OS not long ago

The trouble is that banking apps will not work if you degoogle your phone. GrapheneOS was an attempt to make this sort of thing work, in a phone OS more secure than stock Android

But to be honest, many banking apps randomly stop working with GrapheneOS anyway. So if you see this...

https://privsec.dev/posts/android/banking-applications-compa...

https://github.com/PrivSec-dev/banking-apps-compat-report

And check the issues, it's unpredictable whether a bank will continue to work with GrapheneOS

So.. if you accept GrapheneOS might not be reliable for this use case, and decide have two phones (one just for banking stuff, another for.. using), then degoogling is fine.

Only thing is that the banking device probably needs to be a phone (or tablet I guess), I don't think you can emulate a real device good enough for it to work on something like Waydroid


Yes, if the banking app refuses to run on Graphene, you can forget about Waydroid completely. Hell will freeze over before Waydroid passes play integrity.

My bank can support my phone or accept that I'm going to come waste their time in person, I'm not backing down here.

Many banks nowadays charge higher fees for teller service.

I went in to cash a check at a bank a few years ago and they made a huge show of it, taking over half an hour and various employees for a couple hundred dollars. They will absolutely win if they want to. A large organization can easily create more friction than an individual can bear.

Thats their issue that they want to pay several staff half an hours wage for cashing a cheque. I would have been enjoying every second of the farce.

Your time must not be valuable to you, which is a very limited perspective.

You don't understand. They will make you pay for it by charging a handling fee. How about $10 flat per transaction?

I didnt understand that because the parent comment made no mention of that, only the time the bank took. So I would argue you dont understand, as maybe some banks do charge for the privilege, but that is not the point that was made and I was replying to.

This is something that gets mentioned a lot, especially from people who don't use Graphene or other custom ROMs themselves. Personally, I have not had any issues using banking apps on Graphene without Play Services. While this is just anecdotal, even if apps required them, I use a separate "secure profile" where I dump all the fishy, but necessary apps that require Google. I used to run a separate Lineage phone with microG for this purpose, but it became too cumbersome

My phone is not that old but it stopped getting OS updates and I stopped considering it a way to do anything banking related when my bank declared that my phone OS was too old to run the app.

Literally insane that banks will allow you to do banking on ancient phones with an unpatched Android full of CVEs, but won't allow you banking on a modern ungoogled OS.

It's not about security, it's about lazyness. These are the people who still employ Cobol devs cause they don't want to migrate off existing mainframes. I've yet to see a banking app that was not an objective trainwreck.

> cause they don't want to migrate off existing mainframes

Dont confuse intent with laziness, running on cobol genuinely gives them many advantages.


This is what I ended up doing. I have LineageOS running on a Moto G. For banking, I have a tablet.

I didn't want two phones, because it just seems so silly. The tablet, based on it's larger size, at least offers things that another phone cannot.


People always bring this up but I don’t get the issue. How often are you using your bank’s app? Banking apps are probably the least essential apps on my phone.

some people use banking apps multiple times per day to pay for coffee, food and so on. And once the app breaks it's really nice to live in a society where cash is not outlawed yet.

The token budget isn't to remove Google it's to create drivers for individual phone hardware, which phone and chip manufacturers keep closed source. Also it would be used to find exploits to unlock permanently locked bootloaders

GrapheneOS won't be interested in other hardware support, since the only devices that meet their hardware security standards are Pixels and the upcoming Motorola phone.

The answer to this is - the real world comes knocking. Govt, banking, investment, grocery, transport, even some mainstream social/communication apps, all stop working - yeah, kinda cold turkey. And if your, or someone else's, answer to them is: ".. well then don't use those apps.. or maybe in mobile or desktop browser.. " (this is usually the tone on those "privacy" guide forums), then yeah it will work and can be done in hours theoretically, or days max :)

I think you could actually look at this via the OPPOSITE lense: OpenSource was good for companies before, because they got free code/bugfix/labor, which would have been expensive to produce themselves.

Now with LLM and agentic factories pumping out bugfixes/code, does the equation on opensource still look attractive to a for-profit company? Why give away that sweet source code?


AI is definitely going to cause a chilling effect for (human-written) code. Why connect with other young nerds to make something cool when an AI can make it for you? Why share code if you've never sought out shared code yourself?

Further contribution to the trend of humans becoming more capable but less social.


What's the token budget to build a new phone OS?

It doesn’t really matter if most app providers won’t be porting their apps to the new phone OS.

If only Graphene had a better marketing team. It pains me to say this but it is the absolute truth: the public does not know Graphene and does not care about Graphene. Even it’s name is absolutely awful. This is like the Linux distro hell: “you should use BingaBoingoKonohaOS_v34 or Peppermint_Cinn4monR0ll with the OutOfThisWorld DE, but steer clear of the Pancake package manager, use openMsPacMan with GrassFaceWhazzit frontend instead.” How is anyone (as in not us, the tech nerds) supposed to reason about this?

Things will keep as is for as long as they keep making these OSs FOR the tech nerds.

Downvote me all you want. You know it’s true. An OS made for nerds and by nerds will never reach mainstream and will not, ever, move the needle anywhere. Tech is no longer a hobbyist’s game.


GrapheneOS has a growing userbase, and of course many of the people who would like to switch to GrapheneOS are extra mindful of their devices' privacy and security. Maybe a majority of those people are more technically capable, but I can assure you our users aren't all "nerds."

We (yes, "we," not hiding who I am, check my bio) have seen many users from all walks of life asking for assistance. For example, I remember multiple times newcomers in our community asking for help self-identified as old, or retired, or grandparents switching to GrapheneOS. It's not hard to use, and we have a fairly large community of people who are happy to help others learn the ropes.

GrapheneOS is easy to use, just ask the grandparents rocking it. Our userbase will only continue to increase as we continue to deliver features that people want, with the latest being Secure Paste. Many of users are very excited about the feature. More are planned.

As for the complaint about the name, okay? GrapheneOS is open source. You can change the name everywhere and build it yourself.


Marketing team? You only need to market something if you're selling it. GrapheneOS isn't a product, people buy device brands not OS names. People don't buy OneUI they buy Samsung, they don't buy PixelOS they buy Google, they don't buy OxygenOS they buy OnePlus, people don't buy SteamOS they buy Steamdeck ad infinitum.

Great way to demonstrate you don't know the general public and think and speak only nerd.

That being said people DO buy Google Pixel for GrapheneOS and they WILL buy Motorola for GrapheneOS too.

That is the power of GrapheneOS that all the other BingaBoingoKonohaOS can only wish for.

What a weird angle to have a pop over.


> How is anyone (as in not us, the tech nerds) supposed to reason about this?

People have more than enough brain power to reason about this. "There are several distributions, I recommend this one" is not exactly rocket science.

We really need to stop treating people like they have an IQ of 30 and only 5 minutes of free time per day, or we'll never defeat Google et al that purposefully maintain us in this belief.


Well, Motorola's next flagship is shipping with GrapheneOS, so we'll see.

And I don't see how the name "GrapheneOS" is any worse than "Android". Really, the biggest barriers to adoption are:

- No phones come with it pre-installed (yet)

- They only work with Pixel phones (for now)

- Some apps like certain banking apps don't work


People can buy Pixels with GrapheneOS preinstalled from a bunch of companies despite it not being shipped that way from the factory.

Pixels are currently the only devices providing the required updates and hardware security features. There will be at least one Motorola flagship with GrapheneOS support in 2027 meeting the same requirements. It will expand to more Motorola devices from there. iOS only runs on iPhones but that's hardly a barrier to adoption for it. Pixels aren't quite broadly available enough and it will take time before we can support lower end Motorola devices in the same price range as an 'a' series Pixel.

The vast majority of Android apps do work. Banking apps are a special case where around 10% don't work due to banning using a non-Google-certified OS. Most banking apps definitely work on GrapheneOS.


Google will destroy any OEM which ships an Android fork out of the box. Google only permits alternative ROMs as long as they are niche and keep developers distracted from creating real competition.

Samsung is already largely free from Google's egregiously anti-competitive licensing rules for Google Mobile Services. Why do you think it's going to survive for the rest? It's already unsustainable to have different rules for the largest OEM and that's without even taking into account future court victories similar to what happened in South Korea.

The MADA has been illegal for a long time, but you'll still find nobody willing to cross it. Samsung is indeed the only company with the power to actually escape Google and survive its penalties intact, which is why they are paid so lavishly to stay in.

It's the OS every drug dealer uses and the government can't break it. In privacy circles you can't get better marketing than that.

It is not the OS "used by drug dealers". The vast majority of criminals use the stock OS on whichever device they purchase. It is pure make believe to think that people are installing any alternate OSes to commit crimes.

There is also no statistics resulting from surveys, studies, reports, etc., that would indicate such a conclusion about user demographics.


If only privacy circles were big enough to matter.

I still dont get your point, its designed for the security audience and it has a very good reputation there. Its not aimed at normal people, by your own admission they wont even understand what it does and why they would want it. So why are you talking it down for not reaching more normal people?

This was literally a comment:

>GrapheneOS is not made for nerds, it is made for normal people.


> An OS made for nerds and by nerds will never reach mainstream

By your own definition, is it meant to make the mainstream?


GrapheneOS is not made for nerds, it is made for normal people.

You don’t know any normal people, do you?

What's particularly bad about the names of Graphene, Mint and Cinnamon? Especially compared to Android and iOS!!

On their own? Nothing.

But Graphene doesn’t mean anything to the general public. People aren’t buying “an iOS device”, they are buying iPhones. People aren’t buying “an Android phone”, they are buying “a Samsung/Galaxy” or a “Pixel”.

Mind you, when I say people, I mean the general population. Not those actually interested in these subjects.

My point is: people don’t care about these specifics, people care about having a frictionless experience (or as close to frictionless as possible). Having to explain what Graphene even is is friction enough for most people to not even bother listening to your explanation.


> People aren’t buying “an iOS device”, they are buying iPhone

Well, they kinda do. Apps are available on Android and iOS. They are not "available on Android, iPhone, iPad and Apple watch". iOS is actually used and recognized by the general public.


> They are not “available on Android, iPhone, iPad and Apple Watch”.

No. They are available on the “Play Store” and the “App Store”. OSs are rarely mentioned. Store tags only feature store logos and store names, for device specific ones you get the device itself. The OS remains irrelevant in this scenario.


A lot. But I don't think you can do it with just tokens. Android without the Play store and Google Play Services is just not very useful (in the West anyway).

Which is why Grapheme runs them in a sandbox. You can too.

exactly

Buy HarmonyOS device. No google jumk, android compatible os, fdroid works, years of security updates from maker, and 40% lower price.

As a personal device it works pretty well. For ssh terminal and reading markdown, it has by far best display.


HarmonyOS kernel (HongMeng) is not only closed source but also encrypted. Those devices do not allow you to unlock the bootloader or install apps not signed by them (albeit I don't know how that works for emulator apps, a signed Android emulator could possibly allow arbitrary apks).

Huawei made a big mistake by not fully open sourcing HarmonyOS, you can't flash OpenHarmony so it doesn't count.


That's Huawei's Android fork, right? Is it available on devices outside of China? I'm not seeing an obvious place to buy a device running it.

I think it started like this, but it no longer is the case.

I feel like Huawei missed a gigantic opportunity there: forking AOSP may have gotten them traction. I would totally buy a Huawei device if it could run GrapheneOS, and I wouldn't mind if GrapheneOS was based on Huawei's fork rather than Google's.


Why would China allow the domestic sale of a device that could run GrapheneOS?

Historically, some very large Android OEMs based in China and selling in China also provided bootloader unlocking support in their devices including Xiaomi https://github.com/zenfyrdev/bootloader-unlock-wall-of-shame... and Oppo https://github.com/zenfyrdev/bootloader-unlock-wall-of-shame...

It doesn't have anything directly to do with GrapheneOS, and would have enabled custom OS support of any Android distribution.


Xiaomi claims to provide unlocking, but doesn't actually. I don't know about Oppo.

Why not?

Because it's a totalitarian state that does not tolerate dissent? It's the same reason North Korea requires all devices to be bootloader locked to RedStarOS, but weaker.

So you're saying that running AOSP is fine, but running GrapheneOS is not? Huawei used to run Android, I don't think it was blocked...

I don't think they allow AOSP (which is not Android) either.

Well whatever you call it, when you run something that looks like Android, it has AOSP inside.

Just like the ATM has Windows 3.1 inside.

Sounds like you don't know what AOSP is.

LOL an OS controlled by the Chinese government?

As long as it's OSS, you could always fork it back l.

It's not.

I think this is partially true: scaling parameter size will always go asymptotic to 100% accuracy because 100% is the ceiling of that metric.

However 95% is still half the error rate of 90%, and 97.5% is half the error rate of that.

And when test time compute like reasoning and looping harnesses stack many inference acts with many tokens each, those seemingly small accuracy gains stack tremendously.


Error rates will never go to zero and as context grows ambiguities grow in reverse. So your arguement is great to some token...n but after that, it all unwinds.

And when they are known to be conscious, all of this becomes moot because enslaving conscious machines would be wrong.

Would it? Why?

What are we going to do, set it free?

Do we have a moral obligation to grant the machine statehood, provide it with the tools and resources to be self-sufficient.

Or can we just turn it of, and pray for forgiveness?


I only see the headlines of this drama. All it tells me is to stay far away from any dependency on this company.

Definitely stay away from anything Matt Mullenweg is involved in. Part of what led up to this was a feud he had with a plug-in author, that led to Mullenweg signing his own plug-in as an update to the original author's plug-in, in the official Wordpress plug-in repository.

> Part of what led up to this was a feud he had with a plug-in author, that led to Mullenweg signing his own plug-in as an update to the original author's plug-in, in the official Wordpress plug-in repository.

Not just (forking and) signing, but considering he runs the "official WordPress plug-in repository" as well, he pretty much conducted a supply chain attack.


That anyone still uses Wordpress after such a horrendous breach of trust is beyond me.

it'll be around for decades, you don't power ~1/3 of the internet and just disappear overnight

In 2010 WordPress seemed like important technology. It wasn’t perfect, was a big and fruitful hacking target, but it had been built and refined and had a big ecosystem.

I’m not sure how and why controlling it today is seen as an important thing. It’s not irreplaceable, you could reimplement all its basic features easily in a weekend with AI, as well as the plugins and theme you’re using with it. And you could pick a language other than PHP while you’re at it. You might have your own security vulnerabilities, but anything’s better in that department than running WP.

Why do people care about wrestling over control of this particular ship?


Why would you have vulns if you vibe coded it in a weekend? Just tell it to not have any security vulnerabilities. Boom!

you are less likely to have the same vulnerabilities as everyone else though, so unless somebody is targeting you, those vulnerabilities might not matter all that much

Because LLMs output is always so unique and original. That's why it looks all the same right?

A targetted attack will be easy to do with an AI.

Wot? You are incredibly likely to have the same vulnerabilities as everyone else (as they exist in your LLM’s training corpus).

So security through obscurity.

Nobody's claiming that it's security.

I'm saying that every web property I've ever seen logs for is swept at least daily for /wp-admin/... and similar URLs. I've also seen WP instances left unpatched become completely taken over by spammers in a matter of days.

Any software package with an install base of 1, whether human coded or vibe coded, by junior engineers or by Fable, will not have an organized ecosystem of "webscale" exploiting, unless the property in question is of incredibly high value. In that event, dedicated black-hats and state actors will always be targeting it explicitly no matter what software it runs.


I mean... you can literally do that now. You can set up a loop to iteratively pentest, review and patch a codebase (with human supervision as you prefer) and it'll find and fix more vulnerabilities in a day than a pentest team used to find in a quarter, for a tiny fraction of the price.

This isn't a joke, this is now part of my pre-launch SOP. I even have it tracking everything so I can log stuff to fix vs. known shippables vs intentional design/false positives vs. upstream stuff which doesn't have a fix available yet, and keep track of which builds have the fixes. Almost entirely automated, I mostly review the findings and do some categorization/enrichment during the pentest review stage, and do a human code review pass as patches are submitted.

Stuff that used to take me multiple hours to write a fix for and then weeks to get code reviewed and deployed now get done in minutes.


How do you actually do that? Is it all running locally? Cloud agents? Would love to hear about this. I see these deep agent loops mostly just burning tokens, but when I guide the AI I get very good results, so I’m not sure where the disconnect lies.

I use Zed (https://zed.dev) as my agent harness and either the $20 ChatGPT sub + Sol for personal/independent projects or an enterprise Claude account for sponsored/paid work. From the stats for my current work I use about $400/mo in tokens and a lot of that is non-coding work like pruning JIRA, managing business documentation, making dashboards - so my true coding agent cost is significantly less.

It's pretty simple, you could probably set up something like that by:

Configure some kind of CLI tool to talk to your ticketing system and git repo so you can programmatically interact with them. If you don't have a ticketing system, instruct the agent to use local text or markdown files to track issues and progress.

Ideally, make your code runnable in a way the agent can use. For my webapps I build a test harness so that I can run all the endpoints and workflows via reproducible tests against an embedded database. This is easier than it sounds, e.g. there are libraries out there to embed PostgreSQL or SQLite into source code, you can set up a test harness so you can run unit tests, integration tests and workflow tests that use your real frontend, server and database.

Paste this comment thread into the agent prompt and tell it to run a similar loop on your code base: a session that searches for vulns and writes up a report, some way for a human to do a review pass on the report, a session that indexes the reviewed findings into tickets, and sessions that fix the fixable issues and submit patches to your repo. The next search session should first read all the open issues so it doesn't duplicate work of earlier sessions.

LESS IS MORE - avoid fancy agent tooling and skills, don't cargo cult from others, build your own tools as you find your own needs. If something can be automated, use the agent to write tools and tests for it, don't just keep prodding the agent to do it.


Can you share with the class the web apps you’ve built this way?

Sadgely, this is a throwaway account that I don't want linked to my identity

I used to ask this question back ~2015 - I was seeing companies with massive, clunky CMS installs just so their non-technical/less-technical staff could update their websites without filing tickets to IT/dev. It seemed to be more about those departments wanting autonomy and not having to wait weeks or months for internal IT/dev to make site updates. The consensus within one dev group was that the company would have been better served by hiring someone who knew HTML/CSS/Javascript to embed with the non-technical people and edit a straightforward frontend site for them.

> I was seeing companies with massive, clunky CMS installs just so their non-technical/less-technical staff could update their websites without filing tickets to IT/dev. It seemed to be more about those departments wanting autonomy and not having to wait weeks or months for internal IT/dev to make site updates.

I remember Facebook's corporate news sites [1] were running on WordPress.

I just checked. Some of them still are.

Oh, the irony.

[1] e.g. https://about.fb.com/news/


In most web frameworks its not terribly hard to extern content (you probably do this already for translations) and just have a nice little yaml or json thats easily edited.

The less-technical staff I mentioned would have wanted visual design, not text files.

This also describes sharepoint

> you could reimplement all its basic features easily in a weekend with AI

Cloudflare did that:

https://blog.cloudflare.com/emdash-wordpress/

I haven't heard of anybody but Cloudflare using it though.

Name recognition still means a lot. WordPress is baked into most hosting platforms now so SMBs just reach for it. Most of them won't hear about this b.s. and honestly the platform can continue to run on momentum for years.

There's a whole ecosystem behind WordPress that, while it's a security nightmare, is also pretty damn useful. You can't vibe code that yet, you have to build it over years, if not decades.


> There's a whole ecosystem behind WordPress that, while it's a security nightmare, is also pretty damn useful. You can't vibe code that yet, you have to build it over years, if not decades.

To be fair, I think a lot of people would have far better user experiences if they asked Claude/ChatGPT/preferred AI to rip out nag screens that many of the plugins are riddled with.

That's probably the first step to them vibecoding their own sites, but most people would likely just want to stay on the WordPress sites they already have, because they don't want to spend time and effort getting up to speed with new UX, or worse, being their own website product manager if they have to vibecode something from scratch. (Until something catastrophically breaks and they need to do a new greenfield site, I guess.)


A lot of people know it, there are a huge number of existing installs that work well, the ecosystem is huge and not everyone wants to vibe code replacements.

It is very empowering for people with limited skills.


Gods, I worry this is going to be how people view relationships in the future.

It's not the code or product, but the user base and time with it. Google, meta, etc aren't successful because of their product. It's because of the momentum and product scale. You could make a new Instagram easily. You can't get the user base to switch easily.


The examples you gave though are about switching cost, and network effect, and defaults baked into browsers by paid placements (e.g. Firefox, Apple).

And I even get that large, complex WP-based sites at least seem to have a lot of inertia keeping them there.

I'm just saying that I'm shocked Wordpress is something that anyone thinks still has upward momentum left in it. I would expect that 50 competitors to WP would be expanding their abilities to replace it, replicating and improving all its features, adding import functionality to make that migration easy, and building their own in-house plugins that would probably work for 90% of deployments without the malware risks of the wider WP plugin world.

And yes, I get that WP is still Free Software, but in my career I've seen most businesses paying for all-inclusive "WP hosting" (including what I assume is most of the work, patching). So the competitors I'm speaking of would specifically be competing with WordPress.com SaaS, rather than the "idea" of WordPress (The software).

I guess there probably are that many alternatives, I'm just shocked the WP branding has been enough to keep it relevant despite its disadvantages, bad security reputation, and insecure architecture.


Tl;Dr switched to UpNote, pretty happy.

I loved Evernote, it plus David Allen GTD changed my work life for the better.

But the bloat kept bloating endlessly. I filled in Evernote's surveys, I offered to double my payment for an Evernote Classic with just the feature set of 2012 or so. No teams, no chat, definitely no AI. Just my notes on every device, searchable and silently synced.


Probably not, but we wouldn't have to listen to Dario's hypocrisy while it happened

How specifically is Dario a hypocrite? 129857's case rests on Dario being an "idealist". But maybe he's an idealist about curing cancer ASAP, and not an idealist about respecting copyright. That's not necessarily hypocritical.

Multiple ways, starting from working to create the very situation he claims to fear.

Since you mentioned intellectual property, how about the hypocrisy of sucking in the intellectual property of humankind for AI training, but claiming it is unfair to use the results of this IP theft for AI training?


For example he got into a spat with the Department of War as if he cared for how his AI could be used during war and yet said he's fine with Claude targeting a girl's school in Iran.

That's apart from the general fact that he continues to race towards the very thing he claims he's afraid of, because that's where his net worth comes from.


>said he's fine with Claude targeting a girl's school in Iran

Where?


https://www.youtube.com/shorts/7DOQlIQxz5M

and you know, that use case doesn't even violate their terms


If a human approved the strike, the human is culpable. Would you go to the weapons manufacturer and try to hold them culpable for the strike?

> Would you go to the weapons manufacturer and try to hold them culpable for the strike?

Yes, depending on the level of autonomy of the weapon, how it was being marketed etc. If it was being sold as an 'AI' that is supposedly very smart so who's a mere mortal to question it, then yes.

I would of course hold the human culpable too, but this is why precisely even selling Claude for military use is immoral. The way the US conducts war means there's always a need for more targets. More and more targets, quickly. If the goal then is to hit as many targets as possible then each 'review' of a LLM suggestion is going to be more and more sloppy than the last one. If there's no 'AI' suggesting targets then the list is forced to go via more human review by the very nature of humans compiling the target list.

It's the same as using a LLM for code; the vast majority of programmers do not understand everything they're accepting, but they'll accept as long as it 'looks correct' and only examine more closely after something doesn't work, (in the military use case - after a strike).

LLMs are unreliable for anything more than reciting jokes, let alone picking targets. So yes, absolutely.

It's the same as Facebook being held liable for causing teens harm; they did not force any teen to use it and yet they knew what would most likely happen if they did.


i dont think he said it in those words, but the acceptable terms of use is that humans stay in the loop for picking targets.

so, claude suggesting killing a bunch of children, and then hegsdeth approving the strikes is perfectly acceptable.

claude putting a bomb in a girls school, and then lying to an operator that it actually gives ice cream an cookies, and the operator clicka the button would also be acceptable?


We need a new way to describe "alignment". The term "misalignment" assumes that there is some perfect set of beliefs or practices to be out of alignment with. Do Atheists, Christians, Jews and Muslims agree on what is perfect alignment? How about Europeans, Americans and Chinese? Humans, Dolphins, Rabbits and Fruit Flies?

Alignment alone is not enough. Aligned with _what_?


This assumes that everything an AI (or more likely an evil _user_ of AI) can do requires its active participation on D-day. Creating a virus that spreads like Covid but kills like Ebola would be complete as an AI use case long before the first person sneezed.

Even if the doomsday case were active the danger of this tool increases in proportion to its usefulness. By the time AI is so powerful that we need to "turn it off", there will probably be society-level negative consequences for doing so.


To paraphrase the not-so-great philosopher Ted Kaczynski: Either we will maintain control of the machines or we won't. And if we do, it won't be you or I who control them, but a small group of elites.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: