You sometimes hear people say "I mean, we can't just give an AI a bunch of money/important decisions and expect it to do ok" but this is already happening and has been for years.
Examples:
- Algorithmic trading: I once embedded on an Options trading desk. The head of desk mentioned that he didn't really know what the PnL was during trading hours b/c the swings were so big that only the computer algos knew if the decisions were correct.
- Autopilot: planes can now land themselves to an accuracy that is so precise that the front landing gear wheels "thud" as they go over the runway center markers.
and this has been true for at least 10 years.
In other words, if the above is possible then we are not far off from some kind of "expert system" that runs a business unit (which may be all robots or a mix of robots and people).
This is a piece of science fiction and has its own (inaccurate, IMO) view on how minimum wage McDonald's employees would react to a robot manager. Extrapolating this to real life is naive at best.
"Expert system" running a company is never going to happen unless shareholders are okay with no accountability from the company. You'll always need someone to blame in case things go wrong. You could have an executive using such an "expert system" for literally all their decisions, but it has to be a human being signing off on those decisions. There is no way to prosecute code and unless these expert systems can become sentinent or appear in court, best of luck trying to let it run a company in the real sense of actually making those decisions with full autonomy and responsbility.
But those things were considered on the same level of current LLMs in the sense of "well, a computer might do part of my job but not ALL of it".
No, algorithmic trading didn't replace everything a trader did but it most certainly replaced large parts of the workload and made it much faster and horizontally scalable.
The problem here is that you are cherry picking examples of successful technology.
The inverse would be to list off Theranos, Google Stadia, and other failed tech and claim that people said that there was massive steps that subsequently didn't materialise. In fact a lot of times it was mostly fabricated by people with stuff to gain from ripping off VCs.
Look at how bad it is with Microsoft in Windows despite their "all in on AI".
Ultimately no one really knows how it will pan out, and if we will end up with Enron or an Apple. Or even if it's a combination of a successful tech that ultimately is mishandled by corporations and fails, or a limited tech that regardless captures the imagination through pop culture and takes over.
The two key differences to me are infrastructure and specificity of purpose.
Autoland in plane requires a set of expensive, complex, and highly fine-tuned equipment to be installed on every runway in the world that enables it (which as a proportion is statistically not a majority of them).
And as to specificity, this system does exactly one thing - land a specific model of plane on a specific runway equipped with instrumentation configured a specific way.
The point being: it isn’t a magic wand. Any serious conversation of AI in these types of life or death situations has to recognize that without the corresponding investment in infrastructure and specificity of purpose, things like this blog post are essentially just science fiction. The fact that previous generations of technology considered autoland and algorithmic trading to be magic doesn’t really change anything about that.
I'm saying there's something structurally different form autonomous systems generally and from an LLM corpus which has all of the information in one place and at least in theory extractable by one user.
Considering things like Palantir, and the doge effort running through Musk, it seems inconceivable that this is not already the case.
I think I'm more curious about the possibility of using a special government LLM to implement direct democracy in a way that was previously impossible: collecting the preferences of 100M citizens, and synthesizing them into policy suggestions in a coherent way. I'm not necessarily optimistic about the idea, but it's a nice dream.
Thanks for the comment. Interesting to think about but I am also skeptical of who will be doing the "collecting" and "synthesizing". Both tasks are potentially loaded with political bias. Perhaps it's better than our current system though.
> special government LLM to implement direct democracy
I like your optimism, but I think realistically a special government LLM to implement authoritarianism is much more likely.
In the end, someone has to enforce the things an LLM spits out. Who does that? The people in charge. If you read any history, the most likely scenario will be the people in charge guiding the LLM to secure more power & wealth.
Now maybe it'll work for a while, depending on how good the safeguards are. Every empire only works for a while. It's a fun experiment
Indirectly, this is kind of what I was trying to get at in this weekend project https://github.com/stewhsource/GovernmentGPT using the British commons debate history as a starting point to capture divergent views from political affiliation, region and role. Changes over time would be super interesting - but I never had time to dig into that. Tldr; it worked surprisingly well and I know a few students have picked it up to continue on this theme in their research projects
That looks very interesting. Could use a demo or examples for us short attention spanned individuals. Would be cool to feed it into TTS or video generation like Sora.
Real world LLM's cannot even write a proper legal brief without making stuff up, providing fake references and just spouting all sorts of ludicrous nonsense. Expecting them to set policy or even to provide effective suggestions to that effect is a fool's errand.
>Real world politicians cannot even write a proper legal brief without making stuff up, providing fake references and just spouting all sorts of ludicrous nonsense. Expecting them to set policy or even to provide effective suggestions to that effect is a fool's errand.
This has been a more realistic experience of the average American for the past few years.
This is an interesting and thoughtful article I think, but worth evaluating in the context of the service ("cognitive security") its author is trying to sell.
That's not to undermine the substance of the discussion on political/constitutional risk under the inference-hoarding of authority, but I think it would be useful to bear in mind the author's commercial framing (or more charitably the motivation for the service if this philosophical consideration preceded it).
A couple of arguments against the idea of singular control would be that it requires technical experts to produce and manage it, and would be distributed internationally given any countries advanced enough would have their own versions; but it would of course provide tricky questions for elected representatives in the democratic countries to answer.
There's not a direct tie to what I'm trying to sell admittedly. I just thought it was a worthwhile topic of discussion - it doesn't need to be politically divisive and I might as well post it on my company site.
I don't think there are easy answers to the questions I am posing and any engineering solution would fall short. Thanks for reading.
The really nice thing about this proposal is that at least now we can all stop anthropomorphizing Larry Ellison, and give Oracle the properly robot-identifying CEO it deserves.
> The President sits at the top of the classification hierarchy.
Constitutionally, and in theory as Commander-In-Chief, perhaps. But in practice, it does not seem so. Worse yet, it's been reported the current President doesn't even bother to read the daily briefing as he doesn't trust it.
You're conflating the classification system, established by EO and therefore by definition controlled by the Executive, with the classified products of intel agencies.
A particular POTUS's use (or lack thereof) of classified information has no bearing on the nature of the classification system.
I point that out a little bit when I refer to agencies being discouraged from sharing information. The CIA may be worried about losing HUMINT data to the NSA for example. You may be referring to them worrying about compartmentalizing the information away from the president as well which you are right happens to some extent now but shouldn't 'in theory'. Maybe it's a don't ask don't tell. I think Cheney blew the cover of an intel asset though.
> compartmentalizing the information away from the president as well which you are right happens to some extent now
This is nothing new, and has been happening since at least the 1940s, to multiple administrations from both parties. Roosevelt, Truman, Kennedy, Nixon, Reagan...and that's just some of the instances which were publicly documented.
Let’s assume we live in a hypothetical sane society, and company owners and/or directors are responsible for their actions through this entity. When they decide to delegate management to an LLM, wouldn’t they be held accountable for whatever decisions it makes?
While I have great respect for this piece of IBM literature, I will also mention that most humans are not held accountable for management decisions, so I suppose this idea was for a more just world that does not exist.
Accountability is perhaps irrelevant is my point. You can turn off a computer, you can turn off a human. Is that accountability? Accountability only exists if there are consequences, and those consequences matter. What does it mean for them to "matter"?
If accountability is taking ownership for mistakes and correcting for improved future outcomes, certainly, I trust the computer more than the human. We are never running out of humans incurring harm within suboptimal systems that continue to allow it.
I'd say that the fix then is in creating a more just world where leaders are held accountable than to hand it off to something that, by its very nature, cannot be held accountable.
Because the comment is just a copy/paste of the content at the URL at the top of the comment; which doesn’t add anything to the discussion, because it’s not discussion at all. It’s just a wall of text, it’s not clear to me why you’re posting it as a comment (vs linking to it), and simply regurgitating copy/pasted external information isn’t helpful or interesting.
Next time, write a sentence or two of context about what you’re going to link to — who wrote it and when, why it’s interesting, and how/why it’s relevant to the topic at hand.
There’s almost never a need to copy/paste wholesale external content into an HN comment. Especially true when said content is literally linkable, and actually linked, from your comment!
The reason is that most people interpret the comment as noise. It’s very long, so it takes up a lot of space and makes it harder to find comments by people.
On HN it’s best to just link to the article, no need to also copy and paste anything in comments except for very short quotes.
Examples:
- Algorithmic trading: I once embedded on an Options trading desk. The head of desk mentioned that he didn't really know what the PnL was during trading hours b/c the swings were so big that only the computer algos knew if the decisions were correct.
- Autopilot: planes can now land themselves to an accuracy that is so precise that the front landing gear wheels "thud" as they go over the runway center markers.
and this has been true for at least 10 years.
In other words, if the above is possible then we are not far off from some kind of "expert system" that runs a business unit (which may be all robots or a mix of robots and people).
A great example of this is here: https://marshallbrain.com/manna1
EDIT: fixed some typos/left out words
This is a piece of science fiction and has its own (inaccurate, IMO) view on how minimum wage McDonald's employees would react to a robot manager. Extrapolating this to real life is naive at best.
Why, it's as much of a view of our past adherence to technology without thinking as a well as a view of the future.
"Computer says no" is a saying for a reason.
Current LLMs rarely or seldom say no. Unless, they're specifically configured to block out certain types of requests.
No, algorithmic trading didn't replace everything a trader did but it most certainly replaced large parts of the workload and made it much faster and horizontally scalable.
The inverse would be to list off Theranos, Google Stadia, and other failed tech and claim that people said that there was massive steps that subsequently didn't materialise. In fact a lot of times it was mostly fabricated by people with stuff to gain from ripping off VCs.
Look at how bad it is with Microsoft in Windows despite their "all in on AI".
Ultimately no one really knows how it will pan out, and if we will end up with Enron or an Apple. Or even if it's a combination of a successful tech that ultimately is mishandled by corporations and fails, or a limited tech that regardless captures the imagination through pop culture and takes over.
Autoland in plane requires a set of expensive, complex, and highly fine-tuned equipment to be installed on every runway in the world that enables it (which as a proportion is statistically not a majority of them).
And as to specificity, this system does exactly one thing - land a specific model of plane on a specific runway equipped with instrumentation configured a specific way.
The point being: it isn’t a magic wand. Any serious conversation of AI in these types of life or death situations has to recognize that without the corresponding investment in infrastructure and specificity of purpose, things like this blog post are essentially just science fiction. The fact that previous generations of technology considered autoland and algorithmic trading to be magic doesn’t really change anything about that.
We know very well how to train computers to handle those effectively.
Anything without quick feedback is much more difficult to do this way.
I think I'm more curious about the possibility of using a special government LLM to implement direct democracy in a way that was previously impossible: collecting the preferences of 100M citizens, and synthesizing them into policy suggestions in a coherent way. I'm not necessarily optimistic about the idea, but it's a nice dream.
I like your optimism, but I think realistically a special government LLM to implement authoritarianism is much more likely.
In the end, someone has to enforce the things an LLM spits out. Who does that? The people in charge. If you read any history, the most likely scenario will be the people in charge guiding the LLM to secure more power & wealth.
Now maybe it'll work for a while, depending on how good the safeguards are. Every empire only works for a while. It's a fun experiment
It'd be much better to train an agent per citizen, that's in their control, and have it participate in a direct democracy setup.
This has been a more realistic experience of the average American for the past few years.
That's not to undermine the substance of the discussion on political/constitutional risk under the inference-hoarding of authority, but I think it would be useful to bear in mind the author's commercial framing (or more charitably the motivation for the service if this philosophical consideration preceded it).
A couple of arguments against the idea of singular control would be that it requires technical experts to produce and manage it, and would be distributed internationally given any countries advanced enough would have their own versions; but it would of course provide tricky questions for elected representatives in the democratic countries to answer.
I don't think there are easy answers to the questions I am posing and any engineering solution would fall short. Thanks for reading.
Best meme in hacker space, thanks /u/Cantrill.
Constitutionally, and in theory as Commander-In-Chief, perhaps. But in practice, it does not seem so. Worse yet, it's been reported the current President doesn't even bother to read the daily briefing as he doesn't trust it.
You're conflating the classification system, established by EO and therefore by definition controlled by the Executive, with the classified products of intel agencies.
A particular POTUS's use (or lack thereof) of classified information has no bearing on the nature of the classification system.
This is nothing new, and has been happening since at least the 1940s, to multiple administrations from both parties. Roosevelt, Truman, Kennedy, Nixon, Reagan...and that's just some of the instances which were publicly documented.
<shrug>
no human came out with those tariffs on penguin island
Executives, in contrast, require option strike resets and golden parachutes, no accountability.
Neither will tell you they erred or experience contrition, so at a moral level there may well be some equivalency. :D
I think you are anthropomorphizing here. How does a computer feel when unplugged ? How would a computer take responsibility for its' actions ?
If accountability is taking ownership for mistakes and correcting for improved future outcomes, certainly, I trust the computer more than the human. We are never running out of humans incurring harm within suboptimal systems that continue to allow it.
1. Past decisions and outcomes get into the context window, but that hasn't actually updated any model weights.
2. Your interaction possible eventually gets into the training data for a future LLM. But this is incredibly diluted form of learning.
Next time, write a sentence or two of context about what you’re going to link to — who wrote it and when, why it’s interesting, and how/why it’s relevant to the topic at hand.
There’s almost never a need to copy/paste wholesale external content into an HN comment. Especially true when said content is literally linkable, and actually linked, from your comment!
On HN it’s best to just link to the article, no need to also copy and paste anything in comments except for very short quotes.