Hey everyone, as you know, I comment on
all things science in order to expose
anti-science narratives that polarize
the public against the scientific
community. There's one area I have
commented on very little because of how
nebulous it is, and that's artificial
intelligence. But let's do our best to
rectify that. Now, many of you saw a
piece I put out last summer called,
"Will artificial intelligence destroy
humanity?" which outlined significant
concerns regarding the advent of
technology that the global tech industry
is actively pursuing and which many
experts have declared a very serious
extinction risk. I highlighted some
fascinating research whereby all the
major AI models demonstrated the
tendency to perform blackmail or even
murder people. I talked about what the
CEOs of these tech firms say about AI
and contrasted that with what neutral
academics say. The reaction towards that
video was interesting. Many people
resonated with the concerns while some
others raised objections towards the
reasoning presented in the video. This
prompted a follow-up conversation with
John Perry where we discussed my video
and the lines of thinking on either
side. What makes this topic so tricky is
that it is totally unlike anything else
I address on my channel. If you believe
that vaccines cause autism, you're
wrong. If you believe anthropogenic
climate change isn't real, you're wrong.
If you believe the Earth is flat, you're
wrong. Science deniers are wrong. And my
task as a science communicator is to
convincingly demonstrate the wrongness
of the science denier using as much
legitimate science as is necessary. An
amount which depends on how
sophisticated the anti-science
propaganda campaign is. But this topic
is different. Those who firmly believe
that super intelligent AI is the
greatest threat to mankind and those who
believe it's not a threat at all. These
are projections about future events that
are highly uncertain. And the
uncertainty is compounded by the fact
that nobody really truly understands
exactly how these models work and how
they learn. In fact, I don't really know
what my precise position is on this
topic. Those experiments demonstrating
destructive behavior are very
convincing. Claims that people like Sam
Alman are deliberately overhyping the
tech are also somewhat convincing,
especially given the fact that it has
recently come out that Sam is most
likely a complete psychopath. In
instances like this where it's difficult
to arrive at firm conclusions, the best
we can do is continue to listen to as
many sources as we can. In my first
video on this topic, I closed with a
call to action steering viewers towards
a nonprofit organization called Control
AI that is trying to influence
legislation towards a global ban on
super intelligent AI. first through
local governments and eventually by
international treaty. They have been
making slow but steady progress in
winning over the attention and concern
of politicians and lawmakers in numerous
countries. I'd like to continue the
dialogue on this endeavor and super
intelligent AI in general. I had the
opportunity to chat with Andrea Miotti,
the founder and CEO of Control AI in
order to address some of the common
talking points on both sides of this
issue. Let's check out that discussion
now. Andrea, thanks so much for joining
me. Uh maybe let's just set up some
basic context for some of the viewers
who maybe didn't see my previous video.
What is super intelligence in the
context of AI and how does it pose an
extinction risk?
>> Yeah, and thank you very much Dave for
having me on. So super intelligence is
AI systems that are much smarter than
any human and this is the explicit goal
of almost all of the main AI companies
today. All of them in many companies are
not just building chat bots which is you
know what a lot of people see when they
use uh chat GBT or some of its
competitors. They're trying to build
these AI systems that are smarter than
people across all tasks that are
autonomous. They can take actions on
their own in the real world online. And
in 2023, all of the CEOs of the main AI
companies, Nobel Prize winners, and many
top AI experts all signed a letter
recognizing that AI poses an extinction
risk for humanity on par with nuclear
war. And this was kind of an open secret
in the AI field for many years uh that
finally came out uh on that year to the
broader public. And it's it's very
concerning. It's, you know, this is
where we are headed if we continue
developing AIs that are much smarter
than humans. It's something that kind of
a lot of top people in the field knew
for a long time and so far uh we don't
have the rules in place to make sure
that doesn't happen.
>> Right. Yeah. I mean that letter is
concerning like like I understand the
perspective of like oh these people are
going to say these things but it's very
across the board. You've got academics
that are not aligned necessarily with
any particular special interest. Just
seems to be something resembling a
consensus among the field. I mean,
that's the thing about this topic. It is
kind of a quagmire. I I've been pretty
firmly convinced that that the
extinction risk is significant enough to
at least be taken seriously as a
possibility. But it, you know, very
interestingly putting out that first
video I did, there was kind of a a vocal
minority that they just don't see it
that way. They can't be convinced. Uh
I'm sure that you deal with this all the
time. Of course. Um in your experience,
are there any general trends um in the
way that people voice this uh
disagreement?
>> Yeah. So I would say there's kind of
like three um arguments uh or like three
directions uh against this point. Uh one
of them is expecting that super
intelligence will happen but that
somehow it wouldn't be dangerous. One of
them is uh disagreeing that the policy
focus should be uh preventing super
intelligence right now and one of the
one of them is disagreeing that AI can
ever scale to super intelligence and I
will I will actually start from the one
about super intelligence not being
dangerous. I like this is happened
rarely but almost never happens like
most people if they expect super
intelligence to be possible they see how
dangerous it is. They see how dangerous
it is to make some AI systems that are
much smarter than people that we humans
don't really control. They're running
around. You know, we it's a very big
gamble to make to have entities smarter
than us run our economy and our planet
without us. Uh we get more the other
two. So one of them is the u the whether
AI can ever scale to super intelligence.
You know many people talk about but
isn't AI going to hit a wall? This is
like a common a common turn of phrase
and like I sympathize with like
skepticism towards the tech industry. I
think there's a lot of you know a lot of
reasons to be skeptical of a lot of the
claims from the tech industry but here
we are facing a situation where year and
year we just see the capabilities of AI
advancing. You know it's we have AI
systems this year right now that if we
thought about them like two years ago we
would think they're science fiction. you
know, you can just talk to your computer
and make it write code without you
typing. And, you know, maybe we're going
to hit this wall. I think it's unlikely,
but that's a pretty big gamble to make
of like hoping we're going to hit a
magical wall.
>> Uh, and we're not going to be
>> Yeah, progress seems inevitable. I mean,
I I think that I mean, okay, one of the
most common push backs is just people
saying AI it's just fancy autocomplete.
Uh, it's it they're not actually
thinking, right? They're they're doing
this in the context of LLMs, you know,
chat GBD and stuff. And I understand
that approach, but I mean, we're talking
about different um areas of of
development here, right? Like what what
what would you say to people who who
propose that?
>> Yeah, exactly. I think you're right. But
like I think they they kind of had a
point with like early large language
models that like we're only trained on
online text data. But first of all,
that's just not the case with AI systems
anymore. like even the chupt that you're
using behind it there is a much more
complex AI system that was trained on
basically any type of data these
companies can get their hands on it's
trained on text it's trained on video
it's trained on b any modality you can
think of you know we're kind of far
beyond just the pure uh text models and
especially more and more these a systems
these companies are training them to
actively pursue long-term goals to
actively pursue activities without human
supervision that are very complex
including including using code,
including using computers. So, they're
being made to be this powerful agents
rather than just an autocomplete or just
a chatbot you talk to.
>> I mean, this is hard for me because I
know so little about tech, but all I can
do is look at what is being done. And
when you have I mean, AI has
demonstrated very clear problemolving
ability. So, that just seems to negate
any kind of talking point about, you
know, oh, it's just an autocompleter to
me. Yeah. I that just feels like a like
a mischaracterization. But I I think a
lot of the talk too is about okay
intelligence it can it can figure stuff
out. But then they people want to talk
about sentience or self-awareness,
right? So in my video I cited all these
um experiments where where um you know
behavior was exhibited that that I would
argue resembles sentience, right? When
you have um a concern for
self-preservation, I I don't really see
how that could indicate anything other
than sentience. But some people say no,
that's not it, right? They're not
conscious. They're not sentient. They
never will be in tech. You know, a
program like this can never be sentient.
Um, what do you think about that?
>> Yeah. So, I think on that I think the
question of like consciousness is a very
interesting question for philosophers,
but ultimately like the danger comes
from how capable they are. You know,
whether they are conscious or not, you
know, there's like big question for
philosophy since kind of the beginning
of time. Um but fundamentally as they
get more powerful they become more
dangerous and know whether they're
having internal experiences or not maybe
we will never know but it doesn't really
matter in terms of of the danger that
we're we're facing. All
>> right. Yeah, it's an interesting uh
question. It's an interesting
philosophical question. Are they
sentient or not? But if they murder all
of us, who really cares, right?
>> Exactly.
>> Um
>> Yeah. So, I mean, so, okay, again, with
these with these experiments, so I
showed all of the blackmail and the, you
know, attempted murder and stuff like
that. And in the video, I tried to make
the concession that there that these are
highly specific scenarios or, you know,
you could call them extreme testing
scenarios. um still very chilling but
people respond to that saying okay fine
like they do these weird things in these
ultra specific scenarios but in real
deployment this would never happen it's
not reality is never so constricting but
you know I don't know how like how can
you be so confident about that do they
know something I don't or or what is it
>> yeah I I think that's kind of again it's
a dangerous gamble to make like we see
you know right now we have the weakest
AI systems will ever be if we continue
on this path so We're already seeing
these capabilities where the systems are
not that smart on the grand scheme of
things. You know, I wouldn't call the
things we have now super intelligent and
they already can do things like figure
out that the engineer monitoring them
had an affair uh in the fictional
scenario and then find a way to
blackmail them to get out. Um recently
with Mythos uh uh what during testing
they put it in a sandbox and uh in the
sandbox the AI system could not access
the internet. It was cut off. uh and
some of the engineers like left to have
a break and the AI figured out a bunch
of exploits so it could get out of the
sandbox went on onto the internet and
sent an email to one of these engineers
saying like haha you know I got out I
got out you know gotcha and again this
is you know perhaps innocuous it's still
only in testing uh but we just see the
trend line and the trend line is very
clear as they become more capable they
become smarter as they become smarter
they figure out more and more creative
ways to cheat out of being supervised
ized. Uh just uh today I was seeing one
organization called meter that does
testing of AI systems. They say now most
of the time they spend on their flagship
evaluation is to like make sure that the
systems are not cheating their
evaluation criteria because they find so
many ways to figure out during testing
how to cheat uh uh the test because they
they are aware they're being tested and
it's only going to get harder and in
real world scenarios it's going to be
even more edge cases even more like
dangerous situations that we can't
predict in advance if we can't even
control them when they're kind of in the
box still uh it's only going to be
harder as get smarter.
>> Yeah, this is supposedly the primitive,
not yet super intelligent version. And
they're already fooling us and tricking
us and doing all these incredibly
unexpected things and and evading all of
the checks and balances that were
putting there to to to constrict their
their behavior and their activity. I
mean, you know, it's like some people
are just like, okay, these are these are
fictional scenar they're they're
roleplaying, right? They're just they're
they're playing they're playing a game
or something. And I mean I guess but
like with if we can talk about motive
does anybody have any idea what
motivates the like why would that model
try to why does it want to do that you
know why does it want to do these
things? Yeah, like a big problem here
is, you know, whether whether it's
roleplay like the reality is that we
just don't know. It's it's because right
now the how AI systems are being
developed is that not even their own
developers know exactly why they do what
they do. They they are essentially like
nearly blackboxes to even their own
developers. Uh we understand that
systems very little. our ability to
understand how they uh think internally
is progressing at kind of a snail pace
compared to how powerful they are
becoming. So we have these inscrutable
systems that we don't really know why
they do what they do. They're also built
in a way that makes it kind of really
really hard to understand. They're not
uh uh written line by line by a software
engineer.
>> Exactly.
>> Yeah. Yeah. And you know they're
basically not made by humans. They're
like there's a smaller uh system that
starts a complex pro process of
essentially like similar to evolution
that goes through a bunch of bunch of
data and then spits out a model and it's
a model that its own makers don't really
know how it works internally
>> self-improving and we're not privy to to
all the detail. Yeah. I mean it makes me
feel on the one hand a little bit better
about my own ignorance because truly
none of us know what's going on. But
also it makes me feel worse because I if
they don't know that I mean if nobody
knows then we're really in danger. And
so that kind of leads me to right the
the ultimate gotcha by the by the
naysayers. Well okay worst case
scenario. They do want to kill us all
but then right before they're about to
do it whatever pull the plug it it's a
helpless digital entity. We just you
know controlal altdelete and and and
then it's gone and and no harm done.
Right. So what is that uh does that uh
satisfy you as a as a last resort?
>> Well, so the the kind of sad thing is
that right now we don't even have a big
red button. Like I don't think a big red
button would solve everything, but we
don't even have that. Like right now
these companies, you know, they're
developing these AI systems. They're
developing big data centers. Often the
data center is operated by a different
company, a cloud provider. None of these
none of the AI company CEOs are like you
know the US government nobody has a big
red button and they can just go and be
like wait a minute I want to you know
stop all of this and we're just going to
press this and the data center shuts off
this infrastructure is just not there uh
would it solve the whole problem not
really but it would be a great step
forward and we don't even have uh that
and also like as they become smarter
like we were discussing before they are
finding way more and more clever ways to
circumvent all of this infrastructure
that we could put around them. You know,
they're especially being what the AI
companies are doing is they're making
them more and more competent at software
development which also makes them more
and more competent at hacking and
understanding software. So very often
when you have this situations like you
put them under a sandbox or you can you
you put them with a scenario where they
would be shut down. They in testing
they've shown that they are able to
figure out how to sabotage the shutdown
mechanism. And yeah,
>> this again they're not that smart right
now. As they get smarter, we will need
to have better and better defenses to
even make sure that they cannot be
circumvented by the AI itself.
>> Yeah. Yeah. And I mean they know
everything we know. So they know how to
hack at least as well as we do or
better. Uh they can get onto other
servers. They've already demonstrated
that they can evade whatever kind of
limitations you put on them. I mean, to
me, it's like they you they I mean, they
don't even have to like drop 100 nuclear
bombs on everybody at the same time.
They could just do something like tank
the entire global economy and shut down
a power grid and then watch the chaos
ensue as we all murder each other. You
know, I mean, there's just so many ways
for it to play out. Um,
but uh I don't know. I mean the as as in
so far as a lot of this is speculative
some people are saying like okay you're
distracting from from other more
pressing issues with AI. So right now
we're worried about social media
manipulation and obviously the the
dwindling job market is is of immediate
concern. So what are are we diverting
too much from that or what?
>> Yeah. So I think you know first of all
human extinction is a pretty big deal
and I think right now we are barling
towards AI systems that could wipe us
all out without doing too much about it
and I think that's a massive concern in
itself. This doesn't mean that the other
issues are not valid like you know the
other issues are happening as well. Uh
but you know ultimately we will need to
deal with all of them. uh this is one of
the massive ones that we're facing and I
think having a competition between like
which uh like we don't just have to
focus on one single thing there are
multiple things that we can focus on and
ultimately the the usual winner from
like pitting AI issues one against the
other are just AI companies themselves
they're just trying to be not regulated
at all on any of these issues you know
including on super intelligence and
they're very happy to see uh squables
between people
>> right Yeah.
>> Tank their own pet issue one against the
other.
>> You've started this organization.
Somebody else can start another
organization to help all the people get
find jobs and stuff like come on. It's
uh I mean okay so but you raised an
interesting point right that the AI
companies themselves the optics
associated with all this. So there's a
very interesting thing that people bring
up that's it's hard for me to counter
and it's just that there's a lot of
narratives being spun by the AI
companies, right? They themselves are
saying talking about extinction risk and
and that this is like just hype to
inflate their economic worth. Um I don't
know what is there a threat of truth
there or what is it?
>> So I think it's definitely true that the
companies are hyping up their technology
how powerful it is. you know that's
that's normal and that's you know they
want to raise billions. uh I don't think
the extinction part is part of that also
because if one takes the risk seriously
the answer is to stop them from building
super intelligence which would tank
their valuation as it is right now and
be very very costly like uh and in many
cases also we've seen some of the CEOs
that they you know some years ago they
were more vocal uh about the risks and
then the more they have uh bored
pressure or you know realize uh you know
pressure from investors they become or
the closer they get to it, the more they
backpedal from the from the
>> Exactly. Exactly. And
>> Exactly. And so, you know, ultimately,
if if if somebody told you like Apple is
starting a massive nuclear weapons
program, I don't think people's reaction
would be like, "No, oh wait, this is
just to hype up the next iPhone." Like,
I think people's reaction should be
like, "Wait a minute. Why is Apple
building nuclear weapons? We should make
sure nuclear government stops them right
now." You know, they shouldn't have
nuclear weapons.
don't care about the you know the
valuation or their next iPhone and it's
the same thing here and you know if we
if we ban super intelligence which is
what we call for a lot of these
companies will lose a bunch of the
valuation I don't I don't think they
will be wiped out you know they have
other things they can do they can
>> do other activities are not threatening
to kill us all
>> yeah they're doing fine already yeah no
it's a good point the idea of a of of a
private corporate entity being allowed
to develop technology that threatens the
all of humanity
They they can't do it with nukes. I
mean, yeah, we're comparing something
similar here in scope. I mean, so, okay,
with these narratives, this is the
hardest part for me because it combines
my my my
poor understanding of tech with my poor
understanding of the the or not even my
poor understanding, but the the nebulous
nature of the bureaucracy associate or
the the manipulative the manipulative
nature of the of the rhetoric of these
companies. So what what do they care
about? What are the narratives that
they're spewing? Obviously, they want to
win the race to super intelligence. So
what are they saying to whom? How does
it differ to which group they're talking
to and how does that serve their intent
of manipulating the landscape in their
favor?
>> Yeah, absolutely. So yeah, number one is
exactly what you said like ultimately
what you know what we can model them as
is they want to get to super
intelligence as fast as possible. They
will say many different things. They
will say different things to different
people. You know, executives will say
different things to different people,
but ultimately the whole machine is
pushed towards let's get to super
intelligence as fast as possible. And if
there's something stopping that, they
will fight it. And so they end up having
a few different stories to different
people. You know, to investors, it's
it's going to be amazing. You know,
especially if you invest in me and not
the others, it's going to be
particularly amazing. uh and if we get
closer and closer, we're going to get to
automate the entirety of the economy
which uh and you know maybe not too many
implications beyond that because that
those will be scary. They they maybe
stop at the
>> at the at that point u to their
engineers. Sometimes the engineers raise
concerns. Sometimes you know people
internally many of them like start to be
concerned about uh the risks and they
will say well you know there are some
risks but if we don't do it this other
company or this other company a
different country will do it and those
are obviously worse so we should be
doing it. uh you know our hands are tied
and anyways all of those great benefits
as well.
>> And to the public and to politicians
they are playing a tactic that was kind
of pioneered by the tobacco industry uh
which is just to spread confusion uh
FUD, fear, uncertainty and doubt. This
is in many ways exactly the same
playbook that kind of big tobacco used
to avoid uh regulation especially when
it came out that internally in tobacco
companies they realized that smoking
causes cancer and they spent a long time
trying to like delay uh kind of this
societal awakening about the issue as
much as possible and AI companies are
doing the same you know in many cases
they're kind of really replicating the
playbook onetoone it's just spread many
different narratives say different
things to different people not really
try to maintain coherence, just try to
make it more confusing. And I think a
lot of people feel it. They feel it
like, you know, this person tells me
that about AI, you know, this person
says it's going to wipe out all jobs.
This person says it's not really going
to wipe out all jobs. It's not really
intelligent. It just make people
confused. If people are confused, if
politicians are confused, they're going
to wait
>> and they're not going to act. And that's
what the companies want.
>> Ren render the public kind of impotent
to to Yeah. Just because there's so many
competing nerves, fear, uncertainty, and
doubt. Yeah, that's uh Yeah, that
checks. and and so much easier today
than it was, you know, 50 or 70 years
ago or whenever that was happening with
the tobacco industry. Um because of the
internet and social media, it's just
really
>> it's just astounding how much certain
narratives can gain traction, especially
with, you know, economic backing and
everything. Um you mentioned, so you you
touched on this a moment ago, just uh
these narratives, if we don't if we
don't get it, the other company will or
the other country will. And so obviously
the other country is China. They're
always talking about we got to be China.
We got to be China. We got to be China.
If they get it first, it'll be a
disaster for the Western world. It will
ruin all of Western civilization. Um, it
seems like a little bit of a copout, but
is I is there a nugget of of of uh of
sound logic in there or do we have to
beat them or you what what can we say?
>> Yeah. Yeah. So I would say two things
like one the way that companies use it
is yes just completely self- serving and
we've seen it over time that you know
these companies don't really seem to
have loyalty to any specific country
they have loyalty to themselves and they
will use a narrative that uh benefits
them you know we've seen like
>> patriotism we gota
>> exactly and they're like there's a
recent article in the New Yorker uh
where one of the stories told is at some
point Greg Brockman president of OpenAI
in the early days of OpenAI I he floated
the idea uh internally. But you know
what about auctioning AGI to the highest
bidder and like pit countries against
each other. We auction it to to Russia.
We auction it to China. We auction to
the US and then we just pit everybody
against each other to get the the
highest bidder. You know this is not you
know this um now you know now they might
be more patriotic in the rhetoric but
that's the reality. They're all in for
themselves. Um I think on the on the you
know what about China question I think
it's important that nobody develops
super intelligence. So I do think it's
important like there is a there is a
part of truth here of if anybody
develops super intelligence we are
probably screwed. You know we are facing
an enormous risk of human extinction.
And so I think the correct uh answer
from the US should be to say you know no
we will not let China develop this. It's
not just we are not going to do it.
We're also going to make sure that other
countries don't do it. It's
unacceptable. This would be a massive
violation of national and global
security. And you know, I think the way
to do it is to start the usual uh ladder
of engagement while also making very
clear this is a big red line. You know,
it's this is bigger than nukes. We will
not let anyone develop this including if
it's a foreign country.
>> Yeah. Yeah. The goal the goal is, you
know, beyond legislation, but an
international treaty. Um, I mean, yeah,
that it it does kind of I I think I
think it's naive the narrative that like
one country like China could develop
this and use it to wield like economic
supremacy. I mean, the thing is going to
just escape and permeate the the global
internet. So, it's like, yeah, whoever
does it and I think they know that. But,
I mean, in ter, you know, if everybody
knows it, I guess just the narrative of,
okay, just it's inevitable. It's going
to happen. Relax. it's pointless to try
to stop it. Just, you know, whatever
happens happens. Um there's people who
actually you say that, you know, I mean
it that also seems like a gigantic
copout. Uh it seems like the AI
companies are kind of like, "Yeah, don't
worry about it. Just uh everybody say
this now on social media." What what
what can we say about that?
>> Yeah, that's Yeah, that's another one.
like it's it's really playing into like
building nihilism like the kind of in
some ways the biggest the biggest danger
that they face is that people actually
wake up and realize this is happening
and start uh taking action and that you
know governments wake up and start
saying like no wait a minute why you're
developing a technology that could wipe
everybody out and could overthrow the
government like we're not going to let
you do that and so this nihilism of it's
going to happen anyways you know might
as well be me is a very very convenient
narrative to try to again kind of modify
uh a reaction and make people think like
you know it's just going to happen like
you have to deal with it just don't even
try. Um and the reality is that no
obviously we do have a choice like we
had a choice before with our
technologies you know I'm I'm not saying
that we did everything right with nukes
but we haven't had a nuclear war uh and
you know dropping nukes since 1945.
nuclear nonproliferation was achieved.
You know at the time when the nuke was
developed, many people had the idea of
it's going to be inevitable. Every
country is going to have nukes. Every
country is going to be a nuclear power
and therefore then we will be have a
extremely unstable world where we can
just face nuclear annihilation every
day. And that didn't happen because and
not just by magic but by the hard work
of people in multiple countries working
really hard to prevent nuclear
nonproliferation, educating the public,
educating policy makers about the fact
of the big dangers of having nukes uh
proliferate and in fact we didn't have
nuclear war since then. So I think we
can do it again.
>> Yeah, it's a perfect example. Yeah, I
feel like if the if the if the logic
held of of this inevitability narrative,
then then we should have already faced
the apocalypse, we we could have killed
ourselves in the 1970s and we didn't uh
because we decided not to. So, I think
that logic holds magnificently. Okay.
So, I you know, I I've already stated uh
in my original video and and again
multiple times now, I am convinced of
the risk. Uh I would imagine that most
of my viewers are convinced of of the
risk. Maybe not all, but uh you know,
probably a sizable portion. So, uh we
arrive at the million-dollar question.
You know, what do we do? You've made it
your career to think about this
question. We've talked a little bit
about it, but um let's talk about it now
in even more detail as much as you can.
>> Yeah. Yeah. Absolutely. So, the step
one, which is also kind of what we
touched on before, is right now most
people just don't know. And that's the
main thing that we need to address. you
know, most people uh most average
people, most politicians, they just not
aware that this is happening. Like we've
had some of these expert warnings uh
they've been helpful, but they've been
kind of limited to uh within the AI.
>> Yeah. Mhm.
>> Exactly. Like you know, and you know,
one letter is not like is is a great
step and it's not going to change
everything. It's not going to just
>> uh be installed in everybody's brains
immediately, right? And so what's what's
missing is this awareness. So the first
thing is to build this awareness. build
this common knowledge. It's important
that everybody knows that everybody else
knows or at least enough people. And
that's exactly what like we focus on at
control AI both with lawmakers and with
the public. So with lawmakers we're
focusing on building this this awareness
in uh multiple countries so we can bring
about this international agreement. We
are working in the US, we're working in
the UK, we're working in Canada and
Germany and you know looking to expand
to more countries. So far we've in just
over a year we briefed uh hundreds of
lawmakers in these countries. We briefed
170 plus in the UK uh more than 80 in
Canada uh that where we only started at
the end of last uh in the second half of
last year. We started in the US in the
second half of last year with only one
person. We now briefed uh over 90
congressional offices and 18 members of
Congress and uh senators and more to do.
So this just this big work of informing
lawmakers so they know about the risk
and in most of these meetings like the
most of the meetings it was maybe the
first time like the first time that they
had accurate 30 minutes with somebody to
actually explain them what's going on
with AI where things are going they
might you know they might have a some
understanding of AI they might have used
GPD but they don't know that you know
even the CEOs admit this could kill
everybody they don't know that these
experts have made these warnings
>> they have no sophisticated understanding
I mean they they have a very demanding
job. They can't they're not able to keep
up with with all this stuff.
>> Exactly.
>> Okay. So, so the the the information
campaign has has been a great success.
All of the lawmakers are on board. We're
starting to see um legislation in
individual countries, but of course, as
we said, the the goal is an
international effort here. So, what does
that look like? I'm not very familiar.
Obviously, there are international
organizations that exist. Can this be
operating through an existing
organization? Does a new organization
have to be set up? How do we get the
intern the international ban going? How
does what does that look like?
>> Yeah. So, I think the the best way for
that to get going is for countries to
commit to it directly. I think right now
we're in a situation where uh existing
international organizations are either
not up to the task or you know they're
not trusted by like one or another
country. So, but I think an
international agreement here can be done
by a group of countries that bands
together, you know, especially if the US
either leads the charge or joins after
after others lead the charge and says
very clearly, you know, we think super
intelligence is a massive threat to our
national security. So, it's against our
individual interest. You they can be uh
individually interested first and also
to global security. You know, if anybody
builds this
>> anywhere around the world, they are
compromising our security at home and
abroad. And we say no to that. And if we
have
>> enough countries on board with this, the
AI supply chain is quite narrow. Like
right now, most of the chips, the kind
of supercomputers use to develop these
systems are very complicated to make.
They're made by a handful of companies.
There's uh you know, Nvidia basically
designs almost all of the advanced
chips. uh they're fabricated mostly in
uh Taiwan in a few other locations. The
machines to make them are mostly made in
the Netherlands. So there's like a very
clear supply chain that can be
controlled. Um
>> and there can be a way to make sure that
countries that join the agreement, you
know, they monitor each other and they
make sure that no uh signatory or no
member of this deal is violating the
deal. And even if there are countries
that are outside of the agreement that
don't join the deal, the countries that
make this agreement, they can make sure
that those countries don't get the
resources that they could use to build
super intelligence. So you can either
join the deal and be monitored and uh
keep developing things or if you don't
join the deal, you will be limited from
accessing the AI resources. A bit like
we deal with nuclear weapons where we
monitor uranium, we monitor uranium
stockpiles. We make sure that you know
countries don't enrich uranium so they
cannot get to building the the bomb and
we're and but the way we got there kind
of regardless of the details of the
agreement the most important thing is
that we need the first step that we're
missing is that we need the political
>> backing for this and the political
backing is not going to happen without
the information right now most
politicians just don't know if they
don't know they're not going to act with
nukes they understand this is a top
level national security threat and they
you know you have uh intelligence
agencies you have militaries like
focusing their work uh on making sure
that this doesn't happen the nukes don't
proliferate we just don't have this with
super intelligence and so the first step
is to inform these governments inform
the public so the public can can build
can build the the the pressure and can
build a wave to know to say to their
governments we want this to happen
>> right that I think that's the key yeah
so everything you said makes perfect
sense so I I think it's very clear what
the path is for the governing bodies and
and for the bureaucratic machinery to to
get moving on this. And so then really
the the the the the catalyst for all of
this is is um the urgency among the
public and and and and speaking out and
uh you know making making this an
important issue. So I guess uh you know
in my in my previous video we did talk
about control AI's call to action and
and contacting lawmakers. let's maybe re
reiterate what they can do there and
then in addition is there anything else
that that the viewers at home can can do
to to accelerate and amplify this.
>> Yeah. So I think the the most important
thing that people can do is contact your
lawmaker wherever you are. You know if
you're in the United States but also in
any other country around the world at
Control Air we made tools to make it
really easy to contact your lawmakers in
the US in the UK in Germany. I think
we're launching the Canadian tool soon
and we're going to make more if there is
demand. You know, if you're in a
different country, let us know. And
that's, you know, that's a thing that
can take you less than five minutes.
It's quick. It's very important is, you
know, the best way that as a citizen you
can get engaged. And if you know, if
politicians never hear from you, they
they don't know that people care about
this. They're
>> they're
>> they don't have a strong reason.
>> Yeah. and vice versa. You know, many of
them might start to be concerned and but
they want to know that there is public
backing for this and this public backing
is like super super important. Then if
you have more time and uh you know the
link is at controli.com
uh take-action
or just to go to control.com and there's
a big button to do it. Um, and if you
have more time, we made a platform
called uh microcomit that is made by
some partners of ours where you can sign
up to take a few more actions a week.
You know, let's say you have like 10 15
minutes a week. It's actions like like
these types of civic action things. They
come to your inbox and you can quickly
go through them and you know accept them
or refuse them and do them. And if you
have even more time, there is a great
volunteer community called Torchbearer
that I am a member of that focusing on f
focus on this where the commitment is
around two hours a week uh of working on
this issues and if you want to help out
please check it out torchbearer
community and uh become a member.
>> Okay. Micro commit and torchbearer.
Yeah. beyond. I think everybody should
just, you know, really quick just uh
yeah, contact their lawmaker, get that
going, add add your voice to the to the
pile.
>> Um yeah, I mean that that's all I had. I
mean, I definitely I I I want to I'm
very thankful of people like you who
kind of devote all of your energy and
efforts towards this very particular
problem. Me as a science communicator
and a generalist, I'm just dealing with
So, it feels like spinning plates
sometimes. I'm just running around
putting out all of these different fires
and here's another one. Oh, we might all
die from AI. Um it's just it's it's hard
to uh to process all of this stuff
simultaneously. The ongoing political
turmoil and then these threats from the
private sector. It's just uh it's a lot.
But um as as long as I can kind of um
you know shine shine a spotlight on
whoever is working on each individual
problem with all of their with all of
their might, uh I I think that's kind of
the best I can do. And and I do gain a
tinge of optimism hearing kind of this
very clear plan um that is in place to
to get this initiative going. So thanks
again for all your effort and uh yeah
thanks for talking to me.
>> Yeah thank you so much and thank you so
much for sh sh sh sh sh sh sh sh sh sh
sh sh sh sh sh sh sh sh sh sh sh sh sh
sh sh sh sh shining light on these
issues. Again, the work of people like
you is super important because it lets
people, you know, know what's actually
happening and it, you know, reaches
millions of people and that's what
actually makes change possible. So thank
you so much and thank you.
>> That's all I can do. Yeah, thanks again.
Well, I hope that was as informative and
eye opening for you as it was for me.
I'm doing my best to approach this topic
with humility, partially because tech is
very much not my forte and partially
because nobody knows exactly what's
going to happen. But Andrea and others
make compelling arguments such that I am
definitely convinced of a credible
extinction risk. Is it a certainty? No.
But when you are dealing with the
destruction of the entire human species,
even a singledigit probability should be
enough to take it very seriously. And
this seems higher than that to me. I
find it very difficult to comprehend how
anyone could be completely unconcerned
about this eventuality. As to what will
happen next, the vast majority of us
will just have to wait on the sidelines
and hope for the best. But in the
meantime, check out Control AI and voice
your concerns with your local
legislators. Until next time.