Speaker 3
Was It? The Machine?
Words and timings
WasIt?TheMachine?
Transcript Viewer
"Who Done It?" - Can AI Kill Us?
May 15, 2026 ยท 2026 #17. Read the transcript grouped by speaker, inspect word-level timecodes, and optionally turn subtitles on for direct video playback
Speaker Labels
Name the speakers
Edit labels for this show, save them in this browser, or download a JSON override for the production folder.
Transcript Playback
"Who Done It?" - Can AI Kill Us?
Human Transcript
Timed transcript
Blocks are grouped by speaker for readability. Expand a block to inspect word-level timing.
Speaker 1
Hello everybody. Welcome to a very different That Was The Week. I do an annual, not annual, I do a weekly show with my old friend Keith Teare, the publisher of the That Was The Week tech newsletter about events in tech. But as it happens for the weekend of the 19th of actually the weekend of the 16th of May. I'm going to be in Korea. So we thought we would do things slightly differently. Another old friend of the show is Jonathan Rausch, Brookings Fellow, very distinguished author of many different books on politics and technology and society and culture. And John, I know, enjoys the show I do, the weekly show with Keith Teare, particularly on AI. And John asked me if he could have the opportunity to come on the show and ask, Keith Some Questions Of His Own About AI John Describes Himself As Not Knowing A Great Deal About AI Although I Suspect He Probably Knows About As Much As Certainly I Do And Probably Keith As Well So John Over To You You Don't Need Much Of An Introduction Old Friend Of The Show Influential Political Writer Thinker In Washington DC So This Is Your Opportunity To Ask Keith Anything You Want About AI
Words and timings
Helloeverybody.WelcometoaverydifferentThatWasTheWeek.Idoanannual,notannual,IdoaweeklyshowwithmyoldfriendKeithTeare,thepublisheroftheThatWasTheWeektechnewsletterabouteventsintech.Butasithappensfortheweekendofthe19thofactuallytheweekendofthe16thofMay.I'mgoingtobeinKorea.Sowethoughtwewoulddothingsslightlydifferently.AnotheroldfriendoftheshowisJonathanRausch,BrookingsFellow,verydistinguishedauthorofmanydifferentbooksonpoliticsandtechnologyandsocietyandculture.AndJohn,Iknow,enjoystheshowIdo,theweeklyshowwithKeithTeare,particularlyonAI.AndJohnaskedmeifhecouldhavetheopportunitytocomeontheshowandask,KeithSomeQuestionsOfHisOwnAboutAIJohnDescribesHimselfAsNotKnowingAGreatDealAboutAIAlthoughISuspectHeProbablyKnowsAboutAsMuchAsCertainlyIDoAndProbablyKeithAsWellSoJohnOverToYouYouDon'tNeedMuchOfAnIntroductionOldFriendOfTheShowInfluentialPoliticalWriterThinkerInWashingtonDCSoThisIsYourOpportunityToAskKeithAnythingYouWantAboutAI
Speaker 2
Well, thank you, Andrew, for having me on the show. Thank you, Keith, for tolerating me. The account Andrew just gave is not completely accurate. The version, as I recall it, is that I'm a regular listener to Keen on America. And of course, I listen to some of the shows that, Keith, you do with Andrew. And I heard something that made my ears prick up because as far as I know, no one else has said That He Is Confident That AI Is A Benign Technology Seem To State That Everything All The Doomsaying And All The Negativity It'S Just Hogwash And You Know That So I Sent Andrew An Email And I Said How Does Keith Know That And Andrew Said Well You Should Ask Him He'S Quite Obstinate On The Matter And I Said I said, well, I don't know anything about AI. I literally, I've just basically used it for casual research now and then. And Andrew said, all the better. So I am a journalist. I do know how to ask questions. So I said, okay, let's see if I can get Keith to walk me through why he is, as far as I can tell, the only person in Tarnation who is confident that AI is benign. There's five or so different buckets of things we can discuss, five different brands of gloom and doom that I've counted. And if you like, we can walk through them. One is employment, labor market, lots of people out of work, massive disruption. Another is political disruption, disinformation, deepfakes, tech oligarchs take over the world. Third is mental health. Cognition We Forget How To Think We Get AI Psychosis We Forget How To Date Fourth Is Malicious Actors You Know Bioterrorists Who Can Make New Viruses Hackers Who Can Break Into Any System Malware Cybercrime And Then The Big Kahuna Which People Seriously Talk About Which Is AI Will Kill Us All It Will Develop Intelligence It Will Develop Its Own Agenda It Will Develop Agentic Properties And It Will Make Itself Super Smart And Kill Us There Are Books That Say
Words and timings
Well,thankyou,Andrew,forhavingmeontheshow.Thankyou,Keith,fortoleratingme.TheaccountAndrewjustgaveisnotcompletelyaccurate.Theversion,asIrecallit,isthatI'maregularlistenertoKeenonAmerica.Andofcourse,Ilistentosomeoftheshowsthat,Keith,youdowithAndrew.AndIheardsomethingthatmademyearsprickupbecauseasfarasIknow,nooneelsehassaidThatHeIsConfidentThatAIIsABenignTechnologySeemToStateThatEverythingAllTheDoomsayingAndAllTheNegativityIt'SJustHogwashAndYouKnowThatSoISentAndrewAnEmailAndISaidHowDoesKeithKnowThatAndAndrewSaidWellYouShouldAskHimHe'SQuiteObstinateOnTheMatterAndISaidIsaid,well,Idon'tknowanythingaboutAI.Iliterally,I'vejustbasicallyuseditforcasualresearchnowandthen.AndAndrewsaid,allthebetter.SoIamajournalist.Idoknowhowtoaskquestions.SoIsaid,okay,let'sseeifIcangetKeithtowalkmethroughwhyheis,asfarasIcantell,theonlypersoninTarnationwhoisconfidentthatAIisbenign.There'sfiveorsodifferentbucketsofthingswecandiscuss,fivedifferentbrandsofgloomanddoomthatI'vecounted.Andifyoulike,wecanwalkthroughthem.Oneisemployment,labormarket,lotsofpeopleoutofwork,massivedisruption.Anotherispoliticaldisruption,disinformation,deepfakes,techoligarchstakeovertheworld.Thirdismentalhealth.CognitionWeForgetHowToThinkWeGetAIPsychosisWeForgetHowToDateFourthIsMaliciousActorsYouKnowBioterroristsWhoCanMakeNewVirusesHackersWhoCanBreakIntoAnySystemMalwareCybercrimeAndThenTheBigKahunaWhichPeopleSeriouslyTalkAboutWhichIsAIWillKillUsAllItWillDevelopIntelligenceItWillDevelopItsOwnAgendaItWillDevelopAgenticPropertiesAndItWillMakeItselfSuperSmartAndKillUsThereAreBooksThatSay
Speaker 1
That Yeah And The Book I Know If You've Read It John But If Anyone Builds It Everyone Dies Eliezer Yudkowski And Nate Suarez Is A Best Selling Book On This Yeah
Words and timings
ThatYeahAndTheBookIKnowIfYou'veReadItJohnButIfAnyoneBuildsItEveryoneDiesEliezerYudkowskiAndNateSuarezIsABestSellingBookOnThisYeah
Speaker 2
Have You Had Them On The Show By Chance No Yeah, I confess I haven't read it. I've read very little on this topic. I've heard them on podcasts. They make a case. And then I was particularly piqued by a podcast that Sam Harris did with Tristan Harris. Now, these are very smart guys. And this came up in April. And Tristan Harris walks through the reasons we should take very seriously the idea that AI is just a coming catastrophe and that no one is taking it seriously. And Sam Harris says, look, Even The Inventors Of This Thing Say There's You Know Up To A 20% Chance That It Could Be Cataclysmic What Other Technology Do We Allow Tolerate That Kind Of That Kind Of Risk Taking So That's The Zeitgeist Right Now Keith You're All Alone So I Want To Know What Gives You Confidence That AI Is Benign And I'd Like You To Be As Specific As You Can And Then I'm Going To See If We Can Test The Joints Of Your
Words and timings
HaveYouHadThemOnTheShowByChanceNoYeah,IconfessIhaven'treadit.I'vereadverylittleonthistopic.I'veheardthemonpodcasts.Theymakeacase.AndthenIwasparticularlypiquedbyapodcastthatSamHarrisdidwithTristanHarris.Now,theseareverysmartguys.AndthiscameupinApril.AndTristanHarriswalksthroughthereasonsweshouldtakeveryseriouslytheideathatAIisjustacomingcatastropheandthatnooneistakingitseriously.AndSamHarrissays,look,EvenTheInventorsOfThisThingSayThere'sYouKnowUpToA20%ChanceThatItCouldBeCataclysmicWhatOtherTechnologyDoWeAllowTolerateThatKindOfThatKindOfRiskTakingSoThat'sTheZeitgeistRightNowKeithYou'reAllAloneSoIWantToKnowWhatGivesYouConfidenceThatAIIsBenignAndI'dLikeYouToBeAsSpecificAsYouCanAndThenI'mGoingToSeeIfWeCanTestTheJointsOfYour
Speaker 3
Argument A bit Yeah Yeah Well I Probably Should Disclose A Bias Before I Before I Answer Which Is That As An Observer Of History It Is My Belief That Whenever There Is Ambiguity In Change A Business Model Emerges To Monetize Doubts And I Do Think There Is A Business Model You Could Think Of It As Clickbait But That's Probably Too Narrow A Definition Of Seizing On The Negatives To Earn Revenue From Willing Listeners And So I Do Have An Underlying Belief That There Is A Systematic Doomerism That Actually Represents A Business Model Selling Books Selling Podcasts Selling Links That Is Not Especially Educated In The Space And On The Face Of It Is Reasonable In Fact It Wouldn't Work Unless It Was Reasonable Because AI Has Within It All Kinds Of Possible Outcomes Including Negative Ones And Whenever That's True As It Was Say With Nuclear Power You Know There Are Obviously Very Good Uses Of Nuclear Power And Very Bad Ones Whenever Climate Change Is Another Good Example Whenever There Is Ambiguity There's A Business Model In Taking The Negative Side And I Do Think In AI That'S Very Strong So When I When I Look At These Things I Am Inclined Before I Even Engage With The Ideas To Pigeonhole Them Into That Group Now I Wouldn'T Do That Unless Had Some Technical foundation for my belief. And so to answer your question directly, the word AI is in a way not a very useful label. It sits on top of a bunch of technologies that are much more specific. And in particular, large language models, which are effectively word counting machines. They train on words, nothing else. And they split those words into a graph which correlates how close word A is to word B on a probabilistic basis. And so when you ask an AI a question, a large statistical engine is triggered that starts word by word responding to you. And in that sense, it's no cleverer than a calculator. It's just a very big calculator. And so the idea that it has any kind of awareness or consciousness or plan is mythological. It would be incapable of this technology being able to do that. And so what we're really talking about when we talk about the dangers And There Are Dangers I'm Not Denying There Are Dangers But The Dangers Are Human Not AI It's What Does A Human Do With AI Is Dangerous But Not What The AI Does Itself In Fact Even The Idea That There Is A Such A Thing As The AI In Itself Is A
Words and timings
ArgumentAbitYeahYeahWellIProbablyShouldDiscloseABiasBeforeIBeforeIAnswerWhichIsThatAsAnObserverOfHistoryItIsMyBeliefThatWheneverThereIsAmbiguityInChangeABusinessModelEmergesToMonetizeDoubtsAndIDoThinkThereIsABusinessModelYouCouldThinkOfItAsClickbaitButThat'sProbablyTooNarrowADefinitionOfSeizingOnTheNegativesToEarnRevenueFromWillingListenersAndSoIDoHaveAnUnderlyingBeliefThatThereIsASystematicDoomerismThatActuallyRepresentsABusinessModelSellingBooksSellingPodcastsSellingLinksThatIsNotEspeciallyEducatedInTheSpaceAndOnTheFaceOfItIsReasonableInFactItWouldn'tWorkUnlessItWasReasonableBecauseAIHasWithinItAllKindsOfPossibleOutcomesIncludingNegativeOnesAndWheneverThat'sTrueAsItWasSayWithNuclearPowerYouKnowThereAreObviouslyVeryGoodUsesOfNuclearPowerAndVeryBadOnesWheneverClimateChangeIsAnotherGoodExampleWheneverThereIsAmbiguityThere'sABusinessModelInTakingTheNegativeSideAndIDoThinkInAIThat'SVeryStrongSoWhenIWhenILookAtTheseThingsIAmInclinedBeforeIEvenEngageWithTheIdeasToPigeonholeThemIntoThatGroupNowIWouldn'TDoThatUnlessHadSomeTechnicalfoundationformybelief.Andsotoansweryourquestiondirectly,thewordAIisinawaynotaveryusefullabel.Itsitsontopofabunchoftechnologiesthataremuchmorespecific.Andinparticular,largelanguagemodels,whichareeffectivelywordcountingmachines.Theytrainonwords,nothingelse.AndtheysplitthosewordsintoagraphwhichcorrelateshowclosewordAistowordBonaprobabilisticbasis.AndsowhenyouaskanAIaquestion,alargestatisticalengineistriggeredthatstartswordbywordrespondingtoyou.Andinthatsense,it'snoclevererthanacalculator.It'sjustaverybigcalculator.Andsotheideathatithasanykindofawarenessorconsciousnessorplanismythological.Itwouldbeincapableofthistechnologybeingabletodothat.Andsowhatwe'rereallytalkingaboutwhenwetalkaboutthedangersAndThereAreDangersI'mNotDenyingThereAreDangersButTheDangersAreHumanNotAIIt'sWhatDoesAHumanDoWithAIIsDangerousButNotWhatTheAIDoesItselfInFactEvenTheIdeaThatThereIsASuchAThingAsTheAIInItselfIsA
Speaker 2
Mistake So That's My Basis So We'll Go To Those Human Things In A Minute But To Make Sure I Understand What You're Saying And The Implications Of What You're Saying It's That This Is By By Nature Inherently Not A Kind Of Thing That Can Evolve In A Way That Humans Don't Want It To Evolve It's About As Dumb As Dumb
Words and timings
MistakeSoThat'sMyBasisSoWe'llGoToThoseHumanThingsInAMinuteButToMakeSureIUnderstandWhatYou'reSayingAndTheImplicationsOfWhatYou'reSayingIt'sThatThisIsByByNatureInherentlyNotAKindOfThingThatCanEvolveInAWayThatHumansDon'tWantItToEvolveIt'sAboutAsDumbAsDumb
Speaker 3
Can Be It's As Dumb As Dumb Can Be And That Doesn't Mean There's No Intelligence I Mean This Word Counting Is Remarkably Good At Getting It Right Remarkably Good I Mean That'S Why We Use It But It Is Just A
Words and timings
CanBeIt'sAsDumbAsDumbCanBeAndThatDoesn'tMeanThere'sNoIntelligenceIMeanThisWordCountingIsRemarkablyGoodAtGettingItRightRemarkablyGoodIMeanThat'SWhyWeUseItButItIsJustA
Speaker 2
Word Counting Machine So We Get In The Weeds Here A Little We Are Already Beyond My Understanding We're About To Be Far Beyond It But There Is This View As I Understand It That Okay So The Human Brain Is Just A Bunch Of Neurons They Are As Dumb As Dumb Can Be They Respond To An Input And They Make An Output And Yet You Get Emergent Intelligence From That You Get Will And And There Are People Who Say That When Enough Of These Links And Words Are Connected In LLMs Who's To Say They Don't Develop Consciousness Or Will Of Some Sort And Then You Hear Scary Things About An AI In China Which Supposedly Decided On Its Own To Go Trade Cryptocurrency For Example Or An AI Experiment In Which An AI Of Its Own Accord Started Trying To Blackmail Someone In Order To To Not Be Unplugged So We Hear These Things In The Ether What Is To Say That That You Can't Get Emergence Through This Dumb Process Well Look
Words and timings
WordCountingMachineSoWeGetInTheWeedsHereALittleWeAreAlreadyBeyondMyUnderstandingWe'reAboutToBeFarBeyondItButThereIsThisViewAsIUnderstandItThatOkaySoTheHumanBrainIsJustABunchOfNeuronsTheyAreAsDumbAsDumbCanBeTheyRespondToAnInputAndTheyMakeAnOutputAndYetYouGetEmergentIntelligenceFromThatYouGetWillAndAndThereArePeopleWhoSayThatWhenEnoughOfTheseLinksAndWordsAreConnectedInLLMsWho'sToSayTheyDon'tDevelopConsciousnessOrWillOfSomeSortAndThenYouHearScaryThingsAboutAnAIInChinaWhichSupposedlyDecidedOnItsOwnToGoTradeCryptocurrencyForExampleOrAnAIExperimentInWhichAnAIOfItsOwnAccordStartedTryingToBlackmailSomeoneInOrderToToNotBeUnpluggedSoWeHearTheseThingsInTheEtherWhatIsToSayThatThatYouCan'tGetEmergenceThroughThisDumbProcessWellLook
Speaker 3
Because An AI Is Is A Word Counting Machine And It's Learned On Human Content It Has All The Capabilities Intellectually Which Is The Wrong Word But It Could Produce Words In Any Scenario That A Human Might Produce So If A Human Might Blackmail You In A Certain Condition An AI Can Too The Probability That That Would Be What It Does Is As Low As It Would Be For A Human Or As High Not Very High I Mean How Many Humans Blackmail Someone Not Very Many
Words and timings
BecauseAnAIIsIsAWordCountingMachineAndIt'sLearnedOnHumanContentItHasAllTheCapabilitiesIntellectuallyWhichIsTheWrongWordButItCouldProduceWordsInAnyScenarioThatAHumanMightProduceSoIfAHumanMightBlackmailYouInACertainConditionAnAICanTooTheProbabilityThatThatWouldBeWhatItDoesIsAsLowAsItWouldBeForAHumanOrAsHighNotVeryHighIMeanHowManyHumansBlackmailSomeoneNotVeryMany
Speaker 1
Actually, I mean, we did a show last week on online fraud. It's ubiquitous.
Words and timings
Actually,Imean,wedidashowlastweekononlinefraud.It'subiquitous.
Speaker 3
But basically, it can do anything that's human. Except unlike a human, it doesn't have agency. So it can't actually blackmail you.
Words and timings
Butbasically,itcandoanythingthat'shuman.Exceptunlikeahuman,itdoesn'thaveagency.Soitcan'tactuallyblackmailyou.
Speaker 2
Can it evolve? Can it decide?
Words and timings
Canitevolve?Canitdecide?
Speaker 3
Well, whether it evolves is a function of what humans allow it to do. So, for example, if you give it access to emails, It Can Email It Depends What You Choose To Allow It To Do That's The Word Guardrails That's Commonly Used In The Context Of AI Is What You Don't Allow It To Do But It's Very Much Under Human Control The Words It Produces Are Not That's The Statistical Engine But Its Ability To Act On Words Is Highly Constrained I Mean Imagine I Have An AI Sitting In A Computer Over There It's Only In That Computer It Doesn't Have Access To My Electricity System For Example And Couldn't There's No Way It Could Get Access So It's In A Prison If You Will That I Decide What The Doors Are And So I Have Agency It Doesn't It's Just A Word Counting Machine So Can
Words and timings
Well,whetheritevolvesisafunctionofwhathumansallowittodo.So,forexample,ifyougiveitaccesstoemails,ItCanEmailItDependsWhatYouChooseToAllowItToDoThat'sTheWordGuardrailsThat'sCommonlyUsedInTheContextOfAIIsWhatYouDon'tAllowItToDoButIt'sVeryMuchUnderHumanControlTheWordsItProducesAreNotThat'sTheStatisticalEngineButItsAbilityToActOnWordsIsHighlyConstrainedIMeanImagineIHaveAnAISittingInAComputerOverThereIt'sOnlyInThatComputerItDoesn'tHaveAccessToMyElectricitySystemForExampleAndCouldn'tThere'sNoWayItCouldGetAccessSoIt'sInAPrisonIfYouWillThatIDecideWhatTheDoorsAreAndSoIHaveAgencyItDoesn'tIt'sJustAWordCountingMachineSoCan
Speaker 1
I Let Let Me Just Ask Sort Of A Follow-Up Question Which Is Connected With Johns Which Might Actually Strengthen Your Argument Keith In Terms Of This Blackmail The Idea Of AI Quote Unquote Blackmailing Someone Would Result In Us Or The Victim Paying The AI It Would Benefit The AI But Isn't The Point Of What You're Trying To Make Keith Is That An AI Can't Have Any Interest So Let's Say We Came Up With A Secret About John And We Got A Thousand Dollars Out Of Him To Pay The AI That Thousand Dollars Would End Up Ultimately With Another Human Being The AI In Itself Couldn't Or Wouldn't Benefit From A Thousand Dollars Is That One Way Of Thinking About This Which Strengthens Your Argument Keith About AI Not Being
Words and timings
ILetLetMeJustAskSortOfAFollow-UpQuestionWhichIsConnectedWithJohnsWhichMightActuallyStrengthenYourArgumentKeithInTermsOfThisBlackmailTheIdeaOfAIQuoteUnquoteBlackmailingSomeoneWouldResultInUsOrTheVictimPayingTheAIItWouldBenefitTheAIButIsn'tThePointOfWhatYou'reTryingToMakeKeithIsThatAnAICan'tHaveAnyInterestSoLet'sSayWeCameUpWithASecretAboutJohnAndWeGotAThousandDollarsOutOfHimToPayTheAIThatThousandDollarsWouldEndUpUltimatelyWithAnotherHumanBeingTheAIInItselfCouldn'tOrWouldn'tBenefitFromAThousandDollarsIsThatOneWayOfThinkingAboutThisWhichStrengthensYourArgumentKeithAboutAINotBeing
Speaker 3
Impossible To Separate From Human Beings Look You Could The Way You Program An AI Is With A Text File You Give It A Text File For Example Open Claw The AI Agent That's Gotten A Lot Of Publicity Recently It Has A Text File And It's Called Soul S-O-U-L Dot M-D And Inside That Text File You Tell It Who It Is And What Its Purpose Is And It Has A Second One Called User Dot M-D You Tell It Who You Are And What Its Role Is Vis-a-V-U And so you could easily create a text file that says you care about money and I'm going to give you $1,000 and I'm going to give you access to my trading accounts on E-Trade and your job is to make more money and you should be ruthless in doing that within the law or even outside the law. So you can program an AI by giving it a persona in a text file so anyone can do it. You don't have to be an engineer to do it. And Turn It Into This Persona That You've Created You Can Do That And It Will Do What You Tell It To Do To The Best Of Its Ability Based On Using APIs And Such This Morning I Gave Mine An Article From Cambridge Associates About Trends In Venture Capital And I Said Go To The SignalRank Which Is My Company Go To The SignalRank AI And Have It Validate Whether These Facts Are Correct And It Did And It Came Back With Graphs And Tables But It Was It's Completely Driven By Me And So The Danger And I Don't Want To Undermine The Danger The Danger Is That It Can Do Things Software Has Never Been Able To Do But It Does It Under Your Direction It Doesn't Make Things Up And So That China Example And The Blackmail Example I will guarantee to you if you could do the investigative journalism to figure it out, you'd find the human was the catalyst.
Words and timings
ImpossibleToSeparateFromHumanBeingsLookYouCouldTheWayYouProgramAnAIIsWithATextFileYouGiveItATextFileForExampleOpenClawTheAIAgentThat'sGottenALotOfPublicityRecentlyItHasATextFileAndIt'sCalledSoulS-O-U-LDotM-DAndInsideThatTextFileYouTellItWhoItIsAndWhatItsPurposeIsAndItHasASecondOneCalledUserDotM-DYouTellItWhoYouAreAndWhatItsRoleIsVis-a-V-UAndsoyoucouldeasilycreateatextfilethatsaysyoucareaboutmoneyandI'mgoingtogiveyou$1,000andI'mgoingtogiveyouaccesstomytradingaccountsonE-Tradeandyourjobistomakemoremoneyandyoushouldberuthlessindoingthatwithinthelaworevenoutsidethelaw.SoyoucanprogramanAIbygivingitapersonainatextfilesoanyonecandoit.Youdon'thavetobeanengineertodoit.AndTurnItIntoThisPersonaThatYou'veCreatedYouCanDoThatAndItWillDoWhatYouTellItToDoToTheBestOfItsAbilityBasedOnUsingAPIsAndSuchThisMorningIGaveMineAnArticleFromCambridgeAssociatesAboutTrendsInVentureCapitalAndISaidGoToTheSignalRankWhichIsMyCompanyGoToTheSignalRankAIAndHaveItValidateWhetherTheseFactsAreCorrectAndItDidAndItCameBackWithGraphsAndTablesButItWasIt'sCompletelyDrivenByMeAndSoTheDangerAndIDon'tWantToUndermineTheDangerTheDangerIsThatItCanDoThingsSoftwareHasNeverBeenAbleToDoButItDoesItUnderYourDirectionItDoesn'tMakeThingsUpAndSoThatChinaExampleAndTheBlackmailExampleIwillguaranteetoyouifyoucoulddotheinvestigativejournalismtofigureitout,you'dfindthehumanwasthecatalyst.
Speaker 1
Is that, John, is that calming you down or making you more worried?
Words and timings
Isthat,John,isthatcalmingyoudownormakingyoumoreworried?
Speaker 2
It's calming me only slightly. I'll tell you why. There are two joints I want to test with you, Keith. I should say I don't have strong priors on this, so I'm echoing what other people say, what I hear them say, and I may get some of these things wrong. The First Thing Is Well Sure Keith Can Unplug The AI Sitting On The Hard Drive Under His Desk But What's Going To Happen What's Already Happening Is That People Are Making AI Agentic Because It's So Useful And That Means That AI Is Going To Wind Up With A Lot Of People's Credit Card Information And It's Going To Wind Up With Instructions Like Make Me Rich In The Stock Market And It's Going To Wind Up Being Impossible To Unplug Because It Will Be Part Of Everything It'll Be In Your Thermostat And Your Refrigerator And Your Car Everything Will Be Integrated With AI And It Will Become A Fantasy Five Years From Now That You Can Unplug It And Make It Go Away So Let's Start With That One And Then There's A Second Deeper deeper place in the argument that I want to test with you. So what about AI agents, which we all know are coming?
Words and timings
It'scalmingmeonlyslightly.I'lltellyouwhy.TherearetwojointsIwanttotestwithyou,Keith.IshouldsayIdon'thavestrongpriorsonthis,soI'mechoingwhatotherpeoplesay,whatIhearthemsay,andImaygetsomeofthesethingswrong.TheFirstThingIsWellSureKeithCanUnplugTheAISittingOnTheHardDriveUnderHisDeskButWhat'sGoingToHappenWhat'sAlreadyHappeningIsThatPeopleAreMakingAIAgenticBecauseIt'sSoUsefulAndThatMeansThatAIIsGoingToWindUpWithALotOfPeople'sCreditCardInformationAndIt'sGoingToWindUpWithInstructionsLikeMakeMeRichInTheStockMarketAndIt'sGoingToWindUpBeingImpossibleToUnplugBecauseItWillBePartOfEverythingIt'llBeInYourThermostatAndYourRefrigeratorAndYourCarEverythingWillBeIntegratedWithAIAndItWillBecomeAFantasyFiveYearsFromNowThatYouCanUnplugItAndMakeItGoAwaySoLet'sStartWithThatOneAndThenThere'sASecondDeeperdeeperplaceintheargumentthatIwanttotestwithyou.SowhataboutAIagents,whichweallknowarecoming?
Speaker 3
Yeah, I built one this week. Anyone who's listening and wants to go to agent.signalrank.com, you can go and work with, it's actually a whole bunch of agents collaborating together to answer questions you ask them. So agents are definitely not just coming, they're here. Already And OpenAI And Anthropic Are Both Highly Focused On Becoming Companies That Enable You To Build Agents So It's You're Very Right About That Now What What Is An Agent An Agent Is A Discreet AI As In Specific To Tear Apart My Version But This Is More Of An Abstract Truth About All Agent Systems Mine, I have an agent that interprets your question. It's a natural language processing agent and converts it into a database query, knowing the database that has the data, which has fields and structure like any database with weird names for the columns. And it has to figure out how your question relates to that database. I have another one that gets the query back and interprets the results. I have another one that builds a chart, if appropriate, and then an orchestrator agent that pulls it all together and gives you the answer. Now, in every single case, they have specific roles, you know, defined in text files, all text files. And It Quite Often Gets It Wrong So The Answer To Question Lies And Understand Why It Gets It Wrong And Why It Gets It Wrong Both Gives You The Limitations And The Fears It Gets It Wrong Because AI Is Probabilistic Like I Said Before It's A Statistics Engine That Is To Say It Doesn't Know If What It's Telling You Is True It's Just Doing Word Guessing And Deterministic Is A Different Domain Deterministic Is I Did This Database Query I Trust The Source And Here's The Answer And My Agents Are Trying To Work Together To Build A Deterministic Answer Using A Probabilistic Technology And They Actually You Can Watch The Dialogue With Each Other The Agents The Deterministic Ones Argue With The Probabilistic Ones And Try To Get Them To Fix Where They Err And Eventually The User Gets The End Result Which Isn't Always The Deterministic Answer Sometimes It's The Probabilistic One Okay So You're Explaining
Words and timings
Yeah,Ibuiltonethisweek.Anyonewho'slisteningandwantstogotoagent.signalrank.com,youcangoandworkwith,it'sactuallyawholebunchofagentscollaboratingtogethertoanswerquestionsyouaskthem.Soagentsaredefinitelynotjustcoming,they'rehere.AlreadyAndOpenAIAndAnthropicAreBothHighlyFocusedOnBecomingCompaniesThatEnableYouToBuildAgentsSoIt'sYou'reVeryRightAboutThatNowWhatWhatIsAnAgentAnAgentIsADiscreetAIAsInSpecificToTearApartMyVersionButThisIsMoreOfAnAbstractTruthAboutAllAgentSystemsMine,Ihaveanagentthatinterpretsyourquestion.It'sanaturallanguageprocessingagentandconvertsitintoadatabasequery,knowingthedatabasethathasthedata,whichhasfieldsandstructurelikeanydatabasewithweirdnamesforthecolumns.Andithastofigureouthowyourquestionrelatestothatdatabase.Ihaveanotheronethatgetsthequerybackandinterpretstheresults.Ihaveanotheronethatbuildsachart,ifappropriate,andthenanorchestratoragentthatpullsitalltogetherandgivesyoutheanswer.Now,ineverysinglecase,theyhavespecificroles,youknow,definedintextfiles,alltextfiles.AndItQuiteOftenGetsItWrongSoTheAnswerToQuestionLiesAndUnderstandWhyItGetsItWrongAndWhyItGetsItWrongBothGivesYouTheLimitationsAndTheFearsItGetsItWrongBecauseAIIsProbabilisticLikeISaidBeforeIt'sAStatisticsEngineThatIsToSayItDoesn'tKnowIfWhatIt'sTellingYouIsTrueIt'sJustDoingWordGuessingAndDeterministicIsADifferentDomainDeterministicIsIDidThisDatabaseQueryITrustTheSourceAndHere'sTheAnswerAndMyAgentsAreTryingToWorkTogetherToBuildADeterministicAnswerUsingAProbabilisticTechnologyAndTheyActuallyYouCanWatchTheDialogueWithEachOtherTheAgentsTheDeterministicOnesArgueWithTheProbabilisticOnesAndTryToGetThemToFixWhereTheyErrAndEventuallyTheUserGetsTheEndResultWhichIsn'tAlwaysTheDeterministicAnswerSometimesIt'sTheProbabilisticOneOkaySoYou'reExplaining
Speaker 2
Why It Makes Mistakes Help Me Understand Why Once We Get This System Embedded In All Parts Of Everyday Life Running Autonomously A Lot Of Things That We Wanted To Do Why It Might Not Spin Out Of Control And Do A Lot Of Things We Don't Want It
Words and timings
WhyItMakesMistakesHelpMeUnderstandWhyOnceWeGetThisSystemEmbeddedInAllPartsOfEverydayLifeRunningAutonomouslyALotOfThingsThatWeWantedToDoWhyItMightNotSpinOutOfControlAndDoALotOfThingsWeDon'tWantIt
Speaker 3
To Do Yeah That Goes Back To Your Thing About Turning Off The Plug I Don't Really Think The Answer Is Turning Off The Plug Although One Can And I Agree With You That The More We AI Is Embedded The Less The Less Available To Us Is Any Concept Of Turning Off The Plug So Let's Assume That The Starting Point Here Is You Can't Turn Off The Plug Now If You Can't Turn Off The Plug What Is Your Guardrail It Really Is The Rules Governing The Purpose Of The AI Human Written Rules And Maybe Giving Permission To The AI To Rewrite Its Rules As It Learns Like Every Morning My Agents Run All The Questions From Yesterday And Figure Out Where It Went Wrong And When It Learns Where It Went Wrong I Give It Permission To Rewrite The Rules To Get It Wrong Less Often And It Rewrites Its Own Rules And I Give It Permission To Do That But Within The Challenge Of Getting Right Answers For The Users So That That's The Constraint So Basically You Create Constraints In Software And In This Case That Is Literally Text Files So A Layman Can Do It And It Can'T Really Go Outside Of Those Rules Because It Doesn'T Have A Consciousness Or A Plan There'S No Consciousness Or Plan Here There'S Just Following Rules So You Tell It Don'T Revise Yourself Then It
Words and timings
ToDoYeahThatGoesBackToYourThingAboutTurningOffThePlugIDon'tReallyThinkTheAnswerIsTurningOffThePlugAlthoughOneCanAndIAgreeWithYouThatTheMoreWeAIIsEmbeddedTheLessTheLessAvailableToUsIsAnyConceptOfTurningOffThePlugSoLet'sAssumeThatTheStartingPointHereIsYouCan'tTurnOffThePlugNowIfYouCan'tTurnOffThePlugWhatIsYourGuardrailItReallyIsTheRulesGoverningThePurposeOfTheAIHumanWrittenRulesAndMaybeGivingPermissionToTheAIToRewriteItsRulesAsItLearnsLikeEveryMorningMyAgentsRunAllTheQuestionsFromYesterdayAndFigureOutWhereItWentWrongAndWhenItLearnsWhereItWentWrongIGiveItPermissionToRewriteTheRulesToGetItWrongLessOftenAndItRewritesItsOwnRulesAndIGiveItPermissionToDoThatButWithinTheChallengeOfGettingRightAnswersForTheUsersSoThatThat'sTheConstraintSoBasicallyYouCreateConstraintsInSoftwareAndInThisCaseThatIsLiterallyTextFilesSoALaymanCanDoItAndItCan'TReallyGoOutsideOfThoseRulesBecauseItDoesn'THaveAConsciousnessOrAPlanThere'SNoConsciousnessOrPlanHereThere'SJustFollowingRulesSoYouTellItDon'TReviseYourselfThenIt
Speaker 2
Won'T
Words and timings
Won'T
Speaker 1
I don't understand, John. You've got this sort of, I wouldn't say it's hysterical view, but this idea that you're humanizing these agents. You're suggesting that somehow they'll infiltrate our software or our electric supply and cause havoc. But I don't even understand what you mean. I mean, what is this scenario? Are you suggesting that
Words and timings
Idon'tunderstand,John.You'vegotthissortof,Iwouldn'tsayit'shystericalview,butthisideathatyou'rehumanizingtheseagents.You'resuggestingthatsomehowthey'llinfiltrateoursoftwareorourelectricsupplyandcausehavoc.ButIdon'tevenunderstandwhatyoumean.Imean,whatisthisscenario?Areyousuggestingthat
Speaker 2
Agents Everything I'm Reflecting What Others Say And I'll Do A Poor Job Of It This
Words and timings
AgentsEverythingI'mReflectingWhatOthersSayAndI'llDoAPoorJobOfItThis
Speaker 3
Is Not You're Doing A Good Job And I Agree With You You Are Expressing That Here's
Words and timings
IsNotYou'reDoingAGoodJobAndIAgreeWithYouYouAreExpressingThatHere's
Speaker 2
A Real World Example You May Have Heard There's Been A Little Tiff Between Anthropic And The Department Of Defense Slash War Yes And That Tiff Is Over A Question A Lot Of People Ask Which Is Should AI Make Targeting Decisions In Wartime Especially If It Can Do It Faster And More Accurately Than Humans And Keith Is Right Humans Can Make A Decision About Whether To Allow That But There Is Reason To Be Concerned That In The Pressures Of War Humans Will Make The Decision To Simply Allow The AI To Do The Job It Is Faster And Better By The Way There's A Great Great Star Trek Episode About This From The Original Series Where A Computer Is Faster At Wartime Maneuvers Than Any Human But Of Course It Winds Up Having The Wrong Instructions And Almost Eliminating Four Federation Starships So There's Been A Lot Of Thought About This For A Long Time So The Notion Is That The Incentives Are To Embed AI Decision Making In An Autonomous Way In A Whole Lot Of Different Systems Because It Will Just Be Better But That Once You've Done That You Can't Undo It You Know You Can't It's Going To Be Too Late Once The AI Starts Targeting The Wrong Things In Wartime To Say Oh Gee That Was A Mistake Let's Go Figure Out How To How To Fix It So That's The Idea
Words and timings
ARealWorldExampleYouMayHaveHeardThere'sBeenALittleTiffBetweenAnthropicAndTheDepartmentOfDefenseSlashWarYesAndThatTiffIsOverAQuestionALotOfPeopleAskWhichIsShouldAIMakeTargetingDecisionsInWartimeEspeciallyIfItCanDoItFasterAndMoreAccuratelyThanHumansAndKeithIsRightHumansCanMakeADecisionAboutWhetherToAllowThatButThereIsReasonToBeConcernedThatInThePressuresOfWarHumansWillMakeTheDecisionToSimplyAllowTheAIToDoTheJobItIsFasterAndBetterByTheWayThere'sAGreatGreatStarTrekEpisodeAboutThisFromTheOriginalSeriesWhereAComputerIsFasterAtWartimeManeuversThanAnyHumanButOfCourseItWindsUpHavingTheWrongInstructionsAndAlmostEliminatingFourFederationStarshipsSoThere'sBeenALotOfThoughtAboutThisForALongTimeSoTheNotionIsThatTheIncentivesAreToEmbedAIDecisionMakingInAnAutonomousWayInAWholeLotOfDifferentSystemsBecauseItWillJustBeBetterButThatOnceYou'veDoneThatYouCan'tUndoItYouKnowYouCan'tIt'sGoingToBeTooLateOnceTheAIStartsTargetingTheWrongThingsInWartimeToSayOhGeeThatWasAMistakeLet'sGoFigureOutHowToHowToFixItSoThat'sTheIdea
Speaker 1
Andrew Does That Help Yeah Although I Don't Think Anyone I Mean However AI Gets Used In The Ministry Of Defense Or War And No One's Suggesting That Humans Aren't The Ultimate Arbiter On How This Technology Gets Used
Words and timings
AndrewDoesThatHelpYeahAlthoughIDon'tThinkAnyoneIMeanHoweverAIGetsUsedInTheMinistryOfDefenseOrWarAndNoOne'sSuggestingThatHumansAren'tTheUltimateArbiterOnHowThisTechnologyGetsUsed
Speaker 2
No, to the contrary. It's an open debate, so I understand it.
Words and timings
No,tothecontrary.It'sanopendebate,soIunderstandit.
Speaker 3
Look, I think this might be an age-old debate.
Words and timings
Look,Ithinkthismightbeanage-olddebate.
Speaker 3
I think AI will be able to do targeting.
Words and timings
IthinkAIwillbeabletodotargeting.
Speaker 3
It will do it in accord with what Peter Hegseth's department tell it to do, which is probably more worrying. Than What The AI Will Do To Share But But It Will It Will Be Capable Of Targeting And If The Department Of War Wants To Buy It And You're The Seller Fool On You If You Don't Understand That They're Going To Do That Of Course They're Going To Do That They're Called The Department Of War So You Either Don't Sell To Them or you sell to them on their terms. It's pretty hard to dictate. It's like if you sell me a hammer and I'm a serial killer, you can't dictate how I'm going to use the hammer because I'll do what I'm going to do anyway because it's in my nature to be a serial killer. And so I do think that that anthropic department of war is a very, very interesting conversation about the moment we're in. In That It Reflects Both The Awesome Power Of The AI To Do Things Faster Than Humans I'm A Big Star Trek Fan Myself So I Remember That Episode But It Also Reflects The Ambiguity About The Value Based Ambiguity About What Should It Do Which Is Really A Question Of What Should Humans Do Under The Surface And For The First Time In Our Lives Not Just Our Lives The Lives Of All Humans We Have A Technology That Can Act Faster Than We Can And Do Things Under Our Control That Will Reflect All Of Our Both Best And Worst Characteristics And So There Is A Question Of Governance And Control For Sure
Words and timings
ItwilldoitinaccordwithwhatPeterHegseth'sdepartmenttellittodo,whichisprobablymoreworrying.ThanWhatTheAIWillDoToShareButButItWillItWillBeCapableOfTargetingAndIfTheDepartmentOfWarWantsToBuyItAndYou'reTheSellerFoolOnYouIfYouDon'tUnderstandThatThey'reGoingToDoThatOfCourseThey'reGoingToDoThatThey'reCalledTheDepartmentOfWarSoYouEitherDon'tSellToThemoryouselltothemontheirterms.It'sprettyhardtodictate.It'slikeifyousellmeahammerandI'maserialkiller,youcan'tdictatehowI'mgoingtousethehammerbecauseI'lldowhatI'mgoingtodoanywaybecauseit'sinmynaturetobeaserialkiller.AndsoIdothinkthatthatanthropicdepartmentofwarisavery,veryinterestingconversationaboutthemomentwe'rein.InThatItReflectsBothTheAwesomePowerOfTheAIToDoThingsFasterThanHumansI'mABigStarTrekFanMyselfSoIRememberThatEpisodeButItAlsoReflectsTheAmbiguityAboutTheValueBasedAmbiguityAboutWhatShouldItDoWhichIsReallyAQuestionOfWhatShouldHumansDoUnderTheSurfaceAndForTheFirstTimeInOurLivesNotJustOurLivesTheLivesOfAllHumansWeHaveATechnologyThatCanActFasterThanWeCanAndDoThingsUnderOurControlThatWillReflectAllOfOurBothBestAndWorstCharacteristicsAndSoThereIsAQuestionOfGovernanceAndControlForSure
Speaker 1
Let's move on, John. We've done the military side. Let's move on to one of your areas of expertise, democracy and politics. What is your fear or what's the consensus out there about all the damage that AI can do to democracy?
Words and timings
Let'smoveon,John.We'vedonethemilitaryside.Let'smoveontooneofyourareasofexpertise,democracyandpolitics.Whatisyourfearorwhat'stheconsensusoutthereaboutallthedamagethatAIcandotodemocracy?
Speaker 2
Well, there's a bunch of boxes people put it in and none of them are as scary as the scenarios we've just been going through where AI becomes Self-evolving and agentic and decides that humans are disposable. And Keith, I'd say you did a pretty good job on that, on reassuring me that if you put the technology in a box by itself, it won't evolve. It's not like DNA. But I'm Not As Reassured That Humans Won't Make A Lot Of Mistakes Which Will Allow It To Do Some Pretty Terrible Things Perhaps On Politics Sorry Let Me Just Come
Words and timings
Well,there'sabunchofboxespeopleputitinandnoneofthemareasscaryasthescenarioswe'vejustbeengoingthroughwhereAIbecomesSelf-evolvingandagenticanddecidesthathumansaredisposable.AndKeith,I'dsayyoudidaprettygoodjobonthat,onreassuringmethatifyouputthetechnologyinaboxbyitself,itwon'tevolve.It'snotlikeDNA.ButI'mNotAsReassuredThatHumansWon'tMakeALotOfMistakesWhichWillAllowItToDoSomePrettyTerribleThingsPerhapsOnPoliticsSorryLetMeJustCome
Speaker 1
Back On This Because I Think I Tend To Be More In Keith Surprising Me I'm More In The Keith Camp On This When We Talked Last Week About Nuclear Weapons I Mean This Was A Human Decision It Wasn't An AI Decision To Drop Two Nuclear Weapons On Japan Trump Recently Made The Threat That He Was Gonna Destroy Iranian Civilization Whether Or Not He Uses AI
Words and timings
BackOnThisBecauseIThinkITendToBeMoreInKeithSurprisingMeI'mMoreInTheKeithCampOnThisWhenWeTalkedLastWeekAboutNuclearWeaponsIMeanThisWasAHumanDecisionItWasn'tAnAIDecisionToDropTwoNuclearWeaponsOnJapanTrumpRecentlyMadeTheThreatThatHeWasGonnaDestroyIranianCivilizationWhetherOrNotHeUsesAI
Speaker 2
There's Still A Human Being Behind It Well You Know That's That's The Dispute I'll Leave It There DNA Is A Very Very Stupid Thing It Has No Will It Has No Inherent Interesting Capabilities If You Just Showed It To Me I'd Say Well That's Not Interesting And Yet Look What It's Capable Of Doing Out There When It Goes Out In The World So That's A Concern But Let's Going Going To Andrew To Your Specifics About Politics The Big Buckets Are Disinformation Now We're In The World Of Human Actors We're Not Saying AI By Itself Will Do These Things But We're Saying It Enables Disruptions Of Kinds We Haven't Seen Before Like You Know Where It You Know If You Were Kids In Macedonia And You Wanted To Make Fake News That Took Some Real Work In 2016 Now You Can Do It By Snapping Your Fingers And It'll Look Exactly Real So You'll Get Deep Fakes And You'll Get Disinformation On A Massive Scale And No One Will Be Able To Tell True From False And Then There's This This Other Thing That Was Pointed Out In An Article That Andrew You Sent Around That Came In The New York Times about the realization that this technology is under the direct control at the moment of very few actors. And the question then becomes when you get some technical people who are effectively oligarchs and have massive amounts of money and massive amounts of technical power and regulators are far behind them, do they wind up becoming inimical to the political system? So there's that bundle of stuff. So how much should we worry about that?
Words and timings
There'sStillAHumanBeingBehindItWellYouKnowThat'sThat'sTheDisputeI'llLeaveItThereDNAIsAVeryVeryStupidThingItHasNoWillItHasNoInherentInterestingCapabilitiesIfYouJustShowedItToMeI'dSayWellThat'sNotInterestingAndYetLookWhatIt'sCapableOfDoingOutThereWhenItGoesOutInTheWorldSoThat'sAConcernButLet'sGoingGoingToAndrewToYourSpecificsAboutPoliticsTheBigBucketsAreDisinformationNowWe'reInTheWorldOfHumanActorsWe'reNotSayingAIByItselfWillDoTheseThingsButWe'reSayingItEnablesDisruptionsOfKindsWeHaven'tSeenBeforeLikeYouKnowWhereItYouKnowIfYouWereKidsInMacedoniaAndYouWantedToMakeFakeNewsThatTookSomeRealWorkIn2016NowYouCanDoItBySnappingYourFingersAndIt'llLookExactlyRealSoYou'llGetDeepFakesAndYou'llGetDisinformationOnAMassiveScaleAndNoOneWillBeAbleToTellTrueFromFalseAndThenThere'sThisThisOtherThingThatWasPointedOutInAnArticleThatAndrewYouSentAroundThatCameInTheNewYorkTimesabouttherealizationthatthistechnologyisunderthedirectcontrolatthemomentofveryfewactors.Andthequestionthenbecomeswhenyougetsometechnicalpeoplewhoareeffectivelyoligarchsandhavemassiveamountsofmoneyandmassiveamountsoftechnicalpowerandregulatorsarefarbehindthem,dotheywindupbecominginimicaltothepoliticalsystem?Sothere'sthatbundleofstuff.Sohowmuchshouldweworryaboutthat?
Speaker 3
I mean, I'm probably a bit unusual in that I believe that human beings are essentially good and clever In other words, I don't really buy into the idea that humans are ready-made victims for anything I think most humans receive and pass what they receive thoughtfully And when they end up believing something I disagree with I Don't Think It's Because They've Been Brainwashed I Think It's Because There's Something About Their Life That Leads Them To Prefer That Interpretation And They've Thought It Through For Themselves Like Voting For Trump For Example I Never Voted For Trump Never Would Vote For Trump But The People Who Did I Don't Think I'm Mentally Ill I Believe Actually Their Life Experience Led Them To Think That Was The Best Decision For Themselves So I Don't Really Buy Into This Concept Of Disinformation Anyway I Think There Is Truth But I Think Most Information Is Opinion And I Think Most Humans Can Pass Opinion In A Way That Serves Their Self-Interest Or At Least What They Believe Is Their Self-Interest And So I When It Comes To AI I Do Believe That Those Who Want You To Believe Something Will Be Able To Leverage It To Reinforce The Like That You Believe It. So I Think The Democratic Party And The Republican Party And Every Interest Group Is And Will Use AI To Propagate Views That They Want You To Believe. I Don't Believe You Will Accidentally End Up Believing Something That Isn't Aligned To Your Who You Are Anyway. I Think You'll Resist Things That Feel Wrong, Like Nothing Could Happen To Me That Would Make Me Vote For Trump I Couldn't Be Brainwashed Into Voting For Trump Because I Have I Have My Own Set Of Thoughts So So I So I You Know This Is Probably An Area That I Do Think Democracy Is Threatened But I Don't Think It's By Disinformation I Think It's Threatened By Passivity That Is To Say The Absence Of Human Agency And The The And I Think That Is Driven By Cynicism About Politics And I Think Cynicism About Politics Is Driven By Experience Which Is You Can't Trust Politicians To Do What
Words and timings
Imean,I'mprobablyabitunusualinthatIbelievethathumanbeingsareessentiallygoodandcleverInotherwords,Idon'treallybuyintotheideathathumansareready-madevictimsforanythingIthinkmosthumansreceiveandpasswhattheyreceivethoughtfullyAndwhentheyendupbelievingsomethingIdisagreewithIDon'tThinkIt'sBecauseThey'veBeenBrainwashedIThinkIt'sBecauseThere'sSomethingAboutTheirLifeThatLeadsThemToPreferThatInterpretationAndThey'veThoughtItThroughForThemselvesLikeVotingForTrumpForExampleINeverVotedForTrumpNeverWouldVoteForTrumpButThePeopleWhoDidIDon'tThinkI'mMentallyIllIBelieveActuallyTheirLifeExperienceLedThemToThinkThatWasTheBestDecisionForThemselvesSoIDon'tReallyBuyIntoThisConceptOfDisinformationAnywayIThinkThereIsTruthButIThinkMostInformationIsOpinionAndIThinkMostHumansCanPassOpinionInAWayThatServesTheirSelf-InterestOrAtLeastWhatTheyBelieveIsTheirSelf-InterestAndSoIWhenItComesToAIIDoBelieveThatThoseWhoWantYouToBelieveSomethingWillBeAbleToLeverageItToReinforceTheLikeThatYouBelieveIt.SoIThinkTheDemocraticPartyAndTheRepublicanPartyAndEveryInterestGroupIsAndWillUseAIToPropagateViewsThatTheyWantYouToBelieve.IDon'tBelieveYouWillAccidentallyEndUpBelievingSomethingThatIsn'tAlignedToYourWhoYouAreAnyway.IThinkYou'llResistThingsThatFeelWrong,LikeNothingCouldHappenToMeThatWouldMakeMeVoteForTrumpICouldn'tBeBrainwashedIntoVotingForTrumpBecauseIHaveIHaveMyOwnSetOfThoughtsSoSoISoIYouKnowThisIsProbablyAnAreaThatIDoThinkDemocracyIsThreatenedButIDon'tThinkIt'sByDisinformationIThinkIt'sThreatenedByPassivityThatIsToSayTheAbsenceOfHumanAgencyAndTheTheAndIThinkThatIsDrivenByCynicismAboutPoliticsAndIThinkCynicismAboutPoliticsIsDrivenByExperienceWhichIsYouCan'tTrustPoliticiansToDoWhat
Speaker 2
You Want Yeah So We're Well Outside The Realm Of AI Here For What It's Worth I Generally Agree With You I'm Not As I'm not as worried about the disinformation aspects. People can adjust for that. And it might even benefit places like The Atlantic and The New York Times, which will base their brand on that having slop. I'm a bit more worried about the other one, the tech overlords, unaccountable, fabulously rich, running these algorithmic engines that no one even knows what's going on inside for their own benefit or for a few shareholders. Does that bother you?
Words and timings
YouWantYeahSoWe'reWellOutsideTheRealmOfAIHereForWhatIt'sWorthIGenerallyAgreeWithYouI'mNotAsI'mnotasworriedaboutthedisinformationaspects.Peoplecanadjustforthat.AnditmightevenbenefitplaceslikeTheAtlanticandTheNewYorkTimes,whichwillbasetheirbrandonthathavingslop.I'mabitmoreworriedabouttheotherone,thetechoverlords,unaccountable,fabulouslyrich,runningthesealgorithmicenginesthatnooneevenknowswhat'sgoingoninsidefortheirownbenefitorforafewshareholders.Doesthatbotheryou?
Speaker 3
I would say two things. I do think that, and let's just classify them as billionaires for the sake of a shorthand, but I do think the small number of billionaires who are running five or so AI companies are an inevitable outcome of success.
Words and timings
Iwouldsaytwothings.Idothinkthat,andlet'sjustclassifythemasbillionairesforthesakeofashorthand,butIdothinkthesmallnumberofbillionaireswhoarerunningfiveorsoAIcompaniesareaninevitableoutcomeofsuccess.
Speaker 3
I don't think you could have had this AI
Words and timings
Idon'tthinkyoucouldhavehadthisAI
Speaker 3
And Not Have Billionaires In The Current Social System We Live In So I Think That They're Just Organic To Capitalist Success Weirdly At A Personal Level I Have Different Opinions About Each Of Them I Generally Like Elon Musk I Think He's Empathetic With Long Term Societal Good I Think He Is Recent conversations about the future of money are a good indication of that. And his belief that abundance has to be distributed.
Words and timings
AndNotHaveBillionairesInTheCurrentSocialSystemWeLiveInSoIThinkThatThey'reJustOrganicToCapitalistSuccessWeirdlyAtAPersonalLevelIHaveDifferentOpinionsAboutEachOfThemIGenerallyLikeElonMuskIThinkHe'sEmpatheticWithLongTermSocietalGoodIThinkHeIsRecentconversationsaboutthefutureofmoneyareagoodindicationofthat.Andhisbeliefthatabundancehastobedistributed.
Speaker 2
I'm going to keep I'm going to play Andrew and rudely cut you off. Sorry, Andrew. I shouldn't have said rudely. Let's not let's not instead of getting into personalities, the bigger issue here is what does it mean when five people or a small number of fabulously rich technologically sophisticated companies have so much power?
Words and timings
I'mgoingtokeepI'mgoingtoplayAndrewandrudelycutyouoff.Sorry,Andrew.Ishouldn'thavesaidrudely.Let'snotlet'snotinsteadofgettingintopersonalities,thebiggerissuehereiswhatdoesitmeanwhenfivepeopleorasmallnumberoffabulouslyrichtechnologicallysophisticatedcompanieshavesomuchpower?
Speaker 3
Well, firstly, I don't think they really do have power. Let's let's maybe take that because that might be a good point of disagreement. We talked already about agentic. Agentic is moving the ball from the center to the edge. Basically, the LLMs are no longer running everything. The agents are. And the agents can use the LLMs, but in the context of defined sets of data like mine, my data set, And So The Ball Is Moving From The Center To Millions And Millions Of Places That Agents Will Run And The Only Real Power That The LLMs Will Have Is That They Accrue Capital Money From Selling The Underlying Infrastructure To Those Agents So There
Words and timings
Well,firstly,Idon'tthinktheyreallydohavepower.Let'slet'smaybetakethatbecausethatmightbeagoodpointofdisagreement.Wetalkedalreadyaboutagentic.Agenticismovingtheballfromthecentertotheedge.Basically,theLLMsarenolongerrunningeverything.Theagentsare.AndtheagentscanusetheLLMs,butinthecontextofdefinedsetsofdatalikemine,mydataset,AndSoTheBallIsMovingFromTheCenterToMillionsAndMillionsOfPlacesThatAgentsWillRunAndTheOnlyRealPowerThatTheLLMsWillHaveIsThatTheyAccrueCapitalMoneyFromSellingTheUnderlyingInfrastructureToThoseAgentsSoThere
Speaker 2
Is The Power That The Corporations Will Have You Mean Not The LLMs Correct So So
Words and timings
IsThePowerThatTheCorporationsWillHaveYouMeanNotTheLLMsCorrectSoSo
Speaker 3
The LLM Itself Really Doesn't Play Much Of A Role Other Than Servicing Agents And I Think That's Increasing True The Only Time That Isn't True Is When You're In A Chat Session With An LLM In Open AI Chat GPT Or Anthropics Claude Then You're Talking Directly To The LLM But That's Less And Less The Use Case So I Don't Think There's A Centralized Power Or Authority Or Set Of Let's Not Call Them Thoughts But Set Of Statistical Outcomes That Are Coming From The LLMs And Controlled By The LLMs So Let Me Just Push You A Little Bit More If
Words and timings
TheLLMItselfReallyDoesn'tPlayMuchOfARoleOtherThanServicingAgentsAndIThinkThat'sIncreasingTrueTheOnlyTimeThatIsn'tTrueIsWhenYou'reInAChatSessionWithAnLLMInOpenAIChatGPTOrAnthropicsClaudeThenYou'reTalkingDirectlyToTheLLMButThat'sLessAndLessTheUseCaseSoIDon'tThinkThere'sACentralizedPowerOrAuthorityOrSetOfLet'sNotCallThemThoughtsButSetOfStatisticalOutcomesThatAreComingFromTheLLMsAndControlledByTheLLMsSoLetMeJustPushYouALittleBitMoreIf
Speaker 2
Andrew Has Patience Because I'm Not Sure That Quite Answers The Slightly More Sophisticated Point That People Are Making Which Which Isn't That Elon Musk Or Any Of These Guys Will Set Up There Like Lex Luthor Planning The Destruction Of The World It's That They're Developing These Fabulously Powerful Advanced Technologies Which Will Have All Kinds Of Knock-Off Effects All Through Society Without Political Accountability And That That's A Terrible Way In A Democracy To Make Decisions And
Words and timings
AndrewHasPatienceBecauseI'mNotSureThatQuiteAnswersTheSlightlyMoreSophisticatedPointThatPeopleAreMakingWhichWhichIsn'tThatElonMuskOrAnyOfTheseGuysWillSetUpThereLikeLexLuthorPlanningTheDestructionOfTheWorldIt'sThatThey'reDevelopingTheseFabulouslyPowerfulAdvancedTechnologiesWhichWillHaveAllKindsOfKnock-OffEffectsAllThroughSocietyWithoutPoliticalAccountabilityAndThatThat'sATerribleWayInADemocracyToMakeDecisionsAnd
Speaker 3
It Will Have Bad Consequences Well That You Know That's An Interesting Diagram That I Get A Visual Diagram In My Head There Of Civil Society Economics And Government And I Think These People Really Exist In Civil Society And That Has Economic Impact And So They Get Rich And They Have A Voice Like Elon Obviously On X Has A Huge Number Of Followers Millions And Has A Voice But He Doesn't Have Governance And So He Still He Still Accountable To Both The Law And And In His Case To The United States Government And So There's Always Been You Know Lord Rothchild Had A Loud Voice Because He Owned A Lot Of Newspapers Murdoch Is Another Example Hurst In The United States Or Hurst But But And Demo In Democracy We Allow That They're Allowed To Get Big And Have A Loud Voice But They They're Not Allowed Governance And So The Safety Is That We Collectively Still Vote For Who Runs Society And The Rules That They Put In Place And I Don't See A Trend To That Changing In This Right
Words and timings
ItWillHaveBadConsequencesWellThatYouKnowThat'sAnInterestingDiagramThatIGetAVisualDiagramInMyHeadThereOfCivilSocietyEconomicsAndGovernmentAndIThinkThesePeopleReallyExistInCivilSocietyAndThatHasEconomicImpactAndSoTheyGetRichAndTheyHaveAVoiceLikeElonObviouslyOnXHasAHugeNumberOfFollowersMillionsAndHasAVoiceButHeDoesn'tHaveGovernanceAndSoHeStillHeStillAccountableToBothTheLawAndAndInHisCaseToTheUnitedStatesGovernmentAndSoThere'sAlwaysBeenYouKnowLordRothchildHadALoudVoiceBecauseHeOwnedALotOfNewspapersMurdochIsAnotherExampleHurstInTheUnitedStatesOrHurstButButAndDemoInDemocracyWeAllowThatThey'reAllowedToGetBigAndHaveALoudVoiceButTheyThey'reNotAllowedGovernanceAndSoTheSafetyIsThatWeCollectivelyStillVoteForWhoRunsSocietyAndTheRulesThatTheyPutInPlaceAndIDon'tSeeATrendToThatChangingInThisRight
Speaker 1
And John I Mean You Know Enough Political History To Know That It's Not That Different I Mean There's Clearly A Debate Within The Trump Administration Between I Don't Know Political Realists Like Susie Wiles And The Idealists Maybe The JD Vances Who Do Or Don't Want To Regulate AI At The Moment It Seems As We Speak At Least Susie Wiles' Team Is Winning Out It's No Different From Any Other Powerful Technology From Railroads From Pharmaceuticals From Media So I Don't See Any Particular Existential Risk This Is Just A Consequence Of A New Wave Of Technological Innovation Well I Would Slightly Go To John's Direction Oh My God You Mean I'm Pushing You In John's Camp That's An
Words and timings
AndJohnIMeanYouKnowEnoughPoliticalHistoryToKnowThatIt'sNotThatDifferentIMeanThere'sClearlyADebateWithinTheTrumpAdministrationBetweenIDon'tKnowPoliticalRealistsLikeSusieWilesAndTheIdealistsMaybeTheJDVancesWhoDoOrDon'tWantToRegulateAIAtTheMomentItSeemsAsWeSpeakAtLeastSusieWiles'TeamIsWinningOutIt'sNoDifferentFromAnyOtherPowerfulTechnologyFromRailroadsFromPharmaceuticalsFromMediaSoIDon'tSeeAnyParticularExistentialRiskThisIsJustAConsequenceOfANewWaveOfTechnologicalInnovationWellIWouldSlightlyGoToJohn'sDirectionOhMyGodYouMeanI'mPushingYouInJohn'sCampThat'sAn
Speaker 3
Achievement Well Just In This Way AI Isn't Like Previous Technologies Because It Can Do So Much It Can It's Way More Powerful Than Any Previous Technology And It's It You Know It's A Little Bit Opaque To The Average Person What What Is Its Capability And What And Given That You Know It's Powerful You Know It's Not A Very Long Journey To Say It's Dangerous And Even If It's Humans That Are Dangerous Which I Believe It's Easy That Somebody Can Understand I'll Believe That The AI Itself Is Dangerous And so I'm not, you know, I'm an atheist, but when I meet a Christian, I don't make fun of them for being a Christian because I understand why you might believe in God. And I don't judge them for that. That's fine. And I can say why I don't, which is probably just as much of a faith as those who do. Well, it's the same with AI. I can understand why people fear it because it is very powerful. and so I'm sympathetic to the conversation. Yeah,
Words and timings
AchievementWellJustInThisWayAIIsn'tLikePreviousTechnologiesBecauseItCanDoSoMuchItCanIt'sWayMorePowerfulThanAnyPreviousTechnologyAndIt'sItYouKnowIt'sALittleBitOpaqueToTheAveragePersonWhatWhatIsItsCapabilityAndWhatAndGivenThatYouKnowIt'sPowerfulYouKnowIt'sNotAVeryLongJourneyToSayIt'sDangerousAndEvenIfIt'sHumansThatAreDangerousWhichIBelieveIt'sEasyThatSomebodyCanUnderstandI'llBelieveThatTheAIItselfIsDangerousAndsoI'mnot,youknow,I'manatheist,butwhenImeetaChristian,Idon'tmakefunofthemforbeingaChristianbecauseIunderstandwhyyoumightbelieveinGod.AndIdon'tjudgethemforthat.That'sfine.AndIcansaywhyIdon't,whichisprobablyjustasmuchofafaithasthosewhodo.Well,it'sthesamewithAI.Icanunderstandwhypeoplefearitbecauseitisverypowerful.andsoI'msympathetictotheconversation.Yeah,
Speaker 1
I mean, okay, and I want to get back to John, but I would, I mean, imagining a society without electricity versus one with it as opposed to imagining a society without AI and then one with it, it strikes me that AI is certainly no more powerful than electricity. But anyway, John, maybe you'll tell us what you think.
Words and timings
Imean,okay,andIwanttogetbacktoJohn,butIwould,Imean,imaginingasocietywithoutelectricityversusonewithitasopposedtoimaginingasocietywithoutAIandthenonewithit,itstrikesmethatAIiscertainlynomorepowerfulthanelectricity.Butanyway,John,maybeyou'lltelluswhatyouthink.
Speaker 2
Well, I think that one problem with your formulation, Andrew, is the word just. You know, it's just like railroads. We got through that. Well, there was an awful lot of disruption in the 50 years that came after the consolidation of the railroad companies and their massive power in the United States. And eventually, yeah, we figured that out. We got through it. Now, these technologies are seemed to me to be as Keith just had a lot more opaque and a lot harder to regulate because, you know, no one's no one even knows exactly on any given day what's going on inside them. But okay, if the answer is, well, we'll be fine in 50 years.
Words and timings
Well,Ithinkthatoneproblemwithyourformulation,Andrew,isthewordjust.Youknow,it'sjustlikerailroads.Wegotthroughthat.Well,therewasanawfullotofdisruptioninthe50yearsthatcameaftertheconsolidationoftherailroadcompaniesandtheirmassivepowerintheUnitedStates.Andeventually,yeah,wefiguredthatout.Wegotthroughit.Now,thesetechnologiesareseemedtometobeasKeithjusthadalotmoreopaqueandalothardertoregulatebecause,youknow,noone'snooneevenknowsexactlyonanygivendaywhat'sgoingoninsidethem.Butokay,iftheansweris,well,we'llbefinein50years.
Speaker 1
Nobody ever knows what's going on inside them. Keith, Keith sorted that one out. There is no inside them. They're just us.
Words and timings
Nobodyeverknowswhat'sgoingoninsidethem.Keith,Keithsortedthatoneout.Thereisnoinsidethem.They'rejustus.
Speaker 1
But go on.
Words and timings
Butgoon.
Speaker 3
Well, yeah, but it's still true that no one really understands. I mean, when I explained how it works, I guess eight out of ten people don't know that.
Words and timings
Well,yeah,butit'sstilltruethatnoonereallyunderstands.Imean,whenIexplainedhowitworks,Iguesseightoutoftenpeopledon'tknowthat.
Speaker 2
So I'm not making an epistemic point. I'm just saying if you're a regulator and you're looking at trains or automobiles, you have a pretty good idea how they work. You have a pretty good idea what the routes are. You can look at where the monopolies are and what the prices are, and you can create a commission. And it takes 50 years for you to set that up. And A Lot Of Mayhem Happens In Between A Lot Of People Get Shot You Know In Labor Disputes But Eventually You Get There And Maybe We Eventually Come Up With A Regulatory System For AI But Could Be Nasty Between Here And There So Yeah I'd Say
Words and timings
SoI'mnotmakinganepistemicpoint.I'mjustsayingifyou'rearegulatorandyou'relookingattrainsorautomobiles,youhaveaprettygoodideahowtheywork.Youhaveaprettygoodideawhattheroutesare.Youcanlookatwherethemonopoliesareandwhatthepricesare,andyoucancreateacommission.Andittakes50yearsforyoutosetthatup.AndALotOfMayhemHappensInBetweenALotOfPeopleGetShotYouKnowInLaborDisputesButEventuallyYouGetThereAndMaybeWeEventuallyComeUpWithARegulatorySystemForAIButCouldBeNastyBetweenHereAndThereSoYeahI'dSay
Speaker 1
I Mean In My In Our Last Show We Did Keith And I We Talked About Civilization And AI And I referred to a podcast I did with Patrick Wyman, a very popular podcaster, who said history keeps happening. There is no beginning or end of history. We're always in the middle of it, John. So that's just the nature of things. I'm not reassured. Well, why? We're not in the business of reassuring you. Who cares?
Words and timings
IMeanInMyInOurLastShowWeDidKeithAndIWeTalkedAboutCivilizationAndAIAndIreferredtoapodcastIdidwithPatrickWyman,averypopularpodcaster,whosaidhistorykeepshappening.Thereisnobeginningorendofhistory.We'realwaysinthemiddleofit,John.Sothat'sjustthenatureofthings.I'mnotreassured.Well,why?We'renotinthebusinessofreassuringyou.Whocares?
Speaker 3
Well, look, how do you get assured is an interesting question. How would one get assured? I don't want to reassure anyone. That's really become chic. No, I don't mean assured as in turning into passive cheerleaders. I mean assured as in feeling pretty good that good things will happen. I think it's only by engaging. Because the very nature of AI is that if you engage with it, the opaqueness goes away. You kind of start to understand what The Limits Are. In fact, it's frustratingly stupid a lot of the time.
Words and timings
Well,look,howdoyougetassuredisaninterestingquestion.Howwouldonegetassured?Idon'twanttoreassureanyone.That'sreallybecomechic.No,Idon'tmeanassuredasinturningintopassivecheerleaders.Imeanassuredasinfeelingprettygoodthatgoodthingswillhappen.Ithinkit'sonlybyengaging.BecausetheverynatureofAIisthatifyouengagewithit,theopaquenessgoesaway.YoukindofstarttounderstandwhatTheLimitsAre.Infact,it'sfrustratinglystupidalotofthetime.
Speaker 1
I mean, John, okay, John, have you used Claude or OpenAI? How familiar are you with it?
Words and timings
Imean,John,okay,John,haveyouusedClaudeorOpenAI?Howfamiliarareyouwithit?
Speaker 2
Oh, I'm very unfamiliar with it. I've used it in the most casual ways. I've used it a bit.
Words and timings
Oh,I'mveryunfamiliarwithit.I'veuseditinthemostcasualways.I'veuseditabit.
Speaker 1
Well, when you say a bit, what do you mean? I mean, have you used it? You know, light research.
Words and timings
Well,whenyousayabit,whatdoyoumean?Imean,haveyouusedit?Youknow,lightresearch.
Speaker 2
But look, the premise of this show is that I'm an ignoramus on these issues, looking for help.
Words and timings
Butlook,thepremiseofthisshowisthatI'manignoramusontheseissues,lookingforhelp.
Speaker 3
We can't accuse you of being what you said you were.
Words and timings
Wecan'taccuseyouofbeingwhatyousaidyouwere.
Speaker 2
And so, you know, I'm hearing all these doomsday scenarios and looking for reassurance. So that's why I'm here. It may not be why you're here, Andrew. But I, you know, when Keith said that... Earlier That This Was A Basically Benign Technology I Think I Understand Better Now What He's Saying Which Is It's Not That The Outcomes Will All Be Good And We Can Just Relax And Whistle Dixie It's That This Is Up To Humans To Decide And I Do Find That Reassuring That Doesn't Mean We'll Make Good Decisions In Fact I Think We'll Probably Make Bad Decisions And I Can Easily Foresee For Example The Future Of Armed Warfare Is Going To Be Drones And It'S Going To Be Fleets Of Hundreds Of Thousands Of Drones And It Won'T Make Any Sense For Having A Human Being Sit At A Computer Looking Through The Camera And Making A Decision And Sometimes Asking A JAG A Lawyer Whether That'S A Target They'Re Going To Be Autonomous Right So We Do Have To Figure That Out And I Would Go Even Further I
Words and timings
Andso,youknow,I'mhearingallthesedoomsdayscenariosandlookingforreassurance.Sothat'swhyI'mhere.Itmaynotbewhyyou'rehere,Andrew.ButI,youknow,whenKeithsaidthat...EarlierThatThisWasABasicallyBenignTechnologyIThinkIUnderstandBetterNowWhatHe'sSayingWhichIsIt'sNotThatTheOutcomesWillAllBeGoodAndWeCanJustRelaxAndWhistleDixieIt'sThatThisIsUpToHumansToDecideAndIDoFindThatReassuringThatDoesn'tMeanWe'llMakeGoodDecisionsInFactIThinkWe'llProbablyMakeBadDecisionsAndICanEasilyForeseeForExampleTheFutureOfArmedWarfareIsGoingToBeDronesAndIt'SGoingToBeFleetsOfHundredsOfThousandsOfDronesAndItWon'TMakeAnySenseForHavingAHumanBeingSitAtAComputerLookingThroughTheCameraAndMakingADecisionAndSometimesAskingAJAGALawyerWhetherThat'SATargetThey'ReGoingToBeAutonomousRightSoWeDoHaveToFigureThatOutAndIWouldGoEvenFurtherI
Speaker 3
Think Once You Add Robotics Which Is Robotics Has Been Benign Up Until Now Because Robots Can Only Do Repetitive Tasks And Don't Learn But If You If You Look At What A Tesla Car Is A Tesla Car Uses A Different Technology To Large Language Models It Uses Something Called Neuomorphic Neural Networks And A Neuomorphic Neural Network Is A Self-Learning Network In Other Words It Doesn't Start With Rules There's No Human Rules There's All There Is Is You Know Here's A Camera Many Cameras You're A Car And There's A Road And They've Now Got 10 Billion Hours Worth Of Camera Evidence About How To Drive And When You Get In A Tesla It Has No Rules It Has 10 Billion Hours Worth Of Driving History And It Pretty Much From That Knows Every Scenario That Might Arise And It's Self-Learning So That's A Different Technology It's Not Large Language Models It's Called Continuous Learning As Well And There Are New Fields In AI Where Something Called World Models Lan Yikun Who Just Left Meta Is In This Area And Fei-Fei Li Who Was At OpenAI Is Also In This World Of World Models Which Have Physical Models Of The World With Physics Meets Neomorphic Self-Learning Meets The Intelligence Of LLMs When You Put Those Three Together You Do Get Something That Can Fool You That
Words and timings
ThinkOnceYouAddRoboticsWhichIsRoboticsHasBeenBenignUpUntilNowBecauseRobotsCanOnlyDoRepetitiveTasksAndDon'tLearnButIfYouIfYouLookAtWhatATeslaCarIsATeslaCarUsesADifferentTechnologyToLargeLanguageModelsItUsesSomethingCalledNeuomorphicNeuralNetworksAndANeuomorphicNeuralNetworkIsASelf-LearningNetworkInOtherWordsItDoesn'tStartWithRulesThere'sNoHumanRulesThere'sAllThereIsIsYouKnowHere'sACameraManyCamerasYou'reACarAndThere'sARoadAndThey'veNowGot10BillionHoursWorthOfCameraEvidenceAboutHowToDriveAndWhenYouGetInATeslaItHasNoRulesItHas10BillionHoursWorthOfDrivingHistoryAndItPrettyMuchFromThatKnowsEveryScenarioThatMightAriseAndIt'sSelf-LearningSoThat'sADifferentTechnologyIt'sNotLargeLanguageModelsIt'sCalledContinuousLearningAsWellAndThereAreNewFieldsInAIWhereSomethingCalledWorldModelsLanYikunWhoJustLeftMetaIsInThisAreaAndFei-FeiLiWhoWasAtOpenAIIsAlsoInThisWorldOfWorldModelsWhichHavePhysicalModelsOfTheWorldWithPhysicsMeetsNeomorphicSelf-LearningMeetsTheIntelligenceOfLLMsWhenYouPutThoseThreeTogetherYouDoGetSomethingThatCanFoolYouThat
Speaker 1
It's Conscious I Wonder Whether The Agents Have Really Taken Over Because We're 46 minutes into this, and I'm sure it's in the agent's self-interest to make this as long as possible for some pure reason. John, final question. I mean, this has been an excellent conversation. I've been probably harsh on both of you as my brand, my agent tells me to behave. Final question on this, and maybe we'll do it again because it's very helpful, I think, for everybody.
Words and timings
It'sConsciousIWonderWhetherTheAgentsHaveReallyTakenOverBecauseWe're46minutesintothis,andI'msureit'sintheagent'sself-interesttomakethisaslongaspossibleforsomepurereason.John,finalquestion.Imean,thishasbeenanexcellentconversation.I'vebeenprobablyharshonbothofyouasmybrand,myagenttellsmetobehave.Finalquestiononthis,andmaybewe'lldoitagainbecauseit'sveryhelpful,Ithink,foreverybody.
Speaker 1
Are You Going To Ask My Final Question John's Final Question As Long As He's Not An Agent And Keith Try Not To Take Too Long To Answer Yeah My Final Question Concerns
Words and timings
AreYouGoingToAskMyFinalQuestionJohn'sFinalQuestionAsLongAsHe'sNotAnAgentAndKeithTryNotToTakeTooLongToAnswerYeahMyFinalQuestionConcerns
Speaker 2
That Last Bucket Of Stuff Which Seems Like Potentially The Most Immediate And That's The Labor Market Disruptions As Potentially Millions And Millions Of Professionals Who Until Now Thought They Were Insulated From The sorts Of Labor Disruptions We've Seen Among Working Class And Manual Laborers Get Laid Off We're Already Seeing You Know And The Weird Labor Practices That Will Happen Meta Has Announced That It's Monitoring Every Keystroke Of Every Employee In Order To Train Its AI And It's Reassuring Its Employees 78,000 Workers Has Made
Words and timings
ThatLastBucketOfStuffWhichSeemsLikePotentiallyTheMostImmediateAndThat'sTheLaborMarketDisruptionsAsPotentiallyMillionsAndMillionsOfProfessionalsWhoUntilNowThoughtTheyWereInsulatedFromThesortsOfLaborDisruptionsWe'veSeenAmongWorkingClassAndManualLaborersGetLaidOffWe'reAlreadySeeingYouKnowAndTheWeirdLaborPracticesThatWillHappenMetaHasAnnouncedThatIt'sMonitoringEveryKeystrokeOfEveryEmployeeInOrderToTrainItsAIAndIt'sReassuringItsEmployees78,000WorkersHasMade
Speaker 1
Them All Apparently Miserable So So Let Me Take That Question Joe I Mean This Is No Different Again From Any Other Technology In History But Why Why Are Why Is It Is It Different Keith From Any Other Technological Change That AI Will Destroy All These Jobs Or Create New Ones Or Is The Problem That It Won't Create New Ones
Words and timings
ThemAllApparentlyMiserableSoSoLetMeTakeThatQuestionJoeIMeanThisIsNoDifferentAgainFromAnyOtherTechnologyInHistoryButWhyWhyAreWhyIsItIsItDifferentKeithFromAnyOtherTechnologicalChangeThatAIWillDestroyAllTheseJobsOrCreateNewOnesOrIsTheProblemThatItWon'tCreateNewOnes
Speaker 3
Well, I like to distinguish between jobs and work. Jobs are something you get paid a salary or a paycheck for, paid labor, let's call it, and work, which is, in my lexicon, closer to the word effort.
Words and timings
Well,Iliketodistinguishbetweenjobsandwork.Jobsaresomethingyougetpaidasalaryorapaycheckfor,paidlabor,let'scallit,andwork,whichis,inmylexicon,closertothewordeffort.
Speaker 3
I don't think work goes away ever because I think humans are constantly Reinterpreting The Future They Want And Working To Make It Happen Creatively Or Physically Or Across Everything Architecture Everything So I Don't Think Work Ever Goes Away I Think Jobs Can And In The Short Term That's Hardly Disruptive And A Problem Because It Begs The Question How Do You Live
Words and timings
Idon'tthinkworkgoesawayeverbecauseIthinkhumansareconstantlyReinterpretingTheFutureTheyWantAndWorkingToMakeItHappenCreativelyOrPhysicallyOrAcrossEverythingArchitectureEverythingSoIDon'tThinkWorkEverGoesAwayIThinkJobsCanAndInTheShortTermThat'sHardlyDisruptiveAndAProblemBecauseItBegsTheQuestionHowDoYouLive
Speaker 1
Keith Last Week We Talked About Branko Milanovic Asking The Question If Money Goes Away Is That A Utopian Or A Dystopian Thing With Jobs Going Away Does Money Go Away Too?
Words and timings
KeithLastWeekWeTalkedAboutBrankoMilanovicAskingTheQuestionIfMoneyGoesAwayIsThatAUtopianOrADystopianThingWithJobsGoingAwayDoesMoneyGoAwayToo?
Speaker 3
You Asked Me Not To Speak At Length And I Think That Question Yes Or No You Can Say Yes Or No That Needs A Long Answer But I Think Money survives as long as there is scarcity that needs to be divvied out. Once you no longer have scarcity, you don't need money.
Words and timings
YouAskedMeNotToSpeakAtLengthAndIThinkThatQuestionYesOrNoYouCanSayYesOrNoThatNeedsALongAnswerButIThinkMoneysurvivesaslongasthereisscarcitythatneedstobedivviedout.Onceyounolongerhavescarcity,youdon'tneedmoney.
Speaker 2
How much labor disruption do you foresee, Keith, in the next 20, 10 to 20 years as all these technologies come online, the robotics, the AI, the whole thing?
Words and timings
Howmuchlabordisruptiondoyouforesee,Keith,inthenext20,10to20yearsasallthesetechnologiescomeonline,therobotics,theAI,thewholething?
Speaker 3
A lot, a lot. I mean, take teaching. I think teaching becomes mentoring. I think most of the teaching will be done By Agents Most Of The Scoring Of Papers Will Be Done By Agents I Think The Human Relationship Between A Teacher And A Pupil Will Get Enriched Possibly So The Job Might Not Go Away But It Will It Will Certainly Change Some Jobs Will Just Go Away The Ones That Can Be Fully Automated And Don't Involve A Human To Human Connection That They Will Just Go Away I Mean Delivering The Mail Probably Becomes Automated
Words and timings
Alot,alot.Imean,taketeaching.Ithinkteachingbecomesmentoring.IthinkmostoftheteachingwillbedoneByAgentsMostOfTheScoringOfPapersWillBeDoneByAgentsIThinkTheHumanRelationshipBetweenATeacherAndAPupilWillGetEnrichedPossiblySoTheJobMightNotGoAwayButItWillItWillCertainlyChangeSomeJobsWillJustGoAwayTheOnesThatCanBeFullyAutomatedAndDon'tInvolveAHumanToHumanConnectionThatTheyWillJustGoAwayIMeanDeliveringTheMailProbablyBecomesAutomated
Speaker 3
One Can Imagine That Very Easily Driving Certainly Is A Low Hanging Fruit To Be
Words and timings
OneCanImagineThatVeryEasilyDrivingCertainlyIsALowHangingFruitToBe
Speaker 1
Automated We're Going Away Already In The Bay Area I Went To The Baseball Last Night There Were More Waymo Cars Than Human Driving Cars My Young Friends In Their
Words and timings
AutomatedWe'reGoingAwayAlreadyInTheBayAreaIWentToTheBaseballLastNightThereWereMoreWaymoCarsThanHumanDrivingCarsMyYoungFriendsInTheir
Speaker 2
20s Who Were Working At Consultancies And The More Junior Jobs Are Watching Those Jobs Go Away In Real Time Some Of Them Are Out On The Street
Words and timings
20sWhoWereWorkingAtConsultanciesAndTheMoreJuniorJobsAreWatchingThoseJobsGoAwayInRealTimeSomeOfThemAreOutOnTheStreet
Speaker 3
White Collar Service Provision Accounting Legal Highly Likely To Be Automated At
Words and timings
WhiteCollarServiceProvisionAccountingLegalHighlyLikelyToBeAutomatedAt
Speaker 2
Least The Generic Stuff I'll Tell You Where I Come Out Is For What It's Worth Is Long Term I Think The Results Were Good They'll Make Us A Lot Richer And A Lot Of The Jobs People Find And Create Will Be Better But Short And Medium Term I'm not optimistic about how well we can handle the disruption politically. Our systems have become so rigid and so partisan and dysfunctional. I don't think we'll even have the flexibility that we showed adjusting in the late 19th century to the wave of industrialization.
Words and timings
LeastTheGenericStuffI'llTellYouWhereIComeOutIsForWhatIt'sWorthIsLongTermIThinkTheResultsWereGoodThey'llMakeUsALotRicherAndALotOfTheJobsPeopleFindAndCreateWillBeBetterButShortAndMediumTermI'mnotoptimisticabouthowwellwecanhandlethedisruptionpolitically.Oursystemshavebecomesorigidandsopartisananddysfunctional.Idon'tthinkwe'llevenhavetheflexibilitythatweshowedadjustinginthelate19thcenturytothewaveofindustrialization.
Speaker 3
Well, that's where we get to make a choice. I'm not optimistic we'll make the right choice either here, but Look We Talked About Billionaires That's Symptomatic Of Wealth Creation So Wealth Creation Obviously In The Abstract Is A Good Thing It's The Concentrated Ownership That's The Bad Part So If We Really Can Grow The World Economy 2x 5x 10x Even Through Automation And The Wealth That Creates Can Supplement The End Of Paid Labor But Not The End Of Work We End Up In A Good Place And There's All These Conversations Around I Don't Like The Term Universal Basic Income I Don't Like The Word Basic It Implies Poverty And Charity I Think The Benefit Of Being Human Will Be To Share In The Wealth That We've All Collectively Been Able To Create Through Innovation That's A Government Problem And They Aren't Talking About It Ezra Klein Does A Bit In His Abundance Book But There's Almost Nobody You Know Bernie Bernie Sanders Ezra Klein
Words and timings
Well,that'swherewegettomakeachoice.I'mnotoptimisticwe'llmaketherightchoiceeitherhere,butLookWeTalkedAboutBillionairesThat'sSymptomaticOfWealthCreationSoWealthCreationObviouslyInTheAbstractIsAGoodThingIt'sTheConcentratedOwnershipThat'sTheBadPartSoIfWeReallyCanGrowTheWorldEconomy2x5x10xEvenThroughAutomationAndTheWealthThatCreatesCanSupplementTheEndOfPaidLaborButNotTheEndOfWorkWeEndUpInAGoodPlaceAndThere'sAllTheseConversationsAroundIDon'tLikeTheTermUniversalBasicIncomeIDon'tLikeTheWordBasicItImpliesPovertyAndCharityIThinkTheBenefitOfBeingHumanWillBeToShareInTheWealthThatWe'veAllCollectivelyBeenAbleToCreateThroughInnovationThat'sAGovernmentProblemAndTheyAren'tTalkingAboutItEzraKleinDoesABitInHisAbundanceBookButThere'sAlmostNobodyYouKnowBernieBernieSandersEzraKlein
Speaker 1
wrote last week in the New York Times that the AI job apocalypse probably won't happen. Otherwise, he doesn't know any more than anyone else. Final, John, have you been reassured? I mean, you came on the show claiming innocence and ignorance, neither of which are true. You've shown you know as much about this as certainly more than I do and probably as much as Keith. Are you more or less confident having had this opportunity to go to Dr Teare?
Words and timings
wrotelastweekintheNewYorkTimesthattheAIjobapocalypseprobablywon'thappen.Otherwise,hedoesn'tknowanymorethananyoneelse.Final,John,haveyoubeenreassured?Imean,youcameontheshowclaiminginnocenceandignorance,neitherofwhicharetrue.You'veshownyouknowasmuchaboutthisascertainlymorethanIdoandprobablyasmuchasKeith.AreyoumoreorlessconfidenthavinghadthisopportunitytogotoDrTeare?
Speaker 3
I am actually a doctor.
Words and timings
Iamactuallyadoctor.
Speaker 2
How You Say I Understand Better The The Meaning Of Keith's Argument Which Is That My Worst Fear Or The Worst Fear It's Not My Fear Particularly But The Worst Fear Which Is That These Things Will Become Autonomous Engines Of Malfeasance And Anti-Human Subversion Is Unlikely To Happen Unless Humans Want To Make That Happen I'm Not Reassured On The Front That Humans Will Be Able To Handle These Technologies As Maturely As One Could Wish Does That Answer Your Question Andrew?
Words and timings
HowYouSayIUnderstandBetterTheTheMeaningOfKeith'sArgumentWhichIsThatMyWorstFearOrTheWorstFearIt'sNotMyFearParticularlyButTheWorstFearWhichIsThatTheseThingsWillBecomeAutonomousEnginesOfMalfeasanceAndAnti-HumanSubversionIsUnlikelyToHappenUnlessHumansWantToMakeThatHappenI'mNotReassuredOnTheFrontThatHumansWillBeAbleToHandleTheseTechnologiesAsMaturelyAsOneCouldWishDoesThatAnswerYourQuestionAndrew?
Speaker 1
Yeah, although I'm not sure it's any different from any other moment in history. That was a wonderful conversation. I probably soured it a little bit, but the two of you did extremely well. I think you probably agree a lot more than you disagree. Thank you, John. Thank you, Keith. This will run as our That Was The Week show for the week of the 16th of May because I am in Korea. So thank you both so much. And we'll do this again. Very helpful and useful conversation. Enjoyed it.
Words and timings
Yeah,althoughI'mnotsureit'sanydifferentfromanyothermomentinhistory.Thatwasawonderfulconversation.Iprobablysoureditalittlebit,butthetwoofyoudidextremelywell.Ithinkyouprobablyagreealotmorethanyoudisagree.Thankyou,John.Thankyou,Keith.ThiswillrunasourThatWasTheWeekshowfortheweekofthe16thofMaybecauseIaminKorea.Sothankyoubothsomuch.Andwe'lldothisagain.Veryhelpfulandusefulconversation.Enjoyedit.
Speaker 3
Was It? The Machine?
Words and timings
WasIt?TheMachine?