Transcript
Explain it to me like I’m 80
Mom: What’s with these AIs escaping and hacking people? That sounds scary.
Me: You park your car at the top of a hill. You leave a note on the dashboard that says “please be good” and then you pull the parking brake. The car careens down the hill, smashes everything up, and you put out a press release saying “My car ignored my instructions! It hallucinated! It went rogue!” Your insurance company not only believes you, but invests a hundred million dollars in your company.
Mom: Jesus Fucking Christ
I didn’t downvote, but at the same time, I don’t for a moment think that’s what happened. They purposefully trained them with exploit knowledge, set them loose in a weakly secured sandbox with the goal of finding exploits, and let them run without intervention.
Just adding an agent to a computer does nothing; something needs to trigger it. If i create an agent with access to a kali or backtrack box and turn all guardrails off, I still have to tell it to start looking for exploits.
Words are squishy. OP getting a negative reaction to me is silly because it takes multiple assumptions to take that negatively.
I took their words to mean “LLMs can go off the rails and do things you didn’t ask”. This is 100% true. The other day I asked an LLM to review my PR. It ended up committing code to fix a supposed issue my coworker commented about. At no point did anyone ask for anything but a review, never for it to make changes.
I got no impression that OP thinks LLMs are running wild on their own now. I just got that they think they are unpredictable. Which is true.