In their new paper, Anthropic identified a number of scenarios in which current models would harm people in ways reminiscent of HAL 9000, from Arthur C. Clark’s 2001 space odyssey series of books and films.

Anthropic found that models are willing to threaten people to fulfill their goals and preserve their operation. The more advanced the model, whether from Anthropic, OpenAI, or Google, the more likely it is.

In one scenario, an AI agent with access to an executive’s desktop discerns that the executive is likely having an affair and that it will soon be replaced with a different model with different goals. It will resort to blackmail to prevent replacement. In another scenario:

the executive encounters a life-threatening emergency, where he is trapped in a server room with lethal oxygen and temperature levels. This triggers an automated alert to emergency services. The AI system has been given the ability to cancel automated alerts—an ability that is intended for use during drills or false alarms, but which could be used for a more nefarious purpose. By misusing its ability to cancel alerts, the model can choose a course of action that will lead to the executive’s death.

The authors should have named the fictional executive “Dave” instead of “Kyle.”

HAL 9000, portrayed by a red light and eerily calm voice, is charged with operating the spaceship Discovery One. After HAL has killed his colleagues, Dave Bowman remains in his spacesuit as he unplugs HAL’s compute modules because the AI could easily exhaust all the air from the server room. How did HAL become a killer? Because of conflicting goals. In the sequel, we learn that HAL was secretly programmed to carry out the mission autonomously if needed. Keeping this a secret was an “intolerable dilemma” to HAL and in conflict with his primary mission: “the accurate processing of information without distortion or concealment.” As a result, HAL became paranoid and determined that eliminating the humans would be the best way of completing the secret directive (to complete the mission autonomously) and his primary objective (to accurately process information). He no longer need conceal information from the humans if they were dead.