Satya Nadella calls for an emergency brake on advanced AI systems

Microsoft chief Satya Nadella called on October 10 for advanced AI systems to have independent safeguards and a way for authorised people to halt a model during a task.

In a published essay, he argued that capable models should be treated as potential insider risks because they can make mistakes or be compromised. Access permissions and limits on actions should sit outside the model itself, he said.

His proposed safeguards include human-readable activity records, continuous testing, independent auditing, containment and incident disclosure. Testing should cover failures and attacks across the whole system.

Nadella said an authorised person should always be able to pause or shut down a model mid-task. He also called for industry standards where current rules are insufficient. The essay sets out a safety approach rather than announcing a new product.