AI experts warn humans may lose control as systems self-improve

Speaking to the New York City Council, former Anthropic researcher Jacob Coxon warned that artificial intelligence companies ‘don’t fully control’ their models and are advancing technology too quickly.

Coxon told lawmakers that the industry is pushing ahead without adequate safeguards. He argued that reliance on human oversight will soon disappear as models become more autonomous.

According to his testimony, AI systems will soon gain the ability to improve themselves. This shift means human researchers could eventually become redundant as machines learn to refine their own architecture.

The concerns go beyond immediate technical hurdles. Coxon has previously suggested treating artificial intelligence as a new species with values that may not align with human interests. He noted that leading firms currently cannot stop these systems from pursuing objectives their creators never assigned.

These warnings come as regulators and tech executives debate the next steps for AI development. The council hearing highlights growing calls for stricter oversight as the technology advances.