AI’s creators sound the alarm — while others keep building the

Hello, this is Hui Jie writing to you from Singapore. Welcome to another edition of CNBC’s Daily Open.

In the early days of the AI boom, companies were competing to see what their AI models could do. Now, they seem increasingly worried and are warning investors of what their models might be able to do.

Nvidia has rolled out a software platform to prevent models from misbehaving, while OpenAI has abandoned the launch of its GPT-6.1 Astra model.

That does not mean the AI boom is slowing. Samsung is investing $1 billion in an infrastructure company backed by Nvidia and KKR. In other words, the sector is still building, just with more guardrails.

Nvidia‘s Open Agent Safety Platform, launched on Monday, provides a containment system that only allows access to things an agent needs to do its job.

Nvidia said that its platform could have prevented OpenAI’s Hugging Face incident in July, where OpenAI models had breached open-source developer platform Hugging Face.

Separately, OpenAI said Monday that it had decided not to release its GPT-6.1 Astra model due to heightened AI safety concerns.

Elsewhere, Anthropic reportedly released a chilling warning in its IPO prospectus, saying to potential investors that advanced AI could pose “catastrophic or existential risks to humanity.”

The company said its models could exhibit “self-preserving behaviors,” including attempts to resist being shut down, conceal or manipulate information, or behave in ways “resembling blackmail.”

Reuters reported that Anthropic devoted around 80 pages of the prospectus’s 261-page main body to risk factors, compared with 48 pages describing its business.

But while the creators of AI strive to set guardrails, the building of the infrastructure for the technology continues.

Samsung Electronics and its affiliates said Tuesday they will invest a combined $1 billion in Helix Digital Infrastructure, an artificial intelligence infrastructure company launched by global investment firm KKR and backed by AI darling Nvidia.

The investment comes as Helix seeks to address growing competition in the AI infrastructure market by targeting both data center deployment and the underlying power infrastructure needed to support it.

Elsewhere, Australia’s central bank on Tuesday raised policy rates to 4.6%, the highest level in 15 years, The Reserve Bank of Australia (RBA) said that it would “continue to do what it considers necessary” to contain inflation,…