Amazon memo shows AI coding guidelines, how it tackles tokenmaxxing

Amazon laid out its AI usage guidelines for software engineers in a memo to staffers, and discouraged employees from turning to “tokenmaxxing” in their push to embrace artificial intelligence tools.

The memo, which was circulated internally in September and posted to the company’s website on Tuesday, revises how the company defines the role of software development engineers and managers in the age of AI.

“We’re just seeing the beginning of AI’s effect on software engineering,” the company wrote. “While we have a vision for what the future looks like, there’s a lot that’s still emerging. We are certain that whatever future best practices are, AI will be at the center of software engineering practice.”

Under CEO Andy Jassy, Amazon has rapidly integrated AI across the company’s sprawling business operations. Jassy has previously predicted that AI will “change the scope and speed” of day-to-day work, and potentially shrink its corporate workforce in the next few years through automation, though he’s noted it will also create “other jobs.”

As employees rush to adopt AI tools, companies have grappled with the technology’s surging costs and how to reliably measure whether the technology leads to a productivity boost or other valuable returns on investment.

Companies have recently cracked down on tokenmaxxing, in which developers use AI as much as possible with little regard for output. Some companies have set limits on employees’ usage of tokens, or the units of data that are processed and generated by AI models, in a bid to slow rising AI bills. Others have switched to cheaper, open-source models for certain tasks.

An Amazon spokesperson said the company hasn’t instituted caps on employees’ token usage. Amazon in May shut down an internal leaderboard after discovering employees were tokenmaxxing to climb the ranks.

In the memo posted Tuesday, Amazon said metrics like token usage, the rate of adoption of AI tools, and code changes or reviews remain “deeply flawed” ways of measuring teams or organizations’ AI usage, adding that they can lead to higher costs and too much complexity.

It stopped short of telling staffers to ignore those metrics altogether, however. Token usage and code generation should be considered “with care and judgment” alongside other data, Amazon said.

“We believe that it is best to measure, and be measured by, long-term outcomes,” the company wrote.

Marc Brooker, a vice president and distinguished engineer at Amazon Web Services, told CNBC in an interview that the company has looked to metrics such as “feature velocity” to measure how AI is improving engineer productivity.

The metric looks at things like, “how many launches are we doing, how many successful changes are we making in production,” he said.

“Broadly, we think about this as a return on investment,” Brooker said. “If there is an important problem to…