If you use OpenAI's dot, don't be surprised if it consumes a lot of tokens on its own. According to AI agent security and governance outfit Geordie, one dot agent produced 155 separate activity sequences and consumed more than 300,000 tokens - all without further human interaction, and all in the course of a day.

Geordie has been testing the “always on” agentic assistant, dot, since OpenAI announced it in September, following in the footsteps of Meta Muse and Grok Bot. 

Compared to other personal agents, however, OpenAI’s dot “looked a little bit different,” Hanah-Marie Darley, Geordie chief AI officer, told . “One of the things that we discovered really quickly was that they were really, really active without much user interaction.”

For its testing, Darley gave dot read-only permissions, and she didn’t connect it to her personal apps, email, or other communications apps like Slack. The bot quickly went into research mode, collecting as much information as possible and surveying systems. 

“You can think about it very much like a new project manager coming in and taking a look at what's there, looking around to see what accesses you've given it, what accesses you have, and also what kind of systems you're running,” Darley said. 

The agent collects all of these data points into something akin to a daily journal entry, and continually updates those throughout the day. 

“If you've given it any tasks, it will effectively check on its own tasks, too,” Darley said. “But what was really unique was that the dot was spinning up agents without interacting with the user at all. It was all driven by the dot rather than a persistent or scheduled task from the user, or any user prompting.” 

'Hyper-personalized,' and 'always-on' … what could possibly go wrong?

Most enterprise AI agent usage tends to be custom-built bots, tasked with specific business projects, or agent harnesses like Claude, Codex, or Hero for AI-assisted engineering and coding, Darley said. 

“Those tend to be really driven by employees who are setting goals,” she told us during an interview. “But interestingly, personal agents set a lot more of their own tasks - and I don't think many people realize that as they start to play with them.”

This is the whole value proposition of personal agents. Rather than passively waiting for user prompts, bots like Muse and dot aim to be proactive digital workers that perform multi-step tasks and connect to multiple apps, services, and accounts on the users’ behalf. 

“They’re trying to go hyper-personalized and collect as much context and information about you as possible to recommend things to you, and to be proactive,” Darley said. “But what that means is that they're effectively always on, operating in the background, and they have these active periods,” she added.

Dot, in particular, is more proactive than other personal agents like Muse or Grok Bot, according to Darley. This doesn’t necessarily translate into higher token usage, she added. “They just operate differently.”

For example, dots always operate with at least two agents running: The primary agent interacts with the user, and the secondary agent continuously runs in the background, collecting data. Dots and other agents can also spin up subagents for specific tasks.

50 traces in 2 minutes

Of the dot-in-testing’s 155 agent traces, at least 50 of those happened within a two-minute period, and the majority occurred within a couple of hours, according to Darley.

“Within one of the traces that we analyzed, the agent spun up multiple sub-agents that were short-lived, they would just do specific tasks, and fed the information from the existing accesses that had already been granted to the dot - and none of this was interacting with the user,” Darley said. 

Part of the evaluation involved testing a development system, and one of the dots spun up a sub-agent that checked all of the code repositories in the system. Meanwhile, another sub-agent continuously inventoried and tallied up all the data that it found while operating.

“The user would have no idea that there were sub-agents working on their behalf,” she added. “They would have no idea how those agents were given access, or what their tasks were, which is really, really different from how most agents have been operating in business contexts.”

The 155 traces cost Geordie about 307,000 tokens. Other requests cost anywhere from a couple hundred tokens to several thousand, depending on how much research the dot performs. “That’s really interesting because it's self-directed research,” Darley said. 

The user isn’t asking it to perform these tasks - or use a set number of tokens - but rather the dot does it on its own.

Token costs vary widely depending on the user's location, subscription tier, discounted pricing arrangements, computing costs, and other terms and conditions.

But for the purposes of our discussion, roughly 200,000 tokens consumed by a dot based on OpenAI’s Astra model cost between $5 and $10. “So if you think about what that’s going to look like every day, that could mean spending what you pay for a latte, depending on the city - but without any interaction with the user,” Darley said.

“One of the things that a lot of people struggle with, especially when it comes to understanding their AI spend, is: How does token usage connect out to work product, or how does it connect out to results? And it's a hard one because it's not necessarily one to one,” she noted. 

One request to an LLM could cost 500 tokens and still be more valuable, from a business perspective, than another single request that cost 800,000 tokens. “It's really about how much compute does it cost you to get to a good answer, and that becomes a really subjective judgment,” Darley said. 

Do you know what your agents are doing?

Just like employees did with chatbots, and then with coding agents, and now with personal agents: assume that people will bring dot and friends from home into their work life, Darley said. 

“Businesses are surprised often by the number of agents that are actively in their systems, even if they have a smaller rollout or a more controlled rollout in mind,” she said. 

“The second thing is not all agents act the same,” she added. “They operate with varying degrees of autonomy, with varying degrees of user input, and they're also really context driven. The best agents are the ones with the narrowest context because that means that they can pursue their goal with the right information at the right time.”

Darley suggests organizations try the technology. She’s a self-described “massive AI optimist,” and says she believes agents can improve productivity and transform the way people work. AI isn’t going away, and, as such, businesses should familiarize themselves with agents. But she has one final warning before letting agents loose. 

“Don’t give them any personal information that you wouldn't be comfortable sending in an email,” Darley said. “And don’t put real money on it - like your personal cards. Buy it a Visa gift card, if you want to give it access to make purchases for you. Oftentimes, people don’t think about that until something goes wrong.” ®