finops.work

Free AI for a nation · Part 3 of 7

Agents are a different workload, and waiting is the only price

One measured coding-agent session wrote about 95 times the output of an assumed chat user's day, which is why agents need their own forecast. The rules forbid charging and usage caps, so if peak demand exceeds usable capacity, slower answers and queues do the rationing.

A chat user’s day, estimated

No one publishes how many tokens a person’s chat uses, so I build the estimate from two measured numbers and one assumption. Koreans who use the ten most popular AI apps spent 149.59 minutes in them in September 2026, by WiseApp’s panel, about 5 minutes a day.[69] On OpenRouter, a marketplace for model APIs, the average answer in late 2025 was 400 output tokens.[79] I assume one prompt a minute while the app is open. That gives about 2,000 output tokens per monthly user a day; a slow reader halves it, a heavy user doubles it.

One agent task, measured

The consortia are promising agents: the ministry’s example is a chatbot that plans a trip and an agent that books and pays for it; SK Telecom’s will phone a shop for you; Kakao wants reservations, applications and payments inside KakaoTalk.[18, 27] None has been measured, by the operators in public or by me.

The agent task I can measure is my own, a heavy coding-agent stress case: the session that built this site’s skeleton, from the first deep dive. It made 92 model calls and wrote 189,449 output tokens, thinking included, about 95 times the assumed chat user’s day.[70] That compares output volume, not cost per equivalent task, and four units are easy to confuse: one chat answer, one chat user’s day, one agent task, and one such task every day. The session is one task, once.

Output is the small part. An agent sends its whole history with every call: this session’s calls carried 222,741 input tokens each on average, 98.6% of them read from a cache, which spares the GPU most of the reading. A long history still slows the writing: in NVIDIA’s tables for Llama 3.3 70B on one B200, output per second falls 5 times with prompts of about 8,000 tokens and 25 times at about 33,000, in runs that reuse no cache.[36] What a day of such tasks would mean for the fleet is in the appendix; it counts output, not completed work.

Nobody has measured a public-service agent, so the chart puts the measured session next to two hypothetical tasks, to show the range, not estimate it: a simple application at five model calls and a multi-step booking at 20, each call writing one chat answer. The calls are my assumptions. An honest forecast needs traces from the operators’ own agents: calls per task, tokens in and out, cache hits and completion rate. Until then the agent line in any capacity plan is a guess with wide error bars, which is the reason to keep it separate from chat.

Output per task, in chat answers: two hypothetical, one measuredOutput tokens per task, in chat answers of 400 tokens (OpenRouter’s average completion, December 2025). Measured: this site’s build session of 6 October 2026. Estimated: a chat user’s day, from WiseApp’s 149.59 minutes per user in September 2026 at one prompt a minute. Hypothetical: the two public-service tasks, at the assumed number of calls.
Output per task, in chat answers: two hypothetical, one measured One chat answer: 1; A chat user’s day: 5; Simple application: 5; Multi-step booking: 20; Heavy coding session: 474. One chat answer: 1 One chat answer 1 A chat user’s day: 5 A chat user’s day 5 Simple application: 5 Simple application 5 Multi-step booking: 20 Multi-step booking 20 Heavy coding session: 474 Heavy coding session 474

Waiting is the only lever the rules allow

The tender’s FAQ says operators may not charge ordinary users, limit their use or sort them by purpose. They may slow responses, cap simultaneous connections and block abnormal automated mass requests, as long as people using the service normally don’t feel it.[22] Rationing by waiting is the only kind the rules allow, and the rules ask that no one notice it.

Speed is a strong lever because a GPU writes for many conversations at once; serving more of them per copy of the model slows each one and raises the total. InferenceX measured that trade-off for Llama 3.3 70B on one B200; the chart shows its nine points, each the best configuration it found at that speed.[56] At 102 tokens a second per user, a 400-token answer takes about 4 seconds; at 33 it takes about 12 seconds, and the GPU writes 2.7 times as much in all. The gain comes from batching more users onto each copy, not from the delay; slowing answers without adding users wastes capacity. And the seconds are writing time only: reading the prompt, the time to the first token and any queue come on top, and the rules say nothing about those.

Whether the lever will be needed is open. A zero price removes one constraint on demand; attention, usefulness, trust, skill and the free rivals remain, and a free service can stay underused. If peak demand does exceed usable capacity, congestion becomes a non-monetary price, and it falls unevenly: a chat user barely notices a few seconds, while an agent that makes 92 calls in a row waits for each. The ministry can add GPUs and, from March 2027, move them between consortia by usage;[11] neither replaces a published service level at the busiest hour, which is what would tell a citizen whether “no usage limit” holds when everyone is awake.

A GPU’s output at each speed per userLlama 3.3 70B in FP8 on one B200 with TensorRT-LLM, prompts and answers of about 1,000 tokens, output tokens only. Each point is the best configuration InferenceX measured at that speed on its run of 29 October 2025, not every configuration it ran.
A GPU’s output at each speed per user 33 tokens a second: 4,073; 53 tokens a second: 3,238; 75 tokens a second: 2,269; 102 tokens a second: 1,528; 125 tokens a second: 944; 136 tokens a second: 519; 153 tokens a second: 296; 176 tokens a second: 170; 184 tokens a second: 88. 33 tokens a second: 4,073 33 tokens a second 4,073 53 tokens a second: 3,238 53 tokens a second 3,238 75 tokens a second: 2,269 75 tokens a second 2,269 102 tokens a second: 1,528 102 tokens a second 1,528 125 tokens a second: 944 125 tokens a second 944 136 tokens a second: 519 136 tokens a second 519 153 tokens a second: 296 153 tokens a second 296 176 tokens a second: 170 176 tokens a second 170 184 tokens a second: 88 184 tokens a second 88