In the current AI-driven development ecosystem, token optimization has become an obsession for companies and developers. Every token consumed by models like Claude or GPT incurs a direct cost, and any tool promising drastic reductions generates excitement. Recently, rtk (Rust Token Killer) has attracted attention—a command-line proxy that claims to compress typical Bash command outputs for AI agents by up to 60–90%. But does it actually work in real workflows? Our team conducted an independent battery of tests to verify, and the results are surprising.
rtk presents itself as an elegant solution: it intercepts communication between the AI agent and the terminal, replacing verbose outputs from commands like 'git status' or 'pytest' with summarized, cleaner versions. The idea is that the model consumes fewer tokens by receiving less text. There is even a PreToolUse hook for Claude Code that allows the agent to use rtk without knowing it. In theory, it sounds perfect for projects that heavily use the terminal. However, controlled tests revealed a very different reality.
To evaluate rtk, we used a benchmark of 86 tasks based on SkillsBench, executed with Claude Code 2.1.201 and the claude-sonnet-5 model at two reasoning effort levels (low and high). Each task was run once with the standard agent and once with rtk enabled, measuring cost, tokens, number of turns, and output quality. In total, 425 billed trials were completed at an approximate cost of $320. Data were processed using paired analysis to eliminate noise from task variability.
The first crucial finding is that rtk can only touch a minimal fraction of the tokens actually consumed in a session. Claude Code itself uses internal tools like Read and Grep to read files and search text, which bypass the Bash hook entirely. Moreover, many commands such as 'python3', pipes, and here-docs fall outside rtk's scope. After analyzing transcripts of 83 baseline sessions, we discovered that barely 20% of tool output characters are compressible by rtk. And even then, the model already truncates very long outputs before billing. Therefore, the theoretical maximum savings on total input tokens is about 3%.
The second result is even more revealing: far from saving costs, rtk caused a median increase of +7.6% in cost per task at low effort (p=0.004), accompanied by +13.8% more turns and +14.3% more cache reads. At high effort, the penalty nearly vanished (+0.1% difference, p=0.99), but no savings were observed in either case. Why does this happen? rtk's compression, though real, sometimes removes crucial information that forces the model to take extra steps: re-reading files, re-running commands, or choosing alternative paths. These extra moves consume more tokens than are saved by summarizing the output.
A noteworthy aspect is that task quality was not significantly affected: scores were statistically identical in both arms, with 71 ties at low effort and 62 at high. However, a serious compatibility issue emerged: in one specific container rtk could not run because it required a newer glibc version, preventing a task from completing. Such problems can be critical in heterogeneous development or deployment environments.
The tool itself reports internally having saved 96.2 million tokens, but our invoice showed the opposite. The discrepancy is explained because rtk compares against an output that would never exist: it counts uncompressed total characters as if the model had received them all, when in reality most would have been truncated or cached. It is a clear example of measuring the wrong scenario.
What lessons can companies using AI agents in their software development draw? First, not all token-saving promises withstand empirical verification. Second, real optimization requires understanding the full consumption flow: internal read tools, caching mechanisms, and above all the interaction between the agent and the system. At Q2BSTUDIO, as a software development and technology company, we know that implementing AI efficiently requires a holistic approach. Simply integrating a proxy that compresses outputs is not enough; the architecture must be designed considering the model, the cost of each call, caching, and output quality.
For example, in cloud AWS/Azure projects where AI agents may run in CI/CD pipelines, indiscriminate use of tools like rtk can introduce hidden costs greater than the imagined savings. Our experience has taught us that proper measurement of agent performance must be based on actual billed cost, not on theoretical savings declared by the tool. Q2BSTUDIO offers process automation, cybersecurity, and BI with Power BI solutions that integrate AI in a controlled and auditable manner, avoiding billing surprises.
In conclusion, rtk is an honest engineering effort that solves a real problem (terminal verbosity) but, when applied to modern AI agents, not only fails to save tokens but can increase costs. Its dashboard tells a flattering story, but the actual invoice tells a very different one. For companies seeking to optimize AI agent usage in custom software development, the recommendation is clear: always test with realistic benchmarks, measure total cost, and do not trust self-reported savings. At Q2BSTUDIO we apply this philosophy in every project, ensuring technology delivers tangible value without inflating expectations.
This analysis is part of a broader series where we independently evaluate tools that promise to reduce token consumption in AI agents. If you are considering implementing agents in your organization or need advice on efficient AI architectures, feel free to contact our team. Transparency and rigorous measurement are the only shortcuts that truly work.
