When an organization starts managing secrets with HashiCorp Vault, the volume of operations is usually modest: a few services retrieving credentials or issuing certificates. However, as adoption expands toward CI/CD pipelines, automated machine provisioning, and dynamic PKI signing, concurrency skyrockets. Understanding how Vault behaves under increasing concurrent workloads becomes a critical factor for infrastructure sizing and capacity planning. At Q2BSTUDIO, a company specialized in custom software development and cloud solutions, we have thoroughly analyzed Vault's performance patterns to help our clients design resilient deployments.
Recently, a set of benchmarks conducted on a Vault Enterprise cluster (v1.17.3+ent) with integrated Raft storage in AWS us-west-2 provided revealing data. The tests followed a gradual ramp-up model, from a single virtual user up to 500 for KV workloads, and between 200 and 300 for SSH and PKI. The results, monitored with k6 and Datadog, show clear inflection points: the knee point (where latency begins to outpace throughput growth), the saturation point (where throughput plateaus), and the failure point (where requests start timing out). Below we break down the key findings and practical implications for platform teams.
1. KV reads consistently outperform writes under concurrency
The first clear finding is that read operations in the KV secrets engine far surpass writes under concurrency. In the benchmarks, reads were approximately 2.3 times faster than writes and remained stable even at 500 virtual users. Writes, on the other hand, reached stress points earlier and were the first to fail. The main reason is that writes must be committed and replicated across Raft nodes, generating disk and network overhead that reads do not incur. For heavy write workloads — for example, in CI systems that constantly rotate credentials — proper storage sizing is essential. For large payloads such as 1 MB secrets, the KV2 engine began failing between 50 and 100 virtual users, while reads continued operating. This reflects a reality: in environments with abundant writes, carefully dimensioning storage is key.
2. Payload size: the most impactful factor
Secret size has a dramatic effect on performance. Small payloads (1 KB) scaled without saturation up to 500 virtual users for reads. However, when increasing to 100 KB or 1 MB, throughput plummeted: from thousands of requests per second to single digits. The cause is the processing and replication cost of large payloads: more disk writes, more Raft traffic, and more memory pressure. At Q2BSTUDIO we recommend clients keep secrets below 100 KB whenever possible. If they need to store certificate bundles, configuration files, or binaries, a good practice is to externalize them to a cloud AWS or Azure bucket and store only the reference or access credentials in Vault. This also reduces the attack surface and improves governance.
3. Memory and storage: the real bottlenecks
System metrics revealed that memory was the most limiting resource during KV workloads. With 500 concurrent users, memory usage reached 100% on all nodes, while CPU remained between 20% and 40%. This indicates that compute capacity is not the main limiting factor; memory and disk performance are. Because Vault with Raft requires write confirmations to disk before considering an operation successful, large write bursts cause disk activity spikes that correlate with tail latency increases. For teams managing critical infrastructure, scaling node memory and using high-performance SSD/NVMe disks is a necessary investment. At Q2BSTUDIO we help companies design these configurations, also integrating AI solutions and automation to optimize secret management.
4. ED25519 doubles scalability for SSH workloads
In the SSH engine tests, the choice of cryptographic algorithm made a significant difference. RSA-2048 reached its knee point around 100 virtual users, while ED25519 extended it to approximately 200. Saturation with RSA occurred around 250 users, whereas ED25519 continued scaling beyond 300 without reaching full saturation. CPU utilization with RSA peaked at 92-96%, compared to 40-42% with ED25519. This has direct implications for environments that issue SSH certificates en masse, such as server provisioning or CI/CD pipelines. Choosing ED25519 not only improves scalability but also reduces energy consumption and infrastructure costs. At Q2BSTUDIO, when designing cybersecurity systems for our clients, we prioritize efficient algorithms without sacrificing security.
5. PKI: early latency limits, but stable up to a point
PKI operations (certificate issuance and revocation) are inherently more expensive than KV reads. The knee point appeared around 25 concurrent users, with a baseline latency of about 560 ms per certificate. Below 100 users, all operations succeeded, but beyond that revocations began to experience timeouts. There were no significant differences among RSA, ED25519, and ECDSA in base latency, but behavior under load was similar across algorithms. For organizations relying on PKI for machine or workload identities, it is crucial to plan for issuance peaks — for example, during mass rotations — and consider horizontal scaling of the cluster or using a separate PKI engine. At Q2BSTUDIO we offer consulting to integrate Vault with BI/Power BI dashboards that monitor cluster health in real time and anticipate bottlenecks.
Operational recommendations for platform teams
Based on these benchmarks, we can extract several practical guidelines. First, keep payloads small: below 100 KB scales well, and if large objects are needed, externalize them to a cloud storage service. Second, optimize storage: use disks with high write throughput and fast fsync (NVMe) to reduce Raft write latency. Third, scale node memory: at 500 concurrent users, memory was exhausted; each node should have enough RAM to handle connection peaks and read cache. Fourth, choose efficient cryptographic algorithms: ED25519 for SSH offers double the concurrency capacity of RSA. Fifth, plan capacity for PKI considering that latency rises early and revocations are the first to fail; consider separating PKI engines by load type or implementing rate limiting.
Also, remember that these benchmarks were performed with auditing disabled for KV (though SSH and PKI had it enabled). In production, with audit logging enabled, the extra overhead can magnify issues, especially with large payloads. That's why at Q2BSTUDIO we always recommend running representative load tests with the exact configuration that will be used in production, including logging, monitoring integration, and authentication plugins.
The role of artificial intelligence and AI agents in secret management
An emerging trend is the use of AI agents to automate secret rotation, detect anomalies in access patterns, and predict bottlenecks before they occur. At Q2BSTUDIO we are developing solutions that combine Vault with machine learning models to optimize resource allocation and recommend configuration adjustments in real time. For example, an AI agent could analyze latency and throughput metrics and suggest scaling the number of nodes or changing the cryptographic algorithm before saturation affects critical services. This aligns perfectly with our philosophy of providing custom applications that integrate the best of cloud, cybersecurity, and artificial intelligence.
Conclusion
Vault is a robust tool, but its performance is not uniform across all load types. The benchmarks confirm that writes and large payloads are the main scalability limiters, while memory and storage weigh more than CPU. For platform teams, capacity planning should consider expected concurrency, secret size, algorithm selection, and storage subsystem performance. At Q2BSTUDIO, we help companies of all sizes design scalable and secure architectures on AWS, Azure, and hybrid environments, integrating Vault with BI, automation, and AI agents. If you are planning to deploy or scale Vault in your organization, contact us for personalized consulting.
