The arrival of artificial intelligence in companies is no longer a promise of the future: it is a reality that requires immediate decisions. However, when an infrastructure team receives the first request to support AI loads, the temptation is to start with hardware: ask for quotes for servers with GPUs, choose between Kubernetes or virtual machines, or decide whether the cluster will be Dell or VxRail. That approach, while understandable, is often a strategic mistake. The fundamental decision is not technological, but one of operating model. With the arrival of VMware Cloud Foundation 9.1, organizations have more options than ever to run AI in private environments, but leveraging them correctly requires a shift in mindset: first the service, then the cluster.
In this article, we discuss how enterprises can design an AI architecture on top of VCF 9.1 that is scalable, governable, and future-proof. And, along the way, we'll look at how a software development company like Q2BSTUDIO can accompany this process with enterprise AI solutions, cloud integration, and custom applications.
The first common misconception is that the GPU cluster is the service. In reality, the cluster is just an underlying resource. What really matters is defining a service contract: what kind of workload is going to run? What level of insulation do you need? Who is responsible for the life cycle? How are costs measured and allocated? If the platform team doesn't answer these questions before the first server with GPU arrives, every decision not made will become an incident, an exception, or an after-hours discussion.
VCF 9.1, announced by Broadcom, introduces significant improvements in AI observability, support for multi-tenant Kubernetes, high-speed networking, and new storage options such as global vSAN deduplication. These capabilities are platform ingredients, but they don't define the operating model. For example, the platform allows you to allocate entire GPUs using DirectPath, partition with vGPU or MIG, or even run CPU-only inference. The right choice depends on the use case, not the technology available.
A practical framework is to define AI service classes before deploying any infrastructure. For example: a sandbox for experimentation, a shared inference service, a dedicated layer for production, and a regulated environment for sensitive data. Each class of service must specify the GPU access model (shared, dedicated, batch), lifecycle ring (lab, controlled production, regulated), storage type (for models, datasets, checkpoints, logs), and cost model. This prevents each request from becoming a custom design and the cluster from becoming a dumping ground for experiments.
The placement decision is the first major crossroads. Are GPUs added to an existing workload domain? It is fast and cheap, but risky if production loads arrive later. Is a dedicated work domain created for AI? It offers better lifecycle isolation and performance, but increases the initial cost. VCF 9.1, especially in VxRail environments, allows for a middle way: using Dell vSAN Ready Nodes in a separate domain, maintaining consistency within the domain and compatibility with the Broadcom ecosystem. The key is to validate the support matrices before buying.
Another critical aspect is the tenure model. In AI, tenure isn't just authentication. It involves defining which tenants can share GPUs, what data they can mount, what networks they can reach, and what metrics are visible. A good practice is to combine several layers: project + namespace for experimentation environments, NSX segmentation for shared inference, and full working domain for regulated loads. The platform team should lay out these options in a service catalog, so that business teams don't have to understand the full VCF topology.
Storage is another point where many underestimate complexity. Models may be small, but training datasets, embeddings, checkpoints, and inference logs can grow rapidly. VCF 9.1 offers differentiated vSAN policies: a high-performance policy for vectors, a capacity policy for historical datasets, and a retention policy for logs. Not using a generic policy for everything is critical to avoid bottlenecks and hidden costs. In addition, data lifecycle management (retention, deletion, governance) must be integrated into the class of service.
The network also deserves separate attention. Management traffic is not the same as GPU-to-GPU traffic for distributed training, or storage for checkpointing. VCF 9.1 supports GPUDirect RDMA and Enhanced DirectPath I/O for NVIDIA ConnectX and BlueField devices, enabling near-metal performance. But if the first cluster is designed for basic inference only, it may not support the traffic patterns of future distributed training. The recommendation is to design with evolution in mind, but without oversizing for the first use case.
Cost is the great forgotten in many on-premise deployments. While in the public cloud GPU spend is visible on the monthly bill, in a private data center it can be hidden in the capital budget. VCF 9.1 includes enhancements to VCF Operations with cost data normalized to the FOCUS standard, allowing you to compare the cost of private and public cloud and perform showback or chargeback per GPU hour, per endpoint, or per storage. Implementing a cost model from the start prevents the platform from becoming a hidden subsidy for AI experiments.
In this context, having a technology partner that understands both infrastructure and application development is a competitive advantage. Q2BSTUDIO offers tailored software services that can integrate AI models with business processes, develop custom interfaces for data science teams, and build the observability and cost dashboards the organization needs. In addition, its expertise in AWS and Azure cloud services allows you to hybridize workloads, moving heavy training to the cloud when needed and maintaining on-premises inference to meet latency or data residency requirements.
Another aspect where Q2BSTUDIO added value is in the creation of AI agents and virtual assistants that integrate with business systems. These agents, based on language models and augmented recovery (RAG), require an infrastructure that guarantees low latency and security. The company also helps implement cybersecurity in model access, protecting sensitive data and complying with regulations. And, of course, its Business Intelligence with Power BI services allow you to visualize AI performance and cost metrics, connecting infrastructure with decision-making.
In closing, the core message is clear: don't start with the GPU cluster. Start with the operating model. Define classes of service, choose the tenancy model, design the lifecycle, storage, and network, and establish cost visibility. Then, let VCF 9.1 and its partner ecosystem like Q2BSTUDIO handle the implementation. Artificial intelligence isn't just a hardware problem – it's an architecture, governance, and collaboration challenge across teams. Approaching it with a solid strategy will prevent the first successful project from becoming a legacy pattern that no one can manage.
