I ran an LLM locally on my ASUS ROG Ally: real lessons

Running a large language model (LLM) locally on a device like the ASUS ROG Ally reveals not only hardware limitations, but also deep lessons about memory architecture, model selection, and the true value of artificial intelligence in resource-constrained environments. The experience shows that optimizing the UMA frame buffer in the BIOS is the first critical step to unlocking performance on systems with shared memory, an adjustment many tutorials omit. From there, concepts like zRAM or disk swap turn out to be partial solutions: the former barely compresses already quantized weights, and the latter only prevents the system from killing the process, without speeding up generation. Adjusting vm.swappiness eliminates stuttering generation, and choosing the right model depends on the use case: for asynchronous agents, speed takes a backseat to accuracy, and larger models like Phi-4 perform better than fast but shallow options. This technical reflection directly connects to the challenges companies face when implementing AI for businesses on limited infrastructure. At Q2BSTUDIO, we tackle these challenges with custom software and custom applications that integrate AI agents efficiently, whether on-premises or in the cloud. For environments requiring scalability, we offer cloud services aws and azure that allow deploying models with the right resources. Cybersecurity also plays a fundamental role when handling sensitive data in local assistants, and business intelligence with power bi benefits from lightweight models that process queries privately. Ultimately, setting up an LLM on a ROG Ally teaches that modest hardware can be a valuable ally for routine tasks, while heavy lifting still requires professional infrastructure. At Q2BSTUDIO, we help organizations find that balance with custom artificial intelligence and custom applications that maximize every resource.