At Q2BSTUDIO, a leading technology development and services company, we have explored the use of Large Language Models (LLMs) within Salesforce, resulting in the development of an innovative component. This component integrates a chat interface capable of using Salesforce records as context, running entirely locally on the user's device, ensuring data privacy without relying on third-party services.
The concept of agents in Salesforce, such as Agentforce, inspired us to develop this assistant. While agents can make decisions and execute actions, assistants process information reactively. Although we considered building a local agent with Pico LLM, it involved substantial effort. Therefore, we decided to focus on an efficient and integrated assistant.
Main features:- Multi-model compatibility. Supports any open-source Pico model, such as Gemma, Llama, or Phi, depending on the device's available RAM.
- Access to individual records. When on a record page, the component obtains context based on field values.
- Use of related records. Allows querying additional information from related records to enrich responses.
- Real-time configuration. The component configuration is dynamic and allows adjusting parameters such as token limit, temperature, and top P.
From the user's perspective, operation is simple: upload a model, select a system message, choose Salesforce records, enter the query, and generate responses in real time.
Pico LLM enables efficient execution of models in a browser thanks to its compression technique, optimizing memory and bandwidth usage. In our case, we managed to integrate this technology within Salesforce using an iframe on a Visualforce page. This approach allowed us to overcome the Web Worker restrictions in Lightning Web Components (LWC).
During development, we faced various challenges, especially with optimizing local processing. One of the most interesting aspects was correctly structuring the context within the prompt. Initially, the models did not use Salesforce records effectively, until we discovered that reorganizing the message structure solved the problem.
In terms of performance, tests showed that the initial model loading speed largely depends on RAM speed. Likewise, response generation is fast, almost comparable to cloud solutions. Although we have not fully explored GPU usage for additional optimization, it is clear that this technology has great potential.
At Q2BSTUDIO, we continue exploring new technological solutions and optimizing our tools so that companies can leverage the best of artificial intelligence within the Salesforce ecosystem.
