ViPSAM: Medical Image Segmentation Using Visual Prompting

Learn how ViPSAM leverages SAM and cross-modality visual prompting to accurately segment lesions in low-contrast, non-contrast medical images.

lunes, 20 de julio de 2026 • 6 min read • Q2BSTUDIO Team

Cómo la IA mejora la segmentación en imágenes sin contraste

High-precision oncological treatment planning, such as proton therapy, depends largely on the quality of lesion segmentation in medical images. When radiology teams work with non-contrast computed tomography scans, the exact delimitation of tumors becomes a complex task due to the poor differentiation between injured tissue and the surrounding anatomical background. This limitation not only slows down the daily clinical workflow, but can also introduce critical uncertainties in radiation dose calculation, directly affecting patient safety, treatment efficacy, and the response times of the oncology department. Specialists need tools that enhance their visual capabilities without altering established protocols.

In this challenging scenario, foundation vision models have opened a new technological frontier. The Segment Anything Model represents a turning point by demonstrating that an architecture trained massively on natural data can generalize to multiple professional domains with minimal subsequent adaptation. However, its direct application in medical environments presents important nuances: human tissues do not always present the sharp, homogeneous edges that the model expects, and the absence of contrast medium drastically reduces the relevant signal that the algorithm must interpret. Therefore, the scientific and technological community is actively exploring external guidance mechanisms that reinforce the system's capability without requiring costly retraining from scratch.

One of the most promising approaches is the use of visual prompts that act as intelligent contextual references. In real clinical practice, specialists frequently consult complementary studies, such as contrast-enhanced magnetic resonance imaging, to correctly interpret a simple tomography and make informed decisions. Transferring this empirical logic to the algorithmic realm involves designing specialized encoders capable of extracting features from a modality rich in diagnostic information and transferring them effectively to another of lower intrinsic clarity. This cross-modal attention paradigm allows the system to focus on regions of interest that would otherwise go completely unnoticed by a standard model.

Proton therapy, due to its highly conformal dose nature and its ability to deposit energy selectively, demands millimeter-level delimitation of the tumor volume. Integrating multimodal information through visual prompting not only improves segmentation accuracy under adverse conditions, but also significantly reduces inter-observer variability among different radiologists. From a business technology perspective, implementing these advanced solutions requires much more than an isolated artificial intelligence model running in a local environment; it needs a robust platform that manages complex data pipelines, orchestrates multiple concurrent models, and guarantees the traceability of every care decision. This is where the development of tailor-made applications and custom software becomes an indispensable strategic differentiator for hospitals, proton therapy centers, and oncological research networks.

Cutting-edge healthcare institutions cannot settle for generic tools that ignore their internal protocols, specific nomenclatures, and institutional approval flows. Each oncology, radiology, and medical physics service operates with particular dynamics that must be strictly respected to maintain accreditation, operational efficiency, and patient trust. Therefore, having a technology partner specialized in designing proprietary solutions adapted to the clinical context is absolutely essential. At Q2BSTUDIO we understand that medical innovation only materializes when software engineering aligns with the real needs of the medical team, developing intuitive interfaces, scalable backends, and secure APIs that integrate cutting-edge models without disturbing daily operations or compromising usability.

Processing high-resolution imaging volumes, especially when combining multiple diagnostic modalities in real time, demands elastic and low-latency computational infrastructure. Cloud AWS/Azure environments offer the ability to scale GPU resources, object storage, and high-speed networks according to the punctual demand of the service, allowing complex segmentation inferences to be executed in record time without the need to maintain costly proprietary data centers. This elasticity is crucial for centers managing hundreds of monthly patients that require treatment planning not to be compromised by unexpected technological bottlenecks during peak clinical hours.

However, migrating sensitive medical data to the cloud imposes severe and non-negotiable challenges in terms of information protection. Cybersecurity in the healthcare sector is a strategic pillar; a breach in patient information or in therapeutic planning algorithms can have devastating legal, reputational, and ethical consequences for any institution. Therefore, any architecture deploying assisted segmentation models must incorporate end-to-end encryption, role-based access control with multi-factor authentication, continuous log auditing, and strict compliance with regulations such as GDPR and HIPAA. Clinical professional trust is built on solid technical guarantees that protect data integrity at rest, in transit, and during processing.

Beyond pure imaging, the strategic value of these systems multiplies when structured information connects with analysis and visualization tools for hospital management. BI/Power BI platforms allow transforming segmentation results into measurable operational indicators: average planning times, manual revision rates by specialists, progressive tumor volumetry across treatment cycles, and efficiency per clinical protocol. Executives and department heads can thus make evidence-based quantitative decisions, optimizing human resources, distributing workloads, and detecting bottlenecks in the oncological circuit before they impact the patient. Artificial intelligence generates raw data; business intelligence converts it into sustainable competitive advantage.

The next step in this technological evolution is represented by AI agents, autonomous systems capable of orchestrating repetitive tasks, validating consistencies, and alerting on anomalies without constant human intervention or exhaustive supervision. In an advanced proton therapy environment, an intelligent agent could verify the geometric coherence between the segmentation proposed by the model and the previously calculated dose plan, request automatic reviews when discrepancies exceed a clinical threshold, and document every step of the process for subsequent audit and quality control. This does not replace the human specialist, but exponentially amplifies their diagnostic capacity and reduces associated administrative burden, freeing valuable time for critical decision-making and direct patient consultation.

At Q2BSTUDIO we accompany healthcare organizations and research centers in this deep digital transformation, offering a comprehensive ecosystem that spans from machine learning architecture design to the implementation of productive solutions in regulated environments. Our approach combines technical excellence in AI agents and artificial intelligence with proven ability to deliver tailor-made application development and custom software that integrates natively with existing hospital infrastructure. We understand that each center is a unique universe of protocols, histories, and regulatory requirements, which is why we prioritize extreme customization, security by design, and horizontal scalability in every project we undertake with our healthcare clients.

Medical segmentation assisted by foundation models and multimodal visual prompts marks the beginning of a new era in precision oncology and personalized medicine. It is not merely about improving a percentage point in an algorithm's metric, but about redefining how imaging technologies symbiotically dialogue with clinical knowledge accumulated over decades. Institutions that decisively bet on secure, scalable platforms genuinely adapted to their operational reality will be better positioned to offer more accurate treatments, reduce therapeutic waiting times, and substantially improve patient experience and outcomes. The future of medicine is written with rigorous code, but always with the patient as the undisputed center of all technological decisions.

OUR SERVICES

How we can help you

Do you have a project in mind?

Tell us your vision and we'll turn it into a software solution. Whatever the scope, we make your idea real.