Jamba is a hybrid large language model architecture that combines Transformer, Mamba (state-space), and Mixture-of-Experts (MoE) layers. Designed for high efficiency and long-context processing (up to 256K tokens), it delivers solid benchmark performance with only 12B active parameters and runs on a single 80GB GPU, offering 3 times the capacity of similarly sized models.
Q2BSTUDIO is a company specialized in technological development and services, offering innovative solutions in the field of artificial intelligence and natural language processing.
