Scality launches AI Inference Factory to bring enterprise AI on-premises

Scality’s new AI Inference Factory addresses critical enterprise challenges in adopting artificial intelligence by enabling the deployment of validated open-weight models on private infrastructure. This open-code solution offers organizations a viable alternative to cloud-based services, providing predictable costs and ensuring data sovereignty. By keeping sensitive data and critical inference workloads on-premises, businesses can maintain strict control over their model lifecycles and regulatory compliance without relying on external providers. A key technical advancement is the integration of Scality AI Data Infrastructure, which extends key-value cache storage beyond limited GPU memory. This architecture separates prefill and decode operations, allowing GPUs to retrieve context from high-speed shared storage rather than recomputing it. This approach significantly reduces GPU consumption and operational costs while maintaining high performance, making it feasible to handle large context windows and complex agentic applications efficiently at scale. This development is highly relevant to open data and open-source communities because it leverages validated open-weight models and provides fully transparent, open-code software. It allows organizations to inspect how inference state is managed and contributes to a ecosystem where data handling is not locked behind proprietary black-box services. By supporting various open-source frameworks and hardware, it promotes interoperability and prevents vendor lock-in, empowering enterprises to build sustainable, secure, and adaptable AI infrastructures.

Source: manilatimes.net
Published on 2026-10-09

Scality launches AI Inference Factory to bring enterprise AI on-premises

Related news