# Superlinked SIE Open-source inference for agents, serving embeddings, rerankers, extractors, and open LLMs on your infrastructure. - Findings24: - ウェブサイト: - 料金: Free - カテゴリー: AIエージェント基盤, AI埋め込みAPI ## Superlinked SIEとは？ Superlinked Inference Engine (SIE) is an open-source inference server for running AI models on infrastructure you control, from a laptop to a Kubernetes cluster. It provides a single API for four model operations: **Encode** for text and image vectors, **Score** for reranking query-document pairs, **Extract** for entities and structured data, and **Generate** for text generation. SIE is designed for workloads that use many smaller models, such as semantic search, RAG, document processing, classification, and agent workflows. It handles batching, GPU sharing, and on-demand model switching, with support for more than 100 models. Deployment options include Docker, Helm, Terraform for AWS, GCP, and Azure, and air-gapped installations using mirrored model snapshots. ## Superlinked SIEでできること ### Four unified inference primitives Encode text or images into vectors, score query-document pairs for reranking, extract entities or structured data, and generate text with hosted open models. ### Multi-model GPU sharing Models load on demand and least-recently-used models can be evicted when GPU memory is full, allowing one SIE instance to serve many model types. ### Cluster-wide batching A Kubernetes gateway routes requests to worker pools, where shared queues combine compatible work into batches and distribute GPU capacity across models. ### Flexible deployment Run the same Docker-based setup locally or deploy to Kubernetes with Helm and autoscaling. Terraform support is provided for AWS, GCP, and Azure. ### Model and runtime coverage SIE supports more than 100 models and wraps runtimes including PyTorch, SGLang, and related inference backends, selecting an engine per model. ### Single-layer model evaluation Switch between NLI, text or image embeddings, and cross-encoder rerankers through the same API, supporting structured comparisons without separate serving stacks. ## 利用シーン ### Semantic search and RAG Encode queries and documents, then rerank retrieved candidates to build search and retrieval-augmented generation pipelines on self-hosted infrastructure. ### Agent document processing Extract entities and structured data from unstructured text, and use self-hosted generation models in a custom document-processing pipeline. The separately announced Agent plugin’s automatic task routing and third-party-model redaction are upcoming capabilities. ### Multimodal product classification Evaluate zero-shot NLI, text retrieval, image retrieval, and cross-encoder reranking against hierarchical product taxonomies through one inference layer. ### Product search Build a self-hosted catalog search workflow using attribute extraction, description embeddings, and candidate reranking. ### Production inference operations Run multiple model pools in a Kubernetes cluster with shared GPU capacity, autoscaling, hot model-profile updates, and cloud or air-gapped deployment options. ## よくある質問 ### What infrastructure does SIE run on? Use Docker on CPU or GPU for local development or a single low-traffic server. GPU Kubernetes clusters support production scaling on EKS, GKE or AKS. CPU-only worker pools are for local testing, not a supported production-cluster target. GPU memory, quotas, model size and cache storage must fit the workload. ### What kinds of models does SIE support? SIE supports encoders, rerankers, entity extractors, and generation models, with more than 100 models available out of the box. The model catalog includes text, image, and multimodal workloads. ### Can SIE serve several models on one GPU? Yes. SIE loads models on demand and evicts least-recently-used models under memory pressure. Resident capacity depends on model size and batch settings, and each requested model must still fit in GPU memory. Kubernetes scale-from-zero can introduce documented cold starts of about 3–7 minutes. ### Is SIE open source? Yes. The core inference server is Apache 2.0 open-source software and always free to self-host. You supply and operate the hardware or cloud infrastructure; a free server license does not include free GPUs, cloud capacity or operations. ### How does SIE fit with the Superlinked framework? SIE is the inference layer and can be used independently or underneath the Superlinked framework, which is a higher-level Python SDK for multi-attribute search and recommendation systems. ### Are managed hosting and the Agent plugin generally available? Both are marked Upcoming. Superlinked accepts inference-grant applications for selected free hosted capacity; this does not provide every project with an unrestricted hosted plan. The Agent plugin has a waitlist, including its planned automatic document routing and redaction before third-party model calls. ### What is required for an air-gapped installation? Mirror the container images and model snapshots and configure a pre-populated model cache. Normal deployments need registry and model-download access on first use; gated models require appropriate access tokens. Air-gapped support is a deployment configuration, not an automatic guarantee that every connected agent or external service stays offline. ## クイック情報 - **Product type**: Open-source inference server - **Primary operations**: Encode, Score, Extract, Generate - **Deployment**: Laptop, Docker, Kubernetes, or air-gapped infrastructure - **Cloud targets**: AWS, GCP, and Azure - **Supported model count**: 100+ models out of the box - **Built by**: Superlinked