AI Gateway
AI Gateway documentation
Last updated
Was this helpful?
AI Gateway documentation
Last updated
Was this helpful?
Spice provides a high-performance, OpenAI API-compatible AI Gateway optimized for managing and scaling large language models (LLMs). Additionally, Spice offers tools for Enterprise Retrieval-Augmented Generation (RAG), such as SQL query across federated datasets and an advanced search feature (see ).
Spice supports full OpenTelemetry observability, enabling detailed tracking of data flows and requests for full transparency and easier debugging.
Spice supports a variety of LLMs, including OpenAI, Azure OpenAI, Anthropic, Groq, Hugging Face, and more (see for all supported models).
Custom Tools: Equip models with tools to interact with the Spice runtime.
System Prompts: Customize system prompts and override defaults for .
For detailed configuration and API usage, refer to the .
To use a language model hosted on OpenAI (or compatible), specify the openai
path and model ID in from
.
Example spicepod.yml
:
For details, see .