AI tool guides

73 guides for using LiteLLM with AI coding tools and assistants, from the LiteLLM cookbook. See also the LiteLLM docs.

All guides

• Track Usage for Coding Tools: Track usage and costs for AI-powered coding tools like Claude Code, Roo Code, Gemini CLI, and OpenAI Codex through LiteLLM. (Claude Code, Gemini CLI, OpenAI Codex, LiteLLM)
• Claude Code - WebSearch Across All Providers: Enable Claude Code's web search tool to work with any provider (Bedrock, Azure, Vertex, etc.). LiteLLM automatically intercepts web search requests and executes them server-side. (Claude Code, LiteLLM, Web Search)
• Claude Code - Granular Cost Tracking: Track Claude Code usage by customer or tags using LiteLLM proxy. This enables granular cost attribution for billing, budgeting, and analytics. (Claude Code, LiteLLM)
• Use Claude Code with Non-Anthropic Models: This tutorial shows how to use Claude Code with non-Anthropic models like OpenAI, Gemini, and other LLM providers through LiteLLM proxy. (Claude Code, LiteLLM, OpenAI, Gemini)
• Cursor Integration: Route Cursor IDE requests through LiteLLM for unified logging, budget controls, and access to any model. To connect Cursor to MCP tools, use the direct-client URL and headers in the MCP Configuration Reference (Cursor, LiteLLM, Quickstart)
• GitHub Copilot / VS Code: This tutorial shows two ways to use LiteLLM Proxy from VS Code's chat. The LiteLLM extension adds your gateway as a language model provider, so the chat model picker lists the models your key can reach, with each model's price and a... (Github Copilot, LiteLLM, Quickstart)
• Gemini CLI: This tutorial shows you how to integrate the Gemini CLI with LiteLLM Proxy, allowing you to route requests through LiteLLM's unified interface. (Gemini CLI, Gemini, LiteLLM, Quickstart)
• OpenAI Codex CLI Quickstart: This is a quickstart guide to using OpenAI Codex CLI. (OpenAI Codex CLI, OpenAI, LiteLLM, Quickstart)
• Open WebUI: Rolling Open WebUI out to a whole company with SSO, IdP groups mapped to LiteLLM teams, skills, and artifacts? See Open WebUI for Organizations. (OpenWebUI, LiteLLM, Quickstart)
• Claude Code Quickstart: This tutorial shows how to call Claude models through LiteLLM proxy from Claude Code. (Claude Code, LiteLLM)
• Use Claude Code with MCPs: This tutorial shows how to connect MCP servers to Claude Code via LiteLLM Proxy. For endpoint, transport, and credential selection, see the MCP Configuration Reference (Claude Code, LiteLLM, MCP)
• Using Anthropic File API with LiteLLM Proxy
• Replacing OpenAI ChatCompletion with Completion()
• Claude Agent SDK with LiteLLM: Use Anthropic's Claude Agent SDK with any LLM provider through LiteLLM Proxy.
• Auto Router with Claude Code and Claude Desktop: Route Claude Code and Claude Desktop through a LiteLLM auto router, including gateway discovery's model-name filter and the organization allowlist requirements that decide whether the router is selectable at all.
• Claude Code - Managing Anthropic Beta Headers: When using Claude Code with LiteLLM and non-Anthropic providers (Bedrock, Azure AI, Vertex AI), you need to ensure that only supported beta headers are sent to each provider. This guide explains how to add support for new beta headers or...
• Claude Code - Budget Status Line: Show a user's remaining LiteLLM budget in the Claude Code status line. The status line calls /user/info with the user's virtual key and displays the difference between userinfo.maxbudget and userinfo.spend.
• Claude Code with Bring Your Own Key (BYOK): Use Claude Code with your own Anthropic credentials through the LiteLLM proxy. Claude Code authenticates to Anthropic one of two ways Bearer (tied to your Claude.ai Free, Pro, Max, or Enterprise seat), while a configured Anthropic API key...
• Claude Code - Cut Costs: Claude Code is one of the heaviest consumers of input tokens in a modern engineering org. Long tool loops, large file reads, and MCP catalogs with hundreds of tools push every request toward the top of the context window, and the bill...
• Claude Code SSO: Set up Claude Code with Claude apps gateway sign-in, Lite CLI SSO, or a Claude subscription through LiteLLM.
• Claude Code Gateway reference: Start with Claude Code SSO setup to choose and configure a sign-in flow.
• Use your Claude subscription with Claude Code: Connect Claude Code to LiteLLM with your Claude subscription. Configure the proxy, create a virtual key, sign in, and verify usage.
• Claude Code with Okta SSO (JWT Auth): Route Claude Code through LiteLLM using each developer's Okta identity. Claude Code sends the developer's own Okta access token with every request, LiteLLM validates it against your Okta authorization server and creates the user...
• Claude Code Plugin Marketplace (Managed Skills): LiteLLM AI Gateway acts as a central registry for Claude Code plugins. Admins can govern which plugins are available across the organization, and engineers can discover and install approved plugins from a single source.
• Claude Code - Prompt Cache Routing: Claude's Prompt Caching feature helps to optimize API usage through attempting to cache prompts and re-use cached prompts during subsequent API calls. This feature is used by Claude Code.
• LiteLLM Skills: litellm-skills is a collection of Agent Skills for managing a live LiteLLM proxy. Install them once and any agent that supports the Agent Skills standard (Claude Code, OpenCode, OpenClaw, etc.) can create users, teams, keys, models, MCP...
• Claude Desktop & Cowork SSO: Configure Claude Desktop and Cowork to use LiteLLM through third-party inference with SSO.
• Auto Router with Codex and your ChatGPT subscription: Route Codex through a LiteLLM auto router while each developer keeps their own ChatGPT login, with a virtual key or an SSO sign-in as the gateway credential.
• Codex CLI - Granular Cost Tracking: Track Codex CLI usage by customer or tags using LiteLLM proxy. This enables granular cost attribution for billing, budgeting, and analytics.
• Benchmark LLMs: Easily benchmark LLMs for a given question by viewing
• Comparing LLMs on a Test Set using LiteLLM: LiteLLM allows you to use any LLM as a drop in replacement for
• CopilotKit SDK with LiteLLM: Use CopilotKit SDK with any LLM provider through LiteLLM Proxy.
• Onboard Users for AI Exploration: v1.73.0 introduces the ability to assign new users to Default Teams. This makes it much easier to enable experimentation with LLMs within your company, by allowing users to sign in and create $10 keys for AI exploration.
• Elasticsearch Logging with LiteLLM: Send your LLM requests, responses, costs, and performance data to Elasticsearch for analytics and monitoring using OpenTelemetry.
• Evaluate LLMs - MLflow Evals, Auto Eval
• Using completion() with Fallbacks (Failover) for Reliability: Use completion() with model fallbacks (failover) so a failing provider automatically fails over to a backup model for reliable responses.
• File Search in the Responses API: LiteLLM now supports file_search in the Responses API across both:
• Using Fine-Tuned gpt-3.5-turbo: LiteLLM allows you to call completion with your fine-tuned gpt-3.5-turbo models
• Create your first LLM playground: Create a playground to evaluate multiple LLM Providers in less than 10 minutes. If you want to see this in prod, check out our website.
• Call Gemini Realtime API with Audio Input/Output: 1. Setup config.yaml for LiteLLM Proxy
• Google ADK with LiteLLM: Use Google ADK with the LiteLLM Python SDK or the LiteLLM Proxy.
• Google GenAI SDK with LiteLLM: Use Google's official GenAI SDK (JavaScript/TypeScript and Python) with any LLM provider through LiteLLM Proxy.
• Gradio Chatbot + LiteLLM Tutorial: Simple tutorial for integrating LiteLLM completion calls with streaming Gradio chatbot demos
• CodeLlama - Code Infilling: This tutorial shows how you can call CodeLlama (hosted on Huggingface PRO Inference Endpoints), to fill code.
• Llama2 - Huggingface Tutorial: Huggingface is an open source platform to deploy machine-learnings models.
• Set up environment: Let's get the necessary keys to set up our demo environment.
• Instructor: Combine LiteLLM with jxnl's instructor library for more robust structured outputs. Outputs are automatically validated into Pydantic types and validation errors are provided back to the model to increase the chance of a successful...
• Aporia Guardrails with LiteLLM Gateway: In this tutorial we will use LiteLLM AI Gateway with Aporia to detect PII in requests and profanity in responses
• Qwen Code CLI: This tutorial shows you how to integrate the Qwen Code CLI with LiteLLM Proxy, allowing you to route requests through LiteLLM's unified interface.
• Reliability test Multiple LLM Providers with LiteLLM
• LiveKit xAI Realtime Voice Agent: Use LiveKit's xAI Grok Voice Agent plugin with LiteLLM Proxy to build low-latency voice AI agents.
• Benchmark LLMs - LM Harness, FastEval, Flask
• Mock Completion Responses - Save Testing Costs: Trying to test making LLM Completion calls without calling the LLM APIs ?
• Customize Prompt Templates on OpenAI-Compatible server: You will learn: How to set a custom prompt template on our OpenAI compatible server.
• Model Fallbacks (Provider Failover) w/ LiteLLM: Implement model fallbacks (provider failover) across OpenAI, Anthropic, and Azure with LiteLLM so a failing provider fails over to a backup.
• Chat with sample documents in MongoDB (BETA): Create three fictional policy documents, index them in MongoDB Atlas, test semantic search in the LiteLLM Admin UI, and use the results in a chat completion. The sample text below was written for this tutorial and does not describe real...
• Microsoft SSO: Sync Groups, Members with LiteLLM: Sync Microsoft SSO Groups, Members with LiteLLM Teams.
• Oobabooga Text Web API Tutorial
• OpenAI Agents SDK with LiteLLM: Use OpenAI's Agents SDK with any LLM provider through LiteLLM Proxy.
• Use your ChatGPT subscription with Codex: Connect Codex to LiteLLM with your ChatGPT subscription. Configure a model, create a virtual key, and start using the CLI or desktop app.
• OpenClaw + LiteLLM Integration: OpenClaw is a self-hosted AI assistant that connects chat apps (WhatsApp, Telegram, Discord, and more) to LLM providers. By routing OpenClaw through LiteLLM Proxy, you get access to 100+ providers, cost tracking, spend limits, and...
• OpenCode Quickstart: This tutorial shows how to connect OpenCode to your existing LiteLLM instance and switch between models.
• Open WebUI for Organizations (SSO, Teams, Skills, Artifacts): Roll out Open WebUI on top of LiteLLM as an internal replacement for the Claude and ChatGPT desktop apps, with single sign-on, the signed-in user's IdP token validated by LiteLLM JWT auth, IdP groups mapped to LiteLLM teams and...
• Presidio PII Masking with LiteLLM - Complete Tutorial: This tutorial will guide you through setting up PII (Personally Identifiable Information) masking with Microsoft Presidio and LiteLLM Gateway. By the end of this tutorial, you'll have a production-ready setup that automatically detects...
• Auto-Inject Prompt Caching Checkpoints: Reduce costs by up to 90% by using LiteLLM to auto-inject prompt caching checkpoints.
• provider_specific_params
• Retool Assist: This guide walks you through connecting Retool Assist to LiteLLM Proxy. Retool Assist uses AI to generate and edit apps from within the Retool app IDE. Using LiteLLM with Retool Assist allows you to:
• Scalekit with LiteLLM: Add authenticated tool calls to your LiteLLM-powered agents. Scalekit manages OAuth flows, token storage, and API execution for 100+ third-party apps (Gmail, GitHub, Slack, Salesforce, etc.). Your agent picks tools at runtime and LiteLLM...
• SCIM with LiteLLM: Enterprise: SCIM support requires a premium license.
• [Beta] Routing based on request metadata: Create routing rules based on request metadata.
• Using Text Completion Format - with Completion(): If your prefer interfacing with the OpenAI Text Completion format this tutorial covers how to use LiteLLM in this format
• Llama Together AI Tutorial
• Vertex AI PayGo and Priority