Skip to main content

v1.74.3-stable

Krrish Dholakia
Krrish Dholakia
CEO, LiteLLM
Ishaan Jaffer
Ishaan Jaffer
CTO, LiteLLM

Deploy this version​

docker run litellm
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:v1.74.3-stable

Key Highlights​

  • MCP: Model Access Groups - Add mcp servers to access groups, for easily managing access to users and teams.
  • MCP: Tool Cost Tracking - Set prices for each MCP tool.
  • Model Hub v2 - New OSS Model Hub for telling developers what models are available on the proxy.
  • Bytez - New LLM API Provider.
  • Dashscope API - Call Alibaba's qwen models via new Dashscope API Provider.

MCP Gateway: Model Access Groups​


v1.74.3-stable adds support for adding MCP servers to access groups, this makes it easier for Proxy Admins to manage access to MCP servers across users and teams.

For developers, this means you can now connect to multiple MCP servers by passing the access group name in the x-mcp-servers header.

Read more here

MCP Gateway: Tool Cost Tracking​


This release adds cost tracking for MCP tool calls. This is great for Proxy Admins giving MCP access to developers as you can now attribute MCP tool call costs to specific LiteLLM keys and teams.

You can set:

  • Uniform server cost: Set a uniform cost for all tools from a server
  • Individual tool cost: Define individual costs for specific tools (e.g., search_tool costs $10, get_weather costs $5).
  • Dynamic costs: For use cases where you want to set costs based on the MCP's response, you can write a custom post mcp call hook to parse responses and set costs dynamically.

Get started

Model Hub v2​


v1.74.3-stable introduces a new OSS Model Hub for telling developers what models are available on the proxy.

This is great for Proxy Admins as you can now tell developers what models are available on the proxy.

This improves on the previous model hub by enabling:

  • The ability to show Developers models, even if they don't have a LiteLLM key.
  • The ability for Proxy Admins to select specific models to be public on the model hub.
  • Improved search and filtering capabilities:
    • search for models by partial name (e.g. xai grok-4)
    • filter by provider and feature (e.g. 'vision' models)
    • sort by cost (e.g. cheapest vision model from OpenAI)

Get started

New Models / Updated Models​

Pricing / Context Window Updates​

ProviderModelContext WindowInput ($/1M tokens)Output ($/1M tokens)Type
Xaixai/grok-4256k$3.00$15.00New
Xaixai/grok-4-0709256k$3.00$15.00New
Xaixai/grok-4-latest256k$3.00$15.00New
Mistralmistral/devstral-small-2507128k$0.1$0.3New
Mistralmistral/devstral-medium-2507128k$0.4$2New
Azure OpenAIazure/o3-deep-research200k$10$40New

Features​

Bugs​

  • Github Copilot
    • Fix API base url for Github Copilot - PR
  • Bedrock
    • Ensure supported bedrock/converse/ params = bedrock/ params - PR
    • Fix cache token cost calculation - PR
  • XAI
    • ensure finish_reason includes tool calls when xai responses with tool calls - PR

LLM API Endpoints​

Features​

  • /completions
    • Return ‘reasoning_content’ on streaming - PR
  • /chat/completions
    • Add 'thinking blocks' to stream chunk builder - PR
  • /v1/messages
    • Fallbacks support - PR
    • tool call handling for non-anthropic models (/v1/messages to /chat/completion bridge) - PR

MCP Gateway​

Features​

  • Cost Tracking
    • Add Cost Tracking - PR
    • Add usage tracking - PR
    • Add custom cost configuration for each MCP tool - PR
    • Add support for editing MCP cost per tool - PR
    • Allow using custom post call MCP hook for cost tracking - PR
  • Auth
    • Allow customizing what client side auth header to use - PR
    • Raises error when MCP server header is malformed in the request - PR
  • MCP Server
    • Allow using stdio MCPs with LiteLLM (enables using Circle CI MCP w/ LiteLLM) - PR, Get Started

Bugs​

  • General
  • MCP Server
    • Fix mcp tool separator to work with Claude code - PR, Get Started
    • Add validation to mcp server name to not allow "-" (enables namespaces to work) - PR

Management Endpoints / UI​

Features​

  • Model Hub
    • new model hub table view - PR
    • new /public/model_hub endpoint - PR
    • Make Model Hub OSS - PR
    • New ‘make public’ modal flow for showing proxy models on public model hub - PR
  • MCP
    • support for internal users to use and manage MCP servers - PR
    • Adds UI support to add MCP access groups (similar to namespaces) - PR
    • MCP Tool Testing Playground - PR
    • Show cost config on root of MCP settings - PR
  • Test Key
    • Stick sessions - PR
    • MCP Access Groups - allow mcp access groups - PR
  • Usage
    • Truncate long labels and improve tooltip in Top API Keys chart - PR
    • Improve Chart Readability for Tag Usage - PR
  • Teams
    • Prevent navigation reset after team member operations - PR
    • Team Members - reset budget, if duration set - PR
    • Use central team member budget when max_budget_in_team set on UI - PR
  • SSO
    • Allow users to run a custom sso login handler - PR
  • Navbar
    • improve user dropdown UI with premium badge and cleaner layout - PR
  • General
    • Consistent layout for Create and Back buttons on all the pages - PR
    • Align Show Password with Checkbox - PR
    • Prevent writing default user setting updates to yaml (causes error in non-root env) - PR

Bugs​

  • Model Hub
    • fix duplicates in /model_group/info - PR
  • MCP
    • Fix UI not syncing MCP access groups properly with object permissions - PR

Logging / Guardrail Integrations​

Features​

Bugs​

Performance / Loadbalancing / Reliability improvements​

Bugs​

  • Responses API Bridge
    • add image support for Responses API when falling back on Chat Completions - PR s/o @ryan-castner
  • aiohttp
    • Properly close aiohttp client sessions to prevent resource leaks - PR
  • Router
    • don't add invalid deployment to router pattern match - PR

General Proxy Improvements​

Bugs​

  • S3
    • s3 config.yaml file - ensure yaml safe load is used - PR
  • Audit Logs
    • Add audit logs for model updates - PR
  • Startup
    • Multiple API Keys Created on Startup when max_budget is enabled - PR
  • Auth
    • Resolve model group alias on Auth (if user has access to underlying model, allow alias request to work) - PR
  • config.yaml
    • fix parsing environment_variables from config.yaml - PR
  • Security
    • Log hashed jwt w/ prefix instead of actual value - PR

Features​

  • MCP
    • Bump mcp version on docker img - PR
  • Request Headers
    • Forward ‘anthropic-beta’ header when forward_client_headers_to_llm_api is true - PR

New Contributors​

Git Diff​