Documentation Index
Fetch the complete documentation index at: https://friendli.ai/docs/llms.txt
Use this file to discover all available pages before exploring further.
April, 2026
Pricing Update
We have changed the pricing model forQwen/Qwen3-30B-A3B, DeepSeek-V3.1 to token-based pricing.New Model Family
Added support for the following new model families:- CohereAsrForConditionalGeneration (e.g.,
CohereLabs/cohere-transcribe-03-2026) - Gemma4ForConditionalGeneration (e.g.,
google/gemma-4-31B-it)
March, 2026
Host KV Cache
We now support Host KV Cache. This extends KV capacity beyond GPU memory limits, allowing more tokens to be retained during inference. Read moreSpeculative Decoding with a Draft Model
We now support speculative decoding by pairing the target model with a pre-trained draft model. The draft model proposes multiple candidate tokens verified by the target model, reducing decoding passes and improving throughput and latency. Available for a curated list of target models. Read morePricing Update
We now support cached input pricing for the following Model API.MiniMaxAI/MiniMax-M2.5
February, 2026
January, 2026
New Model Family
Added support for the following new model family:- Glm4MoeLiteForCausalLM (e.g.,
zai-org/GLM-4.7-Flash)
New Model Family
Added support for the following new model family:- ExaoneMoEForCausalLM (e.g.,
LGAI-EXAONE/K-EXAONE-236B-A23B)
December, 2025
New Model Family
Added support for the following new model families:- HunYuanVLForConditionalGeneration (e.g.,
tencent/HunyuanOCR) - MiniMaxM2ForCausalLM (e.g.,
MiniMaxAI/MiniMax-M2) - Gemma3TextModel (e.g.,
google/embeddinggemma-300m) - Phi4MMForCausalLM (e.g.,
microsoft/Phi-4-multimodal-instruct)
November, 2025
Feature Availability Update
Dedicated Endpoints’ Basic plan users can now access the following features that were previously available only to Enterprise plan users:- Request count auto-scaling: Scale endpoints based on request count. Request count scaling strategy adjusts worker numbers according to total requests in the queue and in progress. Read more
- Multi-LoRA serving: Serve multiple LoRA adapters simultaneously on a single endpoint, allowing you to use different fine-tuned models without additional GPU resources. Read more
- Metrics: Track, monitor, and optimize your inference deployment.
- Logs: Track logs and spot issues in real time.
New Model Family
Added support for the following new model families:- FluxKontextPipeline (e.g.,
black-forest-labs/FLUX.1-Kontext-dev) - Olmo3ForCausalLM (e.g.,
allenai/Olmo-3-32B-Think) - LightOnOCRForConditionalGeneration (e.g.,
lightonai/LightOnOCR-1B-1025) - PaddleOCRVLForConditionalGeneration (e.g.,
PaddlePaddle/PaddleOCR-VL) - DeepseekOCRForCausalLM (e.g.,
deepseek-ai/DeepSeek-OCR)
New Model Family
Added support for the following new model families:- Qwen3VLForConditionalGeneration (e.g.,
Qwen/Qwen3-VL-4B-Instruct) - Qwen3VLMoeForConditionalGeneration (e.g.,
Qwen/Qwen3-VL-30B-A3B-Instruct) - GraniteMoeHybridForCausalLM (e.g.,
ibm-granite/granite-4.0-h-small) - DotsOCRForCausalLM (e.g.,
rednote-hilab/dots.ocr)
Pricing Update
We have changed the pricing model forQwen/Qwen3-235B-A22B-Instruct-2507 to token-based pricing.September, 2025
New Model Family
Added support for the following new model families:- Qwen3NextForCausalLM (e.g.,
Qwen/Qwen3-Next-80B-A3B-Instruct) - HunYuanDenseV1ForCausalLM (e.g.,
tencent/Hunyuan-MT-7B) - ApertusForCausalLM (e.g.,
swiss-ai/Apertus-8B-Instruct-2509) - SeedOssForCausalLM (e.g.,
ByteDance-Seed/Seed-OSS-36B-Instruct)
Custom Chat Template Support
We now support custom chat formatting. You can paste or upload a custom Jinja template during endpoint creation. Read more4-Bit Online Quantization Support
We now support 4-bit online quantization. By enabling this feature, you can efficiently run models on smaller instances with negligible quality impact. Read moreAugust, 2025
New Model Family
Added support for the following new model family:- GptOssForCausalLM (e.g.,
openai/gpt-oss-20b)
New Auto-Scaling Type ‘Request count’ Added
Enterprise plan users can now choose to scale their endpoints based on request count. Request count scaling strategy adjusts worker numbers according to total requests in the queue and in progress.Increased Output Token Limits for Reasoning Models
We have increased the output token limits for reasoning models on Model APIs, allowing longer reasoning outputs to be generated.New Model Family
Added support for the following new model family:- HyperCLOVAXForCausalLM (e.g.,
naver-hyperclovax/HyperCLOVAX-SEED-Think-14B)
July, 2025
New Model Family
Added support for the following new model families:- Dots1ForCausalLM (e.g.,
rednote-hilab/dots.llm1.inst) - Glm4vForConditionalGeneration (e.g.,
zai-org/GLM-4.1V-9B-Thinking) - KeyeForConditionalGeneration (e.g.,
Kwai-Keye/Keye-VL-8B-Preview) - HunYuanMoEV1ForCausalLM (e.g.,
tencent/Hunyuan-A13B-Instruct) - PhiMoEForCausalLM (e.g.,
microsoft/Phi-mini-MoE-instruct) - MiniMaxM1ForCausalLM (e.g.,
MiniMaxAI/MiniMax-M1-80k) - Ernie4_5_MoeForCausalLM (e.g.,
baidu/ERNIE-4.5-21B-A3B-Thinking) - Ernie4_5_ForCausalLM (e.g.,
baidu/ERNIE-4.5-0.3B-PT)