> ## Documentation Index
> Fetch the complete documentation index at: https://friendli.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Dedicated Endpoints FAQ and Troubleshooting
> Answers to common questions about Friendli Dedicated Endpoints, including model compatibility, GPU requirements, billing, and troubleshooting tips.
export const RoundedBorderBox = ({children, caption}) =>
{children}
{caption &&
{caption}
}
* Use the repository id of the model. You may select the entry from the list of autocompleted model repositories.
* You may choose a specific branch, or manually enter a commit hash.
## Format Requirements
* A model should be in safetensors format.
* The model should NOT be nested inside another directory.
* Including other arbitrary files (that are not in the list) is totally fine. However, those files will not be downloaded nor used.
| Required | Filename | Description |
| -------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| Yes | *safetensors* | Model weight, e.g. model.safetensors. Use model.safetensors.index.json for split safetensors files |
| Yes | config.json | Model config that includes the architecture. ([Supported Models on Friendli](https://friendli.ai/models)) |
| No | tokenizer.json | Tokenizer for the model |
| No | tokenizer\_config.json | Tokenizer config. This should be present & have a `chat_template` field for the Friendli Engine to provide chat APIs |
| No | special\_tokens\_map.json | Tokenizer’s special tokens to their corresponding token strings |
The dataset should satisfy the following conditions:
1. The dataset must contain a column named **"messages"**.
2. Each row in the "messages" column should be compatible with the chat template of the base model.
For example, [`tokenizer_config.json`](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/blob/41b61a33a2483885c981aa79e0df6b32407ed873/tokenizer_config.json#L42)
of `mistralai/Mistral-7B-Instruct-v0.2` is a template that repeats the messages of a user and an assistant.
Concretely, each row in the "messages" field should follow a format like: `[{"role": "user", "content": "The 1st user's message"}, {"role": "assistant", "content": "The 1st assistant's message"}]`.
In this case, `HuggingFaceH4/ultrachat_200k` is a dataset that is compatible with the chat template.
## Troubleshooting
### Inference Request Errors
Below is a table of common error codes you might encounter when making inference-related API requests.
| Code | Name | Cause | Suggested Solution |
| ----- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `400` | *Bad Request* | The request is malformed or missing required fields. | Check your request payload. Ensure it is valid JSON with all required fields. |
| `401` | *Unauthorized* | Missing or invalid API key. The request lacks proper authentication. | Include a valid Personal API key in the `Authorization` header. Verify the key is active and correct. |
| `403` | *Forbidden* | The API key is valid but does not have permission to access the endpoint. | Ensure your Personal API key has access rights to the endpoint. Use the correct team key or add the `X-Friendli-Team` header if needed. |
| `404` | *Not Found* | The specified endpoint or resource does not exist. This typically occurs when the `endpoint_id` or `team_id` is invalid. | Verify the `endpoint_id` and model name in your request. Ensure they match an existing, non-deleted deployment. Also check for typos in your endpoint ID or team ID. |
| `422` | *Unprocessable Entity* | The request is syntactically correct but semantically invalid (e.g. exceeding token limits, invalid parameter values). | Adjust your request (e.g. reduce `max_tokens`, correct parameter values) and try again. |
| `429` | *Too Many Requests* | You have exceeded rate limits for your plan. | Reduce request frequency or upgrade your plan for higher limits. Wait before retrying after a 429 error. |
| `500` | *Internal Server Error* | A server-side error occurred while processing the request. | Retry the request after a short delay. If the error persists, check endpoint health in the overview dashboard or contact FriendliAI support. |
#### Quick Checklist Before Retrying
* Verify the endpoint URL, `endpoint_id`, and (if applicable) `X-Friendli-Team` header
* Include the `Authorization` header with a valid key
* Confirm the target deployment exists, is healthy, and is not deleted
* Validate request JSON and required fields; reduce `max_tokens` if needed
* Check rate limits; add retry with backoff when receiving `429`
### Model Selection Errors
The repository is gated. Please follow the steps and gain approval from the owner using Hugging Face Hub.
The model does not meet the requirements. Please check if the model follows a correct safetensors format. See the [format requirements](#format-requirements) for details.
The model architecture is not supported. Please refer to the [Supported Models](https://friendli.ai/models?products=DEDICATED) page.
### Endpoint Lifecycle
Endpoints that remain in a sleep state for 48 hours are automatically terminated.
* When `min_replicas = 0`, the endpoint enters a sleep state after the cooldown period if no requests are received.
* A notification is sent after 24 hours of sleep, and the endpoint is terminated after another 24 hours if not reactivated.
This page may not cover all cases. If your issue persists, [contact support](mailto:support@friendli.ai).