> ## Documentation Index
> Fetch the complete documentation index at: https://friendli.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Container Tokenization

> Tokenize text into token IDs using Friendli Container. Run tokenization locally on your own infrastructure for pre-processing and token counting.

By giving a text input, generate a tokenized output of token IDs.


## OpenAPI

````yaml https://github.com/friendliai/friendli-openapi/raw/refs/heads/main/openapi.yaml post /v1/tokenize
openapi: 3.1.0
info:
  title: Friendli Suite API Reference
  description: This is an OpenAPI reference of Friendli Suite API.
  termsOfService: https://friendli.ai/terms-of-service
  contact:
    name: FriendliAI Support Team
    email: support@friendli.ai
  version: 0.1.0
servers:
  - url: https://api.friendli.ai
security: []
tags:
  - name: Serverless.Chat
  - name: Serverless.ToolAssistedChat
  - name: Serverless.Messages
  - name: Serverless.ChatRender
  - name: Serverless.Completions
  - name: Serverless.Token
  - name: Serverless.Audio
  - name: Serverless.Model
  - name: Serverless.Knowledge
  - name: Dedicated.Chat
  - name: Dedicated.Messages
  - name: Dedicated.ChatRender
  - name: Dedicated.Completions
  - name: Dedicated.Embeddings
  - name: Dedicated.TextClassification
  - name: Dedicated.Token
  - name: Dedicated.Image
  - name: Dedicated.Audio
  - name: Dedicated.Endpoint
  - name: Container.Chat
  - name: Container.Messages
  - name: Container.Completions
  - name: Container.TextClassification
  - name: Container.Token
  - name: Container.Image
  - name: Container.Audio
  - name: Cost
  - name: Dataset
  - name: File
paths:
  /v1/tokenize:
    servers:
      - url: http://localhost:8000
    post:
      tags:
        - Container.Token
      summary: Tokenization
      description: By giving a text input, generate a tokenized output of token IDs.
      operationId: containerTokenization
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ContainerTokenizationBody'
        required: true
      responses:
        '200':
          description: Successfully tokenized the text.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ContainerTokenizationSuccess'
              examples:
                Example:
                  value:
                    tokens:
                      - 128000
                      - 3923
                      - 374
                      - 1803
                      - 1413
                      - 15592
                      - 30
        '422':
          description: Unprocessable Entity
components:
  schemas:
    ContainerTokenizationBody:
      properties:
        model:
          anyOf:
            - type: string
            - type: 'null'
          title: Model
          description: Routes the request to a specific adapter.
          examples:
            - (adapter-route)
        prompt:
          type: string
          title: Prompt
          description: Input text prompt to tokenize.
          examples:
            - What is generative AI?
      type: object
      required:
        - prompt
      title: ContainerTokenizationBody
      example:
        prompt: What is generative AI?
    ContainerTokenizationSuccess:
      properties:
        tokens:
          items:
            type: integer
          type: array
          title: Tokens
          description: A list of token IDs.
      type: object
      required:
        - tokens
      title: ContainerTokenizationSuccess

````