MiniMax-M2.7 - 树叶书屋

MiniMax-M2.7 vLLM Deployment Guide

This guide serves MiniMax-M2.7 through an OpenAI-compatible vLLM endpoint. The model weights are available from Hugging Face.

For the latest compatibility and hardware requirements, consult the official vLLM deployment guide. The official guide currently requires Linux, Python 3.9 through 3.12, a GPU with compute capability 7.0 or newer, and about 220 GB of memory for the weights.

Install vLLM

Create an isolated environment and install the latest compatible vLLM release:

uv venv
source .venv/bin/activate
uv pip install vllm --torch-backend=auto

Start the server

For a four-GPU deployment:

SAFETENSORS_FAST_GPU=1 vllm serve \
    MiniMaxAI/MiniMax-M2.7 --trust-remote-code \
    --tensor-parallel-size 4 \
    --enable-auto-tool-choice --tool-call-parser minimax_m2 \
    --reasoning-parser minimax_m2_append_think

For an eight-GPU deployment with expert parallelism:

SAFETENSORS_FAST_GPU=1 vllm serve \
    MiniMaxAI/MiniMax-M2.7 --trust-remote-code \
    --enable_expert_parallel --tensor-parallel-size 8 \
    --enable-auto-tool-choice --tool-call-parser minimax_m2 \
    --reasoning-parser minimax_m2_append_think

The server listens on http://localhost:8000/v1 by default.

Verify the endpoint

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "MiniMaxAI/MiniMax-M2.7",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain mixture-of-experts models briefly."}
        ]
    }'

If vLLM reports that the model is unsupported, upgrade vLLM. If CUDA graph capture causes an illegal memory access, add --compilation-config '{"cudagraph_mode":"PIECEWISE"}' to the server command, as recommended by the official guide.

MiniMax-M2.7 SGLang Deployment Guide

This guide serves MiniMax-M2.7 through an OpenAI-compatible SGLang endpoint. The model weights are available from Hugging Face.

For the latest compatibility and hardware requirements, consult the official SGLang deployment guide. The official guide currently requires Linux, Python 3.9 through 3.12, a GPU with compute capability 7.0 or newer, and about 220 GB of memory for the weights.

Install SGLang

Create an isolated environment and install SGLang:

uv venv
source .venv/bin/activate
uv pip install sglang

Use SGLang 0.5.4.post1 or newer for MiniMax-M2 family support.

Start the server

For a four-GPU deployment:

python -m sglang.launch_server \
    --model-path MiniMaxAI/MiniMax-M2.7 \
    --tp-size 4 \
    --tool-call-parser minimax-m2 \
    --reasoning-parser minimax-append-think \
    --host 0.0.0.0 \
    --trust-remote-code \
    --port 8000 \
    --mem-fraction-static 0.85

For an eight-GPU deployment with expert parallelism:

python -m sglang.launch_server \
    --model-path MiniMaxAI/MiniMax-M2.7 \
    --tp-size 8 \
    --ep-size 8 \
    --tool-call-parser minimax-m2 \
    --reasoning-parser minimax-append-think \
    --host 0.0.0.0 \
    --trust-remote-code \
    --port 8000 \
    --mem-fraction-static 0.85

Verify the endpoint

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "MiniMaxAI/MiniMax-M2.7",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain mixture-of-experts models briefly."}
        ]
    }'

MiniMax-M2.7 Transformers Deployment Guide

This guide runs MiniMax-M2.7 directly with Transformers. The model weights are available from Hugging Face.

For the latest compatibility and hardware requirements, consult the official Transformers deployment guide. The official guide currently requires Linux, Python 3.9 through 3.12, Transformers 4.57.1, a GPU with compute capability 7.0 or newer, and about 220 GB of memory for the weights.

Install dependencies

Create an isolated environment and install the tested Transformers release:

uv venv
source .venv/bin/activate
uv pip install transformers==4.57.1 torch accelerate --torch-backend=auto

Run inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MiniMaxAI/MiniMax-M2.7"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Explain mixture-of-experts models briefly.",
            }
        ],
    }
]

model_inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True,
).to("cuda")

with torch.inference_mode():
    generated_ids = model.generate(model_inputs, max_new_tokens=256)

new_tokens = generated_ids[:, model_inputs.shape[1]:]
print(tokenizer.batch_decode(new_tokens, skip_special_tokens=True)[0])

Keep trust_remote_code=True; the model depends on its repository implementation. If downloading from Hugging Face is unavailable on your network, configure an approved proxy or mirror before running the script.