Prerequisites
- LiteLLM Python SDK or LiteLLM Proxy.
- A NEAR AI Cloud API key from the NEAR AI Cloud Dashboard.
NEARAI_API_KEYexported in the shell or injected into the proxy container.
config.yaml.
Base URL
Use the NEAR AI Cloud gateway base URL:/chat/completions to api_base. LiteLLM and the OpenAI client add the request path when they make chat completion calls.
Model ID
Use LiteLLM’s OpenAI-compatible prefix for the upstream model:openai/ prefix tells LiteLLM to send the request through OpenAI-compatible chat completions. The NEAR AI Cloud model ID after the prefix is z-ai/glm-5.2.
Check Model Discovery and Refresh when you need the freshest NEAR AI Cloud model list.
Configure
Python SDK
Call NEAR AI Cloud directly from the LiteLLM Python SDK:Proxy config.yaml
Expose NEAR AI Cloud through LiteLLM Proxy with a local model alias:
Optional: report capabilities and limits
LiteLLM passes tool calls and reasoning through theopenai/ path regardless of metadata, so the config above is enough to use GLM 5.2. If you want LiteLLM’s introspection endpoints (/v1/models, /v1/model/info, /model_group/info) to report GLM 5.2’s real capabilities — instead of defaulting to false/null because z-ai/glm-5.2 is not in LiteLLM’s built-in model map — add a sibling model_info block:
/v1/models record for z-ai/glm-5.2 (tools and reasoning supported, text-only, context 500000, output 131072). This block is reporting metadata only; it does not change request behavior.
Refresh models
LiteLLM Proxy uses the entries inmodel_list; it does not automatically add every NEAR AI Cloud model to your config. When NEAR AI Cloud releases a new model:
- Run
curl https://cloud-api.near.ai/v1/models. - Copy the new model’s
id. - Add or update a
model_listentry. - Set
litellm_params.modeltoopenai/<model-id>. - Restart or reload LiteLLM Proxy so the new alias appears.
/v1/models and then review them before using the config: