Views:

Configure the LiteLLM integration to enable AI Guard to inspect and protect AI application traffic passing through your LiteLLM proxy, providing consistent security policies across different AI providers and models.

AI application traffic flows through the LiteLLM proxy. Using the AI Guard plugin, LiteLLM sends incoming requests to AI Guard for scanning before forwarding them to the target LLM provider. LLM provider responses are also scanned by AI Guard before being returned to the requestor.
Before configuring the integration, verify that the following requirements are met:
  • LiteLLM version 1.81.0 or later is installed and running. For deployment options, see the LiteLLM documentation.
  • AI Guard is enabled and at least one configuration has been applied and saved:
  • Your TrendAI Vision One™ role includes the AI Security → AI Application Security → AI Guard → Call detection API permission.
    If a role with this permission does not exist, contact your administrator.

Procedure

  1. Obtain your authentication token.
    1. Go to Workflow and Automation → Third-Party Integrations.
    2. Click the LiteLLM card.
    3. Under Authentication token, click Generate and copy the token.
  2. Obtain the endpoint URL for your AI Guard deployment type.
    • Trend-hosted
      1. Go to Workflow and Automation → Third-Party Integrations.
      2. Click the LiteLLM card.
      3. Copy the Endpoint URL.
      Use a TrendAI Vision One™ account in the region where your LiteLLM deployment is located. For non-US regions, the endpoint URL uses the format api.<region>.xdr.trendmicro.com, where <region> is one of: eu, jp, au, in, sg, mea.
    • Self-hosted on Kubernetes
      The endpoint URL depends on where LiteLLM runs relative to AI Guard.
      LiteLLM location Endpoint URL
      Same Kubernetes namespace as AI Guard http://ai-guard:8080
      Different namespace in the same cluster http://ai-guard.trend-ai-security.svc.cluster.local:8080
      Outside the cluster Your external ingress URL
      Replace trend-ai-security with the namespace name if you deployed AI Guard into a custom namespace.
    • Self-hosted on AWS
      1. In the AWS Management Console, open the AI Guard CloudFormation stack.
      2. On the Outputs tab, copy the value of GuardAPIEndpoint. This is your endpoint URL.
  3. Configure and deploy the AI Guard plugin for LiteLLM.
    With your endpoint URL and authentication token ready, follow the TrendAI Guard LiteLLM plugin setup guide for plugin installation, LiteLLM configuration, and deployment options (Docker and Helm).
AI Guard inspects and protects AI application traffic passing through the LiteLLM proxy.