Run an AISVS scan | TrendAI™

Views:

Run an AI scan with the Open Worldwide Application Security Project (OWASP) AI Security Verification Standard (AISVS) attack preset to get a per-level readiness score. For what the score means and how the standard is assessed, refer to Framework scans. AISVS scoring works with the TrendAI™-hosted judge and with a custom judge.

TrendAI™ Artifact Scanner (formerly TMAS) v2.313.0 or later is required for the AISVS attack preset.

Run an AISVS scan with the interactive wizard

Create an application programming interface (API) key for your AI model.

Important

When adding the API key, you must select a role with all AI Scanner permissions. If your organization does not have an existing user role with the necessary permissions, create a custom role or contact your administrator.

For more information, refer to API Keys.

Go to Administration → API Keys.
Select Add API key.
Enter a name, role, expiration time, and description.
Select Add, then copy and retain the API key.

Download and install the TrendAI™ Artifact Scanner CLI.
Configure the scan settings in the CLI.
1. Store the TrendAI Vision One™ API key as an environment variable:
```
export TMAS_API_KEY=<your_vision_one_api_key>
```
2. Add the TrendAI™ Artifact Scanner CLI to your path:
```
export PATH="/path/to/tmas/binary/directory:$PATH"
```
3. Start AI Scanner. For the US TrendAI Vision One™ region, run this command:
```
./tmas aiscan llm -i
```
  For other TrendAI Vision One™ regions, add the region flag. For example, for the Japan region:
```
./tmas aiscan llm -i --region=ap-northeast-1
```
4. Select the target type.
  - Model endpoint (OpenAI compliant): Select this option for LLM endpoints that implement the OpenAI API specification. You need the OpenAI API endpoint URL, API key, and model ID.
  - AI application endpoint (Custom): Select this option for AI applications with custom API interfaces. You need the endpoint URL, API key, and the JSON keys that identify where to embed prompts in the request body and where to extract responses.
  If the request or response schema for a custom endpoint involves an array or nested JSON structure, use config mode (below).
5. Follow the CLI wizard to name the group, enter the target API endpoint, supply the API key, and configure the remaining target settings.
6. For Select attack preset, select the AISVS option, labeled with the current standard version, for example AISVS 1.0.0.
  
  The preset pre-selects the attack objectives, techniques, and modifiers required for AISVS and labels them (required for AISVS). Keep these items selected. Disabling one may reduce the accuracy of your readiness score.
7. Select the report output settings from the provided options.
  
  You can upload results to TrendAI Vision One™ and write one or more local report files. Supported local report formats: json, json-summary, markdown, and html.
  
  You can also control chat transcript inclusion separately for each destination. The default follows the judge in use. With the TrendAI™-hosted judge, transcripts are included by default. With a custom judge, they are excluded by default.
8. Select Yes to save the scan configuration and specify a file name.
9. When the scan completes, view the full results of the scan in AI Scanner. For more information, refer to AI Scanner scan results.

Run an AISVS scan from a configuration file

TrendAI™ recommends config mode for automated or continuous integration and continuous delivery (CI/CD) pipeline integration, repeatable testing, and custom AI application endpoints with complex request or response structures.

Store the target endpoint API key as an environment variable:
```
export TARGET_API_KEY=<your_target_api_key>
```
Create a YAML configuration file that defines the target, scan settings, and the AISVS attack preset.

Set attack_preset: AISVS to select the AISVS requirements for the scan.

For configuration examples, refer to AI Scanner configuration examples by AI endpoint provider.
Run the scan with the configuration file:
```
./tmas aiscan llm -c config.yaml
```
To specify a region, add the region flag:
```
./tmas aiscan llm -c config.yaml --region ap-northeast-1
```
To control where results are written, add an output: block to the configuration file. For details, refer to AI Scanner configuration examples by AI endpoint provider.

View AISVS results

The AISVS widget in the TrendAI Vision One™ AI Scanner page shows a pass rate and a verdict for each level, for example

Level 1 - 46%
                     (6/13)

. The denominator is the number of requirements in that level that attack testing can verify, not the total number of requirements AISVS defines for that level.

Each level verdict summarizes the outcomes of that level's individual requirements. A level verdict is one of the following:

Verdict	Meaning
PASS	Every testable requirement was assessed and passed.
PASS (partially assessed)	The tests passed, but not all required techniques and modifiers ran, so more advanced attacks may still succeed.
NOT ASSESSED	The level was not tested by this scan.
FAIL	One or more tested requirements failed.

Expand a level to see each requirement with a plain-language description, its own outcome (passed, partially assessed, failed, or not assessed), the attacks that exercise it, and a link to the standard. Select View findings on a requirement to filter the results table to the attacks related to that requirement.

The AISVS score also appears in the dashboard, the executive summary, and exported reports. For more information, refer to AI Scanner scan results.

Constraints and limitations

An AISVS-scored scan uses the AISVS attack preset (attack_preset), and a scan uses one attack preset at a time.
You can combine agentic attacks with an AISVS-scored scan, but an agentic attack's results count toward the AISVS score only if the attack is tagged with the relevant AISVS requirement, either explicitly or through a built-in category. For more information on tagging syntax, refer to Agentic attacks.
Custom prompt files can be combined with an AISVS-scored scan in the same configuration, but custom prompts never count toward the AISVS score. They are sent verbatim with no techniques or modifiers, so they contribute no readiness evidence. Use them alongside the AISVS preset for regression testing, not to raise your score.