AI Red Teaming: What It Is, Free Tools, and When You Need It

All five are free and open source. We reviewed each tool’s GitHub repository, license file and documentation on October 24, 2026; we did not run them. Listed A to Z; this is not a ranking.

What AI red teaming tests

AI red teaming tests whether an AI model or app can be pushed into doing something it shouldn’t, using the risks in the OWASP Top 10 for LLM Applications 2025 as the usual checklist.

Prompt injection tops the list because an AI app can’t reliably tell its owner’s instructions from text it reads in an email, web page or document. A red team plants instructions in those places and checks whether the app follows them, for example by forwarding data or calling a tool it shouldn’t. NIST’s Generative AI Profile names prompt injection, data poisoning and model extraction among the attacks AI red-teaming should cover (action MS-2.7-007).

MITRE ATLAS is the attack catalog red teams map their findings to. ATLAS describes itself as a “living knowledge base of adversary tactics and techniques involving AI”, and listed 16 tactics, 208 techniques and 73 case studies on October 24, 2026.

AI red teaming vs penetration testing

AI red teaming targets the model and how it behaves; penetration testing targets the servers, networks and app code around it. Most AI products need both.

A chatbot’s login page, API keys and cloud account are pentest scope; whether the chatbot can be talked into leaking another customer’s order is AI red teaming scope. For the server and app side, compare the pentest firms we reviewed, including AI-led tests, and pentest cost ranges by test type, or the pentest tools you can run yourself.

Open-source AI red teaming tools

Open-source AI red teaming tools are free to run, and the five below are actively maintained, with releases or code changes in the last 12 months.

PyRIT (Microsoft)

PyRIT, the Python Risk Identification Tool, is Microsoft’s open-source framework “to proactively identify risks in generative AI systems”, under the MIT license. Microsoft published a 2026 white paper on PyRIT titled “Democratizing AI Red Teaming Through Open-Source Tooling”. Microsoft archived the older Azure/PyRIT repository; the current home is github.com/microsoft/PyRIT.

garak (NVIDIA)

garak calls itself “the LLM vulnerability scanner”: its README says it probes for hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks, and compares itself to Nmap or Metasploit “but for LLMs”. NVIDIA maintains garak under the Apache-2.0 license.

promptfoo

promptfoo tests prompts, agents and RAG apps and runs red-team scans from simple config files, with command line and CI/CD integration, under the MIT license. The free Community version includes 10,000 probes a month. OpenAI announced on March 9, 2026 that it will acquire Promptfoo and “continue building the open-source project”; the announcement said closing was subject to customary conditions, and we found no closing notice on October 24, 2026.

Giskard

Giskard is an open-source “evaluation and testing library for LLM agents” under the Apache-2.0 license.

DeepTeam (Confident AI)

DeepTeam is Confident AI’s open-source “framework to red team LLMs and AI agents”, under the Apache-2.0 license. DeepTeam’s last tagged release was in November 2025, though its code was updated in October 2026.

Commercial AI red teaming platforms and services

Commercial AI red teaming platforms add managed attack libraries, dashboards and expert-led testing, and none of the four we checked publishes a red teaming price.

Listed A to Z, unranked. Each row uses the vendor’s own description and was checked on October 24, 2026. Ownership changes fast in this market, so check it before you sign.

How an AI red team exercise works

An AI red team exercise runs in six steps: scope, threat model, automated probes, manual attacks, fixes, and a retest.

1. Scope: list the model, the app around it, its data sources and every tool or API it can call.
2. Threat model: decide who would attack it and what they want, such as customer data, free usage or a harmful answer with your logo on it.
3. Automated probes: run a scanner such as garak or promptfoo across thousands of known attack prompts.
4. Manual attacks: human testers chain attacks the scanners miss, especially multi-step ones against agents.
5. Fix: tighten system prompts, filter inputs and outputs, and cut the app’s permissions.
6. Retest: rerun the same attacks after every model or prompt change.

The retest matters because no model is immune. NIST’s Center for AI Standards and Innovation (CAISI) analysed a public competition run by Gray Swan, in which more than 400 participants made over 250,000 attack attempts on 13 frontier models in agent scenarios, and at least one attack succeeded against every model (NIST, March 23, 2026).

Who is required to red team AI?

Few organizations are legally required to red team AI today. The clearest rule is the EU AI Act, whose Article 55(1)(a) requires providers of general-purpose AI models with systemic risk to perform model evaluation “including conducting and documenting adversarial testing”.

The Article 55 duty is narrow. A model is presumed to have systemic risk when the compute used to train it exceeds 10^25 floating point operations (Article 51(2)), which describes frontier models, not a company’s chatbot. The general-purpose AI chapter has applied since August 2, 2025 (Article 113), and models placed on the market before that date have until August 2, 2027 (Article 111(3)).

For everyone else, AI red teaming is voluntary good practice. NIST AI 600-1 recommends it as a risk management action, and ISO/IEC 42001 asks for an AI management system with risk controls; our ISO 42001 guide covers how the standard, the EU AI Act and NIST’s framework fit together.