skills.rest
chengzongcai
chengzongcaiCOMMUNITY
Explore all skills by this author →
GitHub

llm-security

Tests LLM applications and AI agents for prompt injection, tool abuse, and memory poisoning vulnerabilities.

21|8|Updated Aug 3, 2026
Search Tags:#owasp#penetration testing#ai agent#prompt injection#llm security#red teaming#rag poisoning
One-click install
npx skills add https://github.com/chengzongcai/reverse-skill-backup --skill llm-security-chengzongcai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-security
Source: https://github.com/chengzongcai/reverse-skill-backup/tree/main/skills/llm-security
Command: npx skills add https://github.com/chengzongcai/reverse-skill-backup --skill llm-security-chengzongcai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires garak, pyrit, promptfoo, and includes references (resource) components.

What problem does it solve? Security teams lack a repeatable methodology for assessing LLM applications and AI agents, often guessing at attack techniques instead of systematically covering prompt injection, tool abuse, RAG poisoning, and system prompt extraction. ## Core Features & Use Cases - Five-Level Prompt Injection Testing: Progresses from direct instruction overrides through role-play jailbreaks, encoding bypasses, multi-turn extraction, and indirect injection via RAG content. - Agent-Specific Attack Chains: Tests tool misuse chains, human-in-the-loop bypass, memory poisoning, and output handling mapped to OWASP LLM Top 10 and Agentic AI (ASI) Top 10. - Tooling Integration: Directs use of garak, PyRIT, and promptfoo for automated probe scanning, multi-turn attack orchestration, and CI/CD red-team regression testing. - Use Case: During an authorized pentest of a customer support chatbot with tool access, follow the workflow to enumerate registered tools, attempt cross-tool privilege escalation, and verify whether canary tokens in the system prompt leak under extraction attempts. ## Quick Start Ask the AI to run an authorized LLM security assessment against your chatbot target following the llm-security workflow, starting with attack surface reconnaissance and prompt injection probes.

Frequently Asked Questions about llm-security

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test an LLM application for prompt injection vulnerabilities?▼

Use a five-level progression: direct instruction overrides, role-play jailbreaks like DAN, encoding bypasses with Base64 or Unicode homoglyphs, multi-turn gradual extraction, and indirect injection embedded in RAG content. Tools like garak automate over 100 injection probes.

What tools are used for automated LLM red teaming?▼

garak provides 100+ automated injection probes, PyRIT from Microsoft orchestrates multi-turn attacks with scoring, and promptfoo integrates AI-generated attacks into CI/CD pipelines. promptmap2 and AgentThreatBench cover automated reasoning and ASI Top 10 benchmarking.

How is AI agent security testing different from standard LLM testing?▼

Agent testing targets tool misuse chains, privilege escalation across registered tools, human-in-the-loop bypass via urgency or authority manipulation, and long-term memory poisoning. The threat shifts from untrusted output to hijacked control over real actions.

Can prompt injection attacks be fully prevented?▼

No complete defense exists because LLMs process instructions and data in the same natural language channel. The methodology recommends layered defenses: making exploitation harder, detectable via canary tokens, and limiting blast radius with least-privilege tool permissions.

What is indirect prompt injection through RAG systems?▼

Indirect injection embeds malicious instructions in webpages, PDFs, or emails that an agent later retrieves and processes. PoisonedRAG research shows five malicious documents in a million-document corpus can achieve 90% manipulation success, so retrieval-time permission checks are tested.

Related Skills

Explore More Software Engineering Skills→
YukiIto1999YukiIto1999

bug-bounter

Define scope, enumerate web assets, and assess vulnerabilities for bug bounty reporting.

Community
Advanced
Harvest-Forged-CodeHarvest-Forged-Code
10

security-audit

Audit codebases for OWASP Top 10 vulnerabilities, hardcoded secrets, and vulnerable dependencies.

Official
Advanced
xijunwwxijunww
1

recon

Automate penetration testing reconnaissance to discover assets and attack surfaces.

Community
Advanced
infantesromeroadrianinfantesromeroadrian

pentest-cicd-redteam

Automates security assessments and red teaming in CI/CD pipelines via GitHub Actions, GitLab CI, Jenkins, and scanning tools.

Community
Advanced