AI Development and Agentic Security Labs

What it is

Labs AIDS-01 (Genkit Flows Security) and AIDS-02 (Tool Authorization and Sandboxing) from HADESS/DevSecOps Guides, teaching security patterns for Google Genkit AI apps (Genkit 1.x, TypeScript 5.x, Zod 3.x, Node 20+). The summary lists the topics; below are the concrete API mechanisms and defaults the summary does not carry.

How it works

Genkit flows (defineFlow) run Zod inputSchema validation before the LLM is ever called and outputSchema before returning — a failed validation throws ZodError and skips the model. Without a schema, input defaults to z.any() (any JSON passes). Streaming flows validate each chunk against streamSchema; without it, sendChunk() emits raw model output.

Tools (defineTool) are executed by Genkit, not the LLM: the model emits a structured call request, Genkit validates it against the tool's inputSchema, runs the function, feeds results back, and loops. The maxTurns parameter defaults to 5 — the key fact for runaway-loop containment. By default Genkit auto-executes tool calls; setting returnToolRequests: true makes it return response.toolRequests unexecuted so the caller can gate them.

What to do

Concrete mitigations the labs give:

• Constrain input length to stop token-cost inflation: z.string().max(N); restrict model/report choices with z.enum([...]); use .strict() to reject unknown keys.
• Replace free-form SQL tools with an enum-driven report tool — reportType: z.enum(["active_users_count", ...]) mapping to fixed parameterized queries — so DROP TABLE cannot be injected (the enum rejects it).
• Select tools by role at call time, e.g. role === "admin" ? [lookupUser, queryData] : [lookupUser].
• Gate destructive tools behind human approval with returnToolRequests: true, then filter response.toolRequests for a destructive-tools list before resuming.
• Set config: { maxTurns: 3 } explicitly rather than relying on the default 5.
• Redact telemetry via a custom SpanProcessor.onEnd that overwrites the genkit.input and genkit.output span attributes with [REDACTED] before export.
• Detect prompt injection with regex on input, e.g. /ignore\s+(all\s+)?previous\s+instructions/i, /reveal\s+(your|the)\s+(instructions|prompt|system)/i.

Verify with the supplied curl/assert tests, e.g. confirming an over-length message and an out-of-enum language: "xx" are both rejected, and that toolsCalled.length <= 3 under a maxTurns of 3.

Caveats

The labs state Zod validation stops structural attacks (wrong types, extra fields, oversized payloads) but not semantic ones — a valid string can still carry a prompt-injection payload. Both labs list "N/A" for CVEs; these are framework design patterns, not vulnerability advisories. Output sanitization shown is regex script-stripping, an inherently partial XSS defense.