Image evidence & ARGUS

Paste or attach screenshots in chat — PNG, JPEG, or WebP — and ask questions about what you see. ContextMint routes vision through local Ollama by default (no API key required). Optional cloud multimodal is available when you opt in with BYOK. ARGUS runs a structured visual audit on the same image — WCAG-tagged findings, report only, never auto-fix.

Why image evidence

Engineers paste UI mockups, error dialogs, and dashboard screenshots every day. ContextMint treats images as first-class evidence alongside retrieved code chunks — with the same human gates: Context Lens preview before inference, routing badge after, and no silent writes.

  • Local-first — vision models run on your machine via Ollama when installed
  • No cloud key required — attaching an image does not force cloud routing
  • Lane-aware — Work and Hybrid lanes support image + packs; Repo lane combines image + RAG
  • Privacy-conscious — sensitivity notice on first attach; EXIF stripped; images never logged as base64

Attach images in chat

Ways to add an image

  • Paste — copy a screenshot and paste into the compose bar
  • Drag and drop — drop image files onto the compose area
  • File picker — use the attach control and choose an image file

Limits and formats

  • Formats: PNG, JPEG, WebP (SVG and other types are rejected)
  • Up to 3 images per message (configurable)
  • Max 5 MiB per image; large images are resized to a max dimension (default 2048 px)
  • Image chips appear beside file attachment chips in the compose bar

The first time you attach an image in a session, ContextMint may show a sensitivity notice — screenshots can contain credentials, PII, or internal URLs. Review Context Lens before sending when the trust gate is enabled.

Vision routing (local default)

When images are attached, ContextMint selects a vision-capable model — not a text-only coder model. The default path is local Ollama using models such as moondream, llava, or qwen2-vl.

Lens / badge label Meaning
local Local Ollama vision model will run — default and recommended for sovereign workflows
cloud You explicitly chose a cloud vision model and BYOK is enabled with a valid key
needs_vlm No vision model installed — pull one from Engine → Models (primary CTA); cloud is optional secondary

Important: configuring a cloud API key does not automatically send images to the cloud. Images stay on the local vision path unless you explicitly select a cloud vision model. This preserves the sovereign default for regulated and air-gapped environments.

Pull a vision model

If no vision model is installed, Context Lens shows Pull a vision model — a deep link to Engine → Models where you can pull moondream (small, fast) or llava. Vision models are separate from your chat and embed models; both can coexist in Ollama.

Context Lens with images

When Context Lens preview is enabled, attached images appear in the evidence drawer alongside retrieved chunks:

  • Thumbnail preview of each attached image
  • Vision route label (local, cloud, or needs_vlm)
  • Chunk count and RAG evidence (when Repo or Hybrid lane is active)
  • Actions: Send (multimodal chat) and Run visual audit (ARGUS)

The trust gate can require explicit confirmation when images are present — same human-in-the-loop pattern as sensitive text sends. See Context Lens & packs.

Chat answers with images

After you send, the assistant answers using the image plus any retrieved context. The routing badge on the answer shows local or cloud vision tier. Inspector / xAI meta includes image_count and vision_route for audit trails — not raw image bytes.

  • Repo / Hybrid — image + hybrid search (vector + BM25 + grep) when indexed
  • Work — image + session packs and file attachments; no vector repo search
  • Plan / Agent + image — informational banner; images do not trigger autonomous file apply

When images are present, extended “thinking” streams are disabled for compatible local models to avoid incompatible multimodal + think modes.

ARGUS — visual audit

ARGUS (Automated Review for GUI Usability & Standards) is ContextMint’s structured visual audit layer. It uses the same local vision stack as chat but returns a findings report instead of a conversational answer.

Paste audit (from chat)

  1. Attach a UI screenshot in compose.
  2. Open Context Lens (or use the message action after send).
  3. Click Run visual audit.
  4. Review findings inline: severity, category, description, optional WCAG rule reference, screenshot thumb.

The audit reuses the same normalized image blob — no second upload. ARGUS defaults to local vision; it does not require cloud keys.

Sandbox audit (from preview worktree)

When git sandbox preview is enabled, run command palette: ContextMint: Run visual audit on sandbox. ContextMint bootstraps dependencies in the isolated worktree, captures a Playwright screenshot of the running preview, and runs the same VLM audit schema.

  • On dependency bootstrap failure, you get a structured deps_unavailable finding — never a blank-page audit
  • Reports are stored with a TTL (default 7 days) under the operator data directory

Findings format

Each finding includes:

  • severity — e.g. critical, major, minor, info
  • category — accessibility, contrast, layout, copy, interaction, …
  • description — what the model observed
  • wcag_rule — optional WCAG reference when applicable
  • bbox — optional region hint on the screenshot

Safety — report only

ARGUS never auto-fixes your codebase. If you act on a finding, ContextMint can hand off to the assisted patch preview (gate 2) — you still review unified diffs before apply. See Patches & governance.

Optional cloud vision (BYOK)

Teams that approve external inference can enable cloud multimodal when contextmint.cloudEnabled is on and a provider key is configured. You must explicitly select a cloud vision-capable model; images are not silently dispatched to text-only cloud models.

Multimodal context is assembled and redacted in the extension; API keys stay in VS Code SecretStorage. Details: BYOK & enterprise.

Settings reference

Key extension settings (search “ContextMint” in VS Code Settings):

  • contextmint.chat.imageAttachmentsEnabled — enable paste / drop / picker
  • contextmint.chat.imageMaxCountPerMessage — max images per send
  • contextmint.chat.imageMaxBytes — per-file size cap
  • contextmint.chat.imageSensitivityNotice — first-attach notice text
  • contextmint.argus.auditButtonLabel — Lens action label (default “Run visual audit”)

Server tunables (chat.image_*, argus.*) live in operator defaults on the API host. Full list: Settings → Image evidence & ARGUS.

Operator checklist

  1. Engine → Start all — server and Ollama healthy
  2. Engine → Models — pull one vision model (moondream recommended for laptops)
  3. VS Code → paste a screenshot → confirm Lens shows local route
  4. Optional → Run visual audit on a UI screenshot to validate ARGUS

Problems: Troubleshooting → Image evidence & ARGUS. Concepts: Glossary · Chat & lanes.