Image / Vision Analysis

Use this skill when the user sends an image attachment, screenshot, or asks you to analyze visual content.

Trigger Conditions

• User sends an image file (.png, .jpg, .webp)
• User asks “what does this image show?”, “analyse cette image”, “lis ce document”
• User sends a screenshot of an app, website, or error screen
• You need to extract visual information from an image

Tool Selection

✅ Preferred: google-genai Python SDK

# Install (one-time)
uv pip install google-genai

Via execute_code (for programmatic logic):

import os, pathlib
from hermes_tools import terminal

# Read API key from .env (already configured)
import pathlib as pl
env = pl.Path("/home/bf/.hermes/.env").read_text()
for line in env.splitlines():
    if "GEMINI_API_KEY" in line:
        os.environ["GEMINI_API_KEY"] = line.split("=", 1)[1]
        break

from google import genai
client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

uploaded = client.files.upload(file=pathlib.Path("/path/to/image.png"))

response = client.models.generate_content(
    model="gemini-2.5-pro",  # Or fallback to gemini-2.0-flash
    contents=[uploaded, "Describe this image in detail"]
)
print(response.text)

Via terminal (simpler for one-shot queries):

source /home/bf/.hermes/.env && python3 -c "
import os, pathlib, sys
from google import genai
client = genai.Client(api_key=os.environ['GEMINI_API_KEY'])
uploaded = client.files.upload(file=pathlib.Path(sys.argv[1]))
response = client.models.generate_content(
    model='gemini-2.5-pro',
    contents=[uploaded, sys.argv[2]]
)
print(response.text)
" /path/to/image.png "Describe this image"

❌ Avoid: Gemini CLI (npm)

The gemini npm package CLI hangs on every invocation (even --help). Do not attempt to use it.

❌ Avoid: Hermes built-in vision_analyze

This user prefers Gemini directly for vision analysis instead of the Hermes built-in tool.

Model Selection & Quota Handling

The Gemini API key is a free-tier key — quotas are limited.

On 429 (RESOURCE_EXHAUSTED):

1. Try fallback model immediately: gemini-2.0-flash has higher free limits
2. If both exhausted: inform the user clearly — provide the full error message context (which quota was hit: daily vs. per-minute)
3. Do NOT wait/retry silently for 30+ seconds — report the blocker

Error signature:

429 RESOURCE_EXHAUSTED — Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_*

User Preference

This user explicitly instructed: “utilise toujours gemini CLI pour la vision” — always use Gemini for vision analysis. Since the CLI hangs, use the Python SDK with the same API key.

Pitfalls

• File upload required: The Gemini File API requires uploading the image first before analysis. Use client.files.upload() — this returns a handle you pass to contents=.
• Langue: If the user is French-speaking, prompt Gemini in French for better results with their queries.
• .env sourcing: The terminal() subprocess does NOT inherit the Hermes .env file. Always source /home/bf/.hermes/.env && ... or read the file explicitly via Python.
• Key redaction: The actual API key value is redacted in tool output (shows as ***). execute_code can read the file directly and set the env var — this works despite the redaction in terminal output.