There is no screenshot tool in LlamaIndex. Not in the registry, and not — despite what everyone assumes — in llama-index-tools-playwright, whose exposed functions are click, fill, get_current_page, extract_hyperlinks, extract_text, get_elements, navigate_to and navigate_back. No screenshot.
So you write your own, and you hit the interesting part: a LlamaIndex tool can return an image, and that fact is documented nowhere. Not in a guide, not in an example, not on the tools page. It exists in the changelog and in the source, and that is all.
This post is that mechanism, and the two ways it silently does nothing.
The tool
Requires llama-index-core 0.12.45 or later — the release whose changelog line reads "feat: allow tools to output content blocks."
import os
import httpx
from llama_index.core.llms import ImageBlock
def screenshot(url: str) -> ImageBlock:
"""Take a full-page screenshot of a public web page and return the image.
Use this when you need to see how a page actually renders — layout,
visual bugs, whether a banner is covering something — rather than
reading its text.
"""
r = httpx.get(
"https://api.site-shot.com/",
params={
"url": url,
"userkey": os.environ["SITESHOT_API_KEY"],
"full_size": 1,
"no_ads": 1,
"no_cookie_popup": 1,
},
timeout=60.0,
)
r.raise_for_status()
return ImageBlock(image=r.content, image_mimetype="image/png")
Wire it to an agent:
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.anthropic import Anthropic
agent = FunctionAgent(
tools=[screenshot],
llm=Anthropic(model="claude-opus-5"),
system_prompt="You can look at web pages. Use the screenshot tool when the question is visual.",
)
response = await agent.run("Does the pricing page on example.com show a free tier above the fold?")
FunctionAgent, not ReActAgent. That is not a style preference — see below.
Why this works, since no guide will tell you
FunctionTool.call() runs its result through _parse_tool_output, which begins by checking whether the returned object is already a content block:
if isinstance(raw_output, (TextBlock, ImageBlock, AudioBlock, ...)):
return [raw_output]
Anything that is not one of those falls through to return [TextBlock(text=str(raw_output))]. Those blocks land on ToolOutput.blocks, and FunctionAgent forwards them into the conversation verbatim as a tool-role message.
Raw bytes are accepted and base64-encoded for you, so there is no manual encoding step. A hosted URL works too, but is not required.
Returning bytes is the failure that costs you money
This follows directly from the fall-through above, and it is worth stating on its own because nothing raises.
return r.content # DON'T
str(b"\x89PNG\r\n\x1a\n...") is a perfectly valid string. The agent sends the literal Python repr of your PNG to the model as text — thousands of junk tokens, billed at full rate, followed by a model politely hallucinating a description of a page it never saw.
No exception. No warning. Wrap it in ImageBlock.
The provider split
Here is the part that will waste your afternoon: the same correct code behaves differently depending on which model you point it at.
Anthropic forwards the image. The Anthropic integration builds its tool result from all blocks, so an ImageBlock reaches the model.
OpenAI discards it. The OpenAI integration carries this note in its source:
"NOTE: Despite what the openai docs say, if the role is ASSISTANT, SYSTEM or TOOL, 'content' cannot be a list and must be string instead."
The tool-role branch is built from text blocks only. Your image is dropped — no error, no warning, just a model that says it cannot see an image. The Responses API path does the same thing.
If you need this on OpenAI, the workaround is the same one that applies in LangChain: do not return the image from the tool. Have the tool return a reference, and route the actual image into the conversation as a user message, where a list of content blocks is allowed.
An important caveat on scope, because I would rather be narrow than wrong: only the OpenAI and Anthropic integrations were read for this. LlamaIndex ships roughly ninety LLM integrations, and whether Gemini, Bedrock, Vertex, Ollama and the rest forward tool-returned images is genuinely unknown. Test yours; do not assume it inherits either behaviour.
ReActAgent drops images on every provider
Separately from the provider question, the agent class matters.
ReActAgent records each tool result as ObservationReasoningStep(observation=str(tool_call_result.tool_output.content)), and ToolOutput.content is defined as the text blocks joined together — image blocks are filtered out before the model ever sees them.
So a correct tool, on a provider that supports images, still returns nothing visible if you run it under ReActAgent. Use FunctionAgent for any image-returning tool.
Four smaller traps
The built-in HTTP tool cannot fetch an image. RequestsToolSpec.get_request ends in return res.json(), which raises on a PNG body. Every method in that spec does the same. Do not reach for llama-index-tools-requests here.
That same spec drops your key on redirects. It matches auth headers by exact hostname — self.domain_headers.get(self._get_domain(url), {}) — so a redirect to a CDN host silently sends no credentials, and you get a 401 that the agent then narrates as if the page were broken.
Always pass image_mimetype explicitly. ImageBlock guesses the MIME type from a file extension, and a screenshot API URL does not have one. On the Anthropic path a missing MIME type is a hard ValueError. (Passing raw bytes, as above, is the safe route — the content gets sniffed — but naming it costs nothing.)
Never put the key in the function signature. Every parameter becomes part of the JSON schema sent to the model, so a userkey: str argument invites the model to echo or invent it, and puts it into agent memory and traces. Read it from the environment inside the body, and keep the signature def screenshot(url: str).
Honest limits
LlamaIndex imposes no size limit on an image, and that is not reassuring. There is no documented cap beyond a zero-byte check; the real ceiling is whatever the downstream model accepts. A full-page screenshot of a long page is a large base64 payload.
Agents run up to 20 iterations per run() by default, with no timeout. Base workflows default to 45 seconds, but agents explicitly override that to None. A hung screenshot call blocks indefinitely — which is why the example passes timeout=60.0 to httpx rather than relying on the framework.
Whether images are re-sent on every turn is unknown. I could not establish whether FunctionAgent memory replays a returned ImageBlock on each subsequent LLM call or serialises it away. For a screenshot workflow that is a real cost question. If you are running long conversations, measure your token usage rather than trusting either answer.
The multi-modal docs page is stale. It still teaches the deprecated MultiModal LLM classes, which were folded into the base LLM classes in 0.12.47. Prefer the changelog and the source over that page.
FAQ
Can a LlamaIndex tool return an image?
Yes, since llama-index-core 0.12.45. Return an ImageBlock from the tool function and the framework passes it through as a content block on the tool output. Raw bytes are accepted and base64-encoded for you. This capability appears in the changelog and the source code but is not covered by any documentation guide or example page.
Why does my agent say it cannot see the screenshot?
Two likely causes. On OpenAI models the integration builds tool-role content from text blocks only, so the image is silently discarded. And ReActAgent converts every tool result to its text content on all providers, dropping image blocks before the model sees them. Use FunctionAgent, and verify that your provider forwards images.
What happens if a tool returns raw image bytes?
The output falls through to a text block containing the string representation of the bytes, so the model receives the literal Python repr of your PNG as text. Nothing raises an error, you are billed for thousands of junk tokens, and the model describes an image it never received. Always wrap bytes in an ImageBlock.
Does the Playwright tool in LlamaIndex take screenshots?
No. Its exposed functions are click, fill, get_current_page, extract_hyperlinks, extract_text, get_elements, navigate_to and navigate_back. There is no screenshot function in it, and no screenshot tool anywhere in the LlamaIndex tool registry.
Where should the screenshot API key live?
In an environment variable, read inside the tool function body. Never make it a function parameter, because every parameter becomes part of the JSON schema sent to the model, which invites the model to echo or invent the value and persists it into agent memory and traces.
Related reading
- How to Give a LangChain Agent Website Screenshots — the same provider split, in a framework that documents its version of it.
- How to Give a CrewAI Agent Website Screenshots — where a tool cannot return an image at all.
- AI Agent vs Screenshot API — when to render in the agent's own browser instead.
- Screenshot API documentation — every parameter used above.
Site-Shot returns PNG bytes from one GET, which is exactly what ImageBlock wants — no hosting step, no base64 handling, no browser for your agent to drive. See the plans.