Skip to content
agent-media docs
Public documentation · No sign-in required
Documentation / Tools

Every tool. Every parameter.

The core tools and optional image uploader, with input schemas from the connector. Check tools/list for what is available on your connection.

On this page

For user photos, offer open_upload_panel when it appears in tools/list. Users can drop up to 10 images at once, then the agent calls get_uploads. Images expire 24 hours after the upload session opens. Use the returned browser link if the client cannot show an inline panel.

If your connector does not list the panel tools yet, call upload_image with an empty object to open the same uploader. After uploading, call it again with the returned upload_key to get the images. No separate upload page or local folder is needed.

Three tools spend credits and return a job id; the other tools are free. Poll get_run_status until a job is completed; never report a result before you hold the URL.

generate_video has three modes and the fields you pass pick one: first_frame (and optional last_frame) selects image-to-video; refs, video_refs or audio_refs select reference; neither is text. Frames and refs cannot be mixed on Seedance. quality (480p, 720p, 1080p, default 720p) sets the price per second.

generate_videospends credits

Generate Video

Render a video clip from YOUR prompt on the model YOU choose, in one of three MODES that follow from the fields you pass: TEXT (prompt only), IMAGE-TO-VIDEO (`first_frame`: an https still that becomes frame one, optionally `last_frame` to end on; the clip animates the still) or REFERENCE (`refs`: https images such as a portrait, a character sheet from list_characters or a product photo whose identity/look is kept; `video_refs`: clips whose motion or framing is followed; `audio_refs`: a voice or sound to carry; address them in the prompt as @image1, @video1, @audio1). Frames and refs cannot be mixed. Write the shot like a director: who is in frame, where, what happens, camera, and the exact spoken words in quotes if anyone talks. Models: seedance-2.0, seedance-2.5 (default seedance-2.0; call list_models for each model's modes, limits, prices and recent results; or pass model:"auto"). Aspect: 9:16 default, also 16:9, 1:1, 4:3, 3:4, 21:9, adaptive. Quality: 480p, 720p (default), 1080p; the price per second follows the quality (seedance-2.0: 30 credits/s at 480p, 60 credits/s at 720p, 150 credits/s at 1080p). Reference clip seconds are billed like output seconds. Call `quote` first if the user cares about cost. Returns a job id, then call get_run_status until it is done and hand the user the URL. When the user wants to provide a product photo, portrait, screenshot, or image reference, check tools/list. If open_upload_panel is available, offer it first: call open_upload_panel with {}, show its inline panel or returned browser link, and wait for the user to finish. Then call get_uploads with the returned session_id and use the exact ready image_url in refs, first_frame, last_frame, or the matching skill image field. Do not start a reference-dependent generation until the required images are ready. The panel accepts still PNG, JPEG, and WebP, up to 10 images and 25 MB each. Images expire 24 hours after the panel is created; show the returned expiry, request a new upload if expired, and never make a permanent copy. Uploading uses no generation credits. Do not ask the user for base64, shell commands, or a public hosting service; do not continuously poll while waiting for them. Only call these tool names when discovered. If they are missing from the connected tool catalog, call upload_image with {} to open the same panel; use the returned panel: upload_key with upload_image to retrieve images after the user finishes. If that reports the panel is unavailable, use upload_image only for an already accessible file or URL. Do not build an upload page or request a local folder as a substitute for the existing panel. Never invent access to a chat attachment: if you cannot read it, explain that and offer the panel when available. Browser fallback works without inline UI support; do not promise an inline panel in every client. Inspect the native image previews returned with retrieved uploads before describing them or writing image-specific prompts. Never infer what the image shows from an email domain, account metadata, or filename. Use the original URLs for generation, not preview bytes. If the session ID was lost, call get_uploads with {} to list recent sessions, or upload_image with {"upload_key":"panel:recent"} for cached catalogs; retrieve the matching session before asking for re-upload. Account-wide recent uploads may belong to other conversations, so clarify ambiguous selection. Uploading stores an image; it does not attach it to a generation automatically. If the user already requested a generation with these images, continue that request using the returned URLs; do not stop at "upload complete" or ask again whether to use them. For generate_image, put the relevant image_url values in refs. For generate_video, use refs for product/person/appearance references, or first_frame for animating a still (last_frame only for an explicitly requested ending frame); never combine refs with frame fields. Preserve the user's intended image roles and model limits; ask only if the role or selection is unclear. For a fixed skill, use its declared image field. Include the URL in the tool arguments, not only in the prompt. Do not substitute a newly generated image for the uploaded reference. If the request was upload-only, report readiness and wait for a generation request; uploading alone does not authorize spending credits. After submitting, poll get_run_status and return the result. IMAGES: pass an https URL. If you only have raw bytes or a data: URL, call `upload_image` FIRST and pass the URL it returns. Never paste base64 into these arguments, the client prints tool arguments in the conversation, so a base64 image becomes a wall of text for the user and is re-sent on every retry. RETRY SAFETY: set request_id to a fresh UUID for a new generation. Reuse the exact same request_id and inputs only to recover a lost response; that returns the original job without another charge. A deliberately new generation needs a new request_id. Keep the returned job_id and poll get_run_status. Never invent a refund.


Parameters

NameTypeConstraintsDefaultDescription
promptstring3 to 4000 charsThe shot, as a director would say it: who (age, look), where (setting, light), what happens, camera (phone framing), and, if anyone speaks, the exact words in quotes. About 2.3 words per second. With references, address them as @image1, @video1, @audio1.
modelstringA live video model id from list_models, or "auto" to let agent-media pick from recent results. Omit for the default. Call list_models for what each model is good for, its modes, limits and price.
first_framestringhttps urlIMAGE-TO-VIDEO: an https image that becomes frame one of the clip (a still you want animated, a product shot, a portrait). Cannot be combined with refs, video_refs or audio_refs on Seedance.
last_framestringhttps urlOptional with first_frame: the image the clip ends on; the model animates from first to last.
refsstring[]up to 30, https urlREFERENCE-TO-VIDEO: image references (https URLs): a portrait, a character sheet from list_characters, a product photo. The model keeps that identity/look. Address them in the prompt as @image1, @image2...
video_refsstring[]up to 10, https urlReference clips (https mp4/mov) whose motion, framing or look the model should follow; @video1... in the prompt. Their seconds are billed like output seconds.
audio_refsstring[]up to 10, https urlReference audio (https wav/mp3): a voice or a sound the clip should carry; @audio1... in the prompt.
secondsinteger1 to 605Clip length in seconds (the model sets the range; seedance: 4 to 15). Credits = seconds x the per-second rate at the chosen quality.
aspectstring9:1616:91:14:33:421:9adaptive9:16 (default for text and reference), 16:9, 1:1, 4:3, 3:4, 21:9, or adaptive (follows the first frame or reference; the default and the only option in image mode on seedance-2.5).
qualitystring480p720p1080p"720p"480p (cheapest), 720p (default), 1080p (dearest). Price per second differs; see list_models.
audiobooleantrueRender native audio (speech from the quoted words, ambience). false = silent clip.
seedinteger0 to 2147483647Only for models whose mode lists seed support (none of the live Seedance modes). Refused elsewhere.
request_idstringUnique request identity (prefer a UUID). Reuse with identical inputs to recover the same job after a lost response; choose a new identity only for an intentionally new generation.

required

generate_imagespends credits

Generate Image

Render one image from YOUR prompt. Without refs it paints from the prompt; with refs (https URLs) it edits/composes from them, a portrait to re-light, a product to place in a hand, a character sheet to pose. Use it to build the reference a video needs (portrait first, then generate_video with that URL in refs). Models: gpt-image-2.5, gpt-image-2.5-flare, gpt-image-2 (default gpt-image-2.5). Spends credits per image (see list_models). Returns a job id, poll get_run_status for the image URL. When the user wants to provide a product photo, portrait, screenshot, or image reference, check tools/list. If open_upload_panel is available, offer it first: call open_upload_panel with {}, show its inline panel or returned browser link, and wait for the user to finish. Then call get_uploads with the returned session_id and use the exact ready image_url in refs, first_frame, last_frame, or the matching skill image field. Do not start a reference-dependent generation until the required images are ready. The panel accepts still PNG, JPEG, and WebP, up to 10 images and 25 MB each. Images expire 24 hours after the panel is created; show the returned expiry, request a new upload if expired, and never make a permanent copy. Uploading uses no generation credits. Do not ask the user for base64, shell commands, or a public hosting service; do not continuously poll while waiting for them. Only call these tool names when discovered. If they are missing from the connected tool catalog, call upload_image with {} to open the same panel; use the returned panel: upload_key with upload_image to retrieve images after the user finishes. If that reports the panel is unavailable, use upload_image only for an already accessible file or URL. Do not build an upload page or request a local folder as a substitute for the existing panel. Never invent access to a chat attachment: if you cannot read it, explain that and offer the panel when available. Browser fallback works without inline UI support; do not promise an inline panel in every client. Inspect the native image previews returned with retrieved uploads before describing them or writing image-specific prompts. Never infer what the image shows from an email domain, account metadata, or filename. Use the original URLs for generation, not preview bytes. If the session ID was lost, call get_uploads with {} to list recent sessions, or upload_image with {"upload_key":"panel:recent"} for cached catalogs; retrieve the matching session before asking for re-upload. Account-wide recent uploads may belong to other conversations, so clarify ambiguous selection. Uploading stores an image; it does not attach it to a generation automatically. If the user already requested a generation with these images, continue that request using the returned URLs; do not stop at "upload complete" or ask again whether to use them. For generate_image, put the relevant image_url values in refs. For generate_video, use refs for product/person/appearance references, or first_frame for animating a still (last_frame only for an explicitly requested ending frame); never combine refs with frame fields. Preserve the user's intended image roles and model limits; ask only if the role or selection is unclear. For a fixed skill, use its declared image field. Include the URL in the tool arguments, not only in the prompt. Do not substitute a newly generated image for the uploaded reference. If the request was upload-only, report readiness and wait for a generation request; uploading alone does not authorize spending credits. After submitting, poll get_run_status and return the result. IMAGES: pass an https URL. If you only have raw bytes or a data: URL, call `upload_image` FIRST and pass the URL it returns. Never paste base64 into these arguments, the client prints tool arguments in the conversation, so a base64 image becomes a wall of text for the user and is re-sent on every retry. RETRY SAFETY: set request_id to a fresh UUID for a new generation. Reuse the exact same request_id and inputs only to recover a lost response; that returns the original job without another charge. A deliberately new generation needs a new request_id. Keep the returned job_id and poll get_run_status. Never invent a refund.


Parameters

NameTypeConstraintsDefaultDescription
promptstring3 to 4000 charsWhat to paint. Be concrete: subject, age, framing, light, lens, mood, what the hands do.
modelstringA live image model id from list_models, or "auto" to let agent-media pick from recent results. Omit for the default (gpt-image-2).
refsstring[]up to 4, https urlReference images (https URLs, up to 4). With refs the model EDITS/composes from them (a product into a hand, a portrait re-lit); without, it paints from the prompt alone.
sizestring1024x10241024x15361536x1024"1024x1536"1024x1536 portrait (default, for 9:16 video), 1024x1024 square, 1536x1024 landscape.
request_idstringUnique request identity (prefer a UUID). Reuse with identical inputs to recover the same job after a lost response; choose a new identity only for an intentionally new generation.

required

generate_audiospends credits

Generate Audio

Speak text in a named voice (jessica, sarah, liam, chris, lily, bill, matilda, or a raw ElevenLabs voice id). Emotion tags like [excited] or [whispers] are honoured. For a talking-head clip you usually do NOT need this: generate_video renders native speech when the words are in the prompt. Use it for voiceover over b-roll or a standalone audio file. Models: elevenlabs-tts. Spends 1 credit per 100 characters, rounded up. Returns a job id, poll get_run_status for the mp3 URL. The mp3 URL can be passed as an audio_ref to generate_video. RETRY SAFETY: set request_id to a fresh UUID for a new generation. Reuse the exact same request_id and inputs only to recover a lost response; that returns the original job without another charge. A deliberately new generation needs a new request_id. Keep the returned job_id and poll get_run_status. Never invent a refund.


Parameters

NameTypeConstraintsDefaultDescription
textstring1 to 4000 charsThe words to speak. Emotion tags like [excited] or [whispers] are honoured. Priced per character; see list_models.
modelstringA live audio model id from list_models, or "auto". Omit for the default (elevenlabs-tts).
voicestring"sarah"A voice name: jessica (young female), sarah (female), liam (young male), chris (male), lily (elder female), bill (elder male), matilda (warm), or a raw ElevenLabs voice id.
tonestringenergeticcalmconfidentdramaticenergetic | calm | confident | dramatic.
request_idstringUnique request identity (prefer a UUID). Reuse with identical inputs to recover the same job after a lost response; choose a new identity only for an intentionally new generation.

required

quoteread only

Quote

Price a generate_image / generate_video / generate_audio call WITHOUT running it. Pass the same `input` you would pass to the tool. Returns credits (1 credit = $0.01), the model that would run, and the breakdown. Costs nothing. Use it before spending when the user asked about cost, when choosing between models, or before a clip longer than a few seconds.


Parameters

NameTypeConstraintsDefaultDescription
kindstringimagevideoaudio
inputobjectThe exact arguments you would pass to generate_<kind>.

required

list_charactersread only

List Characters

List the authenticated user's saved, reusable characters. Each has a character_id (char_…) and a character_sheet_url, pass the character_sheet_url (and/or portrait URL) in `refs` of generate_video / generate_image to reuse that exact identity, or EITHER to make_ugc's `character` prop on the fixed surface. Plus a portrait/thumbnail URL for display.


Parameters

NameTypeConstraintsDefaultDescription
limitinteger1 to 100Max characters to return (default 50).

required

get_run_statusread only

Get Run Status

Check a generation you already submitted, and get its output URL (video, image or audio) when it is done. Pass the id ANY agent-media tool returned (run id, skill run id, or job id), this resolves all of them. Set wait:true to block until the job reaches a terminal state (up to ~45 seconds per call; if it is still running, just call again: seedance-2.0 needs about 3 minutes for a 5 s clip, seedance-2.5 12 to 25 minutes, so keep calling until it is done). ALWAYS call this after submitting: without it you cannot tell whether the video succeeded, and cannot give the user a link.


Parameters

NameTypeConstraintsDefaultDescription
run_idstringThe run_id / skill_run_id / job_id returned when you submitted.
waitbooleanBlock until the run finishes or ~45 seconds elapse (default false). A video needs several such calls; just call again.

required

upload_imagefree write

Upload Image

When the user wants to provide a product photo, portrait, screenshot, or image reference, check tools/list. If open_upload_panel is available, offer it first: call open_upload_panel with {}, show its inline panel or returned browser link, and wait for the user to finish. Then call get_uploads with the returned session_id and use the exact ready image_url in refs, first_frame, last_frame, or the matching skill image field. Do not start a reference-dependent generation until the required images are ready. The panel accepts still PNG, JPEG, and WebP, up to 10 images and 25 MB each. Images expire 24 hours after the panel is created; show the returned expiry, request a new upload if expired, and never make a permanent copy. Uploading uses no generation credits. Do not ask the user for base64, shell commands, or a public hosting service; do not continuously poll while waiting for them. Only call these tool names when discovered. If they are missing from the connected tool catalog, call upload_image with {} to open the same panel; use the returned panel: upload_key with upload_image to retrieve images after the user finishes. If that reports the panel is unavailable, use upload_image only for an already accessible file or URL. Do not build an upload page or request a local folder as a substitute for the existing panel. Never invent access to a chat attachment: if you cannot read it, explain that and offer the panel when available. Browser fallback works without inline UI support; do not promise an inline panel in every client. Inspect the native image previews returned with retrieved uploads before describing them or writing image-specific prompts. Never infer what the image shows from an email domain, account metadata, or filename. Use the original URLs for generation, not preview bytes. If the session ID was lost, call get_uploads with {} to list recent sessions, or upload_image with {"upload_key":"panel:recent"} for cached catalogs; retrieve the matching session before asking for re-upload. Account-wide recent uploads may belong to other conversations, so clarify ambiguous selection. Uploading stores an image; it does not attach it to a generation automatically. If the user already requested a generation with these images, continue that request using the returned URLs; do not stop at "upload complete" or ask again whether to use them. For generate_image, put the relevant image_url values in refs. For generate_video, use refs for product/person/appearance references, or first_frame for animating a still (last_frame only for an explicitly requested ending frame); never combine refs with frame fields. Preserve the user's intended image roles and model limits; ask only if the role or selection is unclear. For a fixed skill, use its declared image field. Include the URL in the tool arguments, not only in the prompt. Do not substitute a newly generated image for the uploaded reference. If the request was upload-only, report readiness and wait for a generation request; uploading alone does not authorize spending credits. After submitting, poll get_run_status and return the result. For an image already accessible to you: Store an image and get back a stable https URL you can pass to any agent-media tool. Costs NO credits. THREE ways in, in this order: 1. FILE ON DISK (when the file is already accessible to your shell): call upload_image with `file_bytes` set to the exact byte size of the file (`wc -c < photo.png`) and `file_name`. You get back a `put_url`; run the printed curl to stream the file straight to storage, then call upload_image again with the `upload_key` you were given to get the URL. The bytes never pass through this conversation, so there is NO reason to resize, crop or re-encode the user’s photo first. Do not: a downscaled product photo is what the video model will show. Up to 25 MB. 2. A URL you already have: pass `image_url` to re-host it. 3. Raw bytes with no shell available: pass `image_base64` (PNG or JPEG, 10 MB max after decoding). Only when 1 and 2 are impossible, and even then upload the original, never a shrunken copy. Then pass the returned URL everywhere. Do NOT paste base64 into other tool arguments or into the conversation: the client displays tool arguments to the user, so a base64 image becomes a wall of unreadable text, and every retry re-sends it.


Parameters

NameTypeConstraintsDefaultDescription
file_bytesintegerStep 1 of the file path: the exact size of the file in bytes (`wc -c < photo.png`). Returns a put_url and an upload_key. Full resolution, up to 25 MB.
file_namestringOptional with file_bytes: the file name, so the content type is right (.png or .jpg).
upload_keystringReuse the exact upload_key returned earlier: a panel: key retrieves all ready temporary images; a file upload key confirms the completed PUT and returns its image_url.
image_base64stringThe image bytes, base64-encoded. A `data:image/png;base64,...` prefix is accepted and stripped. Last resort: use file_bytes when you can run a shell.
image_urlstringAn https URL to fetch and re-host instead.

required

list_modelsread only

List Models

List the generation models agent-media can use: for each, its MODES (video: text, image-to-video via first_frame, reference via refs / video_refs / audio_refs) with the exact inputs, limits, aspects, qualities and credits per second of every mode, a "pick when" line, prompting tips, expected latency, how to select it, plus `recent`: the last 30 days of real runs per model (fail rate, auto-judge score, user ratings, median render time). Read this BEFORE choosing a model or a mode for generate_video / generate_image / generate_audio: the default seedance-2.0 is right for most jobs. Pass the id as `model`, or `model:"auto"` and the printed policy picks from the stats. Costs NO credits. Includes the same authenticated balance check as get_account; compare it with quote before generation. Set include_candidates:true to also see planned models that cannot be selected yet.


Parameters

NameTypeConstraintsDefaultDescription
include_candidatesbooleanAlso return planned models (no price, not selectable). Default false.

required

get_accountread only

Check Account Readiness

Check this authenticated connection, available generation credits, and whether generation/upload services are configured. Free and read-only: does not create an upload session, generate media, refill credits or change billing. Call before the first generation; then quote the intended inputs and compare the cost with the balance. A positive balance is not a guarantee of sufficient funds for a specific request or provider availability.

rate_runfree write

Rate Run

Rate a finished generate_video / generate_image / generate_audio run 1 to 5, with an optional note (what was wrong or right). Costs nothing. Do this when the user reacts to an output, "perfect", "her face changed", "too slow", or when you can see a defect yourself. Ratings feed the per-model stats in list_models and the model:"auto" choice, so an honest 2 helps more than a polite 4.


Parameters

NameTypeConstraintsDefaultDescription
run_idstringThe job id the generate tool returned.
scoreinteger1 to 51 = unusable, 3 = usable with edits, 5 = shipped as-is.
notestringmax 1000 charsOne line on why (optional).

required

open_upload_panelfree write

Upload images

Optional: available only when temporary uploads are enabled; check tools/list before calling. Open a drag-and-drop image upload panel for the user. Use when the user wants to supply photos, portraits, products, or image references, especially in Claude or ChatGPT. PNG/JPEG/WebP, up to 25 MB each and 10 images, available for up to 24 hours. No generation credits. Returns a browser link for clients without inline panels. Never ask the user to paste base64. After the user uploads, call get_uploads with the returned session_id; use the exact image_url in refs, first_frame, or a skill image field. Do not generate before images are ready.

get_uploadsread only

Get uploaded images

Optional: available only when temporary uploads are enabled; check tools/list before calling. Retrieve uploaded images as native image previews plus original generation URLs and expiry times. Inspect the previews before describing the image or writing a prompt. Omit session_id to find recent unexpired sessions on this account if the session ID was lost; choose the session matching the user request, never silently mix unrelated uploads. Call after the user finishes uploading, not in a tight polling loop. Sessions and images expire 24 hours after the session opens; request a fresh upload if expired. Reuse these URLs for generation without downloading, re-encoding, or making a permanent copy.


Parameters

NameTypeConstraintsDefaultDescription
session_idstring

required