Image Editing (Img2Img, Inpainting, Upscaling)

Edit, upscale, and create variations of existing images across multiple providers with a unified API.

Overview
editImage() API
upscaleImage() API
variateImage() API
Provider Matrix
Img2Img (Style Transfer)
Inpainting
Outpainting
Upscaling
Options Reference
Local Setup (A1111 & ComfyUI)
Custom Provider
Related Documentation

Overview

AgentOS provides three image editing APIs that work across all supported providers. Unlike Image Generation which creates from scratch, these APIs operate on existing images:

API	Purpose
`editImage()`	Img2img, inpainting, outpainting — modify an existing image
`upscaleImage()`	Super-resolution: 2x or 4x upscale with detail enhancement
`variateImage()`	Create N variations of an image while preserving composition

All three accept a Buffer, Uint8Array, or file path as input and return the same ImageResult shape used by generateImage().

editImage() API

import { editImage } from '@framers/agentos';

const result = await editImage({
  // Required
  image: imageBuffer,        // Buffer | Uint8Array | string (file path)
  prompt: 'Make it a sunset scene with warm golden lighting',

  // Optional
  provider: 'openai',        // openai | stability | replicate | stable-diffusion-local
  model: 'gpt-image-1',     // Provider-specific model override
  mask: maskBuffer,          // Mask for inpainting (white = edit, black = keep)
  strength: 0.75,            // How much to transform (0.0 = identical, 1.0 = full regeneration)
  size: '1024x1024',         // Output dimensions
  negativePrompt: 'blurry, low quality',  // What to avoid (Stability, SD Local)
  seed: 42,                  // Reproducible output (provider-dependent)
  output: 'base64',          // 'base64' | 'url' (default varies by provider)
});

// Result shape
console.log(result.images[0].base64);   // Base64-encoded image data
console.log(result.images[0].url);      // Temporary URL (cloud providers)
console.log(result.provider);            // Which provider was used
console.log(result.model);              // Which model was used
console.log(result.usage);              // Token/cost tracking

Strength Parameter

The strength parameter controls the balance between the source image and the prompt. It is supported by all providers, though the exact behavior varies:

Strength	Behavior
`0.0`	Identical to input (no transformation)
`0.1–0.3`	Subtle adjustments — color grading, minor touch-ups
`0.4–0.6`	Moderate changes — style transfer, lighting changes
`0.7–0.9`	Major transformation — composition preserved, content regenerated
`1.0`	Full regeneration guided by prompt (source used only for composition)

upscaleImage() API

import { upscaleImage } from '@framers/agentos';

const result = await upscaleImage({
  // Required
  image: imageBuffer,

  // Optional
  provider: 'stability',     // stability | replicate | stable-diffusion-local
  scale: 4,                  // 2 | 4 (default: 2)
  model: 'esrgan-v1-x2plus', // Provider-specific upscale model
  output: 'base64',
});

// 4x upscaled image
const upscaled = result.images[0];
console.log(`Upscaled to ${upscaled.width}x${upscaled.height}`);

Upscale Models by Provider

Provider	Models	Max Scale
Stability AI	`esrgan-v1-x2plus`, `stable-diffusion-x4-latent-upscaler`	4x
Replicate	`real-esrgan`, `swinir`	4x
Local SD (A1111)	`ESRGAN_4x`, `R-ESRGAN 4x+`, `SwinIR_4x`	4x
Local SD (ComfyUI)	Any upscale model loaded in your workflow	4x

variateImage() API

import { variateImage } from '@framers/agentos';

const result = await variateImage({
  // Required
  image: imageBuffer,

  // Optional
  provider: 'openai',
  n: 3,                       // Number of variations (default: 1, max varies by provider)
  size: '1024x1024',
  strength: 0.6,              // How different each variation should be
});

// Multiple variations returned
for (const variant of result.images) {
  console.log(variant.url || `base64: ${variant.base64?.length} chars`);
}

Provider Matrix

Feature	OpenAI	Stability AI	Replicate	Local SD (A1111)	Local SD (ComfyUI)
Env Var	`OPENAI_API_KEY`	`STABILITY_API_KEY`	`REPLICATE_API_TOKEN`	`STABLE_DIFFUSION_LOCAL_BASE_URL`	`STABLE_DIFFUSION_LOCAL_BASE_URL`
Img2Img	Yes	Yes	Yes	Yes	Yes
Inpainting	Yes	Yes	Yes	Yes	Yes
Outpainting	Yes	Yes	Via model	Yes	Yes
Upscaling	No	Yes	Yes	Yes	Yes
Variations	Yes	Yes	Yes	No	No
Strength	Yes	Yes	Yes	Yes	Yes
Negative Prompt	No	Yes	Model-dependent	Yes	Yes
Seed	No	Yes	Yes	Yes	Yes
Cost Tier	$$$	$$	$$	Free	Free
Latency	~3–8s	~3–6s	~5–15s	~2–10s	~2–10s

Img2Img (Style Transfer)

Transform the style of an image while preserving its composition:

import { editImage } from '@framers/agentos';
import { readFileSync } from 'node:fs';

const photo = readFileSync('./photo.jpg');

// Convert a photograph to oil painting style
const oilPainting = await editImage({
  image: photo,
  prompt: 'Oil painting in the style of Monet, impressionist brushstrokes, warm palette',
  strength: 0.65,
  provider: 'stability',
});

// Convert to anime style
const anime = await editImage({
  image: photo,
  prompt: 'Anime illustration, Studio Ghibli style, vibrant colors',
  strength: 0.7,
  provider: 'stability',
});

Inpainting

Edit specific regions of an image using a mask:

import { editImage } from '@framers/agentos';
import { readFileSync } from 'node:fs';

const image = readFileSync('./room.jpg');
const mask = readFileSync('./mask.png');  // White = area to edit

// Replace the masked area with new content
const result = await editImage({
  image,
  mask,
  prompt: 'A large bookshelf filled with colorful books',
  strength: 0.9,
  provider: 'openai',
});

Mask format: PNG with the same dimensions as the source image. White pixels (#FFFFFF) mark the area to regenerate; black pixels (#000000) mark areas to preserve. Partial transparency (grayscale) controls blending at the boundary.

Outpainting

Extend an image beyond its original borders:

import { editImage } from '@framers/agentos';

// Extend the image to the right
const result = await editImage({
  image: originalBuffer,
  prompt: 'Continue the landscape with rolling hills and a distant village',
  // Outpainting is achieved by providing a larger canvas with the original
  // image placed at an offset, and a mask covering the new area
  mask: outpaintMask,     // White where the extension should go
  size: '1536x1024',      // Wider than the original
  provider: 'stability',
});

Upscaling

Increase image resolution with detail enhancement:

import { upscaleImage } from '@framers/agentos';
import { readFileSync, writeFileSync } from 'node:fs';

const lowRes = readFileSync('./thumbnail-256x256.jpg');

// 4x upscale: 256x256 -> 1024x1024
const result = await upscaleImage({
  image: lowRes,
  scale: 4,
  provider: 'stability',
});

if (result.images[0].base64) {
  writeFileSync('./upscaled-1024x1024.png', Buffer.from(result.images[0].base64, 'base64'));
}

Options Reference

editImage() Options

Option	Type	Default	Description
`image`	`Buffer \| Uint8Array \| string`	required	Source image (buffer or file path)
`prompt`	`string`	required	What to generate / how to transform
`provider`	`string`	Auto-detect	Image provider ID
`model`	`string`	Provider default	Model override
`mask`	`Buffer \| Uint8Array \| string`	—	Inpainting mask (white = edit area)
`strength`	`number`	`0.75`	Transformation strength (0.0–1.0)
`size`	`string`	`'1024x1024'`	Output dimensions (`WxH`)
`negativePrompt`	`string`	—	Content to avoid (Stability, SD Local)
`seed`	`number`	Random	Reproducibility seed
`output`	`'base64' \| 'url'`	Provider default	Return format

upscaleImage() Options

Option	Type	Default	Description
`image`	`Buffer \| Uint8Array \| string`	required	Source image
`provider`	`string`	Auto-detect	Upscale provider ID
`scale`	`2 \| 4`	`2`	Upscale factor
`model`	`string`	Provider default	Upscale model override
`output`	`'base64' \| 'url'`	Provider default	Return format

variateImage() Options

Option	Type	Default	Description
`image`	`Buffer \| Uint8Array \| string`	required	Source image
`provider`	`string`	Auto-detect	Provider ID
`n`	`number`	`1`	Number of variations (max varies by provider)
`size`	`string`	Original size	Output dimensions
`strength`	`number`	`0.6`	How different each variation should be
`output`	`'base64' \| 'url'`	Provider default	Return format

Local Setup (A1111 & ComfyUI)

Automatic1111 (A1111) Web UI

# Start the A1111 server with API enabled
cd stable-diffusion-webui
./webui.sh --api --listen

# Set environment variable
export STABLE_DIFFUSION_LOCAL_BASE_URL=http://localhost:7860

A1111 exposes /sdapi/v1/img2img and /sdapi/v1/extra-single-image endpoints. AgentOS calls these directly.

ComfyUI

# Start ComfyUI
cd ComfyUI
python main.py --listen

# Set environment variable
export STABLE_DIFFUSION_LOCAL_BASE_URL=http://localhost:8188
export STABLE_DIFFUSION_LOCAL_BACKEND=comfyui  # Tell AgentOS to use ComfyUI API

ComfyUI uses workflow-based execution. AgentOS ships with default img2img and upscale workflows. You can override them by placing custom workflow JSON files in ~/.wunderland/comfyui-workflows/.

Custom Provider

import { registerImageProvider } from '@framers/agentos';

registerImageProvider({
  id: 'my-provider',
  name: 'My Image Provider',
  capabilities: {
    edit: true,
    upscale: true,
    variate: false,
    inpaint: true,
  },

  async edit(request) {
    // Call your API
    const response = await fetch('https://my-api.com/edit', {
      method: 'POST',
      body: JSON.stringify({ image: request.image, prompt: request.prompt }),
    });
    const data = await response.json();
    return {
      images: [{ base64: data.result, width: 1024, height: 1024 }],
      provider: 'my-provider',
      model: 'custom-v1',
    };
  },

  async upscale(request) {
    // ...
  },
});

Image Generation — Generate images from text prompts
Multimodal RAG — Image + audio retrieval-augmented generation
Vision Pipeline — OCR and image understanding
High-Level API — Full API reference

Table of Contents​

Overview​

editImage() API​

Strength Parameter​

upscaleImage() API​

Upscale Models by Provider​

variateImage() API​

Provider Matrix​

Img2Img (Style Transfer)​

Inpainting​

Outpainting​

Upscaling​

Options Reference​

editImage() Options​

upscaleImage() Options​

variateImage() Options​

Local Setup (A1111 & ComfyUI)​

Automatic1111 (A1111) Web UI​

ComfyUI​

Custom Provider​

Related Documentation​

Table of Contents