Image generation telemetry: eight models, two cloud GPUs, 506 images

By Edward Grundy · 16 September 2026

Eight open-weight text-to-image models, 506 completed requests, two rented AWS GPU instances, all run through Marigold. Each request recorded inference time, VRAM allocated and peak, mean and peak power draw, and model load time.

Per-step inference time drives the time and cost figures. It is linear in step count, spans 0.008 to 2.534 s/step across the set (over 300-fold), follows architecture more closely than parameter count, and is unchanged by prompt length. Classifier-free guidance multiplies it by 1.7 to 2.1 on the A10G and by 1.0 to 1.5 on the RTX PRO 6000. Cost per image is that time multiplied by an hourly instance rate, and runs from 0.00026 to 0.036 USD.

Setup

Two instances, both in eu-west-2, both running the same Marigold 0.6.10 Docker Compose stack: Postgres for the queue, one worker process, model weights on an EBS volume that survives the instance (the benchmark and the Terraform are public; links at the end).

g5.4xlarge    NVIDIA A10G             22.06 GiB usable VRAM, 64 GiB host
g7e.4xlarge   NVIDIA RTX PRO 6000     96 GiB VRAM, 128 GiB host

The A10G run completed 200 requests across four models. The RTX PRO 6000 run completed 306 across six.

The model cache is an EBS volume attached to whichever instance is running. It held 96 GB of weights for the first run and was reused by the second, so load times below are warm-cache figures.

Methodology

Each model receives the same parameter sweep, submitted serially through the HTTP API and polled to completion. The worker holds one model resident at a time and switches on queue depth, so requests are grouped by model and never interleaved.

steps      1, 4, 8, 16, 25, 40, 100, 200 (set varies by run)
guidance   0.0, 1.0, 3.5, 7.5, 15.0
prompt     8, 30, 90 and 220 CLIP tokens (second run only)
seed       fixed at 20260910 throughout
resolution each model's native default

One warm-up request per model is recorded and excluded from the figures below. The first request after a load ran up to 75 times slower than later ones on one model and no slower on another.

Five repeats of each cell on the first run gave a median coefficient of variation of 0.02 per cent, and the mean of the first three matched the mean of all five with a median error of zero. Inference time on a dedicated GPU with a fixed seed is repeatable to three decimal places.

Models

Weights are the loaded footprint reported by the handler. On disk, FLUX occupies 33.7 GB and z-image 32.9 GB. z-image-turbo returned blank images throughout, so its figures describe the computation as run.

Per-step cost

Inference time is linear in step count across the whole sweep, with a fixed overhead per request. Fitting across every step value at guidance 7.5:

The fixed term is the VAE decode: 0.62 to 0.76 seconds on the A10G for all four models, and 0.01 to 0.17 on the larger card. It puts a floor under the cost of short runs.

Sana 1600M is the only model measured identically on both cards. It takes 0.157 s/step on the A10G and 0.032 on the RTX PRO 6000, a factor of 4.9.

Model size and per-step cost

Sana 600M and Sana 1600M share an architecture, a text encoder and a native resolution. On the same card:

sana_600m     1.18 GB weights    0.0259 s/step
sana_1600m    3.21 GB weights    0.0316 s/step

The larger model has 2.7 times the weights and costs 1.23 times as much per step. Memory follows the same pattern, at 9.10 GB allocated against 7.07, a ratio of 1.29.

Across families the relationship weakens further. z-image-turbo carries 12.31 GB of weights and runs at 2.534 s/step. FLUX carries 5.95 GB quantised and runs at 1.675. SDXL, at 5.13 GB, runs at 0.289, nearly six times faster than FLUX at a similar loaded size. Per-step time in this set follows architecture more closely than weight size.

Seconds per step against loaded weight size at guidance 7.5, one panel per card. The two 1.73 GB models on the RTX PRO 6000 coincide.

Classifier-free guidance cost depends on the card

Guidance above a threshold runs two forward passes per step on a doubled batch, which predicts a doubling of inference time. On the A10G the ratio falls between 1.74 and 2.08 for the three models that use it:

On the RTX PRO 6000 the ratio is close to 1.0 for every model except Sana 1600M:

Inference time at guidance 7.5 divided by inference time at guidance 0.0, 25 steps. RTX is the RTX PRO 6000.

Sana 1600M is the one model present in both runs at the same resolution with the same request: 1.83 on the A10G, 1.52 on the RTX PRO 6000. The doubled batch costs less when the card has capacity to process both halves concurrently.

Sana 600M on the larger card shows 1.06. A smaller model occupies less of the card, so the second pass has more room. The three 512px models, run only on the larger card, show 0.99 to 1.02.

On the A10G a guidance setting roughly doubles inference time. On the RTX PRO 6000 it adds between 0 and 52 per cent.

FLUX.1-schnell is flat at 1.00 across all five guidance values on both cards (it is a distilled model and runs a single forward pass per step).

Prompt length

Four prompt lengths at 25 steps and guidance 7.5, from a single sentence to a paragraph of roughly 220 CLIP tokens (the CLIP encoders truncate at 77 tokens, so the 90 and 220 token prompts reach those models cut short):

The spread is at most 1.1 per cent. CLIP-based models were expected to be flat because CLIP truncates, and the T5 and Gemma encoders to scale with input. FLUX uses T5 and shows a spread of 0.0 per cent. Sana uses Gemma and shows 0.1 to 1.0.

The text encoder runs once per request against a denoising loop of 25 iterations, and even a T5-XXL pass disappears into that (in LLM inference, prompt length drives both time and memory).

Memory

Figures in GB. Peak is the allocator high-water mark during inference.

Allocation follows the model and the request wherever there is headroom. Sana 1600M allocates 9.10 GB on the 96 GB card and 9.07 on the 22 GB card, and FLUX peaks at 16.03 on both.

z-image-turbo peaks at 23.09 GB against 22.06 GiB usable, with allocator warnings throughout, and is the slowest model in the set at 2.534 s/step.

FLUX in fp16 does not fit the A10G. The load reached 21.80 GiB allocated against 22.06 GiB usable and stopped 108 MiB short, about half a per cent. The A10G figures above use NF4 quantisation, which brings the loaded footprint to 5.95 GB and the peak to 16.03.

Power and energy

Energy per image is at 25 steps, guidance 7.5.

Mean power on the A10G stays between 185 and 240 W across four models of very different sizes, with no peak above 273 W on a 300 W card. That agrees with the guidance results: these workloads occupy the card fully. On the RTX PRO 6000 mean power spans 98 to 498 W and rises with model size, because nothing in this set fills that card.

Mean power depends on run length. A short request ends before the card reaches steady state, which lowers the mean. The 0.21 s sdxs request is one of these.

Load time

Warm cache, weights already on the EBS volume:

Load time is reported once per loaded model and copied to every request that model serves, so it cannot be averaged into a per-image figure.

The worker holds one model at a time and unloads on switching. A mixed-model workload pays the load time on every switch: the four A10G models sum to just over six minutes, and the six RTX PRO 6000 models to just under two. A deployment serving one model pays once.

FLUX loaded in 8.1 seconds on the second run against 128.8 on the first. The difference is largely host page cache (the second run had the weights in memory from a preceding request).

Cost per image

At 25 steps and guidance 7.5, assuming continuous utilisation, at approximate on-demand eu-west-2 rates of 2.03 USD per hour for the g5.4xlarge and 4.50 USD for the g7e.4xlarge:

Instance cost per image at 25 steps and guidance 7.5, assuming continuous utilisation. RTX is the RTX PRO 6000.

Cost per image scales with the inverse of utilisation. At 10 per cent utilisation each figure is ten times higher.

FLUX takes 42.46 seconds at 25 steps on the A10G (NF4) and 8.22 on the RTX PRO 6000 (fp16), a factor of 5.2. The RTX PRO 6000 instance costs about 2.2 times as much per hour, so each image costs 43 per cent as much (0.01027 against 0.02394 USD).

Summary

Per-step inference time is linear in step count and repeatable to three decimal places, so two points on a card recover it. It spans 0.008 to 2.534 s/step across the set, and the Sana pair gives 2.7 times the weights for 1.23 times the per-step time. Prompt length changes inference time by at most 1.1 per cent across a 27-fold range of tokens. Guidance multiplies inference time by 1.7 to 2.1 on the A10G and by 1.0 to 1.5 on the RTX PRO 6000, with FLUX at 1.0 on both. At 25 steps and guidance 7.5, cost per image runs from 0.00026 to 0.036 USD at continuous utilisation.

Raw data

[
 {"model":"black-forest-labs/flux.1-schnell","hw":"a10g","s_per_step":1.6745,"fixed_s":0.631,"it_per_s":0.6,"weights_gb":5.95,"vram_gb":13.44,"vram_peak_gb":16.0,"w_mean":239.5,"w_peak":251.1,"load_s":128.8,"cfg_ratio":1.0,"s_25step":42.46,"mwh_per_image":2868.32},
 {"model":"efficient-large-model/sana_1600m_1024px_diffusers","hw":"a10g","s_per_step":0.1569,"fixed_s":0.759,"it_per_s":6.37,"weights_gb":3.21,"vram_gb":9.07,"vram_peak_gb":10.82,"w_mean":214.4,"w_peak":262.6,"load_s":51.4,"cfg_ratio":1.83,"s_25step":4.4,"mwh_per_image":294.2},
 {"model":"stabilityai/stable-diffusion-xl-base-1.0","hw":"a10g","s_per_step":0.2888,"fixed_s":0.637,"it_per_s":3.46,"weights_gb":5.13,"vram_gb":6.95,"vram_peak_gb":11.14,"w_mean":236.6,"w_peak":272.7,"load_s":53.0,"cfg_ratio":1.74,"s_25step":7.85,"mwh_per_image":554.31},
 {"model":"tongyi-mai/z-image-turbo","hw":"a10g","s_per_step":2.5343,"fixed_s":0.621,"it_per_s":0.39,"weights_gb":12.31,"vram_gb":20.53,"vram_peak_gb":23.09,"w_mean":185.7,"w_peak":200.4,"load_s":128.2,"cfg_ratio":2.08,"s_25step":63.95,"mwh_per_image":3304.4},
 {"model":"arrexel/pattern-diffusion","hw":"rtxpro6000","s_per_step":0.0173,"fixed_s":0.036,"it_per_s":57.75,"weights_gb":1.73,"vram_gb":2.61,"vram_peak_gb":3.25,"w_mean":141.9,"w_peak":386.9,"load_s":2.2,"cfg_ratio":0.99,"s_25step":0.47,"mwh_per_image":18.82},
 {"model":"black-forest-labs/flux.1-schnell","hw":"rtxpro6000","s_per_step":0.322,"fixed_s":0.17,"it_per_s":3.11,"weights_gb":5.95,"vram_gb":13.47,"vram_peak_gb":16.03,"w_mean":498.4,"w_peak":574.3,"load_s":8.1,"cfg_ratio":1.0,"s_25step":8.22,"mwh_per_image":1242.89},
 {"model":"dream-textures/texture-diffusion","hw":"rtxpro6000","s_per_step":0.0174,"fixed_s":0.037,"it_per_s":57.47,"weights_gb":1.73,"vram_gb":2.61,"vram_peak_gb":3.25,"w_mean":150.1,"w_peak":388.9,"load_s":0.7,"cfg_ratio":1.02,"s_25step":0.47,"mwh_per_image":22.4},
 {"model":"efficient-large-model/sana_1600m_1024px_diffusers","hw":"rtxpro6000","s_per_step":0.0316,"fixed_s":0.124,"it_per_s":31.65,"weights_gb":3.21,"vram_gb":9.1,"vram_peak_gb":10.85,"w_mean":282.1,"w_peak":572.6,"load_s":50.5,"cfg_ratio":1.52,"s_25step":0.91,"mwh_per_image":96.14},
 {"model":"efficient-large-model/sana_600m_1024px_diffusers","hw":"rtxpro6000","s_per_step":0.0259,"fixed_s":0.128,"it_per_s":38.54,"weights_gb":1.18,"vram_gb":7.07,"vram_peak_gb":8.82,"w_mean":188.4,"w_peak":384.7,"load_s":34.6,"cfg_ratio":1.06,"s_25step":0.78,"mwh_per_image":49.59},
 {"model":"idkiro/sdxs-512-dreamshaper","hw":"rtxpro6000","s_per_step":0.0078,"fixed_s":0.012,"it_per_s":127.84,"weights_gb":0.63,"vram_gb":0.92,"vram_peak_gb":1.08,"w_mean":98.0,"w_peak":232.4,"load_s":5.9,"cfg_ratio":1.01,"s_25step":0.21,"mwh_per_image":7.88}
]

Fields: s_per_step and fixed_s from a linear fit over the step sweep at guidance 7.5; cfg_ratio is inference at guidance 7.5 over guidance 0.0 at 25 steps; mwh_per_image is mean power times duration at 25 steps, guidance 7.5.

Reproducing the run

The benchmark is a Marigold application: a models.yaml declaring the catalogue, a marigold.toml declaring the deployment, a prompt builder and a runner.

git clone https://github.com/bayinfosys/marigold-examples
marigold cache populate marigold-examples/benchmark-image-gen
marigold deployment start marigold-examples/benchmark-image-gen
python3 marigold-examples/benchmark-image-gen/run_benchmark.py \
    --base-url http://localhost:8000 --user-id benchmark \
    --out results/run1.csv

The runner reads the model list from the deployment’s catalogue, sends each parameter set to each model, polls to completion and flattens the response into a row. Sweeps and repeats live in the prompt file, and the analysis groups on the label.

The Terraform for the two instances, including the model cache volume that outlives them, is at github.com/bayinfosys/terraform-aws-marigold.