Edge image generation on NVIDIA Jetson Orin Nano 8GB

Reference architecture. A single 8 GB edge device serves chat, text-to-image and image editing to multiple clients through OpenWebUI. llama-swap keeps one model GPU-resident at a time. Prompt encoding for the image models runs as a dedicated service on an Intel CPU host.

CLIENTS Client 1workstation Client 2laptop Client 3mobile OpenWebUI Front end Chat · image generation Image editing NVIDIA Jetson Orin Nano 8GB Edge inference node · 8 GB unified CPU/GPU memory llama-swap · model orchestration · :8080 Routes by model; one GPU-resident model at a time GPU-resident model None loaded SAM 3 · isolated process Subject mask, then exits intel Core i7 host Text-encoding service · LAN Qwen3-4B encoder :8090 · bf16 weights fp32 compute Serves Z-Image and klein Cache: empty ~8 GB host RAM Frees 2.5-8 GB on the Orin per image model
Orin unified memory utilisation (7.4 GB usable)
OS + CUDA runtime Model weights Peak working set Available (measured minimum)

Event log (animation accelerated; timings are measured)

    Measured performance · 1024x1024

    ModelStepsWarmWeightsAvailable (min)
    SANA-Sprint 0.6B2~5 s~3.1 GB~0.5 GB
    Z-Image-Turbo NF49~75 s3.46 GB1.45 GB
    FLUX.2-klein-4B NF4, edit445-58 s2.46 GB1.85 GB
    klein, subject preservation4~86 s2.46 GB1.5 GB

    Model switch from chat to an image model: ~47 s (SANA-Sprint) to ~116 s (Z-Image-Turbo), first image included. Switch back to chat: ~10 s. Image models are unloaded after 10 minutes idle.

    Edge node: JetPack 7.2, PyTorch 2.12.1 (CUDA 13.2), diffusers 0.39. Encoding host: Intel Core i7-13620H.

    FLUX.2-klein-4B NF4 · Remote text encoder · SAM 3 bf16 · SANA-Sprint encoder NF4 · Z-Image-Turbo bnb-4bit

    NVIDIA, Jetson and Intel are trademarks of their respective owners. Not affiliated.