Structured JSON prompts in GenAI.

Hi all.

When working with GenAI, freeform text prompts are fine for casual use, but structured workflows require more discipline. Using JSON prompts allows you to define tasks clearly, enforce rules, and produce outputs that can feed directly into other models.

In this example, we generate a cozy winter cabin scene across three models: Gemini (image), Veo (video), and Suno (music).

1) Image Generation - Gemini

{
  "task": "image_generation",
  "input": "Winter cabin in a snowy forest during blue hour, warm light
glowing from windows, soft snow falling",
  "requirements": {
    "goal": "Create a detailed scene description suitable for video
 and music generation",
    "rules": {
      "no_people_or_animals": true,
      "no_extra_locations": true
    },
    "quality": {
      "description_detail": "high, vivid, and atmospheric",
      "mood": "peaceful, serene, cozy",
      "style": "concise and clear, visually evocative",
      "view_angle": "wide, showing the cabin and surrounding forest",
      "lighting": "soft blue hour with warm window glow"
    }
  },
  "output_format": {
    "type": "text",
    "example": "A cozy wooden cabin sits quietly in a snowy forest. The soft 
blue light of dusk reflects off the snow, and warm light glows from the windows. 
Snowflakes gently fall, creating a peaceful and serene atmosphere. The scene is 
viewed from a wide angle, showing both the cabin and the surrounding forest."
  },
  "notes": "This description will serve as the base for Veo video generation and 
Suno music generation."
}

2) Video Generation - Veo

{
  "task": "video_generation",
  "input": "USE_OUTPUT(Gemini)",
  "requirements": {
    "goal": "Create a cinematic short video capturing a cozy winter cabin in 
a snowy forest at blue hour",
    "rules": {
      "no_people_or_animals": true,
      "composition": "maintain original from text description"
    },
    "quality": {
      "framing": "cinematic with dynamic but gentle camera pans",
      "view_angle": "wide, capturing cabin and surrounding forest",
      "lighting": "soft ambient lighting reflecting off snow, blue hour with 
warm window glow",
      "mood": "peaceful, serene, cozy",
      "style": "photorealistic with slight cinematic color grading and soft 
focus on snowflakes"
    }
  },
  "output_format": {
    "type": "video",
    "params": {
      "duration": "5s",
      "fps": 24,
      "resolution": "1920x1080"
    }
  },
  "notes": "The video should emphasize atmosphere: gentle snowfall, warm 
cabin light, and tranquil forest surroundings."
}

3) Music Generation - Suno

{
  "task": "music_generation",
  "input": "Winter cabin in a snowy forest video scene (from Veo output)",
  "requirements": {
    "goal": "Create a calming background soundtrack that matches the video scene",
    "rules": {
      "no_percussion": true
    },
    "quality": {
      "mood": "peaceful, serene, cozy",
      "style": "soft ambient, minimalistic, cinematic",
      "instruments": "piano, soft pads",
      "tempo": "slow and relaxing",
      "loop_friendly": true,
      "vocals": "none",
      "intro": "gentle fade-in, 2 seconds",
      "outro": "soft fade-out, 2 seconds"
    }
  },
  "output_format": {
    "type": "audio",
    "params": {
      "duration": "5s",
      "format": "wav",
      "sample_rate": "44.1kHz"
    }
  },
  "notes": "The music should reinforce the atmosphere of a cozy cabin with gentle 
snowfall and warm lighting."
}

Why This Approach Works

1. Clarity: each model has a clear task, rules, and expected output.

2. Consistency: structured key:value pairs ensure outputs are predictable.

3. Reusability: the same JSON structure works for different scenes; only input changes.

4. Automation-friendly: even if switching between GUI tools, the JSON serves as a single source of truth for prompts.

5. Enhanced control: keys like intro, outro, view_angle, vocals allow precise guidance without verbose text instructions.

💡 Using JSON prompts is like moving from freeform sketches to a blueprint: predictable, scalable, and fully structured for multi-model GenAI pipelines.

By the way, I tried JSON in the latest version of the remove background preset in Pixel AI Studio. The result is impressive.

Good luck.