UI Synthesis & Prompt Compiler Architecture

Spec-Driven UI Synthesis vs. Vision Models

A comparative benchmark evaluating frontend code generation. This project contrasts Direct Video-to-Code translation (using multimodal LLMs over 100k tokens of raw video) against a 3-Layer Declarative Specification Pipeline (~350 lines of YAML compiled into a structured prompt context).

Live Interactive Comparison

Interact with both generated outputs in real time below.

Target: 16-Color EGA Retro UI

EXAMPLE 1: DIRECT VIDEO TRANSLATIONRaw Video Data $\rightarrow$ Direct Prompting

Full Screen ↗

⚠️ Hallucinations & Drift Observed:

Hallucinated modern "sci-fi BIOS" glow, rounded corners (`rounded-lg`), and smooth gradients.
Misaligned spatial layouts (portrait box shifted left, map rendered as abstract nodes).
Lost Chinese localization; fallback to English abbreviations (`PROV`, `GOV`, `SOL`).

EXAMPLE 2: SPEC-DRIVEN GENERATION3-Layer YAML Spec $\rightarrow$ Compiled Prompt

Full Screen ↗

✅ Strict Constraint Adherence:

Rigid 90° box corners, flat EGA palette fills, and thick multi-colored borders preserved.
Fixed 55%/45% layout split with structured sidebar header/content/action slots.
Exact Traditional Chinese text localization and 41-province polygon canvas map.

The Underlying Specification (350 Lines Total)

The three modular YAML files compiled by prompt_compiler.py to enforce deterministic layout and state transitions.

# ==============================================================================
# LAYER 1: MULTI-TEMPLATE LAYOUT ARCHITECTURE
# ==============================================================================
width: 640
height: 400
gap: 0

theme:
  colors:
    background: "#000000"
    border_primary: "#FFFF55"   # EGA Yellow
    border_secondary: "#FF55FF" # EGA Magenta
    text_primary: "#FFFFFF"     # EGA White
    text_accent: "#00AAAA"      # EGA Cyan
  directives:
    - "No rounded corners, gradients, CSS drop-shadows, or glowing neon monitors."
    - "All containers and elements must use rigid 90-degree box corners."
    - "All text must render using standard monospace/pixelated styling with standard anti-aliasing disabled."
    - "All borders must be thick, solid, and styled with multi-colored patterns."

templates:
  title_screen:
    direction: row
    regions:
      dragon_pane: { width: "25%" }
      menu_pane: { flex_grow: 1, direction: column, justify: center, align: center }
      tiger_pane: { width: "25%" }

  main_gameplay:
    direction: row
    regions:
      map_pane: { width: "55%" }
      sidebar:
        gap: 8
        padding: 8
        regions:
          header:
            height: 80
            direction: row
            regions:
              info_text: { flex_grow: 3 }
              portrait_box: { width: 80 }
          content:
            regions:
              stats_view: {}
              graphic_view: {}
          action: { height: 120 }

  battlefield:
    direction: row
    regions:
      tactical_grid: { width: "55%" }
      battle_sidebar:
        gap: 8
        padding: 8
        regions:
          battle_stats: { height: "50%" }
          battle_action: { flex_grow: 1 }

# ==============================================================================
# LAYER 2: ELEMENT MANIFEST (Component Catalog & Slot Bindings)
# ==============================================================================
dragon_art:
  type: image
  slot: title_screen.dragon_pane
  selector: "img.dragon-sprite"

title_main_menu:
  type: menu
  slot: title_screen.menu_pane
  selector: "ul.main-menu-options"
  properties:
    localization: "Traditional Chinese"
    options: ["1. 啟動新遊戲", "2. 裝入數據", "3. 結束"]

china_map:
  type: canvas
  slot: main_gameplay.map_pane
  selector: "canvas#province-map"
  properties:
    visual_model: "Solid regional polygon landmass representing 41 provinces with thin black borders."
    palette: "White, Magenta, Cyan, and Yellow flat fills only."
    labeling: "Numeric ID in center using small pixel font."

province_stats_table:
  type: table
  slot: main_gameplay.sidebar.content.stats_view
  selector: "table.stats-grid"
  properties:
    localization: "Traditional Chinese"
    labels: ["人口", "士兵", "在職將領", "在野將領", "黃金", "糧食", "米價", "良駒", "民忠誠度", "地價", "治水度", "城堡"]

battle_hex_map:
  type: grid
  slot: battlefield.tactical_grid
  selector: "canvas#battlefield-map"
  properties:
    visual_model: "10x10 grid map of pixel tiles depicting trees, rivers, mountains, and castles."

# ==============================================================================
# LAYER 3: COMPREHENSIVE BEHAVIOR STATE ROUTING & DIFFS
# ==============================================================================
title_screen_state:
  template: "title_screen"
  active_elements: [dragon_art, tiger_art, title_main_menu]

base_gameplay:
  template: "main_gameplay"
  active_elements: [china_map, ruler_info_label, governor_portrait]

governor_personnel_screen:
  extends: base_gameplay
  active_elements:
    - active_province_highlight
    - province_stats_table
    - personnel_options_menu

battle_combat_phase:
  template: "battlefield"
  active_elements: [battle_hex_map, combatant_matchup_panel, battle_command_menu]

captured_disposal_screen:
  template: "main_gameplay"
  extends: base_gameplay
  active_elements:
    - -ruler_info_label
    - -governor_portrait
    - +captured_general_portrait
    - +captured_general_stats
    - +disposal_action_menu

Research Write-Up & Tradeoff Frontier Analysis

Aesthetic & Stylistic Drift in LLMs

In a visual debugging and code generation setting, direct video-to-code synthesis is highly susceptible to hallucinating modern UI tropes. Because general-purpose LLMs are heavily fine-tuned on modern web frameworks, they default to modern utilities (rounded-lg, box shadows, glowing scanlines, CSS gradients) when given abstract visual instructions.

The Specification Tradeoff Frontier

By establishing a strict 3-layer constraints pipeline in lightweight YAML, developers create a boundary that the LLM cannot bypass. However, an inherent tradeoff exists: increasing specification detail eventually approaches the complexity of writing raw HTML. The ~350-line specification structure used here achieves the optimal Pareto frontier between specification brevity and output stability.