mirror of
https://github.com/comfyanonymous/ComfyUI.git
synced 2026-06-18 22:09:38 +08:00
docs: Add JSON Schema for Prompt API Format
Provides formal validation and documentation for ComfyUI's /prompt API format. Changes: - schemas/prompt.json: Complete JSON Schema for prompt validation - Validates node structure with class_type and inputs - Supports direct values and node links [node_id, output_index] - Includes examples for common workflows - docs/api/prompt-schema.md: Comprehensive documentation - Usage examples and validation guide - Workflow vs Prompt format comparison - Common node type reference - Future enhancement roadmap Benefits: - Enables IDE autocomplete and validation - Provides clear API specification - Foundation for future dynamic schemas - Resolves confusion about workflow vs prompt formats Fixes #8899 [Bounty]
This commit is contained in:
andygoodluck
parent
9a058f7f6e
commit
f689debfcb
213
docs/api/prompt-schema.md
Normal file
213
docs/api/prompt-schema.md
Normal file
@ -0,0 +1,213 @@
|
|||||||
|
# ComfyUI Prompt API Schema Documentation
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This document describes the JSON Schema for ComfyUI's `/prompt` API endpoint. The schema provides formal validation and documentation for the prompt format used when submitting workflows via the API.
|
||||||
|
|
||||||
|
## Schema Location
|
||||||
|
|
||||||
|
- **Schema File**: `schemas/prompt.json`
|
||||||
|
- **Schema ID**: `https://github.com/comfyanonymous/ComfyUI/schemas/prompt.json`
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### Validating a Prompt
|
||||||
|
|
||||||
|
```python
|
||||||
|
import json
|
||||||
|
import jsonschema
|
||||||
|
|
||||||
|
with open('schemas/prompt.json') as f:
|
||||||
|
schema = json.load(f)
|
||||||
|
|
||||||
|
with open('your_prompt.json') as f:
|
||||||
|
prompt = json.load(f)
|
||||||
|
|
||||||
|
# Validate
|
||||||
|
jsonschema.validate(prompt, schema)
|
||||||
|
print("✅ Prompt is valid!")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Format Overview
|
||||||
|
|
||||||
|
The prompt format is a JSON object containing:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"prompt": {
|
||||||
|
"node_id_1": { /* node definition */ },
|
||||||
|
"node_id_2": { /* node definition */ },
|
||||||
|
...
|
||||||
|
},
|
||||||
|
"extra_data": { /* optional */ },
|
||||||
|
"client_id": "optional-client-id"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Node Structure
|
||||||
|
|
||||||
|
Each node has:
|
||||||
|
- `class_type`: The node class (e.g., "KSampler", "CLIPTextEncode")
|
||||||
|
- `inputs`: Input parameters
|
||||||
|
- `_meta`: Optional metadata
|
||||||
|
|
||||||
|
### Input Values
|
||||||
|
|
||||||
|
Inputs can be:
|
||||||
|
1. **Direct values**: Numbers, strings, booleans, arrays, objects
|
||||||
|
2. **Node links**: `["node_id", output_index]` to reference another node
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
### Basic Text-to-Image
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"prompt": {
|
||||||
|
"3": {
|
||||||
|
"class_type": "KSampler",
|
||||||
|
"inputs": {
|
||||||
|
"seed": 123456789,
|
||||||
|
"steps": 20,
|
||||||
|
"cfg": 8.0,
|
||||||
|
"sampler_name": "euler",
|
||||||
|
"scheduler": "normal",
|
||||||
|
"denoise": 1.0,
|
||||||
|
"model": ["4", 0],
|
||||||
|
"positive": ["6", 0],
|
||||||
|
"negative": ["7", 0],
|
||||||
|
"latent_image": ["5", 0]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"4": {
|
||||||
|
"class_type": "CheckpointLoaderSimple",
|
||||||
|
"inputs": {
|
||||||
|
"ckpt_name": "model.safetensors"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"5": {
|
||||||
|
"class_type": "EmptyLatentImage",
|
||||||
|
"inputs": {
|
||||||
|
"width": 512,
|
||||||
|
"height": 512,
|
||||||
|
"batch_size": 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"6": {
|
||||||
|
"class_type": "CLIPTextEncode",
|
||||||
|
"inputs": {
|
||||||
|
"text": "beautiful scenery",
|
||||||
|
"clip": ["4", 1]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"7": {
|
||||||
|
"class_type": "CLIPTextEncode",
|
||||||
|
"inputs": {
|
||||||
|
"text": "text, watermark",
|
||||||
|
"clip": ["4", 1]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"8": {
|
||||||
|
"class_type": "VAEDecode",
|
||||||
|
"inputs": {
|
||||||
|
"samples": ["3", 0],
|
||||||
|
"vae": ["4", 2]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"9": {
|
||||||
|
"class_type": "SaveImage",
|
||||||
|
"inputs": {
|
||||||
|
"filename_prefix": "ComfyUI",
|
||||||
|
"images": ["8", 0]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Workflow vs Prompt Format
|
||||||
|
|
||||||
|
### Workflow Format (UI Internal)
|
||||||
|
|
||||||
|
Used by the web interface for saving/loading workflows.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"last_node_id": 9,
|
||||||
|
"last_link_id": 5,
|
||||||
|
"nodes": [...],
|
||||||
|
"links": [...],
|
||||||
|
"groups": [...]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Prompt Format (API)
|
||||||
|
|
||||||
|
Used by the `/prompt` API endpoint for execution.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"prompt": {
|
||||||
|
"3": { "class_type": "KSampler", "inputs": {...} },
|
||||||
|
...
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key Differences**:
|
||||||
|
- Workflow contains UI metadata (positions, colors)
|
||||||
|
- Prompt is execution-focused (just nodes and connections)
|
||||||
|
- Workflow is converted to prompt format before execution
|
||||||
|
|
||||||
|
## Common Node Types
|
||||||
|
|
||||||
|
### Loaders
|
||||||
|
- `CheckpointLoaderSimple`: Load model checkpoints
|
||||||
|
- `VAELoader`: Load VAE models
|
||||||
|
- `LoraLoader`: Load LoRA models
|
||||||
|
|
||||||
|
### Encoders
|
||||||
|
- `CLIPTextEncode`: Encode text prompts
|
||||||
|
- `CLIPTextEncodeSDXL`: SDXL text encoding
|
||||||
|
|
||||||
|
### Samplers
|
||||||
|
- `KSampler`: Standard sampling
|
||||||
|
- `KSamplerAdvanced`: Advanced sampling options
|
||||||
|
|
||||||
|
### Image Operations
|
||||||
|
- `EmptyLatentImage`: Create empty latent
|
||||||
|
- `VAEDecode`: Decode latent to image
|
||||||
|
- `VAEDecodeTiled`: Tiled decoding for large images
|
||||||
|
- `SaveImage`: Save output image
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
### Dynamic Schemas
|
||||||
|
|
||||||
|
Future versions may support:
|
||||||
|
- Node-specific validation based on installed custom nodes
|
||||||
|
- Input type validation beyond basic JSON types
|
||||||
|
- Required vs optional input validation
|
||||||
|
- Dependency checking between nodes
|
||||||
|
|
||||||
|
### IDE Integration
|
||||||
|
|
||||||
|
The schema enables:
|
||||||
|
- Auto-completion in IDEs (VS Code, PyCharm, etc.)
|
||||||
|
- Real-time validation during development
|
||||||
|
- Documentation tooltips
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
To extend the schema:
|
||||||
|
|
||||||
|
1. Add new node types to the definitions
|
||||||
|
2. Include examples for complex use cases
|
||||||
|
3. Update documentation with validation rules
|
||||||
|
4. Test with existing prompts
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [ComfyUI API Documentation](https://docs.comfy.org/)
|
||||||
|
- [JSON Schema Specification](https://json-schema.org/)
|
||||||
|
- [Original Feature Request #8899](https://github.com/Comfy-Org/ComfyUI/issues/8899)
|
||||||
113
schemas/prompt.json
Normal file
113
schemas/prompt.json
Normal file
@ -0,0 +1,113 @@
|
|||||||
|
{
|
||||||
|
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||||
|
"$id": "https://github.com/comfyanonymous/ComfyUI/schemas/prompt.json",
|
||||||
|
"title": "ComfyUI Prompt API Format",
|
||||||
|
"description": "JSON Schema for ComfyUI /prompt API endpoint",
|
||||||
|
"type": "object",
|
||||||
|
"required": ["prompt"],
|
||||||
|
"properties": {
|
||||||
|
"prompt": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "The prompt structure containing nodes",
|
||||||
|
"additionalProperties": {
|
||||||
|
"$ref": "#/definitions/node"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"extra_data": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "Optional extra data for the prompt execution"
|
||||||
|
},
|
||||||
|
"client_id": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Optional client identifier"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"definitions": {
|
||||||
|
"node": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["class_type", "inputs"],
|
||||||
|
"properties": {
|
||||||
|
"class_type": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The node class type (e.g., KSampler, CLIPTextEncode, etc.)"
|
||||||
|
},
|
||||||
|
"inputs": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "Input parameters for the node",
|
||||||
|
"additionalProperties": {
|
||||||
|
"oneOf": [
|
||||||
|
{
|
||||||
|
"description": "Direct value (string, number, boolean, array, object)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/nodeLink"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"_meta": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "Optional metadata for the node"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"nodeLink": {
|
||||||
|
"type": "array",
|
||||||
|
"description": "Reference to another node's output",
|
||||||
|
"items": [
|
||||||
|
{
|
||||||
|
"type": "string",
|
||||||
|
"description": "The node_id of the source node"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "The output index (0 for first output)"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"minItems": 2,
|
||||||
|
"maxItems": 2
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"examples": [
|
||||||
|
{
|
||||||
|
"prompt": {
|
||||||
|
"3": {
|
||||||
|
"class_type": "KSampler",
|
||||||
|
"inputs": {
|
||||||
|
"seed": 123456789,
|
||||||
|
"steps": 20,
|
||||||
|
"cfg": 8.0,
|
||||||
|
"sampler_name": "euler",
|
||||||
|
"scheduler": "normal",
|
||||||
|
"denoise": 1.0,
|
||||||
|
"model": ["4", 0],
|
||||||
|
"positive": ["6", 0],
|
||||||
|
"negative": ["7", 0],
|
||||||
|
"latent_image": ["5", 0]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"4": {
|
||||||
|
"class_type": "CheckpointLoaderSimple",
|
||||||
|
"inputs": {
|
||||||
|
"ckpt_name": "model.safetensors"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"6": {
|
||||||
|
"class_type": "CLIPTextEncode",
|
||||||
|
"inputs": {
|
||||||
|
"text": "beautiful scenery nature glass bottle landscape, purple galaxy bottle",
|
||||||
|
"clip": ["4", 1]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"7": {
|
||||||
|
"class_type": "CLIPTextEncode",
|
||||||
|
"inputs": {
|
||||||
|
"text": "text, watermark",
|
||||||
|
"clip": ["4", 1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
Loading…
Reference in New Issue
Block a user