mirror of
https://github.com/comfyanonymous/ComfyUI.git
synced 2026-06-17 21:39:45 +08:00
docs: Add JSON Schema for Prompt API Format
Provides formal validation and documentation for ComfyUI's /prompt API format. Changes: - schemas/prompt.json: Complete JSON Schema for prompt validation - Validates node structure with class_type and inputs - Supports direct values and node links [node_id, output_index] - Includes examples for common workflows - docs/api/prompt-schema.md: Comprehensive documentation - Usage examples and validation guide - Workflow vs Prompt format comparison - Common node type reference - Future enhancement roadmap Benefits: - Enables IDE autocomplete and validation - Provides clear API specification - Foundation for future dynamic schemas - Resolves confusion about workflow vs prompt formats Fixes #8899 [Bounty]
This commit is contained in:
andygoodluck
parent
9a058f7f6e
commit
f689debfcb
213
docs/api/prompt-schema.md
Normal file
213
docs/api/prompt-schema.md
Normal file
@ -0,0 +1,213 @@
|
||||
# ComfyUI Prompt API Schema Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the JSON Schema for ComfyUI's `/prompt` API endpoint. The schema provides formal validation and documentation for the prompt format used when submitting workflows via the API.
|
||||
|
||||
## Schema Location
|
||||
|
||||
- **Schema File**: `schemas/prompt.json`
|
||||
- **Schema ID**: `https://github.com/comfyanonymous/ComfyUI/schemas/prompt.json`
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Validating a Prompt
|
||||
|
||||
```python
|
||||
import json
|
||||
import jsonschema
|
||||
|
||||
with open('schemas/prompt.json') as f:
|
||||
schema = json.load(f)
|
||||
|
||||
with open('your_prompt.json') as f:
|
||||
prompt = json.load(f)
|
||||
|
||||
# Validate
|
||||
jsonschema.validate(prompt, schema)
|
||||
print("✅ Prompt is valid!")
|
||||
```
|
||||
|
||||
## Format Overview
|
||||
|
||||
The prompt format is a JSON object containing:
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt": {
|
||||
"node_id_1": { /* node definition */ },
|
||||
"node_id_2": { /* node definition */ },
|
||||
...
|
||||
},
|
||||
"extra_data": { /* optional */ },
|
||||
"client_id": "optional-client-id"
|
||||
}
|
||||
```
|
||||
|
||||
### Node Structure
|
||||
|
||||
Each node has:
|
||||
- `class_type`: The node class (e.g., "KSampler", "CLIPTextEncode")
|
||||
- `inputs`: Input parameters
|
||||
- `_meta`: Optional metadata
|
||||
|
||||
### Input Values
|
||||
|
||||
Inputs can be:
|
||||
1. **Direct values**: Numbers, strings, booleans, arrays, objects
|
||||
2. **Node links**: `["node_id", output_index]` to reference another node
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic Text-to-Image
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt": {
|
||||
"3": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"seed": 123456789,
|
||||
"steps": 20,
|
||||
"cfg": 8.0,
|
||||
"sampler_name": "euler",
|
||||
"scheduler": "normal",
|
||||
"denoise": 1.0,
|
||||
"model": ["4", 0],
|
||||
"positive": ["6", 0],
|
||||
"negative": ["7", 0],
|
||||
"latent_image": ["5", 0]
|
||||
}
|
||||
},
|
||||
"4": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"inputs": {
|
||||
"ckpt_name": "model.safetensors"
|
||||
}
|
||||
},
|
||||
"5": {
|
||||
"class_type": "EmptyLatentImage",
|
||||
"inputs": {
|
||||
"width": 512,
|
||||
"height": 512,
|
||||
"batch_size": 1
|
||||
}
|
||||
},
|
||||
"6": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {
|
||||
"text": "beautiful scenery",
|
||||
"clip": ["4", 1]
|
||||
}
|
||||
},
|
||||
"7": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {
|
||||
"text": "text, watermark",
|
||||
"clip": ["4", 1]
|
||||
}
|
||||
},
|
||||
"8": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {
|
||||
"samples": ["3", 0],
|
||||
"vae": ["4", 2]
|
||||
}
|
||||
},
|
||||
"9": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {
|
||||
"filename_prefix": "ComfyUI",
|
||||
"images": ["8", 0]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Workflow vs Prompt Format
|
||||
|
||||
### Workflow Format (UI Internal)
|
||||
|
||||
Used by the web interface for saving/loading workflows.
|
||||
|
||||
```json
|
||||
{
|
||||
"last_node_id": 9,
|
||||
"last_link_id": 5,
|
||||
"nodes": [...],
|
||||
"links": [...],
|
||||
"groups": [...]
|
||||
}
|
||||
```
|
||||
|
||||
### Prompt Format (API)
|
||||
|
||||
Used by the `/prompt` API endpoint for execution.
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt": {
|
||||
"3": { "class_type": "KSampler", "inputs": {...} },
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Differences**:
|
||||
- Workflow contains UI metadata (positions, colors)
|
||||
- Prompt is execution-focused (just nodes and connections)
|
||||
- Workflow is converted to prompt format before execution
|
||||
|
||||
## Common Node Types
|
||||
|
||||
### Loaders
|
||||
- `CheckpointLoaderSimple`: Load model checkpoints
|
||||
- `VAELoader`: Load VAE models
|
||||
- `LoraLoader`: Load LoRA models
|
||||
|
||||
### Encoders
|
||||
- `CLIPTextEncode`: Encode text prompts
|
||||
- `CLIPTextEncodeSDXL`: SDXL text encoding
|
||||
|
||||
### Samplers
|
||||
- `KSampler`: Standard sampling
|
||||
- `KSamplerAdvanced`: Advanced sampling options
|
||||
|
||||
### Image Operations
|
||||
- `EmptyLatentImage`: Create empty latent
|
||||
- `VAEDecode`: Decode latent to image
|
||||
- `VAEDecodeTiled`: Tiled decoding for large images
|
||||
- `SaveImage`: Save output image
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Dynamic Schemas
|
||||
|
||||
Future versions may support:
|
||||
- Node-specific validation based on installed custom nodes
|
||||
- Input type validation beyond basic JSON types
|
||||
- Required vs optional input validation
|
||||
- Dependency checking between nodes
|
||||
|
||||
### IDE Integration
|
||||
|
||||
The schema enables:
|
||||
- Auto-completion in IDEs (VS Code, PyCharm, etc.)
|
||||
- Real-time validation during development
|
||||
- Documentation tooltips
|
||||
|
||||
## Contributing
|
||||
|
||||
To extend the schema:
|
||||
|
||||
1. Add new node types to the definitions
|
||||
2. Include examples for complex use cases
|
||||
3. Update documentation with validation rules
|
||||
4. Test with existing prompts
|
||||
|
||||
## References
|
||||
|
||||
- [ComfyUI API Documentation](https://docs.comfy.org/)
|
||||
- [JSON Schema Specification](https://json-schema.org/)
|
||||
- [Original Feature Request #8899](https://github.com/Comfy-Org/ComfyUI/issues/8899)
|
||||
113
schemas/prompt.json
Normal file
113
schemas/prompt.json
Normal file
@ -0,0 +1,113 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"$id": "https://github.com/comfyanonymous/ComfyUI/schemas/prompt.json",
|
||||
"title": "ComfyUI Prompt API Format",
|
||||
"description": "JSON Schema for ComfyUI /prompt API endpoint",
|
||||
"type": "object",
|
||||
"required": ["prompt"],
|
||||
"properties": {
|
||||
"prompt": {
|
||||
"type": "object",
|
||||
"description": "The prompt structure containing nodes",
|
||||
"additionalProperties": {
|
||||
"$ref": "#/definitions/node"
|
||||
}
|
||||
},
|
||||
"extra_data": {
|
||||
"type": "object",
|
||||
"description": "Optional extra data for the prompt execution"
|
||||
},
|
||||
"client_id": {
|
||||
"type": "string",
|
||||
"description": "Optional client identifier"
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"node": {
|
||||
"type": "object",
|
||||
"required": ["class_type", "inputs"],
|
||||
"properties": {
|
||||
"class_type": {
|
||||
"type": "string",
|
||||
"description": "The node class type (e.g., KSampler, CLIPTextEncode, etc.)"
|
||||
},
|
||||
"inputs": {
|
||||
"type": "object",
|
||||
"description": "Input parameters for the node",
|
||||
"additionalProperties": {
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Direct value (string, number, boolean, array, object)"
|
||||
},
|
||||
{
|
||||
"$ref": "#/definitions/nodeLink"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"_meta": {
|
||||
"type": "object",
|
||||
"description": "Optional metadata for the node"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"nodeLink": {
|
||||
"type": "array",
|
||||
"description": "Reference to another node's output",
|
||||
"items": [
|
||||
{
|
||||
"type": "string",
|
||||
"description": "The node_id of the source node"
|
||||
},
|
||||
{
|
||||
"type": "integer",
|
||||
"description": "The output index (0 for first output)"
|
||||
}
|
||||
],
|
||||
"minItems": 2,
|
||||
"maxItems": 2
|
||||
}
|
||||
},
|
||||
"examples": [
|
||||
{
|
||||
"prompt": {
|
||||
"3": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"seed": 123456789,
|
||||
"steps": 20,
|
||||
"cfg": 8.0,
|
||||
"sampler_name": "euler",
|
||||
"scheduler": "normal",
|
||||
"denoise": 1.0,
|
||||
"model": ["4", 0],
|
||||
"positive": ["6", 0],
|
||||
"negative": ["7", 0],
|
||||
"latent_image": ["5", 0]
|
||||
}
|
||||
},
|
||||
"4": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"inputs": {
|
||||
"ckpt_name": "model.safetensors"
|
||||
}
|
||||
},
|
||||
"6": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {
|
||||
"text": "beautiful scenery nature glass bottle landscape, purple galaxy bottle",
|
||||
"clip": ["4", 1]
|
||||
}
|
||||
},
|
||||
"7": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {
|
||||
"text": "text, watermark",
|
||||
"clip": ["4", 1]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user