Add ingest_existing_file() to services/ingest.py as a public wrapper for
registering on-disk files (stat, BLAKE3 hash, MIME detection, path-based
tag derivation).
After each prompt execution in the main loop, iterate
history_result['outputs'] and register files with type 'output' as
assets. Runs while the asset seeder is paused, gated behind
asset_seeder.is_disabled(). Populates job_id on the asset_references
table for provenance tracking.
Ingest uses a two-phase approach: insert a stub record (hash=NULL) first
for instant visibility, then defer hashing to the background seeder
enrich phase to avoid blocking the prompt worker thread.
When multiple enrich scans are enqueued while the seeder is busy, roots
are now unioned and compute_hashes uses sticky-true (OR) logic so no
queued work is silently dropped.
Extract _reset_to_idle helper in the asset seeder to deduplicate the
state reset pattern shared by _run_scan and mark_missing_outside_prefixes.
Separate history parsing from output file registration: move generic
file registration logic into register_output_files() in
app/assets/services/ingest.py, keeping only the ComfyUI history format
parsing (_collect_output_absolute_paths) in main.py.
Amp-Thread-ID: https://ampcode.com/threads/T-019cf842-5199-71f0-941d-b420b5cf4d57
Co-authored-by: Amp <amp@ampcode.com>
Write to a temp file in the same directory then os.replace() onto the
target path. If the process crashes mid-write, the original file is
left intact instead of being truncated to zero bytes.
Fixes#11298
* feat(assets): align local API with cloud spec
Unify response models, add missing fields, and align input schemas with
the cloud OpenAPI spec at cloud.comfy.org/openapi.
- Replace AssetSummary/AssetDetail/AssetUpdated with single Asset model
- Add is_immutable, metadata (system_metadata), prompt_id fields
- Support mime_type and preview_id in update endpoint
- Make CreateFromHashBody.name optional, add mime_type, require >=1 tag
- Add id/mime_type/preview_id to upload, relax tags to optional
- Rename total_tags → tags in tag add/remove responses
- Add GET /api/assets/tags/refine histogram endpoint
- Add DB migration for system_metadata and prompt_id columns
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix review issues: tags validation, size nullability, type annotation, hash mismatch check, and add tag histogram tests
- Remove contradictory min_length=1 from CreateFromHashBody.tags default
- Restore size field to int|None=None for proper null semantics
- Add Union type annotation to _build_asset_response result param
- Add hash mismatch validation on idempotent upload path (409 HASH_MISMATCH)
- Add unit tests for list_tag_histogram service function
Amp-Thread-ID: https://ampcode.com/threads/T-019cd993-f43c-704e-b3d7-6cfc3d4d4a80
Co-authored-by: Amp <amp@ampcode.com>
* Add preview_url to /assets API response using /api/view endpoint
For input and output assets, generate a preview_url pointing to the
existing /api/view endpoint using the asset's filename and tag-derived
type (input/output). Handles subdirectories via subfolder param and
URL-encodes filenames with spaces, unicode, and special characters.
This aligns the OSS backend response with the frontend AssetCard
expectation for thumbnail rendering.
Amp-Thread-ID: https://ampcode.com/threads/T-019cda3f-5c2c-751a-a906-ac6c9153ac5c
Co-authored-by: Amp <amp@ampcode.com>
* chore: remove unused imports from asset_reference queries
Amp-Thread-ID: https://ampcode.com/threads/T-019cda7d-cb21-77b4-a51b-b965af60208c
Co-authored-by: Amp <amp@ampcode.com>
* feat: resolve blake3 hashes in /view endpoint via asset database
Amp-Thread-ID: https://ampcode.com/threads/T-019cda7d-cb21-77b4-a51b-b965af60208c
Co-authored-by: Amp <amp@ampcode.com>
* Register uploaded images in asset database when --enable-assets is set
Add register_file_in_place() service function to ingest module for
registering already-saved files without moving them. Call it from the
/upload/image endpoint to return asset metadata in the response.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce023-3384-7560-bacf-de40b0de0dd2
Co-authored-by: Amp <amp@ampcode.com>
* Exclude None fields from asset API JSON responses
Add exclude_none=True to model_dump() calls across asset routes to
keep response payloads clean by omitting unset optional fields.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce023-3384-7560-bacf-de40b0de0dd2
Co-authored-by: Amp <amp@ampcode.com>
* Add comment explaining why /view resolves blake3 hashes
Amp-Thread-ID: https://ampcode.com/threads/T-019ce023-3384-7560-bacf-de40b0de0dd2
Co-authored-by: Amp <amp@ampcode.com>
* Move blake3 hash resolution to asset_management service
Extract resolve_hash_to_path() into asset_management.py and remove
_resolve_blake3_to_path from server.py. Also revert loopback origin
check to original logic.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce023-3384-7560-bacf-de40b0de0dd2
Co-authored-by: Amp <amp@ampcode.com>
* Require at least one tag in UploadAssetSpec
Enforce non-empty tags at the Pydantic validation layer so uploads
with no tags are rejected with a 400 before reaching ingest. Adds
test_upload_empty_tags_rejected to cover this case.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce377-8bde-7048-bc28-a9df063409f9
Co-authored-by: Amp <amp@ampcode.com>
* Add owner_id check to resolve_hash_to_path
Filter asset references by owner visibility so the /view endpoint
only resolves hashes for assets the requesting user can access.
Adds table-driven tests for owner visibility cases.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce377-8bde-7048-bc28-a9df063409f9
Co-authored-by: Amp <amp@ampcode.com>
* Make ReferenceData.created_at and updated_at required
Remove None defaults and type: ignore comments. Move fields before
optional fields to satisfy dataclass ordering.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce377-8bde-7048-bc28-a9df063409f9
Co-authored-by: Amp <amp@ampcode.com>
* Fix double commit in create_from_hash
Move mime_type update into _register_existing_asset so it shares a
single transaction with reference creation. Log a warning when the
hash is not found instead of silently returning None.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce377-8bde-7048-bc28-a9df063409f9
Co-authored-by: Amp <amp@ampcode.com>
* Add exclude_none=True to create/upload responses
Align with get/update/list endpoints for consistent JSON output.
Amp-Thread-ID: https://ampcode.com/threads/T-019ce377-8bde-7048-bc28-a9df063409f9
Co-authored-by: Amp <amp@ampcode.com>
* Change preview_id to reference asset by reference ID, not content ID
Clients receive preview_id in API responses but could not dereference it
through public routes (which use reference IDs). Now preview_id is a
self-referential FK to asset_references.id so the value is directly
usable in the public API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Filter soft-deleted and missing refs from visibility queries
list_references_by_asset_id and list_tags_with_usage were not filtering
out deleted_at/is_missing refs, allowing /view?filename=blake3:... to
serve files through hidden references and inflating tag usage counts.
Add list_all_file_paths_by_asset_id for orphan cleanup which
intentionally needs unfiltered access.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Pass preview_id and mime_type through all asset creation fast paths
The duplicate-content upload path and hash-based creation paths were
silently dropping preview_id and mime_type. This wires both fields
through _register_existing_asset, create_from_hash, and all route
call sites so behavior is consistent regardless of whether the asset
content already exists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove unimplemented client-provided ID from upload API
The `id` field on UploadAssetSpec was advertised for idempotent creation
but never actually honored when creating new references. Remove it
rather than implementing the feature.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Make asset mime_type immutable after first ingest
Prevents cross-tenant metadata mutation when multiple references share
the same content-addressed Asset row. mime_type can now only be set when
NULL (first ingest); subsequent attempts to change it are silently ignored.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Use resolved content_type from asset lookup in /view endpoint
The /view endpoint was discarding the content_type computed by
resolve_hash_to_path() and re-guessing from the filename, which
produced wrong results for extensionless files or mismatched extensions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Merge system+user metadata into filter projection
Extract rebuild_metadata_projection() to build AssetReferenceMeta rows
from {**system_metadata, **user_metadata}, so system-generated metadata
is queryable via metadata_filter and user keys override system keys.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Standardize tag ordering to alphabetical across all endpoints
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Derive subfolder tags from path in register_file_in_place
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Reject client-provided id, fix preview URLs, rename tags→total_tags
- Reject 'id' field in multipart upload with 400 UNSUPPORTED_FIELD
instead of silently ignoring it
- Build preview URL from the preview asset's own metadata rather than
the parent asset's
- Rename 'tags' to 'total_tags' in TagsAdd/TagsRemove response schemas
for clarity
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: SQLite migration 0003 FK drop fails on file-backed DBs (MB-2)
Add naming_convention to Base.metadata so Alembic batch-mode reflection
can match unnamed FK constraints created by migration 0002. Pass
naming_convention and render_as_batch=True through env.py online config.
Add migration roundtrip tests (upgrade/downgrade/cycle from baseline).
Amp-Thread-ID: https://ampcode.com/threads/T-019ce466-1683-7471-b6e1-bb078223cda0
Co-authored-by: Amp <amp@ampcode.com>
* Fix missing tag count for is_missing references and update test for total_tags field
- Allow is_missing=True references to be counted in list_tags_with_usage
when the tag is 'missing', so the missing tag count reflects all
references that have been tagged as missing
- Add update_is_missing_by_asset_id query helper for bulk updates by asset
- Update test_add_and_remove_tags to use 'total_tags' matching the API schema
Amp-Thread-ID: https://ampcode.com/threads/T-019ce482-05e7-7324-a1b0-a56a929cc7ef
Co-authored-by: Amp <amp@ampcode.com>
* Remove unused imports in scanner.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Rename prompt_id to job_id on asset_references
Rename the column in the DB model, migration, and service schemas.
The API response emits both job_id and prompt_id (deprecated alias)
for backward compatibility with the cloud API.
Amp-Thread-ID: https://ampcode.com/threads/T-019cef41-60b0-752a-aa3c-ed7f20fda2f7
Co-authored-by: Amp <amp@ampcode.com>
* Add index on asset_references.preview_id for FK cascade performance
Amp-Thread-ID: https://ampcode.com/threads/T-019cef45-a4d2-7548-86d2-d46bcd3db419
Co-authored-by: Amp <amp@ampcode.com>
* Add clarifying comments for Asset/AssetReference naming and preview_id
Amp-Thread-ID: https://ampcode.com/threads/T-019cef49-f94e-7348-bf23-9a19ebf65e0d
Co-authored-by: Amp <amp@ampcode.com>
* Disallow all-null meta rows: add CHECK constraint, skip null values on write
- convert_metadata_to_rows returns [] for None values instead of an all-null row
- Remove dead None branch from _scalar_to_row
- Simplify null filter in common.py to just check for row absence
- Add CHECK constraint ck_asset_reference_meta_has_value to model and migration 0003
Amp-Thread-ID: https://ampcode.com/threads/T-019cef4e-5240-7749-bb25-1f17fcf9c09c
Co-authored-by: Amp <amp@ampcode.com>
* Remove dead None guards on result.asset in upload handler
register_file_in_place guarantees a non-None asset, so the
'if result.asset else None' checks were unreachable.
Amp-Thread-ID: https://ampcode.com/threads/T-019cef5b-4cf8-723c-8a98-8fb8f333c133
Co-authored-by: Amp <amp@ampcode.com>
* Remove mime_type from asset update API
Clients can no longer modify mime_type after asset creation via the
PUT /api/assets/{id} endpoint. This reduces the risk of mime_type
spoofing. The internal update_asset_hash_and_mime function remains
available for server-side use (e.g., enrichment).
Amp-Thread-ID: https://ampcode.com/threads/T-019cef5d-8d61-75cc-a1c6-2841ac395648
Co-authored-by: Amp <amp@ampcode.com>
* Fix migration constraint naming double-prefix and NULL in mixed metadata lists
- Use fully-rendered constraint names in migration 0003 to avoid the
naming convention doubling the ck_ prefix on batch operations.
- Add table_args to downgrade so SQLite batch mode can find the CHECK
constraint (not exposed by SQLite reflection).
- Fix model CheckConstraint name to use bare 'has_value' (convention
auto-prefixes).
- Skip None items when converting metadata lists to rows, preventing
all-NULL rows that violate the has_value check constraint.
Amp-Thread-ID: https://ampcode.com/threads/T-019cef87-94f9-7172-a6af-c6282290ce4f
Co-authored-by: Amp <amp@ampcode.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Amp <amp@ampcode.com>
Skip entries in the prompt dict that don't contain a class_type key
in apply_replacements(), preventing crashes on metadata or non-node
entries.
FixesComfy-Org/ComfyUI#12517
On Windows, Python defaults to cp1252 encoding when no encoding is
specified. JSON files containing UTF-8 characters (e.g., non-ASCII
characters) cause UnicodeDecodeError when read with cp1252.
This fixes the error that occurs when loading blueprint subgraphs
on Windows systems.
https://claude.ai/code/session_014WHi3SL9Gzsi3U6kbSjbSb
Co-authored-by: Claude <noreply@anthropic.com>
* Brought over minimal elements from PR 10045 to reproduce seed_assets and register_assets_system without adding anything to the DB or server routes yet, for now making everything sync (can introduce async once everything is cleaned up and brought over)
* Added db script to insert assets stuff, cleaned up some code; assets (models) now get added/rescanned
* Added support for 5 http endpoints for assets
* Replaced Optional with | None in schemas_in.py and schemas_out.py
* Remove two routes that will not be relevant yet in this PR: HEAD /api/assets/hash/<hash> and PUT /api/assets/<id>/preview
* Remove some functions the two deleted endpoints were using
* Don't show assets scan message upon calling /object_info endpoint
* removed unsued import to satisfy ruff
* Simplified hashing function tpye hint and _hash_file_obj
* Satisfied ruff
* feat(security): add System User protection with `__` prefix
Add protected namespace for custom nodes to store sensitive data
(API keys, licenses) that cannot be accessed via HTTP endpoints.
Key changes:
- New API: get_system_user_directory() for internal access
- New API: get_public_user_directory() with structural blocking
- 3-layer defense: header validation, path blocking, creation prevention
- 54 tests covering security, edge cases, and backward compatibility
System Users use `__` prefix (e.g., __system, __cache) following
Python's private member convention. They exist in user_directory/
but are completely blocked from /userdata HTTP endpoints.
* style: remove unused imports
* Add get_subgraphs_dir to ComfyExtension and PUBLISHED_SUBGRAPH_DIRS to nodes.py
* Created initial endpoints, although the returned paths are a bit off currently
* Fix path and actually return real data
* Sanitize returned /api/global_subgraphs entries
* Remove leftover function from early prototyping
* Remove added whitespace
* Add None check for sanitize_entry
Adds installed and required workflow templates version information to the
/system_stats endpoint, allowing the frontend to detect and notify users
when their templates package is outdated.
- Add get_installed_templates_version() and get_required_templates_version()
methods to FrontendManager
- Include templates version info in system_stats response
- Add comprehensive unit tests for the new functionality
- Update comfyui-frontend-package from 1.25.9 to 1.25.10
- Revert forced legacy navigation mode from PR #9518
- Frontend v1.25.10 includes proper navigation mode fixes and improved display text
* Added the parameter required_frontend_version in the /system_stats api response
* Update server.py
* Created a function get_required_frontend_version and wrote tests for it
* Refactored the function to return currently installed frontend pacakage version
* Moved required_frontend to a new function and imported that in server.py
* Corrected test cases using mocking techniques
* Corrected files to comply with ruff formatting
* Add '@prerelease' to use latest test frontend
Allows download of pre-release versions.
Will always get the latest pre-release version - even if it's older than the latest stable release.
* nit
* install templates as pip package
* Update requirements.txt
* bump templates version to include hidream
---------
Co-authored-by: Chenlei Hu <hcl@comfy.org>
* Ensuring a 401 error is returned when user data is not found in multi-user context.
* Returning a 401 error when provided comfy-user does not exists on server side.
* improved: better installation guide
- change `pip` to `{sys.executable} -m pip`
modified: To prevent the guide message from being obscured by a complex error message, apply `exit` instead of `raise`.
* ruff fix
