Use the RAM right up to the wire as the community is bit accustomed too.
This trades off headroom for the case where large chunky intermediates
arrive and potenitally hits pagefile/swap, but a lot of people have
"it just fits" workflows out there, so strike a compromise with
75->90%.
Disable the incative cache for all but the very high RAM users.
* openapi: add enum values + FeedbackRequest schema for cloud cutover (PR E)
Adds missing cloud-runtime enum values to vendor schemas that the
cloud runtime emits but vendor declared as plain strings.
Changes:
- JobEntry.status: enum [pending, in_progress, completed, failed, cancelled]
- JobDetailResponse.status: same enum
- BillingStatus: enum [awaiting_payment_method, pending_payment, paid,
payment_failed, inactive]
- FeedbackRequest schema added (with type enum)
- /api/feedback POST: requestBody now $refs FeedbackRequest
All cloud-runtime-emitted; no impact on OSS-local semantics.
Identified via Comfy-Org/cloud's TestCutoverSafe gate (BE-1106) as
the remaining schema-level divergences after PRs A-D landed and got
synced.
* openapi: add type enum to Workspace schema (cutover follow-up)
Cloud's Workspace runtime shape includes a 'type' field with enum
[personal, team] that vendor's Workspace was missing. Cloud handlers
reference the generated ingest.WorkspaceType Go enum.
Same kind of surgical addition as JobEntry.status / BillingStatus /
JobDetailResponse.status in this PR — adds cloud-runtime field to
existing vendor schema.
* openapi: rename cloud-side response schemas to match runtime (PR D)
Follow-up to the BE-1106 stack (#14060/61/63). Cloud's Go handlers
reference response schemas by name (e.g., ingest.WorkflowResponse,
ingest.SubscribeResponse), but vendor's matching operations were
declaring those responses against differently-named vendor-side
schemas (CloudWorkflow, BillingSubscription, etc.). After the stack
landed, schemas like WorkflowResponse exist in vendor but weren't
referenced by any path, so codegen pruned the unreferenced types.
This PR:
1. Updates 34 operation $refs in cloud-runtime paths to point to
the schema names cloud's handlers expect (e.g., CloudWorkflow →
WorkflowResponse on /api/workflows/{workflow_id}).
2. Adds 12 cloud-only schemas that weren't in vendor yet but are
referenced by these renames (e.g., SubscribeResponse,
CancelSubscriptionResponse, BillingOpStatusResponse). Each
copied verbatim from Comfy-Org/cloud's hand-written ingest spec
and tagged x-runtime: [cloud] with a [cloud-only] description
prefix.
Schema renames span the same domains as the operationId renames in
PR A: billing/subscriptions (7 schemas), workflows (5), userdata (3),
jobs (2), hub (2), history (2), auth/workspace (4), and misc cloud
endpoints (9).
Convergent safety check after this lands (against cloud's
TestCutoverSafe gate, BE-1106):
Pre-PR D: 205 missing handler refs
Post-PR D: 105 missing handler refs (-49%)
Cumulative since the original 938-ref baseline: -89%
The remaining 105 are a Phase 3 follow-up (response headers,
text/plain responses, codegen-derived enum sub-types, and a small
set of inline-response-schema operations that vendor declares
inline where cloud has named-schema $refs).
* openapi: drop PR-label comment from new schemas block
PR-internal labels don't belong in committed code — future readers
won't know what 'PR D' means and the marker stops being useful the
moment this PR merges.
* openapi: rename 55 cloud-side operationIds to match runtime handlers
For the 55 operations below, vendor's operationId did not match the
name cloud's runtime handlers expect. Generated types from vendor
therefore had different names (e.g. CreateSubscription200JSONResponse)
than what cloud handlers reference (Subscribe200JSONResponse), which
blocks the post-cutover combined-spec codegen.
All 55 renames target the cloud-runtime-authoritative name. Several
of these endpoints are shared concepts (queue, settings, userdata,
object_info) that OSS local also serves — the rename aligns vendor
with the longstanding cloud handler-side convention to unblock the
shared codegen. No request/response *shape* changes in this PR; only
operationId labels.
Notable categories:
- Billing/subscriptions: 7 renames (subscribe, getBillingPlans, ...)
- Workspace + workflows: 13 renames (createWorkflow, ...)
- Hub: 3 renames
- Auth/users: 5 renames
- Shared OSS surface (settings, queue, view, userdata): 12 renames
- Misc cloud-only: 15 renames
Identified via Comfy-Org/cloud's TestCutoverSafe build-safety gate
(BE-1106), which compares handler type references against codegen
output from the combined spec.
* fix(openapi): resolve getHistory operationId collision
Spectral flagged: both /api/history (OSS local) and /api/history_v2
(cloud) had operationId 'getHistory' after the rename. Rename vendor's
/api/history to 'getPromptHistory' to disambiguate. Cloud's runtime
denies /api/history at the overlay level so combined codegen is
unaffected by this change.
* openapi: add 41 cloud-runtime schemas to components.schemas (PR B of 3) (#14061)
* openapi: add 41 cloud-runtime schemas to components.schemas (cutover prep)
Adds schemas that exist in Comfy-Org/cloud's hand-written ingest spec
but not yet in this vendored OSS spec. All tagged x-runtime: [cloud]
per the field-drift convention and prefixed with [cloud-only] in the
description.
These schemas are referenced by cloud's Go handlers via the generated
ingest.<Schema> Go type names. Codegen from the vendored spec didn't
produce those types because the schemas weren't declared here. Adding
them unblocks the post-cutover combined-spec codegen.
Schemas added (alphabetical):
AssetDownloadResponse, AssetMetadataResponse, BillingBalanceResponse,
BillingPlansResponse, BillingStatusResponse, GetUserDataResponseFull,
HistoryDetailEntry, HistoryDetailResponse, HistoryResponse,
HubLabelInfo, HubProfileSummary, HubWorkflowListResponse,
HubWorkflowStatus, HubWorkflowSummary, HubWorkflowTemplateEntry,
JobStatusResponse, JobsListResponse, LabelRef, LogsResponse, Member,
OAuthRegisterBadRequestResponse, PendingInvite, Plan, PlanAvailability,
PlanAvailabilityReason, PlanSeatSummary, PreviewPlanInfo,
PreviewSubscribeResponse, PublishedWorkflowDetail, SecretResponse,
SubscriptionDuration, SubscriptionTier, UserDataResponseFull,
ValidationError, ValidationResult, WorkflowForkedFrom, WorkflowResponse,
WorkflowVersionContentResponse, WorkspaceAPIKeyInfo, WorkspaceSummary,
WorkspaceWithRole
Identified via Comfy-Org/cloud's TestCutoverSafe build-safety gate
(BE-1106). Companion to PR #14060 (operationId renames).
* fix(openapi): add BindingErrorResponse schema
OAuthRegisterBadRequestResponse references BindingErrorResponse but
that schema wasn't in the original add. Adding it now as a cloud-only
schema matching the cloud runtime's binding-error shape (single
'message' string field).
* openapi: add missing 4xx/5xx response bodies for cloud-emitting endpoints (#14063)
Vendor declares shared endpoints (e.g. /api/queue, /api/settings,
/api/assets/*, /api/billing/*) with success responses but is missing
many of the 4xx/5xx error response bodies that Comfy-Org/cloud's
runtime actually emits. Cloud's Go handlers reference the generated
ingest.Op<StatusCode>JSONResponse types for these missing statuses,
which currently fail to resolve when codegen runs against the
vendored spec.
This PR adds 237 response entries across 117 operations, restoring
the documented error responses that cloud emits. Bodies are copied
verbatim from Comfy-Org/cloud's hand-written ingest spec
(services/ingest/openapi.yaml) and reference a new ErrorResponse
schema also added in this PR (matches cloud's {code, message} runtime
shape, tagged x-runtime: [cloud]).
ErrorResponse is intentionally separate from the existing CloudError
schema. CloudError's shape ({error}) describes one runtime; cloud
emits a different shape ({code, message}). Existing CloudError refs
in vendor are untouched; new cloud-emitting error references use
ErrorResponse.
Identified via Comfy-Org/cloud's TestCutoverSafe build-safety gate
(BE-1106). Companion to PR #14060 (operationId renames) and PR #14061
(cloud-only schema additions).
* openapi: align response declarations with implementation (5 endpoints)
- POST /api/assets/download: replace 200 with 202 + tracking-task body
(endpoint runs asynchronously and returns task_id/status/message).
- POST /api/assets/export: same 200 → 202 + tracking-task body.
- POST /api/assets/from-workflow: change 201 → 200 (handler responds 200,
not 201; no Location header emitted).
- POST /api/feedback: change 200 → 201 (creates a feedback record).
- /api/jobs and /api/jobs/{job_id}: change timestamp fields from
type: number to type: integer + format: int64. Values are Unix
milliseconds — number causes oapi-codegen to emit float64, losing
precision and producing the wrong Go type. Affected fields:
create_time, update_time, execution_start_time, execution_end_time.
Verification: each change reflects what the endpoint observably returns;
no handler changes required. Backwards-compatible for existing clients
(integer is a subset of number; status code shifts within 2xx).
* openapi: align asset download/export 202 status enum with runtime + sibling schemas
CodeRabbit caught a vocabulary mismatch: the two new 202 response schemas
declared `[pending, running, completed, failed]` while the rest of the same
spec uses `[created, running, completed, failed]` for the identical task
lifecycle (download/export progress WebSocket events, /api/tasks, TaskEntry,
TaskResponse — 4 sites total). Cloud's runtime emits `created` on initial
creation (AssetDownloadResponseStatusCreated; task.Status sourced from the
DB enum whose initial value is Created). `pending` would have introduced a
fifth, contradictory vocabulary for the same lifecycle and pushed the spec
further from the implementation it is meant to align with.
Followup tracked separately: extract a shared TaskStatus enum so all five
sites move in lockstep instead of needing per-site edits.
* ModelPatcherDyanmic: purge stale vbar allocs on force cast
* ModelPatcherDynamic: restore backups before load
If doing a clean reload, mutative changes (lora application) could be
applied on-top of the already loaded weight. Restore from backup
unconditionally so that the new load is clean.
The job_ids query parameter on GET /api/assets is tagged x-runtime:
[cloud] and only exists for cloud's variant of this endpoint. Cloud
removed all consumers and the cloud-side handler/codegen/tests in
Comfy-Org/cloud#3778. With cloud no longer accepting this parameter,
the [cloud-only] documentation here is wrong — drop it so the daily
sync to cloud/services/ingest/vendor/openapi.yaml propagates the
removal.
The operation at POST /api/assets/import was defined as `importAssets`
with a URL-list body shape, but no runtime actually serves that
operation at this path. The cloud runtime serves a different operation
here — `importPublishedAssets` — which imports published-workflow
assets into the caller's library by ID, not by URL.
Cloud's URL-based asset ingestion lives at separate paths
(POST /assets/download + GET /assets/remote-metadata) tracked
elsewhere; nothing in this PR affects that work.
Changes:
- Replace the operation at POST /api/assets/import with
`importPublishedAssets`, taking ImportPublishedAssetsRequest
(published_asset_ids + optional share_id) and returning
ImportPublishedAssetsResponse (list of AssetInfo).
- Remove the unused AssetImportRequest component schema (no other
references in the spec).
- Operation and schemas tagged x-runtime: [cloud] with [cloud-only]
description prefix, matching the existing convention for
cloud-runtime-only operations elsewhere in the spec.
Spectral lint passes (0 errors); the two hint-level findings on
the spec are pre-existing and unrelated.
No FE consumer references AssetImportRequest today; this is a pure
spec correction to match what the cloud runtime actually serves.
Add the OAuth 2.1 authorization flow and RFC 7591 Dynamic Client
Registration endpoints to the shared spec, alongside the existing
auth-tagged operations (/api/auth/session, /api/auth/token,
/.well-known/jwks.json). All tagged x-runtime: [cloud] with a
[cloud-only] description prefix, following the established
convention for cloud-runtime-only operations.
Endpoints:
- GET /.well-known/oauth-authorization-server (RFC 8414 metadata)
- GET /.well-known/oauth-protected-resource (RFC 9728 metadata)
- GET /oauth/authorize (consent challenge)
- POST /oauth/authorize (consent submission)
- POST /oauth/token (RFC 6749 §3.2)
- POST /oauth/register (RFC 7591 §3.1 DCR)
Component schemas added:
- OAuthAuthorizationServerMetadata
- OAuthProtectedResourceMetadata
- OAuthConsentChallenge, OAuthConsentChallengeWorkspace
- OAuthAuthorizeRedirectResponse
- OAuthTokenResponse, OAuthTokenError
- OAuthRegisterRequest, OAuthRegisterResponse, OAuthRegisterError
These endpoints are implemented in the cloud runtime today and
are called by browser frontends rendering the consent UI and by
MCP-spec-compliant clients (Claude Desktop, Cursor, etc.) doing
auto-discovery + self-registration. Documenting them in the
shared spec lets the cloud frontend generate types directly from
this spec instead of maintaining a parallel definition.
Spectral lints clean (0 errors). The hint-level findings on
OAuthTokenError / OAuthRegisterError ("standard error schema")
match the same hint on CloudError — these are protocol-specific
RFC-shaped errors, not generic application errors.
The per-tag microsecond stagger preserves intra-batch order, but two
back-to-back write batches on the same reference (e.g.
set_reference_tags for path tags, then add_tags_to_reference for user
tags) call get_utc_now() independently. On Windows the system clock can
return the same datetime for both calls if no OS tick elapsed between
the commits — both batches end up sharing microseconds and
ORDER BY added_at, tag_name falls back to the alphabetic tiebreaker,
sorting user tags ahead of path tags they were meant to follow.
Add _next_added_at_base(reference_id) that reads max(existing added_at)
and returns max(existing + 1us, get_utc_now()), guaranteeing the new
batch sorts strictly after anything previously written for that
reference. Used by set_reference_tags and add_tags_to_reference;
batch_insert_seed_assets stays on raw get_utc_now() since seed inserts
are always the first writes for a new reference.
The accompanying regression test pins get_utc_now() to a frozen value
so the previously-Windows-only race becomes a platform-independent
failure mode under test.
Path-derived tags for nested model layouts (e.g.
models/checkpoints/flux/foo.safetensors) emitted only the slash-joined
shape `["models", "checkpoints/flux"]`, which broke the frontend
combo-widget set-membership filter `include_tags=models,checkpoints` —
the literal `checkpoints` token was no longer present in the asset's
tag set.
Add `expand_bucket_prefixes` at the tag-write layer. When a tag's first
slash segment is a registered model category (or input/output/temp
root), the bucket is inserted as a standalone token immediately after
the slash-joined form. This preserves tag[1] as the slash-joined
positional contract cloud emits while restoring the set-membership
token the frontend filter requires.
The expansion is bounded to known buckets so free-form user labels
with slashes (`my-org/team-a`) pass through unchanged. The helper is
applied uniformly in `set_reference_tags`, `add_tags_to_reference`,
and `batch_insert_seed_assets` so HTTP uploads, user-tag mutations,
and path-scanning ingest all converge on the same canonical shape.
Also align the upload-route category validator with
`resolve_destination_from_tags` by extracting the first slash segment
of tag[1], so HTTP uploads matching cloud's slash-joined emission
shape are no longer rejected as `unknown models category`.
* model_management: disable non-dynamic smart memory
Disable smart memory outright for non dynamic models.
This is a minor step towards deprecation of --disable-dynamic-vram
and the legacy ModelPatcher.
This is needed for estimate-free model development, where new models
can opt-out of supplying a memory estimate and not have to worry
about hard VRAM allocations due to legacy non-dynamic model patchers
This is also a general stability increase for a lot of stray use cases
where estimates may still be off and going forward we are not going
to accurately maintain such estimates.
* pinned_memory: implement with aimdo growable buffer
Use a single growable buffer so we can do threaded pre-warming on
pinned memory.
* mm: use aimdo to do transfer from disk to pin
Aimdo implements a faster threaded loader.
* Add stream host pin buffer for AIMDO casts
Introduce per-offload-stream HostBuffer reuse for pinned staging,
include it in cast buffer reset synchronization.
Defer actual casts that go via this pin path to a separate pass
such that the buffer can be allocated monolithically (to avoid
cudaHostRegister thrash).
* remove old pin path
* Implement JIT pinned memory pressure
Replace the predictive pin pressure mechanism with JIT PIN memory
pressure.
* LowVRAMPatch: change to two-phase visit
* lora: re-implement as inplace swiss-army-knife operation
* prepare for multiple pin sets
* implement pinned loras
* requirements: comfy-aimdo 0.4.0
* ops: remove unused arg
This was defeatured in aimdo iteration
* ops: sync the CPU with only the offload stream activity
This was syncing with the offload stream which itself is synced with the
compute stream, so this was syncing CPU with compute transitively. Define
the event to sync it more gently.
* pins: implement freeing intermediate for pinned memory
Pinning is more important than inactive intermediates and the stream
pin buffer is more important than even active intermediates.
* execution: implement pin eviction on RAM presure
Add back proper pin freeing on RAM pressure
* implement pin registration swaps
Uncap the windows pins from 50% by extending the pool and have a pressure
mechanism to move the pin reservations om demand.
This unfortunately implies a GPU sync to do the freeing so significant
hysterisis needs to be added to consolidate these pressure events.
* cli_args/execution: Implement lower background cache-ram threshold
Limit the amount of RAM background intermediates can use, so that
switching workflows doesn't degrade performance too much.
* make default
* bump aimdo
* model-patcher: force-cast tiny weights
Flux 2 gets crazy stalls due to a mix of tiny and giant weights
creating lopsided steam buffer rotations which creates stalls.
* ops: refactor in prep for chunking
* mm: delegate pin-on-the-way to aimdo
Aimdo is able to chunk and slice this on the way for better CPU->GPU
overlap. The main advantage is the ability to shorten the bus contention
window between previous weight transfer and the next weights vbar
fault.
* bump aimdo
* pinning updates
* specify hostbuf max allocation size
There a signs of virtual memory exhaustion on some linux systems when
throwing 128GB for every little piece. Pass the actual to save aimdo
from over-estimates
* tests: update execution tests for caching
The default caching changed to ram-cache so update these tests
accordingly.
Remove the LRU 0 test as this also falls through to RAM cache.
Smoke test through the real HTTP upload + tag-add path exposed two
ordering bugs the unit-layer tests missed:
1. add_tags_to_reference did `to_add = sorted(want - current)` — an
alphabetical pre-sort defeating the microsecond-stagger fix from the
previous commit. The stagger was encoding alphabetical positions,
not the caller's insertion order. Fix: build to_add by walking the
already-normalized caller list and filtering against the current
set, so the staggered added_at timestamps reflect what the caller
actually requested.
2. get_reference_tags used .order_by(tag_name.asc()) — alphabetical.
It's called by the upload response path; meanwhile
list_references_page and fetch_reference_asset_and_tags were already
updated to order by added_at. The mismatch meant POST /api/assets
returned tags in alphabetical order but a subsequent GET returned
them in insertion order. Fix: order get_reference_tags by added_at
too, so all three response-path helpers agree.
New tests-unit/assets_test/test_user_tag_http_smoke.py exercises the
full HTTP layer: POST /api/assets to upload, POST /api/assets/{id}/tags
to add a user tag (using tag names like "aaa-user-tag" that would jump
to position 0 under alphabetical), GET /api/assets/{id} to verify
ordering. Catches the bugs above in CI going forward.
Full assets suite: 340 passed, 10 pre-existing skipped.
Cursor-reviews follow-up on PR #13994:
1. set_reference_tags / add_tags_to_reference now apply the same
microsecond stagger as batch_insert_seed_assets. Per-tag get_utc_now()
calls can collide at microsecond resolution on fast machines, dropping
retrieval to the tag_name alphabetical tiebreaker. Using a single
base_ts + timedelta(microseconds=i) preserves insertion order for any
batch.
2. Docstring on get_name_and_tags_from_asset_path corrected: only the
subpath is lowercased in code; the root category is lowercase by
construction in get_asset_category_and_relative_path.
3. resolve_destination_from_tags docstring now states explicitly that
hybrid shapes (mix of legacy multi-tag + new slash-joined within a
single call) are accepted and resolve to the same destination.
4. New TestTagRetrievalOrder class in test_asset_info.py exercises the
public write paths (set_reference_tags, add_tags_to_reference,
remove_tags_from_reference) and asserts the public read paths
(list_references_page, fetch_reference_asset_and_tags) return tags
in insertion order rather than alphabetical. Tag names are chosen
to fail loudly under alphabetical regression — "checkpoints" sorts
before "models", "aaa-user-tag" sorts before every path tag, etc.
Full assets suite: 338 passed, 10 pre-existing skipped.
Three bugs surfaced by an end-to-end smoke test of the read+write
round-trip; all in this PR's scope.
1. FK violation on uppercase paths
get_name_and_tags_from_asset_path was preserving case on the
subpath (e.g. "diffusers/Kolors/text_encoder"). ensure_tags_exist
lowercases via normalize_tags before inserting into the tags
table, so the asset_reference_tags.tag_name FK to tags.name
failed for any path containing uppercase letters — including
the diffusers case the PR was designed to support.
Fix: lowercase the slash-joined subpath in
get_name_and_tags_from_asset_path to match the canonicalization
ensure_tags_exist applies. Providers keyed on original-case
subpaths need to normalize their lookup key to lowercase.
2. resolve_destination_from_tags rejected the new tag shape
The inverse function only accepted the legacy one-tag-per-dir
shape (["models", "diffusers", "Kolors", "text_encoder"]).
An upload using the slash-joined shape returned by /api/assets
raised "unknown model category" or "invalid path component".
Fix: pre-split every entry after tags[0] on "/" so both shapes
resolve identically. For models, the first expanded segment is
the category and the rest are subdirs; for input/output the
full expansion becomes the subdirs.
3. Within-batch tag order was lost
bulk_ingest wrote every tag in a single batch with the same
added_at = current_time. The retrieval ORDER BY added_at, tag_name
then fell back to the tag_name tiebreaker, sorting the path-derived
pair alphabetically — putting "checkpoints/..." ahead of "models"
since "c" < "m". The tags[0] = root contract was lost on bulk-
ingested rows.
Fix: stagger added_at by microseconds per tag index within a
reference so the retrieval order matches the input list order.
Path-derived tags now consistently land in position-0 = root,
position-1 = subpath.
Tests
- TestGetNameAndTagsFromAssetPath updated: subpath is now lowercase.
- New TestResolveDestinationFromTags covers both tag shapes, the
unknown-category case for slash-joined input, traversal rejection,
and input/output paths.
- Full suite: 333 passed, 10 pre-existing skipped.
