Skip to content

MVP vertical slice: deterministic-first model selection, placement, and acquisition proof #18

Description

@mark-e-deyoung

Purpose

Turn #16/#17 into one bounded, falsifiable post-identity MVP rather than a broad model platform.

Governing principle from the Generator–Validator thesis:

Deterministic machinery owns what we know. Generator–Validator works the frontier of what we don't yet know. Successful work at that frontier should move the boundary outward.

Critical path

This issue is downstream of #9/#12. Do not start implementation that depends on unresolved artifact identity semantics.

  1. Finish the identity fixture increment (Increment 1: capture metadata-only provider fixtures #12) and minimum identity resolver from MVP: resolve and observe external model identities #9. Prove exact/strongest available identity from captured provider evidence without model downloads.
  2. Define the smallest project-facing request and response contracts. Request: capability + operating envelope + placement policy. Response: selected model/representation/runtime/target plus evidence/qualification state. Keep UNKNOWN first-class.
  3. Define a static private resource inventory contract. Only enough fields to distinguish the first real targets (OS/arch/RAM/VRAM/runtime/provider instance). No scheduler, utilization daemon, autoscaler, or fleet database.
  4. Implement deterministic fast-path planning first. Known qualified tuple lookup, hard constraint rejection, deterministic ranking, exact identity resolution. No LLM/model generator required for known cases.
  5. Add one bounded UNKNOWN/candidate path. Reuse existing public/provider evidence and existing fit tools where useful to propose a small candidate set. Do not build a general crawler/recommender.
  6. Implement one provider-native acquisition path. Hugging Face first, using native shared cache and exact revision; no duplicate Foundry blob store.
  7. Integrate one real consuming project. Remove its direct provider/download/cache-selection mechanics and leave a capability/dependency declaration plus runtime consumption of the returned native handle.
  8. Run the project-owned quality validator plus a bounded performance/resource measurement. Emit a deterministic qualification receipt bound to exact artifact + representation + runtime/toolchain + target + workload.
  9. Prove the determinization ratchet. Repeat an equivalent in-envelope request and require deterministic catalog resolution without candidate generation/research. Then change one material target/envelope variable and require either a known alternate or explicit UNKNOWN -> bounded exploration.
  10. Only after this proof, add a second consumer/provider/runtime to test generality. Statistical prediction, public catalog UI, community submission hosting, broad multimodal support, dynamic placement, and global catalog ingestion remain deferred.

MVP falsification criteria

The concept is not proven merely because it can download a model. The vertical slice should fail/redirect if it cannot demonstrate most of the following:

  • a real consumer loses meaningful provider/model acquisition logic;
  • exact artifact identity is preserved across selection, acquisition, and qualification;
  • at least two plausible model/representation/runtime/placement candidates can be deterministically accepted/rejected for explicit reasons;
  • project-owned quality evidence participates in qualification without being centralized into Foundry;
  • one target/environment change materially changes a selection or produces UNKNOWN;
  • an equivalent repeat request bypasses Generator–Validator exploration;
  • provider-native storage/cache ownership is preserved;
  • the qualification receipt is reproducible and useful enough to drive a future decision.

Candidate-consumer disposition

First integration candidate: SemperSupra/desktop-ui-cv

Preferred first consumer for the broker/interface proof because it already has direct Hugging Face model acquisition (from_pretrained), explicit model/cache/device knobs, model provenance/deployment metadata, benchmark/validation surfaces, and is not currently in the middle of BHADA's release-critical stabilization chain. Keep the first slice to one capability/model path; do not absorb its project-specific registry/evaluators into Foundry.

Second / stronger falsification candidate: mark-e-deyoung/BHADA

BHADA is not too simple. Its multilingual ASR surface has multiple Whisper model sizes, two backend families (faster_whisper / whisper), explicit cache/download behavior, and natural quality/performance measures (e.g. transcription quality + real-time factor). It is a strong test of selection/acquisition determinization and would directly avoid building a separate repo-local artifact catalog/mirror. However, BHADA issue #83 currently has a release/stabilization critical path and issue #88 is follow-on artifact work, so do not inject the first shared-substrate implementation into the current merge/release gates. Use BHADA after the first Foundry slice is stable, and require at least two target profiles/backends so the test is not merely "choose Whisper size".

Later stress test: mark-e-deyoung/ml-depth-pro

Excellent heterogeneity test, poor first MVP consumer. Its current sync_models.py spans direct HTTP, Hugging Face, archives, entire repos, renamed files, unavailable stubs, and multiple checkpoint formats. Use it only after the basic interface/acquisition path is proven; otherwise its edge cases will drive premature abstraction.

Stop / scope guard

Do not build a web catalog, generalized benchmark service, statistical capability predictor, new artifact warehouse, cluster scheduler, broad provider matrix, or generic model ontology to satisfy this issue.

The MVP exists to prove that repeated project-local model research/acquisition can become a progressively deterministic shared engineering capability.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions