feat(modelartifacts): support bounded parallel Hugging Face file downloads (#11162)

* feat(modelartifacts): support bounded parallel Hugging Face file downloads

Closes #11114.

Snapshot materialization fetched every file through the sequential
executor in DownloadFilesWithContext, so a repository split into many
shards spent most of its wall clock in per-file request latency rather
than moving bytes.

Add DownloadFilesWithConcurrency, an errgroup with SetLimit, and keep
DownloadFilesWithContext as a wrapper that passes a limit of 1. That
leaves the two non-artifact callers (core/gallery and the model config
loader) on exactly the path they had: tasks still run in slice order,
and the first failure still returns before any later task starts.

Only whole files run in parallel. A single file is never split, so the
.partial resume machinery and the per-file SHA check in
downloadTaskWithRetry are untouched.

Two details the parallel path forced:

- completedBytes becomes an atomic.Int64. Several AfterDownload hooks
  add to it while other files' progress callbacks read it; without this
  the race detector reports three races on the new specs.
- The caller's status callback is serialized. The sequential path gave
  it an implicit guarantee of never being entered twice at once, and it
  belongs to the caller, so the executor keeps that promise rather than
  pushing locking onto every caller. AfterDownload is deliberately not
  serialized -- it does the verify-and-promote work that parallelism
  exists to overlap.

Manifest order needed no work: each hook already writes its own
manifest.Files slot by snapshot index, so entries stay in snapshot
order whatever the completion order. A spec now pins that.

The default is 1, unchanged behaviour. A shared models volume is often
the bottleneck rather than the link, so raising it is a deployment
decision; --artifact-download-concurrency and
LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY expose it on both `run` and
`models install`.

Not done here, per the issue: no chunk-level parallelism within a single
file, and no throughput measurements across concurrency 1/2/4/8 -- that
needs a representative sharded repo and a real link.

Assisted-by: Claude:claude-opus-5 go-test gofmt
Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>

* feat(modelartifacts): expose download concurrency in settings

Follow-up to review feedback on #11162:

- The CLI flag and docs no longer describe the limit as Hugging Face
  specific. It applies to any artifact source, as @mudler pointed out.
- artifact_download_concurrency is now a persisted runtime setting and
  is editable from the WebUI, so it can be changed without a restart.

The manager's limit becomes an atomic.Int64 behind
SetDownloadConcurrency, because a live runtime setting can be updated
while a materialization is already in flight. Injected materializers
stay compatible through an optional setter interface, so a manager that
does not implement it is simply left alone.

Verified before taking this on: go build, go vet and go test -race all
pass for pkg/modelartifacts, pkg/downloader and core/config. The React
UI builds with vite, artifact_download_concurrency is present in the
built Settings chunk, and eslint reports the same 8 pre-existing
warnings on Settings.jsx as it does without the change.

Implementation contributed by localai-org-maint-bot on the review
thread; reviewed, verified and signed off by me.

Assisted-by: Codex:gpt-5
Assisted-by: Claude:claude-opus-5 go-test vite eslint
Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>

---------

Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>

Author: Dennisadira

core/cli/models.go

@@ -28,6 +28,8 @@ type ModelsCMDFlags struct {
 	Color            string `env:"COLOR" hidden:""`
 	NoColor          string `env:"NO_COLOR" hidden:""`
 	HFToken          string `env:"HF_TOKEN" hidden:""`
+
+	ArtifactDownloadConcurrency int `env:"LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY" help:"How many files of a model artifact to download at once. 1 (the default) downloads sequentially. Raising it helps artifacts split into many files on a fast link, at the cost of more concurrent load on the models volume" group:"storage" default:"1"`
 }
 
 type ModelsList struct {
@@ -87,6 +89,7 @@ func (mi *ModelsInstall) Run(ctx *cliContext.Context) error {
 
 	artifactMaterializer := modelartifacts.NewDefaultManager(
 		modelartifacts.WithHuggingFaceToken(mi.HFToken),
+		modelartifacts.WithDownloadConcurrency(mi.ArtifactDownloadConcurrency),
 	)
 	galleryService := galleryop.NewGalleryService(&config.ApplicationConfig{
 		SystemState:               systemState,

core/cli/run.go

@@ -41,6 +41,7 @@ type RunCMD struct {
 	BackendsPath                 string        `env:"LOCALAI_BACKENDS_PATH,BACKENDS_PATH" type:"path" default:"${basepath}/backends" help:"Path containing backends used for inferencing" group:"backends"`
 	BackendsSystemPath           string        `env:"LOCALAI_BACKENDS_SYSTEM_PATH,BACKEND_SYSTEM_PATH" type:"path" default:"/var/lib/local-ai/backends" help:"Path containing system backends used for inferencing" group:"backends"`
 	ModelsPath                   string        `env:"LOCALAI_MODELS_PATH,MODELS_PATH" type:"path" default:"${basepath}/models" help:"Path containing models used for inferencing" group:"storage"`
+	ArtifactDownloadConcurrency  int           `env:"LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY" help:"How many files of a model artifact to download at once. 1 (the default) downloads sequentially. Raising it helps artifacts split into many files on a fast link, at the cost of more concurrent load on the models volume" group:"storage" default:"1"`
 	GeneratedContentPath         string        `env:"LOCALAI_GENERATED_CONTENT_PATH,GENERATED_CONTENT_PATH" type:"path" default:"${generatedcontentpath}" help:"Location for generated content (e.g. images, audio, videos)" group:"storage"`
 	UploadPath                   string        `env:"LOCALAI_UPLOAD_PATH,UPLOAD_PATH" type:"path" default:"${uploadpath}" help:"Path to store uploads from files api" group:"storage"`
 	DataPath                     string        `env:"LOCALAI_DATA_PATH" type:"path" default:"${basepath}/data" help:"Path for persistent data (collectiondb, agent state, tasks, jobs). Separates mutable data from configuration" group:"storage"`
@@ -278,8 +279,10 @@ func (r *RunCMD) Run(ctx *cliContext.Context) error {
 
 	opts := []config.AppOption{
 		config.WithContext(context.Background()),
+		config.WithArtifactDownloadConcurrency(r.ArtifactDownloadConcurrency),
 		config.WithModelArtifactMaterializer(modelartifacts.NewDefaultManager(
 			modelartifacts.WithHuggingFaceToken(r.HFToken),
+			modelartifacts.WithDownloadConcurrency(r.ArtifactDownloadConcurrency),
 		)),
 		config.WithModelPreloadDisplay(r.Color, r.NoColor != ""),
 		config.WithConfigFile(r.ModelsConfigFile),

core/config/application_artifact_materializer_test.go

@@ -16,6 +16,15 @@ func (*applicationArtifactMaterializer) Ensure(context.Context, string, modelart
 	return modelartifacts.Result{}, nil
 }
 
+type configurableApplicationArtifactMaterializer struct {
+	applicationArtifactMaterializer
+	concurrency int
+}
+
+func (m *configurableApplicationArtifactMaterializer) SetDownloadConcurrency(concurrency int) {
+	m.concurrency = concurrency
+}
+
 var _ = Describe("ApplicationConfig model artifact materializer", func() {
 	It("provides a default materializer", func() {
 		Expect(NewApplicationConfig().ModelArtifactMaterializer).NotTo(BeNil())
@@ -31,4 +40,15 @@ var _ = Describe("ApplicationConfig model artifact materializer", func() {
 		Expect(field.Tag.Get("json")).To(Equal("-"))
 		Expect(field.Tag.Get("yaml")).To(Equal("-"))
 	})
+
+	It("applies runtime download concurrency to configurable materializers", func() {
+		materializer := &configurableApplicationArtifactMaterializer{}
+		appConfig := NewApplicationConfig(WithModelArtifactMaterializer(materializer))
+		concurrency := 4
+
+		appConfig.ApplyRuntimeSettings(&RuntimeSettings{ArtifactDownloadConcurrency: &concurrency})
+
+		Expect(appConfig.ArtifactDownloadConcurrency).To(Equal(4))
+		Expect(materializer.concurrency).To(Equal(4))
+	})
 })

core/config/application_config.go

@@ -35,6 +35,7 @@ type ApplicationConfig struct {
 	// network interfaces (e.g. eth0), filtering out docker0/veth noise.
 	WebRTCICEInterfaces                 []string
 	UploadLimitMB, Threads, ContextSize int
+	ArtifactDownloadConcurrency         int
 	F16                                 bool
 	Debug                               bool
 	EnableTracing                       bool
@@ -58,12 +59,12 @@ type ApplicationConfig struct {
 	// gzip is skipped. 0 keeps middleware.DefaultCompressionMinLength.
 	HTTPCompressionMinLength int
 	PreloadJSONModels        string
-	PreloadModelsFromPath         string
-	CORSAllowOrigins              string
-	ApiKeys                       []string
-	P2PToken                      string
-	P2PNetworkID                  string
-	Federated                     bool
+	PreloadModelsFromPath    string
+	CORSAllowOrigins         string
+	ApiKeys                  []string
+	P2PToken                 string
+	P2PNetworkID             string
+	Federated                bool
 
 	// ExternalBaseURL is the externally visible base URL of this instance
 	// (scheme+host[:port]), set via LOCALAI_BASE_URL. When non-empty it is
@@ -276,11 +277,12 @@ func NewApplicationConfig(o ...AppOption) *ApplicationConfig {
 		// force-enables it). It's a small in-memory ring buffer; the Settings
 		// toggle can still turn it off (a persisted false wins - see
 		// loadRuntimeSettingsFromFile).
-		EnableBackendLogging:     true,
-		AgentJobRetentionDays:    30,               // Default: 30 days
-		LRUEvictionMaxRetries:    30,               // Default: 30 retries
-		LRUEvictionRetryInterval: 1 * time.Second,  // Default: 1 second
-		ModelLoadFailureCooldown: 10 * time.Second, // Default: 10s base cooldown after a failed load
+		EnableBackendLogging:        true,
+		ArtifactDownloadConcurrency: modelartifacts.DefaultDownloadConcurrency,
+		AgentJobRetentionDays:       30,               // Default: 30 days
+		LRUEvictionMaxRetries:       30,               // Default: 30 retries
+		LRUEvictionRetryInterval:    1 * time.Second,  // Default: 1 second
+		ModelLoadFailureCooldown:    10 * time.Second, // Default: 10s base cooldown after a failed load
 		// WatchDogInterval is intentionally left at the zero value here.
 		// The startup loader applies a persisted runtime_settings.json value
 		// only when the interval is still 0 (its "not set by env var"
@@ -685,6 +687,15 @@ func WithModelArtifactMaterializer(materializer ArtifactMaterializer) AppOption
 	}
 }
 
+func WithArtifactDownloadConcurrency(concurrency int) AppOption {
+	return func(o *ApplicationConfig) {
+		if concurrency < 1 {
+			concurrency = modelartifacts.DefaultDownloadConcurrency
+		}
+		o.ArtifactDownloadConcurrency = concurrency
+	}
+}
+
 // WithModelPreloadDisplay configures terminal rendering for model preload output.
 func WithModelPreloadDisplay(renderMode string, disableColor bool) AppOption {
 	return func(o *ApplicationConfig) {
@@ -1190,6 +1201,11 @@ func (o *ApplicationConfig) ApplyRuntimeSettings(settings *RuntimeSettings) (req
 			xsysinfo.SetDefaultVRAMBudget(b)
 		}
 	}
+	if settings.ArtifactDownloadConcurrency != nil {
+		if configurable, ok := o.ModelArtifactMaterializer.(interface{ SetDownloadConcurrency(int) }); ok {
+			configurable.SetDownloadConcurrency(o.ArtifactDownloadConcurrency)
+		}
+	}
 	// Note: ApiKeys need env-merge handling (MergeAPIKeys) - done by the
 	// caller, because the env-provided keys live on the startup config.
 	return requireRestart

core/config/runtime_settings.go

@@ -33,15 +33,16 @@ type RuntimeSettings struct {
 	LRUEvictionRetryInterval *string `json:"lru_eviction_retry_interval,omitempty"` // Interval between retries when waiting for busy models (e.g., 1s, 2s) (default: 1s)
 
 	// Performance settings
-	Threads              *int    `json:"threads,omitempty"`
-	ContextSize          *int    `json:"context_size,omitempty"`
-	VRAMBudget           *string `json:"vram_budget,omitempty"` // Cap VRAM for allocation ("80%" or "12GB"; "" = no cap)
-	F16                  *bool   `json:"f16,omitempty"`
-	Debug                *bool   `json:"debug,omitempty"`
-	EnableTracing        *bool   `json:"enable_tracing,omitempty"`
-	TracingMaxItems      *int    `json:"tracing_max_items,omitempty"`
-	TracingMaxBodyBytes  *int    `json:"tracing_max_body_bytes,omitempty"` // Per-body cap in bytes; 0 disables the cap
-	EnableBackendLogging *bool   `json:"enable_backend_logging,omitempty"`
+	Threads                     *int    `json:"threads,omitempty"`
+	ContextSize                 *int    `json:"context_size,omitempty"`
+	ArtifactDownloadConcurrency *int    `json:"artifact_download_concurrency,omitempty"`
+	VRAMBudget                  *string `json:"vram_budget,omitempty"` // Cap VRAM for allocation ("80%" or "12GB"; "" = no cap)
+	F16                         *bool   `json:"f16,omitempty"`
+	Debug                       *bool   `json:"debug,omitempty"`
+	EnableTracing               *bool   `json:"enable_tracing,omitempty"`
+	TracingMaxItems             *int    `json:"tracing_max_items,omitempty"`
+	TracingMaxBodyBytes         *int    `json:"tracing_max_body_bytes,omitempty"` // Per-body cap in bytes; 0 disables the cap
+	EnableBackendLogging        *bool   `json:"enable_backend_logging,omitempty"`
 
 	// Security/CORS settings
 	CORS             *bool   `json:"cors,omitempty"`

core/config/runtime_settings_registry.go

@@ -227,6 +227,15 @@ var runtimeSettingsFields = []fieldSpec{
 		func(s *RuntimeSettings) **int { return &s.ContextSize },
 		func(o *ApplicationConfig) int { return o.ContextSize },
 		func(o *ApplicationConfig, v int) { o.ContextSize = v }),
+	field("artifact_download_concurrency",
+		func(s *RuntimeSettings) **int { return &s.ArtifactDownloadConcurrency },
+		func(o *ApplicationConfig) int { return o.ArtifactDownloadConcurrency },
+		func(o *ApplicationConfig, v int) {
+			if v < 1 {
+				v = 1
+			}
+			o.ArtifactDownloadConcurrency = v
+		}),
 	// VRAM budget: the cap string ("80%"/"12GB"/"" = uncapped). The live
 	// side effect (xsysinfo.SetDefaultVRAMBudget) is post-processing in the
 	// apply loop, not here - the row only owns the config member, matching

core/config/runtime_settings_registry_internal_test.go

@@ -71,6 +71,7 @@ var _ = Describe("runtime settings registry", func() {
 		src.LRUEvictionRetryInterval = 3 * time.Second
 		src.Threads = 7
 		src.ContextSize = 8192
+		src.ArtifactDownloadConcurrency = 6
 		src.VRAMBudget = "12GiB"
 		src.F16 = true
 		src.Debug = true

core/config/runtime_settings_startup.go

@@ -98,4 +98,9 @@ func (o *ApplicationConfig) ApplyRuntimeSettingsAtStartup(settings *RuntimeSetti
 			xsysinfo.SetDefaultVRAMBudget(b)
 		}
 	}
+	if settings.ArtifactDownloadConcurrency != nil {
+		if configurable, ok := o.ModelArtifactMaterializer.(interface{ SetDownloadConcurrency(int) }); ok {
+			configurable.SetDownloadConcurrency(o.ArtifactDownloadConcurrency)
+		}
+	}
 }

core/http/react-ui/e2e/settings-backend-logging.spec.js

@@ -11,6 +11,13 @@ test.describe('Settings - Backend Logging', () => {
     await expect(page.locator('text=Enable Backend Logging')).toBeVisible()
   })
 
+  test('artifact download concurrency is configurable', async ({ page }) => {
+    const input = page.getByLabel('Artifact Download Concurrency')
+    await expect(input).toBeVisible()
+    await input.fill('4')
+    await expect(input).toHaveValue('4')
+  })
+
   test('backend logging toggle can be toggled', async ({ page }) => {
     // Find the checkbox associated with backend logging
     const section = page.locator('div', { has: page.locator('text=Enable Backend Logging') })

core/http/react-ui/src/pages/Settings.jsx

@@ -394,6 +394,9 @@ export default function Settings() {
               <SettingRow label="Default Context Size" description="Default context window size for models">
                 <input className="input col-w-120" type="number" value={settings.context_size ?? ''} onChange={(e) => update('context_size', parseInt(e.target.value) || 0)} placeholder="2048" />
               </SettingRow>
+              <SettingRow label="Artifact Download Concurrency" description="Maximum artifact files downloaded at once. 1 downloads sequentially.">
+                <input aria-label="Artifact Download Concurrency" className="input" type="number" min="1" style={{ width: 120 }} value={settings.artifact_download_concurrency ?? 1} onChange={(e) => update('artifact_download_concurrency', Math.max(1, parseInt(e.target.value) || 1))} />
+              </SettingRow>
               <SettingRow label="VRAM Budget" description="Cap VRAM used for model allocation on this node. Percentage (e.g. 80%) or absolute (e.g. 12GB). Empty uses all detected VRAM.">
                 <input className="input col-w-120" type="text" value={settings.vram_budget ?? ''} onChange={(e) => update('vram_budget', e.target.value)} placeholder="e.g. 80% or 12GB" />
               </SettingRow>