Skip to content

api-reference.md

Latest commit

 

History

History
75 lines (53 loc) · 2.8 KB

api-reference.md

File metadata and controls

75 lines (53 loc) · 2.8 KB
Raw

API 参考

仅列常用公共接口；完整签名以源码 docstring 为准。

顶层

import foleyflow
foleyflow.__version__
foleyflow.train_demo_generator(epochs=30, backend="flow", ...) -> (FoleyGenerator, TrainState)

FoleyGenerator（foleyflow.pipeline）

FoleyGenerator(model=None, *, audio_config=None, model_config=None, tags=None, backend="flow")
方法 / 属性 说明
generate(prompt, *, timeline=None, config=None) -> np.ndarray 文本 → 波形
generate_mel(prompt, ...) -> np.ndarray 文本 → 归一化梅尔谱 (n_mels, n_frames)
generate_to_file(prompt, path, ...) -> Path 生成并写 WAV
save(path) -> Path / FoleyGenerator.load(path, **kw) 模型持久化
n_frames / fps / native_duration 生成片段的帧数 / 帧率 / 时长

配置（foleyflow.config）

  • AudioConfig(sample_rate, n_fft, hop_length, n_mels, ...)
  • ModelConfig(n_mels, n_frames, hidden_dim, n_layers, cond_dim, ...)
  • GenerationConfig(duration_s, steps, guidance_scale, sampler, seed, align_strength)

音频（foleyflow.audio）

  • MelSpectrogram(config).forward(waveform) -> (n_mels, T)
  • griffin_lim(magnitude, *, n_fft, hop_length, n_iter=60) -> waveform
  • GriffinLimVocoder(config)(mel_amplitude, length=None) -> waveform
  • stft / istft / mel_filterbank / read_wav / write_wav

扩散（foleyflow.diffusion）

  • NoiseSchedule.create(kind="cosine", num_timesteps=1000)
  • GaussianDiffusion(schedule, parameterization="v")
    • .q_sample(x0, t_idx, noise)、.compute_loss(model, x0, cond, rng)
  • ddim_sample(model, diffusion, shape, *, cond, guidance_scale, steps, eta)
  • ddpm_sample(...)

流匹配（foleyflow.flow）

  • make_path("rectified" | "trig")、FlowMatching(path)
  • euler_sample / midpoint_sample / heun_sample(model, shape, *, cond, guidance_scale, steps)

条件（foleyflow.conditioning）

  • TagVocabulary(tags)、EmbeddingTable(vocab, dim)
  • ConditionEncoder(vocab, dim).encode(prompt) -> (dim,)
  • classifier_free_guidance(cond_out, uncond_out, scale)

对齐（foleyflow.align）

  • EventTimeline.from_times([...]).onset_envelope(n_frames, fps)
  • onset_envelope_from_mel(mel)、spectral_flux(mel)
  • dtw(a, b) -> (cost, path)
  • apply_energy_alignment(mel, target_env, strength)、make_alignment_callback(...)

训练（foleyflow.training）

  • SyntheticFoleyDataset(n_mels, n_frames, cond_dim).build(samples_per_class) -> (X, cond, labels)
  • Adam(lr, betas, eps, weight_decay)
  • Trainer(model, objective, optimizer).fit(x, cond, epochs, batch_size) -> TrainState

模型（foleyflow.models）

  • MLPDenoiser(config).predict(x, t, cond) -> np.ndarray
  • sinusoidal_embedding(t, dim)、FiLM(cond_dim, feature_dim, rng)