Skip to content

Latest commit

 

History

History
80 lines (57 loc) · 2.47 KB

File metadata and controls

80 lines (57 loc) · 2.47 KB

使用指南

1. 提取 log-mel 特征

import numpy as np
from soundscape.features import FeatureConfig, logmel_spectrogram

cfg = FeatureConfig(sample_rate=32000, n_fft=1024, hop_length=320, n_mels=64)
signal = np.random.default_rng(0).standard_normal(cfg.sample_rate)
feats = logmel_spectrogram(signal, cfg)   # shape: (帧数, 64)

FeatureConfig 是不可变的 dataclass,frame_rate(帧率)与 effective_fmax (有效上限频率)会自动推导。

2. 声学场景分类

分类器工作在 clip 级嵌入上,需要先把帧级特征池化成一个向量:

from soundscape.asc import PrototypeSceneClassifier, mean_std_pool

X = np.vstack([mean_std_pool(logmel_spectrogram(s, cfg)) for s in train_signals])
clf = PrototypeSceneClassifier().fit(X, train_labels)

pred = clf.predict(mean_std_pool(feats)[None, :])
proba = clf.predict_proba(mean_std_pool(feats)[None, :])

3. 声音事件检测

检测器需要一个满足 FrameModel 协议的帧级模型(features → 每帧每类概率)。 仓库自带一个线性基线,方便先把链路跑通:

from soundscape.sed import LinearFrameModel, SoundEventDetector
from soundscape.sed.labels import DESED_EVENTS

model = LinearFrameModel.random(cfg.n_mels, len(DESED_EVENTS))
detector = SoundEventDetector(model, DESED_EVENTS, cfg, threshold=0.5, median_kernel=5)
events = detector.detect(signal)          # list[Event]

替换成自己的模型时,只要它接受 (帧数, 频带) 并返回 (帧数, 类别数) 概率即可。

4. 评估

from soundscape.metrics import SegmentBasedMetrics, EventBasedMetrics

seg = SegmentBasedMetrics(DESED_EVENTS, time_resolution=1.0)
seg.evaluate(reference_events, estimated_events)
print(seg.results()["overall"])           # precision / recall / f_measure / error_rate

evt = EventBasedMetrics(DESED_EVENTS, t_collar=0.2)
evt.evaluate(reference_events, estimated_events)
print(evt.results())

5. 场景描述与检索

from soundscape.llm import SceneNarrator
print(SceneNarrator(language="zh").describe("park", events))

from soundscape.retrieval import EmbeddingIndex, search
index = EmbeddingIndex(audio_embeddings, ids=clip_ids)
similarity = index.similarity(text_embeddings)

命令行

soundscape scenes
soundscape events
soundscape describe --scene park --events est.txt --lang zh
soundscape evaluate --reference ref.txt --estimated est.txt --mode segment
soundscape demo --seconds 4 --seed 0