Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion apps/light/README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ Enter one JSON object per line in the terminal, using the received channel ID:
{"command":"devices"}
{"command":"set-input","deviceId":"DEVICE-ID"}
{"command":"set-output","deviceId":null}
{"command":"set-volume","sessionId":"SESSION-ID","volume":150}
{"command":"stats","id":"my-query"}
{"command":"leave"}
{"command":"reconnect"}
Expand Down Expand Up @@ -129,6 +130,20 @@ quitting remain available. Denial keeps the microphone muted and emits
`microphone-error`; it does not try to bypass system settings. The macOS
permission flow still requires execution on a Mac.

### Per-participant volume

`set-volume` sets, from 0 to 200%, how loud you hear one voice session
(`sessionId` from the participant list), like the full client's slider. It is a
local preference: neither the server nor the other participants know about it.
The choice is saved in the profile, applies at once to the current call and to
later ones, in both P2P and SFU, and emits `participant-volume`. `stats` lists in
`userVolumes` the non-default volumes of the current call's sessions.

WebRTC applies the gain to the decoded stream before mixing: there is no extra
processing, and 100% costs nothing. Setting 100 again removes the entry. The
profile keeps up to 256 sessions with a changed volume; beyond that, `set-volume`
is rejected until one is set back to 100.

## Tray interface

`--tray` adds the icon to the Windows notification area or the macOS status bar.
Expand Down Expand Up @@ -226,7 +241,7 @@ require explicit recovery, never silent replacement. Directories containing
another application's files are rejected.

`monky-light-settings.json` stores only user preferences (currently audio
devices) and never participates in identity validation. It is replaced
devices and each participant's volume) and never participates in identity validation. It is replaced
atomically under the profile lock. Invalid or unreadable settings emit a
`warning`, fall back to defaults, and are replaced by the next save; an
interrupted settings write does not require recovery.
Expand Down
17 changes: 16 additions & 1 deletion apps/light/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ Digite um objeto JSON por linha no terminal, usando o ID recebido:
{"command":"devices"}
{"command":"set-input","deviceId":"ID-DO-DISPOSITIVO"}
{"command":"set-output","deviceId":null}
{"command":"set-volume","sessionId":"ID-DA-SESSAO","volume":150}
{"command":"stats","id":"minha-consulta"}
{"command":"leave"}
{"command":"reconnect"}
Expand Down Expand Up @@ -130,6 +131,20 @@ precisa transmitir. Enquanto a permissão está pendente, é possível receber
mutado e gera `microphone-error`; não tenta contornar as configurações do sistema.
O fluxo de permissão do macOS ainda requer execução em um Mac.

### Volume por participante

`set-volume` ajusta de 0 a 200% o volume com que você ouve uma sessão de voz
(`sessionId` da lista de participantes), como o controle deslizante do cliente
completo. É uma preferência local: o servidor e os outros participantes não
sabem dela. A escolha é salva no perfil, vale imediatamente na chamada atual e
nas próximas, inclusive em P2P e SFU, e gera `participant-volume`. `stats` traz
em `userVolumes` os volumes diferentes de 100 das sessões da chamada atual.

O ganho é aplicado pelo próprio WebRTC ao fluxo decodificado, antes da mixagem:
não há processamento extra, e 100% não custa nada. Voltar a 100 apaga a entrada.
O perfil guarda até 256 sessões com volume alterado; acima disso, `set-volume`
é recusado até que alguma volte a 100.

## Interface de bandeja

`--tray` adiciona o ícone na área de notificação do Windows ou na barra de status
Expand Down Expand Up @@ -226,7 +241,7 @@ componentes da identidade exigem recuperação explícita, nunca recriação sil
Diretórios com arquivos de outros aplicativos são recusados.

`monky-light-settings.json` guarda apenas preferências do usuário (hoje, os
dispositivos de áudio) e nunca participa da validação da identidade. É
dispositivos de áudio e o volume de cada participante) e nunca participa da validação da identidade. É
substituído de forma atômica sob o lock do perfil. Configurações inválidas ou
ilegíveis geram `warning`, usam os padrões e são substituídas no próximo
salvamento; uma gravação interrompida das configurações não exige recuperação.
Expand Down
37 changes: 36 additions & 1 deletion apps/light/src/headless_application.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,7 @@ class Application final {
config.admitted_participants = roster();
config.policy = audioPolicy();
config.devices = devicePreference();
for (const auto& [session, volume] : settings_.userVolumes) config.volumes.emplace(session, volume / 100.0);
config.network_ignore_mask = audio_.networkIgnoreMask;
if (auth.iceServers) {
for (const auto& value : *auth.iceServers) {
Expand Down Expand Up @@ -512,6 +513,29 @@ class Application final {
if (active_) active_->select_devices(devicePreference());
}

// Same scale and key as the full client: 0-200 percent per voice session.
void setVolume(const Json& value) {
const auto session = value.at("sessionId").get<std::string>();
const auto& level = value.at("volume");
if (!level.is_number_integer() || level.get<int>() < 0 || level.get<int>() > kMaxUserVolume) {
throw std::invalid_argument("volume must be an integer from 0 to 200");
}
const auto volume = level.get<int>();
auto settings = settings_;
if (volume == kDefaultUserVolume) settings.userVolumes.erase(session);
else settings.userVolumes[session] = volume;
try {
identity_.saveSettings(settings);
} catch (const std::invalid_argument&) {
throw;
} catch (const std::exception& error) {
throw std::invalid_argument(std::string("Could not save participant volume: ") + error.what());
}
settings_ = std::move(settings);
if (active_) active_->set_volume(session, volume / 100.0);
output({{"event", "participant-volume"}, {"sessionId", session}, {"volume", volume}});
}

void stopWatchingDevices() {
if (!stopWatchingDevices_) return;
auto stop = std::move(stopWatchingDevices_);
Expand Down Expand Up @@ -720,6 +744,15 @@ class Application final {
result["microphoneAccessPending"] = microphoneRequestPending_;
result["channelId"] = session_.admission() ? Json(session_.admission()->channelId) : Json(nullptr);
result["participants"] = session_.admission() ? session_.admission()->participants.size() : 0;
// Only the current call's sessions: the profile may remember many more.
auto volumes = Json::object();
if (session_.admission()) {
for (const auto& [session, participant] : session_.admission()->participants) {
const auto found = settings_.userVolumes.find(session);
if (found != settings_.userVolumes.end()) volumes[session] = found->second;
}
}
result["userVolumes"] = std::move(volumes);
result["channels"] = session_.authentication() ? session_.authentication()->server.at("channels") : Json(nullptr);
return result;
}
Expand Down Expand Up @@ -752,14 +785,15 @@ class Application final {
if (name.starts_with("fixture-") && audio_.fixtureCommand) audio_.fixtureCommand(value);
else if (name == "set-input") selectDevice(true, value);
else if (name == "set-output") selectDevice(false, value);
else if (name == "set-volume") setVolume(value);
else if (name == "join") session_.join(value.at("channelId").get<std::string>(), Clock::now());
else if (name == "leave") session_.leave();
else if (name == "mute") session_.setMuted(value.at("enabled").get<bool>());
else if (name == "deafen") session_.setDeafened(value.at("enabled").get<bool>());
else if (name == "reconnect") restartConnection();
else if (name != "quit") {
throw std::invalid_argument("Unknown command; use join, leave, mute, deafen, devices, set-input, "
"set-output, stats, channels, reconnect or quit");
"set-output, set-volume, stats, channels, reconnect or quit");
}
output({{"event", "command-accepted"}, {"id", id}, {"command", name}});
if (name == "quit") quit();
Expand Down Expand Up @@ -826,6 +860,7 @@ int runHeadless(std::vector<std::string> arguments, HeadlessConfiguration audio)
"{\"command\":\"deafen\",\"enabled\":false}\n"
"{\"command\":\"devices\"} / {\"command\":\"set-input\",\"deviceId\":\"...\"|null}\n"
"{\"command\":\"set-output\",\"deviceId\":\"...\"|null}\n"
"{\"command\":\"set-volume\",\"sessionId\":\"...\",\"volume\":0-200}\n"
"{\"command\":\"stats\"} / {\"command\":\"channels\"}\n"
"{\"command\":\"leave\"} / {\"command\":\"reconnect\"} / {\"command\":\"quit\"}\n"
"EOF and Ctrl+C release the connection and media resources.\n"
Expand Down
46 changes: 45 additions & 1 deletion apps/light/src/media/voice_engine.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,15 @@ std::vector<SdpSection> Sections(const std::string &sdp) {
}
} // namespace

// A remote audio source scales its decoded stream before the mixer; nothing
// is added to the audio path and the default gain costs nothing.
void SetGain(webrtc::MediaStreamTrackInterface *track, double gain) {
if (!track || track->kind() != webrtc::MediaStreamTrackInterface::kAudioKind)
return;
if (auto *source = static_cast<webrtc::AudioTrackInterface *>(track)->GetSource())
source->SetVolume(gain);
}

struct VoiceEngine::Impl : std::enable_shared_from_this<VoiceEngine::Impl> {
struct Peer;
struct SfuListener;
Expand Down Expand Up @@ -540,6 +549,11 @@ struct VoiceEngine::Impl : std::enable_shared_from_this<VoiceEngine::Impl> {
void Consume(const std::string &id);
void CloseConsumer(const std::string &id);
void Stats();
double Volume(const std::string &session) const {
const auto found = config.volumes.find(session);
return found == config.volumes.end() ? 1.0 : found->second;
}
void ApplyVolume(const std::string &session);
};

struct VoiceEngine::Impl::Peer : webrtc::PeerConnectionObserver,
Expand Down Expand Up @@ -1264,8 +1278,11 @@ void VoiceEngine::Impl::ConfigureTracks(Peer &peer) {
peer.participant.user_id != config.self_user_id &&
!peer.participant.suppressed &&
!token->deafened.load();
if (auto track = transceiver->receiver()->track())
if (auto track = transceiver->receiver()->track()) {
track->set_enabled(listen);
if (listen)
SetGain(track.get(), Volume(peer.participant.session_id));
}
if (transceiver->media_type() == webrtc::MediaType::VIDEO) {
if (!transceiver->stopped())
Check(transceiver->StopStandard(), "Reject video transceiver");
Expand Down Expand Up @@ -1704,6 +1721,7 @@ void VoiceEngine::Impl::Consume(const std::string &id) {
// The transport keeps a pointer to it until it is closed: listed at once,
// whatever happens next, it is closed with the call.
auto &owned = consumers.emplace(id, std::move(consumer)).first->second;
SetGain(owned->GetTrack(), Volume(session));
if (token->deafened.load()) {
owned->Pause();
Notify(msg::SFU_CONSUMER_SET_PAUSED, {{"channelId", config.channel_id},
Expand Down Expand Up @@ -1970,6 +1988,32 @@ void VoiceEngine::update_policy(AudioPolicy policy) {
}
}

void VoiceEngine::Impl::ApplyVolume(const std::string &session) {
if (const auto peer = peers.find(session);
peer != peers.end() && peer->second->mic_transceiver)
SetGain(peer->second->mic_transceiver->receiver()->track().get(),
Volume(session));
for (const auto &[id, consumer] : consumers)
if (const auto found = known_producers.find(id);
found != known_producers.end() &&
Text(found->second, "producerSessionId") == session)
SetGain(consumer->GetTrack(), Volume(session));
}

void VoiceEngine::set_volume(std::string session_id, double gain) {
if (!(gain >= 0.0 && gain <= 2.0))
throw std::invalid_argument("Participant gain must be between 0 and 2");
auto self = owner_->impl;
if (auto call = self->Ingress())
self->ForCall(call, [session = std::move(session_id), gain](Impl &engine) {
if (gain == 1.0)
engine.config.volumes.erase(session);
else
engine.config.volumes[session] = gain;
engine.ApplyVolume(session);
});
}

void VoiceEngine::select_devices(AudioDevicePreference devices) {
auto self = owner_->impl;
if (auto call = self->Ingress())
Expand Down
6 changes: 6 additions & 0 deletions apps/light/src/media/voice_engine.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#include <chrono>
#include <functional>
#include <future>
#include <map>
#include <memory>
#include <optional>
#include <string>
Expand Down Expand Up @@ -54,6 +55,9 @@ struct JoinConfig {
std::vector<IceServer> ice_servers;
AudioPolicy policy;
AudioDevicePreference devices;
// Playback gain per remote session (1.0 when absent), applied to the
// decoded stream before mixing, as the full client's per-user slider.
std::map<std::string, double> volumes;
std::optional<int> network_ignore_mask;
std::chrono::milliseconds operation_timeout{15000};
};
Expand Down Expand Up @@ -127,6 +131,8 @@ class VoiceEngine final {
// Persists for later generations of this call. Missing devices fall back to
// the system default and are retried when the preference is applied again.
void select_devices(AudioDevicePreference devices);
// Gain 0.0-2.0; applies at once and to that session's later streams.
void set_volume(std::string session_id, double gain);
void poll_stats();

private:
Expand Down
46 changes: 41 additions & 5 deletions apps/light/src/profile_identity.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ constexpr std::size_t kMaxMetadataBytes = 4096;
constexpr char kSettingsFile[] = "monky-light-settings.json";
constexpr char kSettingsPendingFile[] = "monky-light-settings.json.pending";
constexpr char kSettingsFormat[] = "monky-light-settings";
constexpr std::size_t kMaxSettingsBytes = 16 * 1024;
constexpr std::size_t kMaxSettingsBytes = 64 * 1024;
constexpr std::size_t kMaxDeviceIdBytes = 1024;
// server-key-auth (#821): the identity key pinned for each server this profile signed in to.
constexpr char kServerKeysFile[] = "monky-light-servers.json";
Expand Down Expand Up @@ -533,12 +533,34 @@ std::optional<std::string> settingsDevice(const Json& data, const char* field) {
return found->get<std::string>();
}

bool validUserVolume(const std::string& session, int volume) {
return !session.empty() && session.size() <= kMaxVolumeSessionBytes && volume >= 0 &&
volume <= kMaxUserVolume && volume != kDefaultUserVolume;
}

std::map<std::string, int> settingsVolumes(const Json& data) {
std::map<std::string, int> result;
const auto found = data.find("userVolumes");
if (found == data.end() || found->is_null()) return result;
if (!found->is_object() || found->size() > kMaxUserVolumes) {
throw std::runtime_error("Profile settings contain invalid participant volumes");
}
for (const auto& [session, volume] : found->items()) {
if (!volume.is_number_integer() || !validUserVolume(session, volume.get<int>())) {
throw std::runtime_error("Profile settings contain invalid participant volumes");
}
result.emplace(session, volume.get<int>());
}
return result;
}

ProfileSettings parseSettings(const std::string& contents) {
Json data;
try {
// Only the participant-volume map nests, one level deep.
data = Json::parse(contents, [](int depth, Json::parse_event_t event, Json&) {
if (event == Json::parse_event_t::array_start ||
(event == Json::parse_event_t::object_start && depth != 0)) {
(event == Json::parse_event_t::object_start && depth > 1)) {
throw std::runtime_error("Profile settings must be a flat JSON object");
}
return true;
Expand All @@ -550,7 +572,12 @@ ProfileSettings parseSettings(const std::string& contents) {
!data.contains("version") || !data["version"].is_number_integer() || data["version"] != 1) {
throw std::runtime_error("Profile settings have an unsupported format or version");
}
return {settingsDevice(data, "inputDeviceId"), settingsDevice(data, "outputDeviceId")};
for (const auto& [key, value] : data.items()) {
if (value.is_object() && key != "userVolumes") {
throw std::runtime_error("Profile settings must be a flat JSON object");
}
}
return {settingsDevice(data, "inputDeviceId"), settingsDevice(data, "outputDeviceId"), settingsVolumes(data)};
}

std::string encodeSettings(const ProfileSettings& settings) {
Expand All @@ -559,16 +586,25 @@ std::string encodeSettings(const ProfileSettings& settings) {
throw std::invalid_argument("Audio device ID must be a nonempty string of at most 1024 bytes");
}
}
if (settings.userVolumes.size() > kMaxUserVolumes) {
throw std::invalid_argument("Too many participant volumes are stored; reset some to 100");
}
for (const auto& [session, volume] : settings.userVolumes) {
if (!validUserVolume(session, volume)) {
throw std::invalid_argument("Participant volume must be 0-200 for a session ID of at most 256 bytes");
}
}
const auto optional = [](const std::optional<std::string>& value) {
return value ? Json(*value) : Json(nullptr);
};
std::string result;
try {
result = Json{{"format", kSettingsFormat}, {"version", 1},
{"inputDeviceId", optional(settings.inputDeviceId)},
{"outputDeviceId", optional(settings.outputDeviceId)}}.dump(2) + "\n";
{"outputDeviceId", optional(settings.outputDeviceId)},
{"userVolumes", Json(settings.userVolumes)}}.dump(2) + "\n";
} catch (const Json::exception&) {
throw std::invalid_argument("Audio device ID must be valid UTF-8");
throw std::invalid_argument("Audio device and session IDs must be valid UTF-8");
}
if (result.size() > kMaxSettingsBytes) throw std::invalid_argument("Profile settings exceed their size limit");
return result;
Expand Down
10 changes: 10 additions & 0 deletions apps/light/src/profile_identity.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
#include <array>
#include <cstdint>
#include <filesystem>
#include <map>
#include <memory>
#include <optional>
#include <string>
Expand All @@ -16,8 +17,17 @@ namespace monky::light {
struct ProfileSettings {
std::optional<std::string> inputDeviceId; // nullopt: system default
std::optional<std::string> outputDeviceId; // nullopt: system default
// Playback volume per voice session, 0-200 percent like the full client.
// 100 is the default and is never stored.
std::map<std::string, int> userVolumes;
};

constexpr int kDefaultUserVolume = 100;
constexpr int kMaxUserVolume = 200;
// Bounds the settings file; setting a volume back to 100 frees its entry.
constexpr std::size_t kMaxUserVolumes = 256;
constexpr std::size_t kMaxVolumeSessionBytes = 256;

// Owns the signing key and exclusive profile lock for its entire lifetime.
// Requires an absolute path and an existing parent; creates only the profile leaf.
// Existing directories must contain only this component's identity and settings files.
Expand Down
Loading
Loading