Skip to content

Add track mute and equalizer. #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jan-ivar wants to merge 1 commit into
base: gh-pages
Choose a base branch
from
Open

Add track mute and equalizer. #33

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 75 additions & 18 deletions gum_test.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,14 +92,24 @@ <h4>getUserMedia is missing!</h4>
video.setAttribute("width", 640);
video.setAttribute("height", 480);

const muted = document.createElement("input");
muted.type = "checkbox";
muted.checked = true;
muted.onclick = () => { video.muted = muted.checked; };
const equalizer = document.createElement("canvas");

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a visualizer, not an equalizer. An equalizer is used to change the gain of certain frequencies, in order to "equalize" them.

Applies throughout.

equalizer.setAttribute("width", 25);
equalizer.setAttribute("height", 480);

const mute = document.createElement("label");
mute.innerHTML = "Volume mute";
mute.insertBefore(muted, mute.firstChild);
function createCheckbox(textContent, checked, type = "checkbox") {
const label = Object.assign(document.createElement("label"), {textContent});
const input = Object.assign(document.createElement("input"), {type});
label.insertBefore(input, label.firstChild);
return Object.assign(input, {checked});
}

const camOn = createCheckbox("📹", true);
const micOn = createCheckbox("🎙️", true);
const elementMuted = createCheckbox("🔇", true);
elementMuted.onclick = () => {
audio.muted = elementMuted.checked;
video.muted = elementMuted.checked;
};

const snapshots = [];

Expand Down Expand Up @@ -156,8 +166,8 @@ <h4>getUserMedia is missing!</h4>
}
video.srcObject = null;
content.removeChild(video);
if (mute.parentNode) {
controls.removeChild(mute);
if (equalizer.parentNode) {
content.removeChild(equalizer);
}
fps.innerHTML = "";
stopbuttons.removeChild(snapshot);
Expand All @@ -170,6 +180,12 @@ <h4>getUserMedia is missing!</h4>
}
audio.srcObject = null;
content.removeChild(audio);
content.removeChild(equalizer);
}
for (const muted of [elementMuted, camOn, micOn]) {
if (muted.parentNode.parentNode) {
controls.removeChild(muted.parentNode);
}
}
saved_stream = null;
stop.style.display = "none";
Expand All @@ -180,22 +196,18 @@ <h4>getUserMedia is missing!</h4>
function pauseMedia() {
if (saved_stream) {
if (saved_stream.getVideoTracks().length) {
video.srcObject = saved_stream;
video.play();
} else {
audio.srcObject = saved_stream;
audio.play();
}
saved_stream = null;
} else {
if (video.srcObject) {
video.pause();
saved_stream = video.srcObject;
video.srcObject = null;
} else if (audio.srcObject) {
audio.pause();
saved_stream = audio.srcObject;
audio.srcObject = null;
}
}
}
Expand All @@ -212,12 +224,18 @@ <h4>getUserMedia is missing!</h4>
try {
const stream = await navigator.mediaDevices[gum](constraints);
message.innerHTML = "<p>Success!</p>";
if (stream.getVideoTracks().length) {
const [audioTrack] = stream.getAudioTracks();
const [videoTrack] = stream.getVideoTracks();
if (audioTrack) {
controls.appendChild(elementMuted.parentNode);
audio.muted = elementMuted.checked;
video.muted = elementMuted.checked;
}
if (videoTrack) {
content.appendChild(video);
if (stream.getAudioTracks().length) {
controls.appendChild(mute);
video.muted = muted.checked;
}
controls.appendChild(camOn.parentNode);
camOn.onclick = () => { videoTrack.enabled = camOn.checked; };
camOn.onclick();

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't the checked state change through other means than click? keyboard navigation and selecting it with space is what I'm thinking. The "change" event looks like the right thing. Also applies to micOn and elementMuted.

video.srcObject = stream;
video.play();
frames.innerHTML = '';
Expand All @@ -231,6 +249,13 @@ <h4>getUserMedia is missing!</h4>
audio.srcObject = stream;
audio.play();
}
if (audioTrack) {
content.appendChild(equalizer);
startEqualizer(stream);
controls.appendChild(micOn.parentNode);
micOn.onclick = () => { audioTrack.enabled = micOn.checked; };
micOn.onclick();
}
running = true;
} catch (err) {
if (location.protocol != "https:") {
Expand Down Expand Up @@ -287,6 +312,38 @@ <h4>getUserMedia is missing!</h4>
setTimeout(captureImage, 2000);
}
}

function startEqualizer(stream) {
const audioCtx = new AudioContext();
const analyser = audioCtx.createAnalyser();
audioCtx.createMediaStreamSource(stream).connect(analyser);

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a MediaStreamTrackSource now. The MediaStreamSource is obsolete really, since it locks onto the first track and basically works as the track source anyway.


const canvasCtx = equalizer.getContext("2d");
const data = new Uint8Array(equalizer.width);

let interval = setInterval(() => {
if (stream.getAudioTracks()[0].readyState != "live") {
clearInterval(interval);

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or use the ended event on the track to clear the interval. Also close the AudioContext at that point please.

return;
}
canvasCtx.fillStyle = "#ffffff";

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clearRect shouldn't need fillStyle, only fillRect, no?

canvasCtx.clearRect(0, 0, equalizer.width, equalizer.height);
analyser.getByteFrequencyData(data);
canvasCtx.lineWidth = 2;
let x = 0;
for (let y of data) {
y = equalizer.height - (y / 128) * equalizer.height / 4;
let c = Math.floor((x*255)/equalizer.width);
canvasCtx.fillStyle = "rgb("+c+",0,"+(255-x)+")";

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it could be psychedelic. Do you have a screenshot?

canvasCtx.fillRect(x, y, 2, equalizer.height - y)
x++;

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you made sure x doesn't overshoot the canvas width? That would be unnecessary. You can control how many frequency buckets you get from the analyser with the fftSize attribute.

}
analyser.getByteTimeDomainData(data);

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data is never used.

canvasCtx.lineWidth = 5;
canvasCtx.beginPath();
canvasCtx.stroke();
}, 1000 * equalizer.width / audioCtx.sampleRate);

@Pehrsons Pehrsons Nov 20, 2019

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait what, why is the width of the canvas part of the equation?

}
</script>

</body>
Expand Down