Building Audio-Based Web Applications with the Web Audio API and WebAssembly

The Web Audio API gives browsers the ability to generate, process, and analyze audio in real-time. Combined with WebAssembly (WASM) modules like ggwave, you can build sophisticated audio applications that run entirely in the browser. This guide covers the key concepts and common pitfalls.

Web Audio API Fundamentals

The Web Audio API is built around a graph of audio nodes. Audio flows from source nodes through processing nodes to a destination (usually the speakers).

AudioContext

Everything starts with an AudioContext. It represents the audio processing graph and controls the sample rate, timing, and state of the audio system.

const audioContext = new AudioContext({ sampleRate: 48000 });

Important considerations:

• Sample rate: Choose 48000 Hz for ggwave compatibility. The default varies by platform.
• Autoplay policy: Browsers block audio playback until the user interacts with the page. You must create or resume the AudioContext inside a click handler.
• State management: An AudioContext can be "suspended" (paused by the browser to save resources) or "running". Always check and resume if needed.

Audio Nodes

The most commonly used nodes for data-over-sound applications:

AudioBufferSourceNode — Plays a pre-generated audio buffer (used for ggwave transmission):

const buffer = audioContext.createBuffer(1, samples.length, 48000);
buffer.getChannelData(0).set(float32Samples);
const source = audioContext.createBufferSource();
source.buffer = buffer;
source.connect(audioContext.destination);
source.start();

AnalyserNode — Provides real-time frequency and time-domain analysis (used for spectrum visualization):

const analyser = audioContext.createAnalyser();
analyser.fftSize = 2048;
source.connect(analyser);
analyser.connect(audioContext.destination);

// Read frequency data
const dataArray = new Uint8Array(analyser.frequencyBinCount);
analyser.getByteFrequencyData(dataArray);

MediaStreamSourceNode — Captures microphone input (used for ggwave reception):

const stream = await navigator.mediaDevices.getUserMedia({
  audio: { sampleRate: 48000, channelCount: 1 }
});
const source = audioContext.createMediaStreamSource(stream);

Sample Format Conversion

This is the most common source of bugs when integrating WASM audio modules with the Web Audio API.

The Problem

The Web Audio API uses Float32 samples in the range [-1.0, 1.0]. Many audio libraries (including ggwave) use Int16 samples in the range [-32768, 32767]. If you pass data between them without conversion, you get silence, distortion, or ear-splitting noise.

Int16 to Float32 (for playback)

When a WASM module returns Int16 audio data that you want to play through the Web Audio API:

// wasmOutput is an Int8Array (raw bytes containing I16 samples)
const bytesCopy = new ArrayBuffer(wasmOutput.byteLength);
new Uint8Array(bytesCopy).set(
  new Uint8Array(wasmOutput.buffer, wasmOutput.byteOffset, wasmOutput.byteLength)
);
const int16 = new Int16Array(bytesCopy);
const float32 = new Float32Array(int16.length);
for (let i = 0; i < int16.length; i++) {
  float32[i] = int16[i] / 32768.0;
}

Key detail: always copy the bytes out of the WASM heap first. WASM memory can be reallocated (invalidating existing views) when the module allocates more memory.

Float32 to Int16 (for processing)

When you capture microphone audio (Float32) and need to pass it to a WASM module expecting Int16:

const float32Input = audioProcessingEvent.inputBuffer.getChannelData(0);
const int16 = new Int16Array(float32Input.length);
for (let i = 0; i < float32Input.length; i++) {
  const s = Math.max(-1, Math.min(1, float32Input[i]));
  int16[i] = s < 0 ? s * 32768 : s * 32767;
}
const bytes = new Uint8Array(int16.buffer);
// Pass bytes to WASM module

Real-Time Spectrum Visualization

Visualizing the frequency spectrum during audio transmission serves two purposes: it confirms that audio is actually being produced, and it provides visual feedback that is synchronized with the sound.

The Correct Approach

Connect an AnalyserNode in the audio graph between the source and the destination. Then read frequency data from the analyser in a requestAnimationFrame loop:

const analyser = audioContext.createAnalyser();
analyser.fftSize = 2048;
analyser.smoothingTimeConstant = 0.3;

// Connect: source → analyser → destination
source.connect(analyser);
analyser.connect(audioContext.destination);

function drawSpectrum() {
  const data = new Uint8Array(analyser.frequencyBinCount);
  analyser.getByteFrequencyData(data);
  // Draw data to canvas...
  requestAnimationFrame(drawSpectrum);
}
drawSpectrum();

This approach guarantees perfect synchronization because the analyser reads from the same audio stream that is being played. There is no timing calculation or animation scheduling to get wrong.

Common Mistakes

• Generating fake visualization data: Calculating frequencies from the input text and animating them independently. This will never be perfectly synchronized with the actual audio.
• Using setInterval instead of requestAnimationFrame: Results in inconsistent frame rates and wasted CPU cycles when the tab is not visible.
• Not setting smoothingTimeConstant: The default (0.8) makes the visualization sluggish. Values between 0.1 and 0.4 provide responsive visual feedback.

Microphone Access

Capturing microphone audio requires HTTPS (except on localhost) and explicit user permission.

const stream = await navigator.mediaDevices.getUserMedia({
  audio: {
    sampleRate: 48000,
    channelCount: 1,
    echoCancellation: false,
    noiseSuppression: false,
    autoGainControl: false
  }
});

Disable echoCancellation, noiseSuppression, and autoGainControl when receiving ggwave signals. These processing algorithms are designed for human speech and will distort the FSK tones.

WASM Integration in Next.js

Loading a WASM module in a Next.js application requires some webpack configuration because WASM modules compiled with Emscripten often reference Node.js modules like fs:

// next.config.ts
webpack: (config, { isServer }) => {
  if (!isServer) {
    config.resolve.fallback = {
      ...config.resolve.fallback,
      fs: false,
    };
  }
  return config;
}

Load the WASM module dynamically to avoid server-side rendering issues:

const factory = (await import('ggwave')).default;
const ggwave = await factory();

Performance Tips

1. Reuse AudioContext: Creating a new AudioContext for each transmission wastes resources and may hit browser limits. Create one and reuse it.

2. Clean up resources: Disconnect nodes and stop media streams when they are no longer needed. Orphaned audio nodes consume memory and CPU.

3. Use appropriate buffer sizes: For ScriptProcessorNode, a buffer size of 4096 provides a good balance between latency and CPU usage. Smaller buffers increase CPU load; larger buffers increase latency.

4. Singleton WASM module: Load the ggwave WASM module once and cache the promise. Multiple calls to the factory function waste memory.

Conclusion

The Web Audio API and WebAssembly together enable sophisticated audio processing in the browser. The main challenges are sample format conversion, browser autoplay policies, and WASM integration quirks. Once these are handled correctly, you can build reliable data-over-sound applications that work across all modern browsers.

---

See these techniques in action on our GGWave Demo page. The source code demonstrates AudioContext management, AnalyserNode visualization, and ggwave WASM integration in a Next.js application.