Overview: Visualizing Sound, Psychoacoustics & Gestural Music Creation
The Polyphonic Musical Drawing Canvas is a biophysical audio workspace and psychoacoustics laboratory engineered to bridge visual spatial gesture with continuous digital audio synthesis. In classical acoustic theory and human auditory perception, musical pitch is perceived logarithmically rather than linearly. Frequency perception conforms closely to the Weber-Fechner Law and the Stevens Power Law, wherein human cochlear hair cells along the basilar membrane respond tonotopically to acoustic waves: high-frequency vibrations displace cells at the stiff base of the cochlea, while low frequencies travel toward the flexible apex.
This application transforms the two-dimensional Cartesian plane of the screen into an expressive, multi-touch musical instrument. The vertical $Y$-axis governs the fundamental frequency ($f_0$) across five musical octaves (C2 through C7), precisely mapping spatial altitude to acoustic frequency. The horizontal $X$-axis controls stereophonic spatialization ($[-1.0, +1.0]$ stereo panning) and dynamic filter dispersion. By pairing the visual mechanics of freehand drawing with multi-timbre synthesis engines—including Frequency Modulation (FM), Amplitude Modulation (AM), and physical waveguide modeling—users experience how continuous geometric brush strokes translate into rich harmonic spectrums in real time.
Technical Architecture: Signal Processing, Mathematics & Rendering Safeguards
The application combines the Web Audio API (via the Tone.js framework) and an optimized HTML5 Canvas 2D rendering loop. The audio pipeline ensures non-blocking, artifact-free real-time synthesis:
1. Equal Temperament and Microtonal Pitch Mapping:
The canvas height $H$ maps the client coordinate $y \in [0, H]$ onto a continuous MIDI scale interval $[M_{\text{min}}, M_{\text{max}}] = [36, 96]$, corresponding to notes C2 ($65.41\text{ Hz}$) through C7 ($2093.00\text{ Hz}$):
$$M(y) = M_{\text{max}} - \left(\frac{y}{H}\right) \times (M_{\text{max}} - M_{\text{min}})$$
When scale quantization is enabled, the calculated continuous MIDI number $M(y)$ is snapped to the nearest valid scale degree $S_k \in \mathcal{S}_{\text{scale}}$ modulo 12 relative to the selected root key $R \in [0, 11]$:
$$M_{\text{quantized}} = 12 \times \left\lfloor \frac{M(y)}{12} \right\rfloor + \arg\min_{s \in \mathcal{S}} |(M(y) \bmod 12) - (s + R \bmod 12)|$$
The resulting MIDI value converts to fundamental frequency $f_0$ in Hertz via the 12-Tone Equal Temperament formula:
$$f_0 = 440 \times 2^{\frac{M - 69}{12}}$$
2. Complex Modulation Synthesizers:
For the FM Synth, the instantaneous acoustic output is governed by frequency modulation of a carrier frequency $f_c$ by a modulator frequency $f_m$ with modulation index $\beta$:
$$y_{\text{FM}}(t) = A(t) \sin\left(2\pi f_c t + \beta \sin(2\pi f_m t)\right)$$
Expanding via Jacobi-Anger series yields harmonic Bessel components $J_n(\beta)$ at sideband frequencies $f_c \pm n f_m$:
$$y_{\text{FM}}(t) = A(t) \sum_{n=-\infty}^{\infty} J_n(\beta) \sin\left(2\pi(f_c + n f_m)t\right)$$
Similarly, for the AM Synth, amplitude modulation produces the carrier and two symmetric sidebands:
$$y_{\text{AM}}(t) = \left[1 + m \cos(2\pi f_m t)\right] A(t) \cos(2\pi f_c t) = A(t) \cos(2\pi f_c t) + \frac{m A(t)}{2} \left[\cos(2\pi(f_c+f_m)t) + \cos(2\pi(f_c-f_m)t)\right]$$
3. Dynamic Parameter Ramping & De-Clicking:
Audio pops and audible clicking caused by instantaneous waveform discontinuities are eliminated by applying exponential parameter ramps (`rampTo(targetFreq, 0.04)`) to internal voltage-controlled oscillators (VCOs) and voltage-controlled amplifiers (VCAs).
4. Canvas Rendering and ResizeObserver Decoupling:
To prevent layout shifts and display distortion, the `
Open Access License: This interactive educational module is released under
CC BY-NC 4.0 (Attribution-NonCommercial)
for non-commercial research, academic study, and clinical education.
Commercial & Enterprise Licensing: For white-labeling, proprietary LMS/course embedding, hardware dashboard telemetry integration, or custom feature engineering, secure a commercial license at
BioniCloud.com or contact
Dr. Yuri Beno.