Draw on the canvas to synthesize audio. High notes reside at the top, low notes at the bottom. Use multiple fingers for full polyphonic expression.
Draw on the canvas to synthesize audio. High notes reside at the top, low notes at the bottom. Use multiple fingers for full polyphonic expression.
The Polyphonic Musical Drawing Canvas is an interactive synthesis platform that bridges the gap between digital painting and sound generation. By combining the HTML5 Canvas API with Tone.js, users can map expressive freehand movements to polyphonic synthesizers in real-time. Each coordinate on the visual space is directly linked to pitch and volume, enabling intuitive musical composition for artists, educators, and creators of all ages.
Select your preferred instrument from the control panel dropdown at the top of the workspace. Use your mouse pointer or multiple fingers on touch devices to draw on the canvas. The vertical position (Y-axis) maps directly to pitch, with higher frequencies located at the top and deeper frequencies at the bottom. The horizontal trail creates temporary soundscapes. Toggle audio generation on or off using the Sound Control button, run the automated generator using the Play Demo button, or clear the drawing space instantly with the Reset Simulation button.
The web application uses the Web Audio API wrapper Tone.js to orchestrate polyphonic synthesizer instances dynamically. When drawing begins, unique oscillator chains (classic synth, frequency modulation, amplitude modulation, membrane/drum, or physical pluck models) are spawned or updated using responsive rate ramping to eliminate audible pops and glitching. Event loops are decoupled using Web API requestAnimationFrame, ensuring an Interaction to Next Paint (INP) score well below 200ms even during complex polyphonic gestures.
Future development will expand the system to include scale constraint options (pentatonic, chromatic, minor/major) so users can easily draw harmonically matched sequences. We also aim to add collaborative painting modes via peer-to-peer WebRTC connections, customizable sound envelopes, direct audio recording export in WAV format, and visual presets that sync the paintbrush graphics with the physical timbre of the chosen synth engine.