LiveSpeak Architectural Specification & Biophysical Framework
1. Overview & Biophysical Acoustic Principles
Human speech is an extraordinarily complex bio-acoustic signal generated by the coordinated dynamics of respiration, vocal fold oscillation, and vocal tract articulation. When air is expelled from the lungs through the trachea, it builds subglottal pressure beneath the vocal folds. As pressure breaches the glottal threshold, mucosal folds vibrate, transforming continuous pulmonary airflow into periodic acoustic pulses. This fundamental vibration establishes the vocal fundamental frequency ($F_0$), perceived psychoacoustically as pitch.
As sound waves propagate through the vocal tract—encompassing the pharyngeal cavity, oral cavity, tongue position, jaw aperture, and nasal passages—specific frequency bands are selectively amplified or attenuated based on dynamic physical acoustic resonance. These resonant peaks are known as formants ($F_1, F_2, F_3, \dots$). The lowest formant $F_1$ correlates primarily with tongue height and jaw opening, while $F_2$ reflects tongue advancement (fronting versus backing). The multi-dimensional visualizer LiveSpeak maps these rapid bio-acoustic transformations in real time, converting raw microphone waveforms or synthetic acoustic streams into high-resolution time-frequency spectrograms, autocorrelation pitch contours, and 2D vowel formant topologies.
2. Interactive Instructions & Controls Guide
The LiveSpeak workspace provides a fully responsive, real-time diagnostic suite designed for speech acoustics, vocal training, and linguistic phonetic research:
- ▶ START DEMO: Launches an automated step-by-step masterclass tutorial. The system synchronizes visual highlights, parameter shifts, audio sonification, and speech telemetry to guide users through speech acoustic mechanics.
- RESET BASELINE: Clears stored word transcripts, resets maximum volume gauges, re-calculates noise floor baselines, and returns audio DSP parameters to initial default values.
- 🔇 / 🔊 SOUND TOGGLE: Controls real-time auditory feedback, including pitch-matched sonification tones and speech event cues generated via the Web Audio API.
- 🎙️ ENABLE MICROPHONE: Requests browser microphone access (`navigator.mediaDevices.getUserMedia`) to process live human speech. If microphone permissions are denied or unavailable, LiveSpeak seamlessly switches to a high-fidelity synthetic voice stream generator.
- Analysis Preset Profiles: Selects targeted signal processing baselines:
- Conversational Speech: Standard baseline for natural dialogue and speech rate monitoring ($120 - 160 \text{ WPM}$).
- Public Speaking Coach: Enhanced gain sensitivity with emphasis on volume dynamic range and pitch inflection.
- Acoustic Phonetics: High-frequency FFT windowing tailored for tracking $F_1$ and $F_2$ vowel formants.
- Whisper Mode: Lowers noise gate threshold ($-60 \text{ dB}$) and boosts gain ($3.0\times$) for unvoiced, low-energy speech.
- High Background Noise: Aggressive spectral noise gating to isolate vocal speech bands ($300 \text{ Hz} - 3.4 \text{ kHz}$).
- Visual View Modes: Switch between the default Quad-Grid layout or expand a single focused visualization: Oscilloscope Waveform, Short-Time Fourier Transform (STFT) Spectrogram, $F_0$ Pitch Tracking Contour, or $F_1/F_2$ Vowel Space Formant Map.
- Real-Time Transcription Box: Renders live speech recognition results. Interim hypothesis words appear in glowing amber, while finalized text converts to bright cyan. Copy or clear text with dedicated buttons.
3. Mathematical Foundations & Signal Processing Pipelines
LiveSpeak utilizes advanced digital signal processing (DSP) algorithms implemented via HTML5 Web Audio API nodes and raw JavaScript typed arrays:
A. Short-Time Fourier Transform (STFT) & Spectrogram
Continuous audio streams are sampled at $f_s = 44.1 \text{ kHz}$ or $48 \text{ kHz}$ and windowed into overlapping frames of length $N \in \{256, 512, 1024, 2048\}$. The Discrete Fourier Transform (DFT) converts discrete time-domain frames $x[n]$ into complex frequency spectra $X[k]$:
$$X[k] = \sum_{n=0}^{N-1} x[n] \cdot w[n] \cdot e^{-j \frac{2\pi}{N} k n}$$
where $w[n]$ represents a Blackman or Hann window function applied to minimize spectral leakage across bin boundaries. The magnitude response $|X[k]|$ is converted to logarithmic decibel intensity ($\text{dBFS}$):
$$\text{Power}(\text{dB}) = 20 \cdot \log_{10}\left( \frac{|X[k]| + \epsilon}{\text{MaxAmplitude}} \right)$$
B. Fundamental Frequency ($F_0$) Autocorrelation Pitch Extraction
To isolate fundamental pitch independent of harmonic overtones, LiveSpeak computes the normalized time-domain autocorrelation function $R(\tau)$ across lag intervals $\tau \in [\tau_{\min}, \tau_{\max}]$ corresponding to human vocal limits ($60 \text{ Hz} \le F_0 \le 500 \text{ Hz}$):
$$R(\tau) = \frac{\sum_{n=0}^{W-1} x[n] \cdot x[n+\tau]}{\sqrt{\sum_{n=0}^{W-1} x[n]^2 \cdot \sum_{n=0}^{W-1} x[n+\tau]^2}}$$
The lag $\tau^*$ maximizing $R(\tau)$ above a periodicity confidence threshold ($R(\tau^*) > 0.45$) yields the fundamental frequency estimate $F_0 = \frac{f_s}{\tau^*}$.
C. Spectral Centroid & Voice Activity Detection (VAD)
The spectral centroid $\bar{f}$, indicating the "brightness" or center of gravity of the acoustic energy spectrum, is derived as:
$$\bar{f} = \frac{\sum_{k=0}^{N/2} f[k] \cdot |X[k]|}{\sum_{k=0}^{N/2} |X[k]|}$$
Voice Activity Detection evaluates frame Root-Mean-Square (RMS) amplitude $\text{RMS} = \sqrt{\frac{1}{N} \sum_{n=0}^{N-1} x[n]^2}$ against the user-configured noise gate threshold $T_{\text{gate}}$.
4. Future Directions & Research Roadmap
The LiveSpeak platform is designed for ongoing expansion into clinical speech pathology, multi-speaker separation, and deep learning neural acoustic analysis. Key development vectors include:
- WASM Neural Speech Recognition (Whisper Integration): Integrating lightweight WebAssembly-compiled OpenAI Whisper models for offline, privacy-preserving speech recognition with near-zero latency.
- Formant LPC Polynomial Linear Predictive Coding: Replacing energy-peak formant heuristics with full Linear Predictive Coding (LPC) filter root extraction for ultra-precise $F_1 - F_4$ formant tracking.
- Clinical Dysphonia & Jitter/Shimmer Telemetry: Adding clinical vocal stability metrics including pitch jitter (percentage cycle-to-cycle frequency variation) and shimmer (amplitude perturbation) for early detection of vocal cord fatigue or pathology.
- 3D Vocal Tract Articulatory Model: Linking real-time formant trajectories directly to a 3D anatomical model of the human tongue, velum, pharynx, and lip position.
5. Related Interactive Laboratories & Modules
Explore complementary acoustic, neurological, and signal processing laboratories across the BioniChaos ecosystem:
Open Access License: This interactive educational module is released under
CC BY-NC 4.0 (Attribution-NonCommercial)
for non-commercial research, academic study, and clinical education.
Commercial & Enterprise Licensing: For white-labeling, proprietary LMS/course embedding, hardware dashboard telemetry integration, or custom feature engineering, secure a commercial license at
BioniCloud.com or contact
Dr. Yuri Beno.