Overview
The Real-Time Signal Amplification Microscope is an educational laboratory platform designed to demonstrate the mechanics of Eulerian Video Magnification (EVM) and remote photoplethysmography (rPPG). By capturing subtle color changes on the human face caused by vascular perfusion, the system transforms an ordinary web camera into an interactive non-contact biological pulse sensor.
As the left ventricle contracts, it propels a pulse wave of oxygenated blood throughout the arterial system. This volumetric change alters the optical density of the skin tissue. Because hemoglobin has a distinct absorption peak in the green spectrum, these rhythmic volumetric shifts emerge as subtle, sub-visual color variations. Digital camera sensors catch these micro-signals, allowing temporal filters and GPU fragment shaders to isolate, filter, and amplify the changes into a visible representation of blood flow.
How to Use
1. Configuration & Alignment: Allow webcam permissions to start the feed. By default, the system uses MediaPipe Face Mesh to locate and isolate the forehead. If the roll or yaw limits are exceeded, the tracking indicator warnings guide you back to center. Switch to manual mode to draw a customized bounding region over other skin areas.
2. Interactive Control Parameters: Adjust the Amplification Factor to visually scale the intensity of the color transformations. Utilize the Low Cutoff and High Cutoff input fields to configure the Butterworth passband filters, optimizing noise exclusion and stabilizing the target spectrum.
3. Diagnostics and Sonification: Switch raw and filtered signals in the oscilloscope to inspect real-time waveforms. Click the Sound ON button to initiate the synthesizer engine, generating auditory representations of your microvascular pulse waves. Utilize the Reset Simulator button to instantly wipe custom settings back to baseline configurations.
Technical Details
The software executes real-time digital signal processing (DSP) steps within a frame-loop paradigm:
- Temporal Resampling Engine: Camera capture rates fluctuate based on lighting and hardware constraints. To preserve filter coefficients and prevent mathematical instability, the system linearly interpolates uneven captures onto a strict, uniform 30 Hz grid virtual clock.
- IIR Butterworth Bandpass: Resampled signals pass through a second-order Infinite Impulse Response (IIR) bandpass filter. High-pass elements remove low-frequency breathing or movement drift, while low-pass elements restrict high-frequency sensor noise.
- High-Resolution FFT and OBB: A 256-sample rolling buffer is windowed with a Hann function and zero-padded to 1024 bins, executing a Cooley-Tukey Fast Fourier Transform (FFT) to achieve an interpolation-enhanced 0.029 Hz resolution. To preserve tracking consistency, the forehead region is calculated using an aspect-ratio-corrected 3D Oriented Bounding Box (OBB), aligning itself with lateral head roll and pitch, preventing tracking dropouts.
- GPU Shading Pipeline: Color amplification operates inside a WebGL fragment shader, executing per-pixel color transformations on the GPU to avoid UI processing latency.
Future Directions
While EVM demonstrates the visibility of microvascular variations, clinical environments face physical limitations. Future optimization paths include implementing dense Optical Flow matrices (such as Lucas-Kanade trackers) to dynamically warp coordinates along with facial expressions, preserving the ROI frame.
Furthermore, combining red, green, and blue components via dynamic chrominance algorithms (such as POS or CHROM) could normalize the signal under fluctuating ambient lighting. Deploying lightweight convolutional feature extraction layers via WebGPU could also enable reliable, low-light rPPG extraction.