Concept

Point the camera at a banana or an apple and the plugin switches presets. Attach a contact mic to the fruit, tap it, and listen to the haptic percussion of the object itself. The fruit is both the controller and the instrument.

Inspired by Haptic Percussion from Sampleson — turning everyday objects into expressive instruments through contact microphones.

Technical Detail

• —8-mode modal synthesis engine in C++17 / JUCE 8 using Bessel zero ratios for circular membrane modes. Onset detection via high-pass differencing with 3 ms peak-hold and 50 ms refractory cooldown. 9 parameters: tuning, decay, damp, strike, attenuation, low-cut, mic gain, output gain, threshold.
• —Svelte 5 UI embedded via WKWebView. JS-to-C++ bridged through URL-scheme interception; C++ pushes parameters, peaks, and a 40-bin FFT spectrum back at 30 Hz. Audio thread reads only atomic pointers — fully lock-free.
• —Camera-based object detection via MediaPipe switches presets in real time. Terminal-style interface with scanlines, bloom, and selectable palettes.
• —Passes pluginval at strictness level 5 (VST3). Builds as VST3, AU, and Standalone.

Learnings

• —Real-time computer vision driving audio parameters inside a DAW is a legitimate architectural pattern — this project is a proof of concept, but the bridge design scales.
• —Translating the physics of a resonant membrane into code made the connection between equations and acoustic reality feel immediate. Physical modeling rewards mathematical intuition.