Sound & Interaction — DSP · Physical Modeling · HCI
Perkung-fu
Physical modeling percussion synthesizer VST plugin with a sci-fi terminal interface driven by real-time camera object detection.
C++JUCEMediaPipeWebView
Work in progress
Concept
Perkung-fu combines martial-arts discipline with percussive synthesis — (joking). That's just the name. The core idea is simpler and weirder: point the camera at a banana or an apple and the plugin switches presets. Each fruit triggers a different instrument configuration. Then, with a contact mic attached to the fruit, you tap it — and listen to the haptic percussion of the object itself. The fruit is both the controller and the instrument.
Technical Detail
- —Physical modeling synthesis engine built in C++/JUCE. Parameters include Tuning (40–2000 Hz), Decay, Damping, and Strike position — mapped to the physical behavior of a resonant membrane.
- —Camera-based object detection pipeline integrated into the WebView UI layer. Detected objects and their positions modulate synthesis parameters in real time via the JS–JUCE bridge.
- —Terminal-style UI with selectable palettes, scanlines, bloom effects, and bilingual label support — the interface is designed to feel alien and performative.
