Concept

Perkung-fu combines martial-arts discipline with percussive synthesis — (joking). That's just the name. The core idea is simpler and weirder: point the camera at a banana or an apple and the plugin switches presets. Each fruit triggers a different instrument configuration. Then, with a contact mic attached to the fruit, you tap it — and listen to the haptic percussion of the object itself. The fruit is both the controller and the instrument.

Technical Detail

• —Physical modeling synthesis engine built in C++/JUCE. Parameters include Tuning (40–2000 Hz), Decay, Damping, and Strike position — mapped to the physical behavior of a resonant membrane.
• —Camera-based object detection pipeline integrated into the WebView UI layer. Detected objects and their positions modulate synthesis parameters in real time via the JS–JUCE bridge.
• —Terminal-style UI with selectable palettes, scanlines, bloom effects, and bilingual label support — the interface is designed to feel alien and performative.

Learnings

• —Combining WebView and MediaPipe inside an audio plugin that runs in a DAW has serious potential beyond this demo. Here it is used for fun, but the underlying architecture — real-time computer vision driving audio parameters inside a production environment — is a legitimate direction. This project is just the tip of the iceberg.
• —Physical modeling is a fascinating world, largely because of the role mathematics plays in it. Translating the physics of a resonant membrane — tuning, damping, strike position — into code that produces convincing percussive sound made the connection between equations and acoustic reality feel immediate and rewarding.