About Gesture Synth

What Gesture Synth is

Gesture Synth is a free, browser-based musical instrument maintained by TideSparrow. It turns webcam-detected hand movement into chords, theremin-style pitch, expression controls, synchronized loops, and downloadable recordings.

The project started with a simple question: can a laptop camera become a musical input without installing an app or buying a controller? The answer became a set of five playable modes rather than a one-effect demo. Gesture Piano turns downward fingertip motion into notes. Gesture Chords separates harmony and expression across two hands. Air Theremin maps hand height to pitch and volume, while Air Flute and Air Drums use visible targets for melody and rhythm.

What is original here

Gesture Synth's value is the complete browser performance path. The project combines camera permission, local hand tracking, instrument-specific gesture detection, Web Audio synthesis and samples, a four-track loop workflow, and local audio or video export in one interface.

The gesture rules are implemented and maintained in the public project source. They include safeguards that are specific to playing music with a camera: short hold times stop unstable chord poses from flickering, lift thresholds prevent one piano tap from retriggering, cooldowns suppress duplicate drum hits, and aspect-ratio correction keeps the visible target aligned with the camera landmark. The engineering walkthrough explains those choices and their limitations.

How it works and what leaves the device

• MediaPipe hand landmarks translate visible movement into musical controls.
• The Web Audio API generates and mixes sound in the browser.
• Camera access begins only after the player grants permission.
• The live camera feed is processed inside the browser session and is not uploaded by Gesture Synth.
• MediaPipe runtime and model files are downloaded from third-party hosts; those hosts receive normal request data, but not camera frames for recognition.
• Recordings are assembled by the browser and downloaded locally rather than stored in a Gesture Synth account.

The Privacy Policy describes the current data flow, analytics and advertising services, and visitor controls in more detail.

How the guides are produced

The guides on this site document controls that exist in the live instrument. They are written around reproducible tasks such as making an I-IV-V-IV chord loop, playing the opening of Ode to Joy, or keeping an Air Drums backbeat. Technical claims are checked against the implementation, and outside platform claims link to first-party documentation when possible.

The site does not publish scraped articles, syndicated feeds, or automatically generated pages. When a page no longer reflects the product or lacks enough first-hand value, it is corrected, redirected, or removed from the public article index. Dates on guides indicate when their content was last reviewed.

Testing and known limits

Deterministic parts of the instrument—such as landmark geometry, target intersection, movement thresholds, and finger-state transitions—have automated tests in the public repository. Browser checks cover page rendering and the permission-to-instrument flow. Real camera performance still varies with lighting, framing, webcam quality, browser support, hand overlap, and movement speed, so the guides describe practical recovery steps rather than promising perfect recognition.

Gesture Synth is a creative instrument, not an accessibility device, music teacher, medical product, or replacement for a physical MIDI keyboard. The Gesture Synth versus MIDI keyboard guide explains where each input method is stronger.

Who maintains it

TideSparrow develops, documents, and maintains Gesture Synth. The project's source history, implementation, tests, and public issue tracker are available in the Gesture Synth repository. The site may use clearly separated advertising to help cover hosting and continued development, but ads do not change the instrument controls or editorial conclusions.

Contact

For a bug report, browser compatibility problem, correction, rights concern, privacy question, or product suggestion, use the public contact page. Please include the page or instrument mode involved and enough detail to reproduce the issue without sharing private recordings or credentials.