Gesture Synth Piano: Play a Hand Gesture Piano Online

Gesture Piano takes a minute to understand. You do not press a physical key. You place a fingertip over an on-screen key, tap down, and lift it again. The camera tracks that short movement and the Web Audio instrument plays the note.

It works well for slow melodies, simple bass notes, and short patterns. It is also a useful way to find out whether camera music suits you before buying a MIDI keyboard. Open the Gesture Synth instrument, or choose Gesture Piano in the hand gesture music library.

Put your hands in a good position

Sit far enough from the webcam that both hands fit in the frame. Face your palms toward the camera and leave some space around the fingertips. Light your hands from the front. A bright window behind you can turn them into silhouettes, which makes tracking harder.

When the browser asks for camera access, choose Allow. Gesture Synth processes the hand landmarks in the browser session and does not upload the live camera feed. Wait for setup to finish, then click once inside the instrument so the browser can start audio.

The preview may look mirrored. That is normal. Use the visible keyboard and fingertip target rather than trying to imagine the view from across the room.

Play one note

Use this four-step movement:

1. Hold your palm toward the camera.
2. Move a fingertip above the key you want.
3. Lower or curl the fingertip into the key.
4. Lift it before moving to the next note.

The lift matters. Keeping a finger over a key does not create a series of notes. If the same note retriggers, lift a little higher between taps. If a note is missed, slow down and make the downward movement clearer.

Begin with three neighboring keys. Play a slow left, middle, right, middle pattern. A small target area makes it easier to see whether the hand is being tracked correctly. After that feels reliable, use more of the keyboard or bring in the second hand.

Add the other hand

All five fingertips on both hands can target keys. One hand can carry a melody while the other repeats a low note. You do not need to copy traditional piano fingering while learning. A clean gesture is more useful than a clever one.

For a simple exercise, play one low note with the left hand on beat one of each bar. Use the right hand for a four-note phrase during the rest of the bar. Keep the tempo slow enough that every finger comes up before the next note.

Do not cross the hands at first. Overlapping fingers are harder for the tracker to separate. Keep the hands apart, then bring them closer once you can play without missed taps.

If notes go missing

Gesture Piano needs to see the whole hand. Turn a sideways palm toward the lens. If a fingertip disappears behind another finger, move the hand higher or farther away. Keep the wrist inside the frame and avoid sudden movements toward the camera.

These small changes usually help:

• Aim for the center of the visible key.
• Leave a clear lift between notes.
• Slow down before changing the lighting or settings.
• Keep neighboring fingers slightly apart.
• Click inside the instrument if the browser is silent.

If the camera is blocked, change the browser's site permission to Allow and reload. If tracking slows down, improve the light on your hand and close another application that may be using the webcam.

Record a short piano part

The loop controls are handy for four-bar ideas. Pick a comfortable tempo, select Track 1, and press R. Wait for the count-in and play a short melody. When the track loops, select Track 2 if you want to add a bass note or a chord part.

You can mute a mistake in the grid without throwing away the whole take. M mutes the selected track, S solos it, and Delete clears it. Export saves the loop mix as an audio WebM file.

Use the top Record control when you want the hand movement in the result. It records the canvas and synthesized sound together. Newer browsers may save MP4, while browsers without MP4 recording support save WebM. The browser creates the file locally.

A one-minute exercise

Try this before playing a full song:

1. Play four neighboring notes with the right hand.
2. Reverse the same four notes at the same tempo.
3. Add one low left-hand note at the start of each group.
4. Record four bars and listen for missed taps.

The exercise is deliberately plain. It lets you fix the movement before a faster melody makes the problem harder to hear. When it feels steady, use the Gesture Synth chords guide to add harmony, or return to the full Gesture Synth tutorial for theremin, drums, and looping.

Keep the taps small and give each note a clear target. Slow playing is not a limitation here. It is the easiest way to learn what the camera can reliably see.