AirBeats — Vision-Powered Music Platform
Browser-based music platform that lets users play instruments using webcam hand tracking — no controllers needed.
🎵 What Is AirBeats?
AirBeats is a browser-based music platform that transforms your webcam into a musical instrument. No controllers, no downloads, no setup — just open the browser, wave your hands, and play. It brings together real-time computer vision, interactive game mechanics, and audio synthesis into a single playful experience.
🧒 Built for Curious Minds
AirBeats is designed to spark curiosity in kids and beginners alike. By mapping hand gestures to musical sounds, it helps build mind-muscle connection — the same kind of coordination that comes from learning a real instrument. Kids discover rhythm, melody, and timing through play, developing spatial awareness and fine motor skills without even realizing they're learning. It makes music education accessible, screen-interactive, and genuinely fun.
🥁 Five Instruments, Infinite Possibilities
Choose from a 6-pad drum kit with crash, ride, snare, bass, toms, and hi-hat — or switch to a traditional Tabla with 20+ articulations like Na, Dhin, Dha, and Ge. The Piano Tiles mode turns MIDI files into a falling-tile rhythm game where your finger count selects columns. A 21-key Piano Grid lets you target individual notes from C3 to B5. And with the Custom Builder, you can create your own instrument — assign from 60+ sound presets, upload your own samples, and design the layout you want.
🤖 How the Magic Works
Under the hood, a MediaPipe-powered hand tracking pipeline identifies 3D landmarks from your webcam feed in real time. A gesture recognizer classifies open palms vs. fists for percussive hits, while a finger-count detector maps extended fingers to Piano Tiles columns. A collision engine checks gesture positions against pads and tiles, a game engine schedules and renders notes based on MIDI timing, and a low-latency audio engine powered by Web Audio API and Tone.js delivers responsive sound — all running entirely in the browser.
🎶 Why It Matters
AirBeats isn't just a tech demo — it's a doorway into music for anyone with a laptop and a webcam. It lowers the barrier to musical exploration, supports learning through play, and proves that powerful creative tools can live entirely in the browser. Whether you're a kid discovering drums for the first time or a developer curious about vision-driven interfaces, AirBeats has something for you.
Tech: React 19, TypeScript, Vite, Tailwind CSS, MediaPipe Tasks Vision, Web Audio API, Tone.js, @tonejs/midi, Canvas 2D