wondersnap

本页保存的是公开项目资料快照，阅读过程不需要连接 GitHub。

WonderSnap

Gesture-controlled 3D models made of glowing light particles, running entirely in your browser.

Snap your fingers in front of your webcam and up to 250,000 GPU particles swirl into existence. Make a fist and they form the Eiffel Tower, a beating heart or a V8 engine. Open your hand and the model morphs into the next one, or explodes into a labelled diagram of every part. No mouse, no controller, no install beyond Node.js.

图片：WonderSnap showing a beating human heart made of particles

Quick start

You need Node.js 18 or newer and a webcam (optional: everything also works with the mouse and keyboard).

git clone https://github.com/AkbarSheikh-debug/wondersnap.git
cd wondersnap
npm install
npm start

Then open http://localhost:5173 in Chrome or Edge and click Start with camera, or Continue without camera to drive it with the on-screen buttons and keyboard.

The browser only allows camera access on localhost or https, which is why the app comes with its own tiny local server. To use a different port: npm start -- 8080.

How it works

Layer	What it does
Hand tracking	MediaPipe Hand Landmarker tracks up to two hands (21 landmarks each) from the webcam, fully on-device
Gesture recognition	Custom classifiers turn landmarks into poses (fist, open, point, pinch, peace), a finger-snap detector, hand twist/tilt and two-hand zoom, fed through a debouncer and a state machine
Particle engine	A hand-written WebGL2 renderer. Particle physics runs on the GPU with transform feedback: no Three.js, no game engine, no framework
Models	33 procedural models built from real measurements and sampled into point clouds, with named parts that can explode, glow and be pulled out
Server	A zero-dependency Node.js static server (server.mjs)

Nothing is sent anywhere: the video never leaves your machine.

Gestures

Gesture	What it does
🫰 Snap	Summon the particles, or dissolve the current model
✊ Fist	Form the wonder, organ or machine
✋ Open hand	Wonders morph to the next one. Organs, engines and vehicles explode: how far you open your hand sets how far the parts fly apart, and closing it puts them back together
🔄 Twist / raise your hand	Turn and tilt the formed model
☝️ Point	Hold your finger on a part to select it. It glows, and a card explains what it does
🤏 Pinch	Pull the selected part out toward you; pinch again to put it back
🙌 Two hands	Move them apart or together to zoom
✌️ Peace	Jump to the next model

Keyboard and mouse

Key	Action
Space	Snap
F / O / V	Fist / open hand / peace sign
← →	Previous / next model
E, ↑ ↓, mouse wheel	Explode amount
+ - 0, ctrl + wheel	Zoom
C	Camera on/off
L	Part labels
R	Auto-rotate
G	Hand rotation on/off
X	Cut-away cross-section (, and . nudge the plane)
Q	Quiz mode
M	Voice commands and read-aloud
K	Record a video
D	Play the demo
I / Esc	Describe / deselect the selected part
H	Help

Click a part to select it, drag to rotate.

Features

• Beating heart and breathing lungs. The heart contracts in a lub-dub rhythm at 72 bpm, the lungs inflate every 4.5 s, and pulses of light travel through them like blood or air.
• Exploded views with named parts. Every part has a leader-line label saying what it does.
• Quiz mode. “Find: Hippocampus”: point at (or click) the right part. Five questions, with a score.
• Voice control. Say “show me the heart”, “open it up”, “where is the right atrium”, “zoom in”, “quiz” and more. Parts are read aloud with speech synthesis (Chrome or Edge).
• Cut-away. A cutting plane follows your hand and reveals a glowing cross-section.
• Recording. Save a WebM video of the scene.

Models

Category	Models
Wonders (11)	Turtle Tower, Eiffel Tower, Statue of Liberty, Burj Khalifa, Great Pyramid, Colosseum, Leaning Tower of Pisa, Taj Mahal, Big Ben, Christ the Redeemer, Sydney Opera House
Anatomy (10)	Brain, beating Heart, Kidney, breathing Lungs, Eye, Ear, Tooth, Skull, Skeleton, Human Body (skin, organs, nerves, arteries, veins, skeleton)
Biology (2)	DNA double helix that unzips, Animal cell
Engines (4)	Inline-4, Supercharged HEMI V8, Turbofan jet, 9-cylinder radial
Vehicles (4)	Sports car, Motorcycle, Airliner, Saturn V (with stage separation)
Machines (2)	Mechanical wristwatch, EV battery pack (280 cells, busbars, cooling, BMS)

URL options

Option	Effect
?n=250000	Particle count
?model=12	Start on a given model
?autostart=camera / ?autostart=nocamera	Skip the start screen
?trails=0	Turn off particle trails
?dpr=1	Force the device pixel ratio (useful on slower GPUs)

Tests

36 end-to-end and unit tests with Playwright, driving the real app with synthetic hands on a deterministic clock.

npx playwright install chromium   # one time
npm test

Spec	Covers
app.spec.js	The full gesture story, every model, explode/contract, keyboard, wheel, tabs, demo, phone layout, hand twist and tilt
features.spec.js	Heartbeat and breathing, two-hand zoom, point-to-pick, pinch-to-pull, quiz, voice commands, cut-away, video recording
camera.spec.js	Real getUserMedia to MediaPipe on Chromium’s fake webcam, plus the camera-denied fallback
gpu.spec.js	The GPU physics shader matches its CPU twin to ~1e-7 in every mode
logic.spec.js	Pose classifiers, snap detector, debouncer, state machine, controller, voice-command parser
models.spec.js	Every model is deterministic, finite and fast, with real measurements and correctly exploding parts

Project structure

index.html, styles.css     page and styles
server.mjs                 zero-dependency static server
models/                    MediaPipe hand landmark model
src/app.js                 render loop, explode, zoom, picking, quiz, cut-away, HUD, labels, demo
src/hands.js               webcam + MediaPipe Hand Landmarker
src/features.js            voice commands, read-aloud, video recorder
src/gl/                    WebGL2 shaders and renderer (transform-feedback physics)
src/logic/                 gestures, state machine, controller, CPU physics twin
src/lib/                   vector math, samplers, procedural shapes
src/models/                wonders, anatomy, biology, engines, vehicles, machines
tests/                     Playwright specs

Troubleshooting

• Camera doesn’t start: open the app via http://localhost:5173, not by double-clicking index.html, and allow camera access when the browser asks.
• Hand tracking never loads: run npm install first; the tracking runtime is served from node_modules.
• Low frame rate: try http://localhost:5173/?n=100000&dpr=1.
• Port already in use: npm start -- 8080 and open http://localhost:8080.

License

MIT © 2026 Akbar Sheikh