Post-Touch
Expressive gesture in touchless creative interfaces · Design Project 1 · Department of Design, IIT Delhi
Long before language we made meaning with our hands. This project asks what that knowledge looks like when the hand, and not a learned device grammar, becomes the interface.
To create
is human.
The oldest marks we have are handmade. People pressed pigment around their own hands on a cave wall. The gesture came first and the image followed.
We point before we can speak. We push away what we fear and shape the air to describe what we mean. The hand is, as Kant put it, the visible part of the brain.
It began as a tool.
It became an instrument.
Post-Touch started as a proposal for a tool, a fluid medium that would mirror the fluid nature of the hand. A canvas with no mouse and no stylus, where brush size and colour mapped to movement. The refinement turned that tool into a research instrument. The question was no longer whether we could paint without hardware. It was what expressive gesture actually is, and why no one had measured it.
Functional gesture
Studied as a command. Swipe, pinch, point. Success is measured by accuracy, agreement, and error rate. The body is asked to be precise and repeatable.
Expressive gesture
Modulated on purpose. Precision is sometimes abandoned and rhythm carries meaning. No prior study used a generative canvas to capture it, so it stayed invisible. The gap is methodological as much as empirical.
What does expressive gesture look like when the body, and not a learned device grammar, is the interface for creative mark making?
- 01
What kinematic parameters, velocity, depth, and dwell, characterise expressive gesture in mid air drawing?
- 02
Which spatial zones of the interaction volume are preferentially used, and how does this relate to fatigue?
- 03
What spontaneous gestural vocabularies do participants develop without prior instruction?
- 04
How should kinematic data be encoded visually so the resulting mark preserves expressive intent?
- 05
What design guidelines can be derived for inclusive expressive touchless interfaces?
The work rests on four ideas. Phenomenology, that the body is the site where perception and meaning happen and not a tool the mind drives. Kinematics, that speed, depth, and dwell describe how a gesture is made and not only what it points at. Creativity, that expressive work depends on uninterrupted attention an interface can support or break. And ergonomics, that every movement made in the air carries a physical cost the design has to account for.
Reading the hand used to
mean wearing the machine.
Each generation removed a layer of hardware until nothing was left between the hand and the camera.
Interaction meant suiting up. The Power Glove was expensive and tethered, and the machine, not the hand, was the focus.
The Kinect dropped the glove but still read the room with a special depth sensor.
A standard webcam is enough. Machine learning finds the hand in plain RGB video, which is what Post-Touch runs on.
The canvas is the instrument,
not the product.
Post-Touch tracks both hands in real time and turns bare movement into painted marks. It runs at thirty frames per second, logging position, velocity, and depth for each hand. The painting is not the goal. It is the trace the research reads.
- 01
Camera
Laptop webcam, 30 fps, 1280×720
- 02
Landmarks
MediaPipe Hands, 21 points per hand
- 03
Kinematics
Per frame speed, depth, dwell
- 04
Gesture
Open, pinch, fist, point
- 05
Brush
Width, alpha, pigment, then canvas
A vocabulary the
hand already knows.
Nothing here has to be learned. The system listens for gestures already in the body, so a first time participant can begin without instruction. This is the principle of primal mimesis.
Draw
The index fingertip is tracked in real time and its position drives the paint mark.
Pinch
Sliding a pinched hand left and right maps to the colour wheel, shifting the active hue.
Fist
Both hands closing at once changes the brush, the left held still while the right advances through the set.
Open palm
The palm becomes an eraser. Any stroke within its radius is lifted from the canvas.
Closing the
proprioceptive gap
Mid air has no surface to rest against, so the hand cannot feel where a mark will land. A ring anchored between the index and the thumb projects the exact point of contact and scales with the pinch aperture.
It is a small piece of pseudo haptics, a stand in for the resistance that is missing, and it resolved the disorientation almost immediately.
Eight brushes are organised into three families, with a ninth neutral line kept as a control. Each answers the same movement differently, mapping depth to weight, velocity to energy, and dwell to bloom. The mapping is where kinematics becomes image.
One gesture, many responses.
| Brush | Family | Depth | Velocity | Dwell |
|---|---|---|---|---|
| Glow | Mark | Stroke weight | Blur radius | Core opacity |
| Blossom | Mark | Branching count | Petal scale | Bloom radius |
| Ribbon | Mark | Nib width | Nib angle | Ink density |
| Neon | Mark | Glow spread | Strobe frequency | Core thickness |
| Scatter | Wet | Particle size | Spread radius | Particle count |
| Watercolor | Wet | Wash opacity | Edge graining | Bloom delay |
| Electric | Atmospheric | Branch depth | Arc frequency | Residual glow |
| Constellation | Atmospheric | Node density | Edge length | Star magnitude |
What the instrument records
Every frame is written to a session log as position, velocity, depth, gesture, and active brush. A thirty minute session becomes a structured record that can be replayed, measured, and compared across participants.
Tracked time · Dominant 72% · Non-dominant 15% · Both hands 10%
Across early sessions the dominant hand carried most of the drawing while the non-dominant hand steadied and framed. Velocity and dwell separated the brushes cleanly, and most activity stayed within a comfortable central zone, with reach to the edges falling off as fatigue set in.
Claude as a witness
When a session ends, the finished canvas and a compact kinematic summary are sent to a vision language model. It returns a short metaphorical title and a two sentence reading grounded in the body and its movement. The human gesture makes the work and the machine reads it afterward.
- Witness. It attends to the gesture as mark. It observes, it does not intervene.
- Interpreter. It reads only the finished trace, never the live hand.
- Authorship preserver. The expression precedes and determines the output. The reading cannot alter it.
- Generator. Post-Touch is not an AI art tool. The model adds no marks of its own.
Expressive gesture is not functional gesture performed artistically. It is a distinct register, marked by continuous modulation, spatial exploration, and rhythmic variation.
Participants consistently moved beyond the functional gesture space. They invented compound gestures the classifier never anticipated, like a slow pinch and draw to blend a wash. Four ideas carried the work. Primal mimesis, the reticle as pseudo haptics, the mapping of kinematics to visual form, and the machine as a respondent rather than a generator.
The system in motion
Live session capture, forthcoming
The synthesis
The final poster gathers the pivot, the system, and the early findings onto one sheet. It is the dark artifact this lighter page unpacks, reproduced as it was presented.
Outputs and what comes next
- D1
Kinematic Heatmap
Instrument built - D2
Spatial Lexicon
Data collection pending - D3
Kinematic to visual mappings
Prototype implemented - D4
Universal reach guidelines
Analysis pending