User-Defined Gestural Interfaces in Human-Computer Interaction

User-defined gestural interfaces empower end users to create, personalise and deploy bespoke movement-based commands for digital systems. Rather than relying on fixed, designer-defined gesture sets, these interfaces employ user-centred elicitation techniques to capture intuitive, context-relevant gestures. This approach accommodates diverse user preferences, cultural norms and ergonomic constraints, yielding higher acceptance and memorability. Core processes include prompting participants to propose gesture-to-command mappings, analysing agreement rates and refining a consensus vocabulary. Gesture tracking may rely on optical sensors, depth cameras, radar-based chips or inertial measurement units, each bringing trade-offs in accuracy, latency and deployment cost. Applications span smart home automation, spatial augmented reality, virtual and augmented reality, wearable computing and public interactive displays. Key challenges lie in ensuring robust recognition across environments, minimising fatigue through ergonomic evaluation, supporting adaptive remapping as user needs evolve and balancing system-level standardisation with personal flexibility. Emerging research addresses comfort evaluation models based on mechanical energy expenditure, radar-based microgesture taxonomies, integration with mixed-reality prototypes and the development of structured datasets to train adaptive interfaces. By harmonising user agency with reliable sensor technologies and well-defined elicitation protocols, the field is charting a path towards more natural, inclusive and efficient gestural interaction paradigms.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

A comprehensive review of mid-air‐gesture elicitation studies has mapped the evolution of user-defined vocabularies, identifying variability in procedure, participant profiles and appropriateness criteria. This work highlights the critical role of context framing and minimal affordance presentation in eliciting high-agreement gestures and points to under-explored questions around cross-cultural consistency and longitudinal memorability.

Advances in radar-based sensing have yielded a taxonomy of microgestures—fine finger and wrist motions—captured via wearable radar chips. Through clustering studies with two participant cohorts, researchers produced a consensus set of microgestures for common Internet-of-Things tasks, achieving high agreement rates and validating the taxonomy with fresh user trials. This classification underpins reliable recognition of radar-derived gestures for next-generation wearables and smart‐environment controls.

In virtual and augmented reality contexts, designers have systematically evaluated 33 ergonomically informed microgestures against 20 common commands. User studies emphasised comfort, preference and error rates, leading to a recommended microgesture set that minimises fatigue and maximises usability. The methodology—combining ergonomic principles with iterative preference testing—offers a blueprint for future gesture-based input schemes across immersive systems.

User-Defined Gestural Interfaces in Human-Computer Interaction publication trend

The graph below shows the total number of articles in user-defined gestural interfaces in human-computer interaction across all publications each year (not limited to Nature Index journals).

Technical terms

Gesture elicitation: A participatory method in which users propose movement-based commands for mapping to system functions.

Mid-air interaction: Touchless control of digital content or devices through freehand gestures captured by sensors.

Microgestures: Fine-scale, often finger-or wrist-centered, motions intended for precise or discreet input.

Gesture vocabulary: The complete set of distinct gestures assigned to specific commands within an interface.

Spatial augmented reality: The projection or superimposition of digital content onto real‐world surfaces, enabling gestural manipulation in a mixed environment.

References

1. Structured dataset of human-machine interactions enabling adaptive user interfaces. Scientific Data (2023).
2. Gesture Elicitation Studies for Mid-Air Interaction: A Review. Multimodal Technologies and Interaction (2018).
3. Eliciting Contact-Based and Contactless Gestures With Radar-Based Sensors. IEEE Access (2019).
4. Mechanical Energy Expenditure‐based Comfort Evaluation Model for Gesture Interaction. Computational Intelligence and Neuroscience (2018).
5. Design of 3D Microgestures for Commands in Virtual Reality or Augmented Reality. Applied Sciences (2021).
6. Mid-Air Gesture Control of Multiple Home Devices in Spatial Augmented Reality Prototype. Multimodal Technologies and Interaction (2020).