Full Portfolio
This portfolio presents my research projects with my stduents and collaborators, spanning wearable haptic interfaces, multimodal interaction and perception, and data-driven texture rendering. These projects explore how touch, thermal feedback, smell, force, motion, and intelligent sensing can create richer interaction in virtual, mixed, and everyday environments. The work has been published and showcased at top venues about computer graphics and human-computer interaction, including SIGGRAPH, IEEE TVCG, IEEE VR and CHI, DIS, CSCW. Select a project below to read more.
Wearable Haptic Interfaces
ThermAirGlove: A Pneumatic Glove for Thermal Perception and Material Identification in Virtual Reality
Shaoyu Cai, Pingchuan Ke, Takuji Narumi, and Kening Zhu
2020 IEEE Conference on Virtual Reality and 3D User Interfaces, pp. 248-257 (IEEE VR).
Keywords: Thermal feedback, pneumatic actuation, material simulation
ViboPneumo: A Vibratory-Pneumatic Finger-Worn Haptic Device for Altering Perceived Texture Roughness in Mixed Reality
Shaoyu Cai, Zhenlin Chen, Haichen Gao, Ya Huang, Qi Zhang, Xinge Yu, and Kening Zhu
IEEE Transactions on Visualization & Computer Graphics, 2024 (TVCG).
Keywords: Vibrotactile feedback, pneumatic actuation, texture rendering
ThermOuch: A Wearable Thermo-Haptic Device for Inducing Pain Sensation in Virtual Reality through Thermal Grill Illusion
Haichen Gao, Shaoyu Cai, Yuhong Wu, and Kening Zhu
SIGGRAPH Asia 2024 Conference Papers, pp. 1-12. 2024 (SIGGRAPH Asia).
Keywords: Thermohaptics, thermal grill illusion, pain sensation
PropelWalker: A Leg-based Wearable System with Propeller-based Force Feedback for Walking in Fluids in VR
Pingchuan Ke, Shaoyu Cai, Haichen Gao, and Kening Zhu
IEEE Transactions on Visualization and Computer Graphics, 29(12), 5149-5164. 2022 (TVCG).
Keywords: Force feedback, weight simultion, liquid simulation
VirCHEW Reality: On-Face Kinesthetic Feedback for Enhancing Food-Intake Experience in Virtual Reality
Qingqin Liu, Ziqi Fang, Jiayi Wu, Shaoyu Cai, Jianhui Yan, Tiande Mo, Shuk Ching Chan, and Kening Zhu
SIGGRAPH 2025 Conference Papers, pp. 1-12. 2025 (SIGGRAPH).
Keywords: Kinesthetic feedback, food texture rendering, pneumatic actuation
🏆 Top 10 Technical Papers Fast Forward — SIGGRAPH 2025
🏆 Best Audience Choice Award — SIGGRAPH Asia 2025 Emerging Technologies
Skin-integrated Multimodal Haptic Interface for Immersive Tactile Feedback
Ya Huang, Jingkun Zhou, Pingchuan Ke, Xu Guo, Chunki Yiu, Kuanming Yao, Shaoyu Cai, Dengfeng Li, Yu Zhou, Jian Li, Tsz Hung Wong, Yiming Liu, Lei Li, Yuyu Gao, Xingcan Huang, Hu Li, Jiyu Li, Binbin Zhang, Zhenlin Chen, Huanxi Zheng, Xingyu Yang, Haichen Gao, Zichen Zhao, Xu Guo, Enming Song, Hui Wu, Zuankai Wang, Zhaoqian Xie, Kening Zhu, and Xinge Yu
Nature Electronics 6, no. 12 (2023): 1020-1031.
Keywords: Flexible electronics, multimodal haptics, texture simulation
Multimodal Interaction & Perception
Modulating Olfactory Perception through Localized Nasal Thermal Stimulation
Rui Zhang, Yitian Ding, Ningchang Xiong, and Shaoyu Cai*
Proceedings of the 2026 Designing Interactive Systems Conference (DIS 2026), pp. 931-946. 2026.
Keywords: Crossmodal perception, thermal feedback, olfactory perception
Signals of Aggression: Modelling Multimodal Cues and Perceptual Effects in Virtual Agents
Shaun Jing Heng Ong, Aiden Tat Yang Koh, Shaoyu Cai*, Felicia Fang-Yi Tan, Patrick Chia, and Eng Tat Khoo
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026), April 13-17, 2026, Barcelona, Spain.
Keywords: Multimodal perception, aggression perception, virtual agents
From Beeping Machines to Caring Companions: Redesigning Social Robots Through Reflective Ethnography in Elderly Care Work
Xinpeng Liu, Haosu Xu, Yiming Luo, and Shaoyu Cai*
Proceedings of the ACM on Human-Computer Interaction 9(7), 1-29. CSCW 2025.
Keywords: Multimodal interaction, HRI, social robots, ederly care
🏆 Best Paper Honorable Mentions — CSCW 2025
Data-driven Texture Rendering
Visual-Tactile Cross-Modal Data Generation using Residue-Fusion GAN with Feature-Matching and Perceptual Losses
Shaoyu Cai, Kening Zhu, Yuki Ban, and Takuji Narumi
IEEE Robotics and Automation Letters (RA-L), vol. 6, no. 4, pp. 7525-7532 with IROS 2021 presentation.
Keywords: Cross-modal visual-tactile generation, Generative adversarial networks (GANs), Tactile intelligence
GAN-based Image-to-Friction Generation for Tactile Simulation of Fabric Material
Shaoyu Cai, Lu Zhao, Yuki Ban, Takuji Narumi, Yue Liu, and Kening Zhu
Computers & Graphics, vol. 102, pp. 460-473, Feb. 2022.
Keywords: Image-to-Friction generation, Generative adversarial networks (GANs), Electrotactile displays
🏆 Best Paper Audience Choice Award — ICAT-EGVE 2020
Multi-modal Transformer-based Tactile Signal Generation for Haptic Texture Simulation of Materials in Virtual and Augmented Reality
Shaoyu Cai and Kening Zhu
2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).
Keywords: Multi-modal visual-tactile generation, Transformer, Texture Simulation