(* indicates equal contribution)
GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment
Lance Ying, Kunal Jha, Shivam Aarya, Joshua B. Tenenbaum, Antonio Torralba, Tianmin Shu.
arXiv preprint, 2024.
MMToM-QA: Multimodal Theory of Mind Question Answering
Chuanyang Jin, Yutong Wu, Jing Cao, Jiannan Xiang, Yen-Ling Kuo, Zhiting Hu, Tomer D. Ullman, Antonio Torralba, Joshua B. Tenenbaum, Tianmin Shu.
62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024
(Outstanding Paper Award)
Risk-Bounded Online Team Interventions via Theory of Mind
Yuening Zhang, Paul Robertson, Tianmin Shu, Sungkweon Hong, Brian C. Williams.
IEEE International Conference on Robotics and Automation (ICRA), 2024.
An asynchronous, automated workflow for looking time experiments with infants
Gal Raz, Sabrina Piccolo, Janine Medrano, Shari Liu, Kirsten Lydic, Catherine Mei, Victoria Nguyen, Tianmin Shu, Rebecca Saxe.
Developmental Psychology, 2024.
Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning
Zhiting Hu* and Tianmin Shu*.
arXiv preprint, 2023.
Paper   NeurIPS 2023 Tutorial
NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants
Xavier Puig*, Tianmin Shu*, Joshua B. Tenenbaum, and Antonio Torralba.
IEEE International Conference on Robotics and Automation (ICRA), 2023.
Paper   Project   Code   TechXplore
Stateful Active Facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning
Dianbo Liu*, Vedant Shah*, Oussama Boussif*, Cristian Meo, Anirudh Goyal, Tianmin Shu, Michael Mozer, Nicolas Heess, and Yoshua Bengio.
11th International Conference on Learning Representations (ICLR), 2023.
Zero-shot linear combinations of grounded social interactions with Linear Social MDPs.
Ravi Tejwani*, Yen-Ling Kuo*, Tianmin Shu, Bennett Stankovits, Dan Gutfreund, Joshua B. Tenenbaum, Boris Katz, and Andrei Barbu.
37th AAAI Conference on Artificial Intelligence (AAAI), 2023.
Show Me What You Can Do: Capability Calibration on Reachable Workspace for Human-Robot Collaboration
Xiaofeng Gao, Luyao Yuan, Tianmin Shu, Hongjing Lu, and Song-Chun Zhu.
IEEE Robotics and Automation Letters (RA-L), 2022.
Paper   Project   TechXplore
Incorporating Rich Social Interactions Into MDPs.
Ravi Tejwani*, Yen-Ling Kuo*, Tianmin Shu, Bennett Stankovits, Dan Gutfreund, Joshua B. Tenenbaum, Boris Katz, and Andrei Barbu.
IEEE International Conference on Robotics and Automation (ICRA), 2022.
(Excellent Paper Award at IROS Cognitive and Social Aspects of Human Multi-Robot Interaction Workshop, 2022)
Social Interactions as Recursive MDPs
Ravi Tejwani*, Yen-Ling Kuo*, Tianmin Shu, Boris Katz, and Andrei Barbu.
5th Conference on Robot Learning (CoRL), 2021.
Paper   Project   MIT News   Discover Magazine
AGENT: A Benchmark for Core Psychological Reasoning
Tianmin Shu, Abhishek Bhandwaldar, Chuang Gan, Kevin A. Smith, Shari Liu, Dan Gutfreund, Elizabeth Spelke, Joshua B. Tenenbaum, and Tomer D. Ullman.
38th International Conference on Machine Learning (ICML), 2021.
Paper   Supp   Project   Code (Data Generation)   ScienceNews for Students   VentureBeat   Blog
Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
Xavier Puig, Tianmin Shu, Shuang Li, Zilin Wang, Joshua B. Tenenbaum, Sanja Fidler, and Antonio Torralba.
9th International Conference on Learning Representations (ICLR), 2021. (Spotlight presentation)
(Best Paper Award at NeurIPS Cooperative AI Workshop, 2020)
Paper   Project   Code   VentureBeat
PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception
Aviv Netanyahu*, Tianmin Shu*, Boris Katz, Andrei Barbu, and Joshua B. Tenenbaum.
35th AAAI Conference on Artificial Intelligence (AAAI), 2021.
(Best Paper Award at NeurIPS Shared Visual Representations in Human and Machine Intelligence Workshop, 2020)
Paper   Supp   Project   Code (Data Generation)
A Unified Psychological Space for Human Perception of Physical and Social Events
Tianmin Shu, Yujia Peng, Song-Chun Zhu, and Hongjing Lu.
Cognitive Psychology, 128: 101398, 2021.
Paper   Code (Stimuli Generation)   Code (Model)   Data
Exploring Biological Motion Perception in Two-stream Convolutional Neural Networks
Yujia Peng, Hannah Lee, Tianmin Shu, and Hongjing Lu
Vision Research, 178: 28-40, 2021.
Learning to Infer Human Attention in Daily Activities
Zhixiong Nan, Tianmin Shu, Ran Gong, Shu Wang, Ping Wei, Song-Chun Zhu, and Nanning Zheng.
Pattern Recognition, 103: 107314, 2020.
VRKitchen: an Interactive 3D Environment for Learning Real Life Cooking Tasks
Xiaofeng Gao, Ran Gong, Tianmin Shu, Xu Xie, Shu Wang, and Song-Chun Zhu.
ICML workshop on Reinforcement Learning for Real Life, 2019.
Paper   Project   Code   Tech Xplore   Bibtex
Social Scene Understanding: Group Activity Parsing, Human-Robot Interactions, and Perception of Animacy
Tianmin Shu.
PhD Thesis, 2019.
Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning
Tianmin Shu, Caiming Xiong, and Richard Socher.
6th International Conference on Learning Representations (ICLR), 2018.
Paper   ScienceNews   Bibtex
Interactive agent modeling by learning to probe
Tianmin Shu, Caiming Xiong, Ying Nian Wu, and Song-Chun Zhu.
NeurIPS Deep Reinforcement Learning Workshop, 2018.
Learning Social Affordance Grammar from Videos:
Transferring Human Interactions to Human-Robot Interactions
Tianmin Shu, Xiaofeng Gao, Michael S. Ryoo, and Song-Chun Zhu.
IEEE International Conference on Robotics and Automation (ICRA), 2017.
Paper   Project   Slides   New Scientist   Bibtex
Joint Inference of Groups, Events and Human Roles in Aerial Videos
Tianmin Shu, Dan Xie, Brandon Rothrock, Sinisa Todorovic, and Song-Chun Zhu.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
(Oral presentation, acceptance rate: 71/2123 = 3.3%)