publications | Kevin Pu

2024

UIST
VizGroup: An AI-Assisted Event-Driven System for Real-Time Collaborative Programming Learning Analytics

Xiaohang Tang, Sam Wong, Kevin Pu , Xi Chen, Yalong Yang, and Yan Chen

2024

Abs Bib PDF

Programming instructors often conduct collaborative learning activities, like Peer Instruction, to foster a deeper understanding in students and enhance their engagement with learning. These activities, however, may not always yield productive outcomes due to the diversity of student mental models and their ineffective collaboration. In this work, we introduce VizGroup, an AI-assisted system that enables programming instructors to easily oversee students’ real-time collaborative learning behaviors during large programming courses. VizGroup leverages Large Language Models (LLMs) to recommend event specifications for instructors so that they can simultaneously track and receive alerts about key correlation patterns between various collaboration metrics and ongoing coding tasks. We evaluated VizGroup with 12 instructors using a dataset collected from a Peer Instruction activity that was conducted in a large programming lecture. The results showed that compared to a version of VizGroup without the suggested units, VizGroup with suggested units helped instructors create additional monitoring units on previously undetected patterns on their own, covered a more diverse range of metrics, and influenced the participants’ following notification creation strategies.
@misc{tang2024vizgroupaiassistedeventdrivenrealtime, title = {VizGroup: An AI-Assisted Event-Driven System for Real-Time Collaborative Programming Learning Analytics}, author = {Tang, Xiaohang and Wong, Sam and Pu, Kevin and Chen, Xi and Yang, Yalong and Chen, Yan}, year = {2024}, eprint = {2404.08743}, archiveprefix = {arXiv}, primaryclass = {cs.HC}, url = {https://arxiv.org/abs/2404.08743}, }
CHI
Behind the Pup-ularity Curtain: Understanding the Motivations, Challenges, and Work Performed in Creating and Managing Pet Influencer Accounts

Suhyeon Yoo, Kevin Pu, and Khai N. Truong

In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 2024

Abs Bib PDF

Creating dedicated accounts to post users’ pet content is a growing trend on Instagram. While these account owners derive joy from this pursuit, they may also struggle with criticisms and challenges. Yet, there remains a knowledge gap on how pet account owners manage their pets’ online presence and navigate these obstacles successfully. Drawing from interviews with 21 Instagram pet account owners, we uncover the motivations behind pet account creation, spanning personal, altruistic, and commercial goals. We learn about the strategies employed for crafting their pets’ online identities and personas, as well as the challenges faced by both owners and their pets in navigating the complexities of digital identity management. We discuss the evolving dynamics between humans and their pets, positioning pet identity cultivation as a form of collaborative work, akin to the “third shift”, highlighting the need to design interfaces that support this unique identity management process.
@inproceedings{10.1145/3613904.3642367, author = {Yoo, Suhyeon and Pu, Kevin and Truong, Khai N.}, title = {Behind the Pup-ularity Curtain: Understanding the Motivations, Challenges, and Work Performed in Creating and Managing Pet Influencer Accounts}, year = {2024}, isbn = {9798400703300}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3613904.3642367}, doi = {10.1145/3613904.3642367}, booktitle = {Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems}, articleno = {736}, numpages = {17}, keywords = {Instagram, human-animal interaction, pet influencers, social media}, location = {Honolulu, HI, USA}, series = {CHI '24}, }

2023

UIST
DiLogics: Creating Web Automation Programs with Diverse Logics

Kevin Pu, Jim Yang, Angel Yuan, Minyi Ma, Rui Dong, Xinyu Wang, Yan Chen, and Tovi Grossman

In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, San Francisco, CA, USA, 2023

Abs Bib PDF Video

Knowledge workers frequently encounter repetitive web data entry tasks, like updating records or placing orders. Web automation increases productivity, but translating tasks to web actions accurately and extending to new specifications is challenging. Existing tools can automate tasks that perform the same logical trace of UI actions (e.g., input text in each field in order), but do not support tasks requiring different executions based on varied input conditions. We present DiLogics, a programming-by-demonstration system that utilizes NLP to assist users in creating web automation programs that handle diverse specifications. DiLogics first semantically segments input data to structured task steps. By recording user demonstrations for each step, DiLogics generalizes the web macros to novel but semantically similar task requirements. Our evaluation showed that non-experts can effectively use DiLogics to create automation programs that fulfill diverse input instructions. DiLogics provides an efficient, intuitive, and expressive method for developing web automation programs satisfying diverse specifications.
@inproceedings{10.1145/3586183.3606822, author = {Pu, Kevin and Yang, Jim and Yuan, Angel and Ma, Minyi and Dong, Rui and Wang, Xinyu and Chen, Yan and Grossman, Tovi}, title = {DiLogics: Creating Web Automation Programs with Diverse Logics}, year = {2023}, isbn = {9798400701320}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3586183.3606822}, doi = {10.1145/3586183.3606822}, booktitle = {Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology}, articleno = {74}, numpages = {15}, keywords = {PBD, Web automation, neurosymbolic programming}, location = {San Francisco, CA, USA}, series = {UIST '23}, }

2022

UIST
SemanticOn: Specifying Content-Based Semantic Conditions for Web Automation Programs

Kevin Pu, Rainey Fu, Rui Dong, Xinyu Wang, Yan Chen, and Tovi Grossman

In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, Bend, OR, USA, 2022

Abs Bib PDF Video

Data scientists, researchers, and clerks often create web automation programs to perform repetitive yet essential tasks, such as data scraping and data entry. However, existing web automation systems lack mechanisms for defining conditional behaviors where the system can intelligently filter candidate content based on semantic filters (e.g., extract texts based on key ideas or images based on entity relationships). We introduce SemanticOn, a system that enables users to specify, refine, and incorporate visual and textual semantic conditions in web automation programs via two methods: natural language description via prompts or information highlighting. Users can coordinate with SemanticOn to refine the conditions as the program continuously executes or reclaim manual control to repair errors. In a user study, participants completed a series of conditional web automation tasks. They reported that SemanticOn helped them effectively express and refine their semantic intent by utilizing visual and textual conditions.
@inproceedings{10.1145/3526113.3545691, author = {Pu, Kevin and Fu, Rainey and Dong, Rui and Wang, Xinyu and Chen, Yan and Grossman, Tovi}, title = {SemanticOn: Specifying Content-Based Semantic Conditions for Web Automation Programs}, year = {2022}, isbn = {9781450393201}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3526113.3545691}, doi = {10.1145/3526113.3545691}, booktitle = {Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology}, articleno = {63}, numpages = {16}, keywords = {user intent, semantics, Web automation, PBD}, location = {Bend, OR, USA}, series = {UIST '22}, }