DiLogics: Creating Web Automation Programs with Diverse Logics
Conditionally accepted to the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23), 2023
Knowledge workers frequently encounter repetitive web data entry tasks, like updating records or placing orders. Web automation increases productivity, but translating tasks to web actions accurately and extending to new specifcations is challenging. Existing tools can automate tasks that perform the same logical trace of UI actions (e.g., input text in each feld in order), but do not support tasks requiring diferent executions based on varied input conditions. We present DiLogics, a programming-by-demonstration system that utilizes NLP to assist users in creating web automation programs that handle diverse specifcations. DiLogics frst semantically segments input data to structured task steps. By recording user demonstrations for each step, DiLogics generalizes the web macros to novel but semantically similar task requirements. Our evaluation showed that non-experts can efectively use DiLogics to create automation programs that fulfll diverse input instructions. DiLogics provides an efcient, intuitive, and expressive method for developing web automation programs satisfying diverse specifcations.