Geonsun Lee

Effective presentations blend gestures, speech, and slides contents harmoniously. However, current tools for embedding presenters into slides often demand substantial manual effort. We introduce TalkDirector, a novel system that dynamically renders the presenter’s video feed within slides, optimizing placement for content relevance, gestures, and layout. Our pipeline leverages automatic speech recognition, real-time segmentation, gesture detection, and large multi-modal models to infer the video feed’s placement and size within the slides. We conducted two workshops (n=5) to understand motivations and decision-making for video placements in presentations. Further, we collected VTalk-68, a multimodal dataset of 68 presentations with 20 presenters to gain insights into user preferences for video placements in various presentation contexts. A user evaluation against a baseline prototype (n=12) showed that TalkDirector significantly reduces cognitive load and enhances presentation effectiveness, underscoring the potential of dynamic video integration to improve online presentations.

Maintaining engagement in immersive meetings is challenging, particularly when users must catch up on missed content after disruptions. Traditional transcription methods, like tabletop panels, distract users from the group, diminishing social presence, while always-on avatar-fixed interfaces fail to provide past context. We developed a context-aware avatar-fixed transcription panel that adapts based on user engagement, offering live transcriptions and summaries to enhance catching up while preserving social presence. A 12-participant formative study in a live VR meeting guided our design, leading to two user studies with small (3 avatars) and mid-sized (7 avatars) groups. Results showed our method significantly improved social presence (p < .05) over baseline and information recall (p < .05) over always-on interfaces, with even greater improvements in mid-sized groups (p < .001). Participants ranked our method as the most preferred. These findings suggest that context-aware transcriptions effectively maintain engagement and support re-engagement, particularly in larger group settings.