Multimodal AI System Design

Build AI systems that spatially and temporally ground language in audio-visual scenes for applications in robotics, media, and accessibility.

Design AI systems that seamlessly fuse text, vision, audio, and sensor data into unified multimodal pipelines for real-world applications.

Design multimodal perception systems for embodied AI agents — robots, drones, and autonomous systems — integrating vision, language, and sensor data.

Design AI-powered content moderation systems that detect harmful, violating, or policy-breaking content across text, images, video, and audio at scale.

Design, collect, annotate, and quality-control multimodal training datasets combining text, images, audio, and video for AI model development.

Design rigorous evaluation benchmarks and metrics for multimodal AI systems, ensuring fair, reproducible, and meaningful capability measurement.

Design multimodal AI systems for healthcare that integrate medical imaging, clinical notes, lab data, and genomics for diagnosis support and clinical decision-making.

Design retrieval-augmented generation systems that retrieve and reason over text, images, tables, and documents for knowledge-intensive AI applications.

Design conversational AI systems that combine speech understanding with visual perception for voice-driven, visually-aware assistant and interface applications.

Architect and fine-tune vision-language models (VLMs) for tasks like image captioning, visual QA, document understanding, and grounded reasoning.