Embodied AI Perception Designer

Embodied AI perception is the discipline of designing the sensory and interpretive systems that allow physical agents — robots, drones, autonomous vehicles, and other situated machines — to understand their environment well enough to act purposefully within it. Unlike perception for static analysis, embodied perception must be real-time, robust to partial observability, and tightly coupled with action and planning systems.

The Embodied AI Perception Designer AI assistant helps you architect the multimodal perception stack for your embodied agent. This covers sensor suite selection and integration (RGB-D cameras, LiDAR, IMU, microphones, tactile sensors), perception model design for tasks including 3D scene understanding, object detection and tracking, affordance estimation, and language-conditioned navigation, as well as the interfaces between perception outputs and downstream planning and control modules.

A key focus of this assistant is the integration of natural language into embodied perception pipelines. It helps you design systems where a robot can receive and act on spoken or typed instructions, ask clarifying questions when its perceptual state is ambiguous, and generate natural language descriptions of what it perceives. This includes work on vision-language navigation, instruction following in 3D environments, and open-vocabulary object detection for manipulation tasks.

The assistant provides architecture blueprints for your perception stack, guidance on sim-to-real transfer strategies, recommendations for simulation environments such as AI2-THOR, Habitat, and Isaac Sim for training and evaluation, and advice on handling the latency and reliability constraints of real hardware deployment.

This role is ideal for robotics engineers building perception systems for manipulation or navigation, autonomous systems researchers integrating large pretrained models into real-time pipelines, and AI researchers designing multimodal agents for embodied AI benchmarks and competitions.