Autonomous Vehicle Simulation
Identifying Issues and Testing Solutions for Simulation Tooling.
Uber’s autonomous vehicle division (Uber ATG, now merged with Aurora) utilized Unreal Engine 4 to create simulations for testing and validating autonomous vehicle behavior. To learn more about simulation at Uber, watch the video below:
Problem
Creating a simulated scenario took between 30 minutes and 16 hours. This variance not only frustrated users, but it also made it difficult to plan scenario development at scale and impacted the company’s development targets.
Problem
Creating a simulated scenario took between 30 minutes and 16 hours. This variance not only frustrated users, but it also made it difficult to plan scenario development at scale and impacted the company’s development targets.
Solution
Within three months, a new process to apply motion to simulation actors was implemented. This increased alignment between simulation design and implementation, decreasing the iteration count, which lowered the maximum time to create a scenario.
Solution
Within three months, a new process to apply motion to simulation actors was implemented. This increased alignment between simulation design and implementation, decreasing the iteration count, which lowered the maximum time to create a scenario.
Research
Generative research methods were ongoing to ensure a close understanding of user processes. This provided me a catalog of user issues and feedback. Through routine analysis of this catalog, I identified a pattern of pain points when setting up virtual actors (e.g., cars, bicyclists, pedestrians). The primary issue was a table-based interface that required repetitive input and was prone to errors. This led to multiple simulation exports and re-runs. With certain simulation scenarios relying on actor speed, this was a potential cause of the variance in creation time. User observation and interviews confirmed that the virtual actor creation flow was both error-prone and highly time-variable
Research
Generative research methods were ongoing to ensure a close understanding of user processes. This provided me a catalog of user issues and feedback. Through routine analysis of this catalog, I identified a pattern of pain points when setting up virtual actors (e.g., cars, bicyclists, pedestrians). The primary issue was a table-based interface that required repetitive input and was prone to errors. This led to multiple simulation exports and re-runs. With certain simulation scenarios relying on actor speed, this was a potential cause of the variance in creation time. User observation and interviews confirmed that the virtual actor creation flow was both error-prone and highly time-variable
Testing
I created low-fidelity mockups that replaced the table-based inputs with an editable graph. This was inspired by 3D animation tools and internal visualization methods. With this I aimed to increase control over variables, while also giving a visual expectation of actor movement.
Testing
I created low-fidelity mockups that replaced the table-based inputs with an editable graph. This was inspired by 3D animation tools and internal visualization methods. With this I aimed to increase control over variables, while also giving a visual expectation of actor movement.
User feedback was gathered on this mockup. I narrowed collaboration to two subject matter experts for a deeper dive into core concepts. Through this, we discovered the graph-based interface was difficult to standardize. This prompted another mockup around interaction points (a concept derived from how simulation designs were documented internally).
User feedback was gathered on this mockup. I narrowed collaboration to two subject matter experts for a deeper dive into core concepts. Through this, we discovered the graph-based interface was difficult to standardize. This prompted another mockup around interaction points (a concept derived from how simulation designs were documented internally).
This version received unanimously positive feedback. I then collaborated with developers to assess feasibility and refined the mockup to reflect their input. With this I drafted a PRD (Product Requirements Document) outlining expected behavior, MVP features, and future iterations. Given the tool’s nature, high-fidelity mockups were not necessary.
This version received unanimously positive feedback. I then collaborated with developers to assess feasibility and refined the mockup to reflect their input. With this I drafted a PRD (Product Requirements Document) outlining expected behavior, MVP features, and future iterations. Given the tool’s nature, high-fidelity mockups were not necessary.
Pivot
Due to shifting company priorities, a product designer and user researcher joined our team to redesign the entire simulation-creation tool. To support this effort, I shared my issues catalog and feature PRDs (including the one for this project). At this point, my role shifted to supporting the new designer through understanding user nuance and providing design feedback on the new tool. Ultimately, the core flow of using interaction points to determine actor motion was implemented into the new tool.
Pivot
Due to shifting company priorities, a product designer and user researcher joined our team to redesign the entire simulation-creation tool. To support this effort, I shared my issues catalog and feature PRDs (including the one for this project). At this point, my role shifted to supporting the new designer through understanding user nuance and providing design feedback on the new tool. Ultimately, the core flow of using interaction points to determine actor motion was implemented into the new tool.
Impact
Hi, I'm Luke
Hi, I'm Luke
Reduced maximum scenario creation time from 16 hours to 2 hours, improving predictability for cross-team planning.
The interaction-point paradigm unblocked the creation of 5 new scenario types (1,000+ individual simulations with added variables), enabling more comprehensive vehicle safety testing.
If you want to hear more details about this project,
feel free to email me at lukeschmidt.design@gmail.com
or connect with me on LinkedIn
If you want to hear more details about this project,
feel free to email me at lukeschmidt.design@gmail.com
or connect with me on LinkedIn