OpenAI - OpenAI DevDay 2024 | Community Spotlight | Grab
Grab, a leading super app in Southeast Asia, is enhancing its mapping services through community-based mapping and AI technologies. Initially, Grab Maps was developed due to the inadequacy of third-party apps in providing localized data. Grab Maps now serves both internal needs and external businesses across Southeast Asia. The approach involves collecting street-level imagery using 360° cameras from its driver network, extracting details like traffic signs and road accessibility to build detailed maps. Recently, Grab has adopted OpenAI's vision fine-tuning capabilities to improve data matching for traffic signs, addressing challenges like intricate geometries and visual occlusions. This involves using a small fine-tuning dataset combining street-level imagery and map tiles to accurately match traffic signs to roads, enhancing the reliability of their maps.
Key Points:
- Grab Maps started in 2017 to address localization issues with third-party apps.
- It uses community-based mapping with 360° cameras to collect detailed street-level data.
- OpenAI's vision fine-tuning is used to improve traffic sign data matching.
- Grab Maps serves both internal and external clients across Southeast Asia.
- The approach enhances map reliability by addressing complex mapping challenges.
Details:
1. 👋 Introduction and Welcome
1.1. Speaker Introduction
1.2. Event Context
2. 🚗 Grab's Journey and Growth
- Grab began 12 years ago with the goal of enhancing taxi safety in Malaysia, initially operating in a single city and country.
- Currently, Grab stands as a leading super app in Southeast Asia, with 1 in 20 people utilizing its services for food, rides, and payments.
- The platform boasts over 41 million monthly transacting users, underscoring its extensive market reach and user engagement.
- Grab is dedicated to propelling Southeast Asia forward by offering services that extend beyond traditional ride-hailing and food delivery, aiming to elevate the region's global stature.
3. 🗺️ Grab Maps: Innovation in Mapping
- Grab Maps was launched in 2017 to address the limitations of third-party mapping applications, which often lacked detailed regional data and quickly became outdated.
- The primary goal of Grab Maps is to provide up-to-date, localized mapping solutions that cater specifically to the unique needs of Southeast Asian regions.
- By focusing on granular regional data, Grab Maps aims to enhance navigation accuracy and relevance for users in these areas.
- Grab Maps leverages local insights and data collection to continuously update and refine its mapping services, ensuring high accuracy and reliability.
- The initiative has significantly improved user experience by offering more precise and contextually relevant navigation options compared to generic mapping solutions.
4. 📸 Community-Based Mapping Approach
- Grab Maps intelligence services cater to internal needs across eight countries and offer enterprise-grade solutions for businesses throughout Asia.
- The mapping approach is based on community involvement, emphasizing precision through the use of street-level imagery captured with 360° cameras.
- By utilizing a large network of drivers, Grab is able to extract critical details such as turn restrictions, traffic signs, speed limits, places, and road accessibility from the collected images.
- This comprehensive data collection process enables the creation of reliable and highly detailed road topology maps.
5. 🤖 Leveraging AI for Mapping Challenges
- OpenAI released Vision fine-tuning capability for customizing Vision models with strong image understanding, enhancing AI's ability to tackle complex visual tasks.
- Early adoption of Vision fine-tuning API has shown promise in solving data matching problems, particularly in mapping applications.
- A specific example involves matching street imagery with traffic signs to the correct road, a task complicated by intricate geometries and visual occlusions.
- Utilized GPT-4 fine-tuning with proprietary data to effectively manage these complexities, demonstrating the potential of AI in improving mapping accuracy and efficiency.
6. 🔍 Experimentation and Fine-Tuning
- Initiated with a small fine-tuning dataset that combines street-level imagery and map tiles to enhance accuracy.
- Utilized consecutive map views and corresponding street-level imagery, labeled as frame one and frame two, to ensure consistency and precision in data alignment.
- Each map tile includes the vehicle's position marked by a red dot and a traffic sign marked by a small letter U, providing clear visual markers for data validation.
- The process involved iterative testing and adjustments to optimize the model's performance, addressing challenges such as data misalignment and marker visibility.
- Results indicated improved model accuracy and reliability in identifying traffic signs and vehicle positions, demonstrating the effectiveness of the fine-tuning approach.