OpenAI - Dev Day Holiday Edition—12 Days of OpenAI: Day 9
OpenAI has introduced several new features and models for developers and startups using their API. These include the launch of the 01 model out of preview, which has been used for building agentic applications, customer support, and financial analysis. Key features launched include function calling, structured outputs, developer messages, and reasoning effort parameters. Function calling allows models to interact with backend APIs, while structured outputs ensure models adhere to specific formats. The reasoning effort parameter optimizes the model's thinking time, saving resources on simpler tasks. Additionally, vision inputs have been introduced to aid in fields like manufacturing and science.
OpenAI also announced the introduction of preference fine-tuning, a method that optimizes models based on user preferences, enhancing performance in areas like customer support and content moderation. This method uses direct preference optimization, allowing developers to guide models towards preferred behaviors. The company has also improved the real-time API with WebRTC support, reducing latency and simplifying integration. New SDKs for Go and Java have been released, and the cost of GPT-4 audio tokens has been reduced. These updates aim to enhance the developer experience and expand the capabilities of applications built on OpenAI's platform.
Key Points:
- OpenAI's 01 model is now available with features like function calling and structured outputs, enhancing API capabilities.
- Preference fine-tuning allows developers to optimize models based on user preferences, improving performance in specific use cases.
- The real-time API now supports WebRTC, reducing latency and simplifying integration for voice applications.
- New SDKs for Go and Java have been released, expanding language support for developers.
- GPT-4 audio tokens are now 60% cheaper, making it more cost-effective for developers to use audio features.
Details:
1. 🎉 Introduction to Developer Day 🎉
- The event is part of the 12 Days series, a structured, multi-day event aimed at engaging developers.
- Olivia Gar is introduced as the leader, serving as a key point of contact and authority for the event.
- The Developer Day is the ninth day in the series, indicating a progression and build-up of activities.
- The series is designed to provide developers with insights, tools, and networking opportunities.
2. 🚀 Focus on Developers and Startups 🚀
- OpenAI's platform product is highly regarded, especially for developers and startups, due to its robust capabilities and potential for innovation.
- The platform enables developers and startups to build on top of OpenAI's technology, offering tools and resources that facilitate development and innovation.
- Specific features such as API access, integration capabilities, and support for various programming languages make it an attractive option for tech development.
- Case studies highlight successful implementations by startups, showcasing increased efficiency and product development speed.
- The sentiment expressed is one of strong bias towards the platform's capabilities and potential for innovation, with a focus on empowering developers and startups.
3. 🌍 API Growth and New Features 🌍
- The API has been available for four years, showing significant growth with 2 million developers using it from over 200 countries.
- New features are being introduced as a thank you to the developers, enhancing the API's functionality and usability.
- The API's impact is evident in its global reach and the diverse applications it supports, contributing to its success and continued development.
4. 🔧 Announcing New API Models and Features 🔧
4.1. Introduction
4.2. Team Members
5. 🛠️ Launching Function Calling and Developer Messages 🛠️
- OpenAI 01 is moving out of preview in the API, enabling developers to build applications in areas such as customer support and financial analysis.
- Developers have been creating agentic applications using the API since its preview in September, indicating strong interest and potential for diverse applications.
- Feedback from developers highlighted missing core features, which are now being addressed with the launch of new functionalities in the API.
- The new features aim to enhance the capabilities of developers in creating more robust and versatile applications, addressing previous limitations.
6. 🧠 Introducing Reasoning Effort and Vision Inputs 🧠
- Developer messages are a new type of system message designed to enhance instruction hierarchy by allowing developers to specify which instructions to follow and in what order.
- These messages improve the model's ability to execute tasks as intended by developers, providing a structured approach to task management.
- By introducing developer messages, the model can better align with developer goals, ensuring more accurate and efficient task execution.
7. 🔍 Vision and Error Detection Demo 🔍
- The introduction of 'reasoning effort' as a new parameter optimizes the model's problem-solving time, leading to cost and time savings on simpler problems while dedicating more resources to complex issues.
- Vision inputs are being introduced to enhance capabilities in fields like manufacturing and science, driven by user demand for more advanced features.
- A live demo showcased the new capabilities, particularly focusing on error detection in text forms using vision inputs, demonstrating practical applications and benefits.
8. 🧮 Tax Calculation and Function Calling Demo 🧮
- The model can detect errors in forms, such as arithmetic mistakes, but is not a substitute for professional judgment.
- An error was identified on line 11 where addition was used instead of subtraction for calculating adjusted gross income (AGI).
- The wrong standard deduction was used, which depends on filing status and the number of checked boxes on the form.
- The model successfully identified both the arithmetic error and the incorrect standard deduction amount.
- The model uses algorithms to cross-verify calculations and ensure deductions align with filing status, enhancing accuracy in tax preparation.
9. 📊 Structured Outputs and Model Evaluations 📊
9.1. Tax Calculation and Function Calling
9.2. Structured Outputs and JSON Schema
9.3. Model Evaluations and Performance
10. 🎤 Real-time API Enhancements and WebRTC 🎤
- The new API uses 60% fewer thinking tokens than the previous version, making it faster and cheaper for applications.
- WebRTC support is introduced, providing benefits like low latency, echo cancellation, and dynamic bit rate adjustment, which are essential for internet-based applications.
- The integration of WebRTC simplifies the code significantly, reducing it from 200-250 lines to just 12 lines, eliminating the need for handling back pressure and other complexities.
- A demo application was shown where a simple script was executed to demonstrate the ease of use and effectiveness of the new API features.
- The code for the demo will be made available for download, requiring only an API token change to run.
11. 🔧 Fine-Tuning and Customization Options 🔧
- The microcontroller used in the demonstration is extremely small, about the size of a penny, and can be integrated into various devices like wearables, cameras, and microphones.
- The setup process for the microcontroller is straightforward, requiring only a token and Wi-Fi details, with no soldering or hardware modifications needed. Users can start building applications in 30 to 45 minutes.
- The cost of GPT-40 audio tokens has been reduced by 60%, and 4 mini audio tokens are now 10 times cheaper than before.
- A new Python SDK for the API has been introduced to simplify integration, along with API changes to enhance function coding and guard rails.
- A new method called preference fine tuning is available, using direct preference optimization to align models with user preferences, improving performance based on user feedback.
- Preference fine tuning differs from supervised fine tuning by using pairs of responses to optimize model behavior, focusing on qualities like response formatting and creativity.
- Typical use cases for preference fine tuning include customer support, copywriting, creative writing, and content moderation, allowing models to be more concise and relevant.
- The fine-tuning process is user-friendly, involving uploading training data in a specific format and selecting hyperparameters, with the process taking from minutes to hours depending on data size.
- Early access to preference fine tuning has shown promising results, with Rogo AI improving accuracy from 75% to over 80% on their internal benchmark using this method.