A preview of the DALL-E image editor with inpainting on ChatGPT

The ChatGPT team is now working on a new image editor mode within the ChatGPT interface. This editor will come with an inpainting feature that allows you to select a specific area of an image to be regenerated with a new prompt.

This editor has a new interface which will become available on DALL-E GPT as well as on other custom GPTs with image generation capabilities.

Current status of the feature

Update from 01.04.24: The feature was released publicly.

The current status of this feature is "under development," which means it is not yet available to users. This feature may be announced by OpenAI at any time as it is almost ready. However, it is not clear if it will be available to all users from the start or only to a small subset initially.

It is also very likely that this editor will only be available to Plus users, similar to DALL-E GPT access overall.

Recently, a hidden blog post announcing this feature was found on OpenAI's website, indicating that this feature is very likely close to being released.

How to use inpainting for DALL-E on ChatGPT?

1. The editor UI appears only when you click on the generated image from the normal Chat UI. In this case, chat conversation will start appearing on the right side of the screen, and the majority of the screen will be dedicated to displaying the selected image.

2. On top, you will see a control bar with previously available options to download an image or see the prompt used to generate the selected image. Apart from this, a new inpainting functionality can be accessed from there as well.
3. At first, you will see a tooltip highlighting this new feature. Activating the inpainting UI will allow you to draw an area of the image to select which part you want to modify. After selection, the chat edit box will start displaying the "edit selection label," indicating that your next prompt will only be applied to the selected area.
4. A newly generated image after inpainting will appear on the main screen afterwards, and you will also be able to modify it further.

History of the inpainting feature

Previously, image inpainting was available as part of a standalone DALL-E 3 tool on OpenAI Labs. After that, DALL-E 3 got integrated into ChatGPT, and inpainting was only accessible via an API.

Shortly after Microsoft Bing was rebranded into Copilot, it also got a new inpainting feature on its Designer tool, where it was possible to apply a different style to the whole image or select an object on the image to regenerate it.

After that, OpenAI started developing its own image editing features on ChatGPT itself to enable the same functionality.

Firstly, they released DALL-E controls for DALL-E GPT, where you can populate your prompt with a selected style and aspect ratio. These controls are only available on DALL-E GPT and are not applied automatically as it happens on Copilot.

At the same time, OpenAI started developing the DALL-E Editor. For quite some time, it was only available as a layout change, while inpainting functionality appeared in the frontend code only recently.

Now, this feature is also available on mobile apps.

How does inpainting work?

Inpainting in the context of image editing, especially with DALL-E 3, involves the algorithm intelligently filling in a selected area of an image based on the surrounding content and a given prompt. This allows for seamless edits, additions, or removals within images, offering a high level of customization and creativity.

About DALL-E

DALL-E is an AI program developed by OpenAI that generates images from textual descriptions, known for its ability to create detailed and imaginative visuals from simple prompts. It represents a significant advancement in AI's ability to understand and interpret human language in the context of visual creativity.

About OpenAI

OpenAI is a leading AI research organization known for its groundbreaking work in the field of artificial intelligence. With a commitment to advancing digital intelligence in a way that benefits humanity, OpenAI has developed a range of influential AI technologies, including GPT (Generative Pre-trained Transformer) models and DALL-E.

About ChatGPT

ChatGPT is a variant of the GPT language model that's optimized for generating human-like text in a conversational format. Developed by OpenAI, ChatGPT has been integrated with various functionalities, including the ability to interact with images, making it a versatile tool for both text and image-based applications.

ChatGPT Alpha

This article can guide you on how to get access to the new alpha and experimental ChatGPT features

TestingCatalogAlexey Shabanov

About the Sources

The screenshots and insights shared in this preview stem from the meticulous work of Alexey Shabanov, who took to reverse-engineering the ChatGPT codebase to uncover these features. His findings align with similar reports shared on X by Tibor and also highlighted by TestingCatalog, further substantiating the anticipated developments within the ChatGPT interface.

About the Author

Alexey Shabanov is a renowned figure in the realm of technology, particularly known for his expertise in reverse engineering, beta testing, and early adoption of emerging technologies. With over eight years at the helm of TestingCatalog - a platform dedicated to unearthing and documenting new apps and features, especially within the AI sector—Alexey has tested and reported on thousands of applications. His deep involvement and insightful analysis make him a credible source for the latest developments in the AI field, particularly those that are still under wraps or in the development phase.