OpenAI plans to add vision capabilities to the conversation mode of ChatGPT on iOS

· 2 min read
Vision mode on ChatGPT for iOS
Vision mode on ChatGPT for iOS

OpenAI is developing vision capabilities for ChatGPT's conversation mode on iOS. This new feature, which is still under development and not yet available to users, will allow individuals to take or upload photos. ChatGPT will analyze these images and incorporate the information into the ongoing conversation. The feature relies on the vision API from the recently released GPT-4 Turbo model, aiming to provide a more comprehensive conversational experience by incorporating visual context.

To utilize this feature, once available, users will likely need to:

  1. Switch to the conversational mode in ChatGPT on an iOS device.
  2. Opt to capture a new picture or select an existing one.
  3. Permit ChatGPT to process and examine the image.
  4. ChatGPT will then utilize the visual context to proceed with the dialogue.

By identifying objects, examining scenes, or responding to questions based on the provided image, users could communicate with ChatGPT in novel ways. Nonetheless, like any new technology, there are possible limitations and concerns. OpenAI has voiced potential issues, especially regarding the analysis of images featuring individuals, and has put in place measures to address these concerns.

ChatGPT Alpha
This article can guide you on how to get access to the new alpha and experimental ChatGPT features

About GPT-4 Turbo with Vision: GPT-4 Turbo with Vision is technology developed by OpenAI that can interpret both text and image inputs. This breakthrough facilitates JSON mode and function calling for vision inquiries, rendering the model more adaptable and convenient for users. It marks a forward leap in AI technology by OpenAI, extending ChatGPT's capacities to incorporate visual aspects along with textual dialogues.

Source: @testingcatalog