OpenAI recently announced an update to ChatGPT (Apple, Android) that includes two new features: AI voice options for hearing the chatbot answer to your prompts and image analysis capabilities. The image function is comparable to what Google’s Bard chatbot already provides for free.
Even after hours of testing the limits and powers of ChatGPT, OpenAI’s chatbot continues to surprise and frighten me. Yes, I was impressed with the web browsing beta offered by ChatGPT Plus, but I was concerned about the tool’s implications for people who write for money online, among other things. I had similar mixed sentiments about the new image feature that was made available to OpenAI subscribers.
While I have yet to test the new audio capabilities (other outstanding reporters on staff have), I was able to test the soon-to-arrive image features. Here’s how to utilize the new picture search feature coming to ChatGPT, as well as some pointers to get you started.
How to Access ChatGPT’s Image Features
While the upgrade is due before the end of the year, it is unknown when the image and speech features will be available to the public. As with most OpenAI updates, such as the GPT-4 version of ChatGPT, paying subscribers are given priority access.
In the ChatGPT smartphone app, there are three options for uploading photographs. To begin, select the camera icon to the left of the message bar and take a new photo with your smartphone. You can use your finger to create a circle around whatever you want the chatbot to focus on before uploading the image.
You can also upload images from your device as well as anything stored to your phone. ChatGPT desktop browser users can simply upload stored photographs from their computer. Although you cannot yet post films to the chatbot, you can submit several photographs in a single inquiry.
Suggestions for Experimenting With New AI Tools
This isn’t the first time a type of “computer vision” has been made available to the public, but the user interface, combined with a sophisticated chatbot, gives the idea that something unique, and perhaps transformative, is going on here. Before we go any further, keep in mind that you should never upload personal or sensitive photographs to ChatGPT when experimenting with the image feature.
Do you want to limit the amount of time OpenAI saves your data and AI interactions in order to develop its chatbot? Turn off Chat History & Training in Settings, then Data Controls. With this turned off, your data is destroyed after one month. This procedure must be repeated for each browser used to access ChatGPT, both PC and mobile.
ChatGPT produced the greatest results for me when I uploaded photographs that were clear and well-lit. The chatbot was able to classify many of the objects in my apartment, including an orchid plant and international currency, as well as a wayward charging cord and a Steve Irwin Funko Pop.
Despite the tool’s ability to search through information, do not immediately trust its results. ChatGPT mislabeled my daily multivitamin as an erectile dysfunction treatment medication.
ChatGPT clearly has limitations. When shown a random photo of a mural, it couldn’t identify the artist or location; however, ChatGPT was able to pinpoint the location of photographs of several San Francisco landmarks, including Dolores Park and the Salesforce Tower. Although it may still feel a little gimmicky, anyone on a trip to a new city or country (or even just a different neighborhood) could enjoy experimenting with the visual side of ChatGPT.
One of the key constraints imposed by OpenAI on this new feature is a restriction on the chatbot’s capacity to answer inquiries that identify humans. “I’m set up to prioritize user privacy and safety.” In order to maintain these priorities, identifying genuine persons based on photographs, even if they are renowned, is limited,” ChatGPT explained. While the chatbot did not refuse to answer every question when given pornography, it did hesitate to make any precise descriptions of the adult performers other than explaining their tattoos.
It’s worth mentioning that one chat I had with an early version of ChatGPT’s image functionality looked to dance past some of OpenAI’s guardrails. Initially, the chatbot refused to recognize a Bill Hader meme. ChatGPT then deduced that a photo of Brendan Fraser in George of the Jungle was actually a shot of Brian Krause in Charmed. When the chatbot was questioned if it was certain, it switched to the proper response.
ChatGPT went crazy trying to explain a visual from RuPaul’s Drag Race during the same session. I posted a screenshot of Kylie Sonique Love, one of the drag queen candidates, and ChatGPT correctly identified her as Brooke Lynn Hytes, another participant. When I questioned the chatbot’s response, it went on to guess Laganja Estranja, India Ferrah, Blair St. Clair, and Alexis Mateo.
“I apologize for the oversight and incorrect identifications,” ChatGPT responded when I pointed out the recurrence of its faulty answers. ChatGPT refused to identify Jared Kushner as I extended the conversation and submitted a photo of him.
If the guardrails are removed, either through a jailbroken ChatGPT or a future open source model, the privacy consequences might be rather troubling. What if every photograph of you taken and posted online could be easily linked to your identity with only a few clicks? What if someone took a snapshot of you in public without your permission and instantly found your LinkedIn profile? Women and other minorities are expected to face an increase in abuse from people utilizing chatbots for stalking and harassment if sufficient privacy safeguards are not maintained for these new picture features.
