Random Image Display on Page Reload

How to Use ChatGPT’s New Image Features

Sep 30, 2023 7:00 AM

How to Use ChatGPT’s New Image Features

OpenAI’s new image analysis update for its chatbot is both impressive and frightening. Here’s how to use it, and some advice for your experiments.


Photograph: Maja Stankovic/Getty Images

OpenAI recently announced an upgrade to ChatGPT (Apple, Android) that adds two features: AI voice options to hear the chatbot responding to your prompts, and image analysis capabilities. The image function is similar to what’s already available for free with Google’s Bard chatbot.

Even after hours of testing the limits and capabilities of ChatGPT, OpenAI’s chatbot still manages to surprise and scare me at the same time. Yes, I was quite impressed with the web browsing beta offered through ChatGPT Plus, but I remained anxious about the tool’s ramifications for people who write for money online, among many other concerns. The new image feature arriving for OpenAI’s subscribers left me with similarly mixed feelings.

While I’ve not yet had the opportunity to experiment with the new audio capabilities (other great reporters on staff have), I was able to test the soon-to-arrive image features. Here’s how to use the new image search coming to ChatGPT and some tips to help you start out.

While the update is expected to arrive before the end of the year, it’s unclear exactly when the image and voice features will roll out to the public. As with most of OpenAI’s updates, like the GPT-4 version of ChatGPT, paying subscribers are first in line to receive access.

There are three ways to upload photos in the ChatGPT mobile app. First, you can select the camera option located to the left of the message bar and take a fresh photo with your smartphone. Before uploading the image, you can use your finger to draw a circle around whatever you’d like the chatbot to focus on.

screenshot of ChatGBT analyzing photo

The image analysis from ChatGPT made plenty of mistakes, but it sure can identify some random cords.

ChatGPT via Reece Rogers

You can also choose photos from your device to upload and select files that are saved to your phone. Desktop browser users of ChatGPT can just upload saved photos from their computer. Even though there’s no option yet to upload videos to the chatbot, you can submit multiple images in a single prompt.

Most Popular

Screenshot of chatgbt anayzing photo of plug

Another example of ChatGPT’s newest update being able to identify random gear in my apartment.

ChatGPT via Reece Rogers

This isn’t the first time a kind of “computer vision” has been made available to the public, but the accessible user interface paired with a powerful chatbot leaves the impression that there might be something distinct, and potentially transformative, happening here. Before we go any further, remember to never upload personal, sensitive photos to ChatGPT as you try out the image feature.

Want to limit how long OpenAI stores your data and AI interactions to train its chatbot? Go into Settings, then Data Controls, and turn off Chat History & Training. With this disabled, your information is deleted after one month. This process must be done individually for every browser that you use to access ChatGPT, for both PC and mobile.

I received the best results from ChatGPT when uploading images that were clear and well lit. It fumbled a few guesses, but the chatbot was able to label many of the objects in my apartment: from an orchid plant and international coins to a stray charging cable and Steve Irwin Funko Pop.

Despite the tool's power for searching through information, refrain from immediately trusting its answers. ChatGPT mislabeled my daily multivitamin as a pill for treating erectile dysfunction.

Screenshot of Chatgbt analyzing photo

ChatGPT, are you so sure about that?

ChatGPT via Reece Rogers

Most Popular

ChatGPT definitely has its limits. When given a random photo of a mural, it couldn’t identify the artist or location; however, ChatGPT easily clocked where images of multiple San Francisco landmarks were taken, like Dolores Park and the Salesforce Tower. Although it may still feel a bit gimmicky, anyone out on an adventure in a new city or country (or just a different neighborhood) might have fun playing around with the visual aspect of ChatGPT.

One of the major guardrails OpenAI put around this new feature is a limit on the chatbot’s ability to answer questions that identify humans. “I’m programmed to prioritize user privacy and safety. Identifying real people based on images, even if they are famous, is restricted in order to maintain these priorities,” ChatGPT told me. While it didn’t refuse to answer every question when shown pornography, the chatbot did hesitate to make any specific descriptions of the adult performers, beyond explaining their tattoos.

It’s worth noting that one conversation I had with the early version of ChatGPT’s image feature seemed to skirt around part of the guardrails put in place by OpenAI. At first, the chatbot refused to identify a meme of Bill Hader. Then ChatGPT guessed that an image of Brendan Fraser in George of the Jungle was actually a photo of Brian Krause in Charmed. When asked if it was certain, the chatbot switched over to the correct response.

In this same conversation, ChatGPT went wild trying to describe an image from RuPaul’s Drag Race. I shared a screenshot of Kylie Sonique Love, one of the drag queen contestants, and ChatGPT guessed that it was Brooke Lynn Hytes, a different contestant. I questioned the chatbot’s answer, and it proceeded to guess Laganja Estranja, then India Ferrah, then Blair St. Clair, then Alexis Mateo.

“I apologize for the oversight and incorrect identifications,” ChatGPT replied when I pointed out the repetitiveness of its wrong answers. As I continued the conversation and uploaded a photo of Jared Kushner, ChatGPT declined to identify him.

If the guardrails are removed, either through some kind of jailbroken ChatGPT or an open source model released in the future, the privacy implications could be quite unsettling. What if every picture taken of you and posted online was easily tied to your identity with just a few clicks? What if someone could snap a photo of you in public without consent and instantly find your LinkedIn profile? Without proper privacy protections remaining in place for these new image features, women and other minorities are likely to receive an influx of abuse from people using chatbots for stalking and harassment.

Get More From WIRED

Reece Rogers is WIRED's service writer, focused on explaining crucial topics and helping readers get the most out of their technology. Prior to WIRED, he covered streaming at Insider.
Service Writer

More from WIRED

How ChatGPT Can Help You Do More With PDFs

The AI chatbot can search, summarize, and create PDF documents with a few handy plug-ins.

David Nield

The Best Tea Accessories for Sipping Steamy Beverages

Brewing tea can be as simple as dropping a bag in a cup. But if you want to elevate your experience, try a few of our favorite handy gadgets.

Michael Calore

Our Favorite Online Gift Cards and Digital Presents

Forget the wrapping paper and shipping. Go digital this year with subscriptions, ebooks, apps, and more.

Simon Hill

The Best Multi-Tools for Any Task

Most tasks don't require a trip to the garage. With a multi-tool, you can keep a toolbox's worth of tools in your pocket, desk drawer, or glove compartment.

Matt Jancer

The Best Turntables for Your Vinyl Collection

Streaming music giving you the blues? These entry-level turntables will help you rock on to analog audio at home.

Parker Hall

Roblox Builds Out Its Metaverse Vision With Video Chat

This week, we discuss how adding animated chats to Roblox will advance the company’s plan to enrich its virtual world.

Michael Calore

Nintendo Reportedly Gave Private Switch 2 Demos to Developers

Plus: Google leaked its own images of the Pixel 8 and Pixel Watch, and you can now learn music in Duolingo.

Boone Ashworth

12 Great Deals on Upcycled Products, Headphones, and Smartwatches

Treat yourself while saving money on these recycled accessories—the planet will thank you. Or take advantage of eye-watering price cuts on Samsung smartwatches and earbuds.

Medea Giordano

Credit belongs to : www.wired.com

Check Also

Metal Prices Are Soaring. So Is Metal Theft

It’s a multi-billion-dollar global problem, and in a rapidly electrifying world, the profits—and ease—of stealing …