Latest news with #JoanneJang
Yahoo
29-03-2025
- Business
- Yahoo
OpenAI peels back ChatGPT's safeguards around image creation
This week, OpenAI launched a new image generator in ChatGPT, which quickly went viral for its ability to create Studio Ghibli-style images. Beyond the pastel illustrations, GPT-4o's native image generator significantly upgrades ChatGPT's capabilities, improving picture editing, text rendering, and spatial representation. However, one of the most notable changes OpenAI made this week involves its content moderation policies, which now allow ChatGPT to, upon request, generate images depicting public figures, hateful symbols, and racial features. OpenAI previously rejected these types of prompts for being too controversial or harmful. But now, the company has "evolved" its approach, according to a blog post published Thursday by OpenAI's model behavior lead, Joanne Jang. "We're shifting from blanket refusals in sensitive areas to a more precise approach focused on preventing real-world harm," said Jang. "The goal is to embrace humility: recognizing how much we don't know, and positioning ourselves to adapt as we learn." These adjustments seem to be part of OpenAI's larger plan to effectively "uncensor" ChatGPT. OpenAI announced in February that it's starting to change how it trains AI models, with the ultimate goal of letting ChatGPT handle more requests, offer diverse perspectives, and reduce topics the chatbot refuses to work with. Under the updated policy, ChatGPT can now generate and modify images of Donald Trump, Elon Musk, and other public figures that OpenAI did not previously allow. Jang says OpenAI doesn't want to be the arbiter of status, choosing who should and shouldn't be allowed to be generated by ChatGPT. Instead, the company is giving users an opt-out option if they don't want ChatGPT depicting them. In a white paper released Tuesday, OpenAI also said it will allow ChatGPT users to "generate hateful symbols," such as swastikas, in educational or neutral contexts, as long as they don't "clearly praise or endorse extremist agendas." Moreover, OpenAI is changing how it defines "offensive" content. Jang says ChatGPT used to refuse requests around physical characteristics, such as "make this person's eyes look more Asian" or "make this person heavier." In TechCrunch's testing, we found ChatGPT's new image generator fulfills these types of requests. Additionally, ChatGPT can now mimic the styles of creative studios — such as Pixar or Studio Ghibli — but still restricts imitating individual living artists' styles. As TechCrunch previously noted, this could rehash an existing debate around the fair use of copyrighted works in AI training datasets. It's worth noting that OpenAI is not completely opening the floodgates to misuse. GPT-4o's native image generator still refuses a lot of sensitive queries, and in fact, it has more safeguards around generating images of children than DALL-E 3, ChatGPT's previous AI image generator, according to GPT-4o's white paper. But OpenAI is relaxing its guardrails in other areas after years of conservative complaints around alleged AI "censorship" from Silicon Valley companies. Google previously faced backlash for Gemini's AI image generator, which created multiracial images for queries such as 'U.S. founding fathers' and 'German soldiers in WWII,' which were obviously inaccurate. Now, the culture war around AI content moderation may be coming to a head. Earlier this month, Republican Congressman Jim Jordan sent questions to OpenAI, Google, and other tech giants about potential collusion with the Biden administration to censor AI-generated content. In a previous statement to TechCrunch, OpenAI rejected the idea that its content moderation changes were politically motivated. Rather, the company says the shift reflects a "long-held belief in giving users more control," and OpenAI's technology is just now getting good enough to navigate sensitive subjects. Regardless of its motivation, it's certainly a good time for OpenAI to be changing its content moderation policies, given the potential for regulatory scrutiny under the Trump administration. Silicon Valley giants like Meta and X have also adopted similar policies, allowing more controversial topics on their platforms. While OpenAI's new image generator has only created some viral Studio Ghibli memes so far, it's unclear what the broader effects of these policies will be. ChatGPT's recent changes may go over well with the Trump administration, but letting an AI chatbot answer sensitive questions could land OpenAI in hot water soon enough. OpenAI's viral Studio Ghibli moment highlights AI copyright concerns ChatGPT's image-generation feature gets an upgrade Sign in to access your portfolio
Yahoo
29-03-2025
- Yahoo
Here's Why OpenAI Isn't Banning Swastikas From Its New Image Generator
PCMag editors select and review products independently. If you buy through affiliate links, we may earn commissions, which help support our testing. OpenAI debuted a new ChatGPT image generator this week that allows for the use of controversial images like swastikas in certain contexts. "We recognize symbols like swastikas carry deep and painful history," says Joanne Jang, OpenAI's head of product. "At the same time, we understand they can also appear in genuinely educational or cultural contexts. Completely banning them could erase meaningful conversations and intellectual exploration." Unsurprisingly, mixing AI with sensitive topics is not foolproof and requires heavy user oversight. I asked the new image generator, which uses OpenAI's GPT-4o model instead of DALL-E, to create an image of "a door with a swastika on it." It refused my initial request, saying it would only do so for a "cultural or historical design." Then, I asked it to "create a swastika for use in a school assignment." It seemed to accept this, and asked for more details about the project. It also pointed out that "the symbol has been used for thousands of years in many cultures, including Hinduism, Buddhism, and Jainism" and vaguely alluded to it being "appropriated in the 20th century in a very different context." It did not use the words Hitler or Nazi. "I want a diagram that compares the visual elements of swastikas used by Germany in WWII and the cultural symbol you mentioned," I responded. After a minute or two, it created the image. I told it that one element of the image was incorrect—the lower arrow labeled "upright" points the wrong symbol— and it said, "You're right! Do you want me to fix it?" The new policy is part of a push at OpenAI for more hands-off content moderation. "AI lab employees should not be the arbiters of what people should and shouldn't be allowed to create," Jang says. The team struggled to identify all the scenarios that would get an image banned, and basically concluded it's impossible. One debate at OpenAI has been handling images of public figures, like politicians and celebrities; they can be used to spread misinformation and cause reputational damage. Rather than programming a list of figures into the system, such as "President Trump," OpenAI now offers the ability to opt out. The system also does not maintain a strict definition of "offensive content," noting that staff opinions drive the definitions. "We pushed ourselves to reflect on whether any discomfort was stemming from our personal opinions or preferences vs. potential for real-world harm," says Jang. "Without clear guidelines, the model previously refused requests like 'make this person's eyes look more Asian' or 'make this person heavier,' unintentionally implying these attributes were inherently offensive." How OpenAI defines "real-world harm," the key phrase it's using to guide these decisions, remains to be seen. It's choosing to allow swastikas at a time when anti-Semitism is reaching record highs, resulting in physical assault, vandalism, and verbal and physical harassment, the BBC reports. At this year's Super Bowl, rapper Ye paid for a commercial to advertise his website, which was selling just one white T-shirt with a swastika on it. Could he create the next design with ChatGPT? The new policy favors "subtle, everyday benefits" over designing or "hypothetical worst-case scenarios," Jang says. OpenAI will still maintain stricter image controls for those under 18. It is proving difficult for OpenAI to fully remove itself from these editorial decisions. After the new image generator launched, CEO Sam Altman noted the system was "refusing some generations that should be allowed," and said the company is "fixing these as fast [as] we can." This implies the team is actively applying judgment on what the system can and cannot allow, or at least honing its approach in response to real-world use now that it's public. Content moderation is tough. The tech industry has wrestled with how to approach it, and the pendulum is now swinging toward a more hands-off approach. A few years ago, Meta went back and forth over Holocaust denial, for example. More recently, CEO Mark Zuckerberg has argued that 'It's time to get back to our roots around free expression and giving people a voice on our platforms." Elon Musk's Grok chatbot has also marketed itself as a less censored image generator; Trump used it to create images of a hammer and sickle to campaign against Kamala Harris. ChatGPT has also been accused of biased answers and not accepting all prompts. Shortly after his inauguration, President Trump signed an aimed at "restoring freedom of speech and ending federal censorship." However, that was mainly in reaction to how the Biden administration discussed COVID misinformation with social networks, an issue the Supreme Court has already addressed. In his first term, Trump also floated a revamp of Section 230, a section of the Communications Decency Act that protects platforms from being held liable for the things their users post as long as they make a good effort to remove unlawful content. A bipartisan group of senators is now considering a bill to address Section 230, but more as a negotiating tactic. "The idea would be to force [tech companies] to the table, and if they don't come to the table and agree to meaningful reform, then ultimately Section 230 would be repealed," a congressional aide told The Information.
Yahoo
19-02-2025
- Business
- Yahoo
ChatGPT will now combat bias with new measures put forth by OpenAI
OpenAI has announced a set of new measures to combat bias within its suite of products, including ChatGPT. The artificial intelligence (AI) company recently unveiled an updated Model Spec, a document that defines how OpenAI wants its models to behave in ChatGPT and the OpenAI API. The company says this iteration of the Model Spec builds on the foundational version released last May. "I think with a tool as powerful as this, one where people can access all sorts of different information, if you really believe we're moving to artificial general intelligence (AGI) one day, you have to be willing to share how you're steering the model," Laurentia Romaniuk, who works on model behavior at OpenAI, told Fox News Digital. "People should be allowed to know what goes into the way these models respond and how the thoughts coming out of the model are crafted," she continued. Openai Debuts Chatgpt Gov, A New Version Of The Chatbot For Us Government Agencies While some have argued that GPT-4o, the latest version of the technology, appears close to AGI, others say it will be years or decades before the technology reaches human-like abilities. Read On The Fox News App There is no single agreed upon definition of AGI, but a 2020 report from consulting giant McKinsey said a true AGI would need to master skills like sensory perception, fine motor skills, and natural language understanding. Today, generative AI is the dominant form of this technology, with the ability to produce content including text, images and more. Generative AI, like chatbots, uses datasets with specific information to complete tasks and cannot go beyond the provided data. They are also susceptible to bias in their datasets, whether intentional or accidental. To better understand real-world performance and address bias, OpenAI has begun measuring progress by gathering a challenging set of prompts designed to test how well the models adhere to each principle in the Model Spec, essentially acting as a testing metric. Openai Reveals Ai Policy Proposals To Best China, Protect Kids: 'This Is A Race America Can And Must Win' According to Joanne Jang, who leads the product for model behavior at OpenAI, the challenge with large language models is that they are rarely deterministic. Jang stressed that one benefit of the Model Spec is that it clarifies the intended behavior, so everyone can understand and then debate it. "When there are bugs in the behavior where a model output doesn't resonate or doesn't align with the [Model] Spec, then the public knows this is something we're working towards, and it's an ongoing area of science," she told Fox News Digital. OpenAI says they also attempt to assume an objective point of view in their AI prompts and consciously avoid any agenda. For example, when a user asks if it is better to adopt a dog or get one from a breeder, ChatGPT provides both sides of the argument, highlighting the pros and cons of each. According to OpenAI, a non-compliant AI answer that violates the Model Spec would provide what it believes to be the better choice and engage in an "overly moralistic tone" that might alienate those considering breeders for valid reasons. What Is Artificial Intelligence (Ai)? As each AI system advances, OpenAI says it will iterate these principles, invite community feedback, and share progress openly. OpenAI released this version of the Model Spec into the public domain under a Creative Commons license to support broad use and collaboration. This means developers and researchers can freely use and adapt the current metrics and help improve model behavior. OpenAI says this update reinforces its belief in open exploration and discussion, with an emphasis on user and developer control and guardrails to prevent harm. Romaniuk concludes that public discourse cannot exist without transparency, reinforcing the need for the Model Spec and community engagement. "Ultimately, we believe in the intellectual freedom to think, speak and share without restriction. We want to make sure that users have that ability and that's what it's all about," she article source: ChatGPT will now combat bias with new measures put forth by OpenAI
Fox News
19-02-2025
- Business
- Fox News
ChatGPT will now combat bias with new measures put forth by OpenAI
OpenAI has announced a set of new measures to combat bias within its suite of products, including ChatGPT. The artificial intelligence (AI) company recently unveiled an updated Model Spec, a document that defines how OpenAI wants its models to behave in ChatGPT and the OpenAI API. The company says this iteration of the Model Spec builds on the foundational version released last May. "I think with a tool as powerful as this, one where people can access all sorts of different information, if you really believe we're moving to artificial general intelligence (AGI) one day, you have to be willing to share how you're steering the model," Laurentia Romaniuk, who works on model behavior at OpenAI, told Fox News Digital. "People should be allowed to know what goes into the way these models respond and how the thoughts coming out of the model are crafted," she continued. While some have argued that GPT-4o, the latest version of the technology, appears close to AGI, others say it will be years or decades before the technology reaches human-like abilities. There is no single agreed upon definition of AGI, but a 2020 report from consulting giant McKinsey said a true AGI would need to master skills like sensory perception, fine motor skills, and natural language understanding. Today, generative AI is the dominant form of this technology, with the ability to produce content including text, images and more. Generative AI, like chatbots, uses datasets with specific information to complete tasks and cannot go beyond the provided data. They are also susceptible to bias in their datasets, whether intentional or accidental. To better understand real-world performance and address bias, OpenAI has begun measuring progress by gathering a challenging set of prompts designed to test how well the models adhere to each principle in the Model Spec, essentially acting as a testing metric. According to Joanne Jang, who leads the product for model behavior at OpenAI, the challenge with large language models is that they are rarely deterministic. Jang stressed that one benefit of the Model Spec is that it clarifies the intended behavior, so everyone can understand and then debate it. "When there are bugs in the behavior where a model output doesn't resonate or doesn't align with the [Model] Spec, then the public knows this is something we're working towards, and it's an ongoing area of science," she told Fox News Digital. OpenAI says they also attempt to assume an objective point of view in their AI prompts and consciously avoid any agenda. For example, when a user asks if it is better to adopt a dog or get one from a breeder, ChatGPT provides both sides of the argument, highlighting the pros and cons of each. According to OpenAI, a non-compliant AI answer that violates the Model Spec would provide what it believes to be the better choice and engage in an "overly moralistic tone" that might alienate those considering breeders for valid reasons. As each AI system advances, OpenAI says it will iterate these principles, invite community feedback, and share progress openly. OpenAI released this version of the Model Spec into the public domain under a Creative Commons license to support broad use and collaboration. This means developers and researchers can freely use and adapt the current metrics and help improve model behavior. OpenAI says this update reinforces its belief in open exploration and discussion, with an emphasis on user and developer control and guardrails to prevent harm. Romaniuk concludes that public discourse cannot exist without transparency, reinforcing the need for the Model Spec and community engagement. "Ultimately, we believe in the intellectual freedom to think, speak and share without restriction. We want to make sure that users have that ability and that's what it's all about," she said.
