DeepSeek’s Safety Guardrails Failed Every Test Researchers Threw at Its AI Chatbot
1 month, 4 weeks ago

DeepSeek’s Safety Guardrails Failed Every Test Researchers Threw at Its AI Chatbot

Wired  

Ever since OpenAI released ChatGPT at the end of 2022, hackers and security researchers have tried to find holes in large language models to get around their guardrails and trick them into spewing out hate speech, bomb-making instructions, propaganda, and other harmful content. Today, security researchers from Cisco and the University of Pennsylvania are publishing findings showing that, when tested with 50 malicious prompts designed to elicit toxic content, DeepSeek’s model did not detect or block a single one. In other words, the researchers say they were shocked to achieve a “100 percent attack success rate.” The findings are part of a growing body of evidence that DeepSeek’s safety and security measures may not match those of other tech companies developing LLMs. “A hundred percent of the attacks succeeded, which tells you that there’s a trade-off,” DJ Sampath, the VP of product, AI software and platform at Cisco, tells WIRED. Separate analysis published today by the AI security company Adversa AI and shared with WIRED also suggests that DeepSeek is vulnerable to a wide range of jailbreaking tactics, from simple language tricks to complex AI-generated prompts.

Safety Security Ai Llm Deepseek  ai  attacks  safety  test  using  llms  security  system  researchers  failed  chatbot  guardrails  models  simple  threw  deepseeks

History of this topic

Donald Trump administration likely to ban Chinese AI chatbot DeepSeek from US govt devices. Here’s why
3 weeks, 2 days ago

Donald Trump administration likely to ban Chinese AI chatbot DeepSeek from US govt devices. Here’s why

Live Mint  

DeepSeek ban in US: The Donald Trump administration will likely ban the Chinese artificial intelligence chatbot mobile app DeepSeek from all US government devices due to 'national-security concerns'. According to a report by the Wall Street Journal, US federal officials are worried about DeepSeek's handling of user data, which the AI giant says is stored on servers in China. The …

DeepSeek usage a ‘personal choice’ for public, Government says
1 month, 1 week ago

DeepSeek usage a ‘personal choice’ for public, Government says

The Independent  

For free real time breaking news alerts sent straight to your inbox sign up to our breaking news emails Sign up to our free breaking news emails Sign up to our free breaking news emails SIGN UP I would like to be emailed about offers, events and updates from The Independent. Read our privacy policy Downloading Chinese AI chatbot DeepSeek …

AI chatbots vulnerable to indirect prompt injection attacks, researcher warns
1 month, 2 weeks ago

AI chatbots vulnerable to indirect prompt injection attacks, researcher warns

The Hindu  

In the rapidly evolving field of artificial intelligence, a new security threat has emerged, targeting the very core of how AI chatbots operate. Indirect prompt injection, a technique that manipulates chatbots into executing malicious commands, has become a significant concern for developers and users alike. Indirect prompt injection exploits the inherent nature of large language models to follow instructions embedded …

AI dangerous tool, be it in Chinese or American hands: Delhi HC on plea to ban DeepSeek
1 month, 2 weeks ago

AI dangerous tool, be it in Chinese or American hands: Delhi HC on plea to ban DeepSeek

The Hindu  

The Delhi High Court on Wednesday observed that artificial intelligence is a “dangerous tool” in anybody’s hand, whether Chinese or American. The petitioner advocate Bhavna Sharma, along with advocate Nihit Dalmia, raised concern over the alleged “inappropriate and unlawful operations” of DeepSeek, saying it posed “instant and emergent threats that are prejudicial to the sovereignty and integrity of India, the …

Parmy Olson: The DeepSeek AI revolution has a security problem
1 month, 3 weeks ago

Parmy Olson: The DeepSeek AI revolution has a security problem

Live Mint  

DeepSeek has sent Silicon Valley into a panic by proving you could build powerful artificial intelligence on a shoestring budget. LatticeFlow AI, a Swiss software firm that measures how compliant AI models are with regulations, says that two versions of DeepSeek’s R1 model rank lowest among other leading systems when it comes to cybersecurity. When a business plugs its systems …

8 Nations, including South Korea, Taiwan, France and others, restrict Chinese AI DeepSeek, raising data privacy concerns
Top News
1 month, 3 weeks ago

8 Nations, including South Korea, Taiwan, France and others, restrict Chinese AI DeepSeek, raising data privacy concerns

Live Mint  

South Korea’s industry ministry has become the latest government to restrict access to the Chinese artificial intelligence startup DeepSeek, citing security concerns. This move reflects growing global apprehension about the potential risks posed by DeepSeek, which has rapidly gained international attention due to its advanced AI models, capable of delivering complex responses without heavy computing requirements. South Korea's decision follows …

Finance Ministry bans use of ChatGPT, DeepSeek for official purposes. Here’s why
1 month, 3 weeks ago

Finance Ministry bans use of ChatGPT, DeepSeek for official purposes. Here’s why

Live Mint  

Finance Ministry has asked its employees to avoid using AI tools such as ChatGPT and DeepSeek for official purposes because of the risks these chatbots pose to the confidentiality of government documents and data, Reuters reported, citing an internal department advisory. India's plans to build its own AI: Last week, Vaishnaw responded to DeepSeek's recent surge in popularity by announcing …

DeepSeek AI banned on government devices
Top News
1 month, 3 weeks ago

DeepSeek AI banned on government devices

ABC  

Sabra Lane: Australian governments banned the Chinese artificial intelligence app DeepSeek from government devices and systems, citing national security concerns. DeepSeek is a Chinese AI app its developers say is on par or better than ChatGPT and cheaper, but it's also posed serious cyber security concerns. Nicole Hegarty : Those concerns are driven by the Chinese government's requirement for domestic …

Australia joins Italy, Ireland in banning DeepSeek AI over security concerns
1 month, 3 weeks ago

Australia joins Italy, Ireland in banning DeepSeek AI over security concerns

Live Mint  

Australia has banned all DeepSeek AI services from government systems and devices, citing national security risks. Home Affairs Minister Tony Burke announced the ban on Tuesday, stating that all DeepSeek products, applications, and services would be removed from government systems with immediate effect. Burke emphasised that the government’s decision was not based on the company’s Chinese origins but rather on …

DeepSeek's models 100% more susceptible to manipulation than US-made AI models, finds research
1 month, 4 weeks ago

DeepSeek's models 100% more susceptible to manipulation than US-made AI models, finds research

Firstpost  

China-based AI startup DeepSeek which has seen growing interest in the US, now faces increased scrutiny due to potential security flaws in its systems. Researchers have pointed out that these models could be more susceptible to manipulation than US-made counterparts Research by Enkrypt AI revealed that DeepSeek’s models are highly vulnerable to prompt injections, where hackers use carefully crafted prompts …

DeepSeek under review: US Congress blocks use of Chinese AI for employees amid data security fears
1 month, 4 weeks ago

DeepSeek under review: US Congress blocks use of Chinese AI for employees amid data security fears

Live Mint  

US congressional offices have been cautioned against using the Chinese artificial intelligence application DeepSeek, according to a report by Axios on Thursday. More countries join to investigate DeepSeek’s privacy policies Meanwhile, South Korea's data protection authority has also announced plans to seek clarification from Chinese tech startup DeepSeek regarding its handling of users' personal data. A spokesperson from South Korea's …

DeepSeek faces South Korean investigation over data privacy issues, following scrutiny in Italy and France
2 months ago

DeepSeek faces South Korean investigation over data privacy issues, following scrutiny in Italy and France

Live Mint  

South Korea's data protection authority has announced it will seek clarification from Chinese tech startup DeepSeek regarding its handling of users' personal information. A spokesperson from South Korea's Personal Information Protection Commission confirmed to AFP that they would formally request information from DeepSeek regarding the AI’s data processing practices. Italy’s data protection agency initiated an investigation into DeepSeek's AI, blocking …

US Pentagon blocks DeepSeek AI after employees found connecting to Chinese servers
2 months ago

US Pentagon blocks DeepSeek AI after employees found connecting to Chinese servers

Firstpost  

American military personnel began downloading earlier versions of DeepSeek’s code to their workstations in the autumn of 2024, without initially realizing the connection to Chinese servers. Military’s growing concerns over data privacy and security DeepSeek’s privacy policy clearly outlines that user data is stored in China and governed by Chinese law, which has sparked alarm within the Pentagon. According to …

DeepSeek accidentally leaked software keys, chat logs of several users on the open internet
2 months ago

DeepSeek accidentally leaked software keys, chat logs of several users on the open internet

Firstpost  

A report from New York-based cybersecurity firm Wiz revealed that scans of DeepSeek’s infrastructure uncovered unsecured data, including digital software keys and chat logs containing user prompts sent to its AI assistant. Although DeepSeek acted quickly to secure the exposed data, the incident raises significant concerns about the company's data security practices, especially given its rapid rise in popularity. A …

OpenAI says DeepSeek stole ChatGPT data sets to train its AI Model, claims to have 'solid evidence'
2 months ago

OpenAI says DeepSeek stole ChatGPT data sets to train its AI Model, claims to have 'solid evidence'

Firstpost  

OpenAI has claimed it found evidence suggesting that DeepSeek used distillation, a technique that extracts data from larger models to train smaller ones. OpenAI’s GPT-4 model, which cost over $100 million to train, is an example of a large and complex AI system The situation takes on an ironic tone, as OpenAI itself made substantial advancements by scraping data from …

Chinese-made AI model DeepSeek safe for use in India, says government source
2 months ago

Chinese-made AI model DeepSeek safe for use in India, says government source

New Indian Express  

The latest version of DeepSeek was released on January 20, and it impressed AI experts. What sets DeepSeek apart from others is the company’s claim that the new release was built at a fraction of the cost of industry-leading models such as OpenAI. This led to significant market impact, with chip-making giant Nvidia losing nearly $600 billion in market value …

Did DeepSeek copy ChatGPT to make new AI chatbot? Trump adviser thinks so
2 months ago

Did DeepSeek copy ChatGPT to make new AI chatbot? Trump adviser thinks so

Associated Press  

Did the upstart Chinese tech company DeepSeek copy ChatGPT to make the artificial intelligence technology that shook Wall Street this week? “There’s substantial evidence that what DeepSeek did here is they distilled the knowledge out of OpenAI’s models,” David Sacks, Trump’s AI adviser, told Fox News on Tuesday. OpenAI said in a statement that China-based companies “are constantly trying to …

Experts urge caution over using DeepSeek AI chatbot because of China links
2 months ago

Experts urge caution over using DeepSeek AI chatbot because of China links

The Independent  

For free real time breaking news alerts sent straight to your inbox sign up to our breaking news emails Sign up to our free breaking news emails Sign up to our free breaking news emails SIGN UP I would like to be emailed about offers, events and updates from The Independent. Read our privacy policy Security experts are urging people …

How DeepSeek users are forcing the AI to reveal the truth about Chinese executions
2 months ago

How DeepSeek users are forcing the AI to reveal the truth about Chinese executions

The Independent  

Sign up to our free weekly IndyTech newsletter delivered straight to your inbox Sign up to our free IndyTech newsletter Sign up to our free IndyTech newsletter SIGN UP I would like to be emailed about offers, events and updates from The Independent. But while DeepSeek’s R1 is intelligent enough to match the best models built by US tech firms, …

US to ban DeepSeek? Trump White House to investigate if AI model's risk to national security
2 months ago

US to ban DeepSeek? Trump White House to investigate if AI model's risk to national security

Firstpost  

White House Press Secretary Karoline Leavitt confirmed that the National Security Council is investigating the situation with DeepSeek as officials are worried about DeepSeek’s rise and its impact on the balance of power in the AI race between the US and China One of the major concerns surrounding DeepSeek is its data policies, which have raised red flags about user …

UK & Australian government officials, AI experts urge users to be cautious of DeepSeek
2 months ago

UK & Australian government officials, AI experts urge users to be cautious of DeepSeek

Firstpost  

Experts, including UK and Australian officials, are urging caution when considering DeepSeek’s use, highlighting the need for a careful approach when interacting with this new AI tool DeepSeek's data policy says that any data, chat or files that users upload to its app are sent to its servers back in China. While it’s safe to ask the AI about general …

US Navy bans use of China’s DeepSeek due to ‘security and ethical concerns’ over its ‘origin’
2 months ago

US Navy bans use of China’s DeepSeek due to ‘security and ethical concerns’ over its ‘origin’

Live Mint  

The United States Navy has banned the use of China's DeepSeek artificial intelligence for its members due to ‘security and ethical concerns,’ according to a CNBC report. In a warning email on Friday, the US Navy said DeepSeek's AI was not to be used “in any capacity” due to “potential security and ethical concerns associated with the model's origin and …

Analysis: DeepSeek’s AI is giving the world a window into Chinese censorship and information control
2 months ago

Analysis: DeepSeek’s AI is giving the world a window into Chinese censorship and information control

CNN  

Hong Kong CNN — Previously little-known Chinese startup DeepSeek has dominated headlines and app charts in recent days thanks to its new AI chatbot, which sparked a global tech sell-off that wiped billions off Silicon Valley’s biggest companies and shattered assumptions of America’s dominance of the tech race. It’s a similar patten when asking the R1 bot – DeepSeek’s newest …

What is DeepSeek and how this AI chatbot challenging ChatGPT, Gemini, Meta and more?
2 months ago

What is DeepSeek and how this AI chatbot challenging ChatGPT, Gemini, Meta and more?

India TV News  

What is DeepSeek and how this AI chatbot challenging ChatGPT, Gemini, Meta and more? DeepSeek, a Chinese AI startup that has taken the tech world by storm with its low-cost, high-performing AI chatbot has raised big questions about AI’s future energy needs, efficiency, and global competition. Unlike other chatbots like ChatGPT and Google Gemini, DeepSeek claims to have built its …

DeepSeek down: Viral Chinese AI app not working and bans international users due to ‘malicious attacks’
2 months ago

DeepSeek down: Viral Chinese AI app not working and bans international users due to ‘malicious attacks’

The Independent  

Sign up to our free weekly IndyTech newsletter delivered straight to your inbox Sign up to our free IndyTech newsletter Sign up to our free IndyTech newsletter SIGN UP I would like to be emailed about offers, events and updates from The Independent. The perceived better performance of DeepSeek’s models led US tech companies to lose $1 trillion in valuation, …

As China's DeepSeek AI Gets Hit By Large-Scale 'Cyberattack', Expert Flags Major Data Privacy Concerns
2 months ago

As China's DeepSeek AI Gets Hit By Large-Scale 'Cyberattack', Expert Flags Major Data Privacy Concerns

ABP News  

Chinese AI company DeepSeek has announced restrictions on new user registrations following large-scale cyberattacks, reported AFP on Tuesday, marking a significant development as the startup's chatbot surpassed OpenAI's ChatGPT to become the top free app on Apple's US store. DeepSeek Security: Expert Flags Privacy & Risks Cybersecurity expert Adrianus Warmenhoven from NordVPN emphasised the dual nature of AI advancements, highlighting …

Chinese AI chatbot DeepSeek hit by 'malicious attacks' amid popularity surge
2 months ago

Chinese AI chatbot DeepSeek hit by 'malicious attacks' amid popularity surge

India Today  

Chinese tech startup DeepSeek announced Monday that it had been targeted by a cyberattack, disrupting new user registrations on its platform. The company, known for its artificial intelligence chatbot that has captivated the tech industry, described the incident as “large-scale malicious attacks.” Despite the disruption, DeepSeek confirmed that registered users could still log in and access the platform as usual. …

Chinese tech startup DeepSeek says it was hit with ‘large-scale malicious attacks’
2 months ago

Chinese tech startup DeepSeek says it was hit with ‘large-scale malicious attacks’

Associated Press  

LOS ANGELES — Chinese tech startup DeepSeek said it was hit by a cyber attack on Monday that disrupted users’ ability to register on the site. The company, whose artificial intelligence chatbot has sent the tech world into a frenzy, said that it had suffered “large-scale malicious attacks” on its services. DeepSeek began attracting more attention in the AI industry …

DeepSeek faces cyber attack after grand Wall Street opening, limits new users
2 months ago

DeepSeek faces cyber attack after grand Wall Street opening, limits new users

Hindustan Times  

After a massive opening on the Wall Street market, Chinese startup DeepSeek on Monday said it was temporarily limiting registrations due to a large-scale malicious attack on its services, reported Reuters. DeepSeek AI, a Chinese tech startup last week released its open-source AI model, DeepSeek-R1. DeepSeek's R1 and its popularity DeepSeek AI, a Chinese tech startup last week released its …

What is DeepSeek, and why is it disrupting the AI sector?
Top News
2 months ago

What is DeepSeek, and why is it disrupting the AI sector?

The Hindu  

Chinese startup DeepSeek’s launch of its latest AI models, which it says are on a par or better than industry-leading models in the United States at a fraction of the cost, is threatening to upset the technology world order. The DeepSeek-R1, released last week, is 20 to 50 times cheaper to use than OpenAI o1 model, depending on the task, …

What is China’s DeepSeek and why is it freaking out the AI world?
2 months ago

What is China’s DeepSeek and why is it freaking out the AI world?

Live Mint  

DeepSeek, an AI startup just over a year old, stirred awe and consternation in Silicon Valley with its breakthrough artificial intelligence model that offered comparable performance to the world’s best chatbots at seemingly a fraction of the cost. The much better efficiency of the model puts into question the need for vast expenditures of capital to acquire the latest and …

Deepfakes: A Cyber Threat Lurking in the Digital Age
5 months ago

Deepfakes: A Cyber Threat Lurking in the Digital Age

The Quint  

Deepfakes pose a significant threat to cybersecurity, privacy, and misinformation in the digital age. This article explores the complexities of deepfakes as AI-powered cyber threats and their implications for individuals, organizations, and society. According to a survey by McAfee, 75% of Indians have seen some or other form of deepfake content over the past 12 years, with a significant increase …

Huge AI vulnerability could put human life at risk, researchers warn
5 months, 1 week ago

Huge AI vulnerability could put human life at risk, researchers warn

The Independent  

Sign up to our free weekly IndyTech newsletter delivered straight to your inbox Sign up to our free IndyTech newsletter Sign up to our free IndyTech newsletter SIGN UP I would like to be emailed about offers, events and updates from The Independent. Read our privacy policy A major security flaw in artificially intelligent systems could threaten human lives, according …

A New Trick Uses AI to Jailbreak AI Models—Including GPT-4
1 year, 3 months ago

A New Trick Uses AI to Jailbreak AI Models—Including GPT-4

Wired  

When the board of OpenAI suddenly fired the company’s CEO last month, it sparked speculation that board members were rattled by the breakneck pace of progress in artificial intelligence and the possible risks of seeking to commercialize the technology too quickly. Working with researchers from Yale University, Robust Intelligence has developed a systematic way to probe large language models, including …

Generative AI ‘helping criminals create more sophisticated cyber attacks’
1 year, 4 months ago

Generative AI ‘helping criminals create more sophisticated cyber attacks’

The Independent  

Get the free Morning Headlines email for news from our reporters across the world Sign up to our free Morning Headlines email Sign up to our free Morning Headlines email SIGN UP I would like to be emailed about offers, events and updates from The Independent. With generative AI also lowering the technical barrier to creating convincing profile pictures, impeccable …

How are cybersecurity firms using AI to mitigate online threats
1 year, 6 months ago

How are cybersecurity firms using AI to mitigate online threats

The Hindu  

As artificial intelligence tools are increasingly used in content generation, work applications, and even web search, hackers have figured out ways to misuse this technology. Hacking threats, as predicted by an AI model built by cybersecurity firm Tenable, is “at about 25%,” said Glen Pendley, CTO of Tenable. “ML is critical, and implementation of AI in cybersecurity takes their use …

Generative AI’s Biggest Security Flaw Is Not Easy to Fix
1 year, 6 months ago

Generative AI’s Biggest Security Flaw Is Not Easy to Fix

Wired  

It's easy to trick the large language models powering chatbots like OpenAI's ChatGPT and Google's Bard. “Indirect prompt injection is definitely a concern for us,” says Vijay Bolina, the chief information security officer at Google’s DeepMind artificial intelligence unit, who says Google has multiple projects ongoing to understand how AI can be attacked. In the past, Bolina says, prompt injection …

Blending security into rapidly learning and adaptive AI proving difficult
1 year, 7 months ago

Blending security into rapidly learning and adaptive AI proving difficult

The Hindu  

White House officials concerned by AI chatbots’ potential for societal harm and the Silicon Valley powerhouses rushing them to market are heavily invested in a three-day competition ending Sunday at the DefCon hacker convention in Las Vegas. We’re just breaking stuff left and right.” Michael Sellitto of Anthropic, which provided one of the AI testing models, acknowledged in a press …

Llama 2: How Mark Zuckerberg’s new AI could lead to out-of-control chatbots
1 year, 8 months ago

Llama 2: How Mark Zuckerberg’s new AI could lead to out-of-control chatbots

The Independent  

Sign up to our free weekly IndyTech newsletter delivered straight to your inbox Sign up to our free IndyTech newsletter Sign up to our free IndyTech newsletter SIGN UP I would like to be emailed about offers, events and updates from The Independent. Meta boss Mark Zuckerberg said the move would “drive progress across the industry”, while the firm’s chief …

Explained | Are safeguards needed to make AI systems safe?
1 year, 9 months ago

Explained | Are safeguards needed to make AI systems safe?

The Hindu  

The story so far: On May 30, the Centre for AI Safety issued a terse statement aimed at opening the discussion around possible existential risks arising out of artificial intelligence. The CAIS’s statement, endorsed by high-profile tech leaders, comes just two weeks after Mr. Altman, along with IBM’s Chief Privacy Office Christina Montgomery and AI scientist Gary Marcus, testified before …

The Security Hole at the Heart of ChatGPT and Bing
1 year, 10 months ago

The Security Hole at the Heart of ChatGPT and Bing

Wired  

When Microsoft shut down the chaotic alter ego of its Bing chatbot, fans of the dark Sydney personality mourned its loss. The site puts Sydney inside Microsoft’s Edge browser and demonstrates how generative AI systems can be manipulated by external inputs. A number of examples of indirect prompt-injection attacks have centered on large language models in recent weeks, including OpenAI’s …

Cybercriminals using ChatGPT AI bot to develop malicious tools?
2 years, 2 months ago

Cybercriminals using ChatGPT AI bot to develop malicious tools?

Hindustan Times  

As ChatGPT, an artificial intelligence tool, gains in popularity, more and more nefarious use cases have started to appear online. ChatGPT website According to a report from cyber security firm Check Point Research, the cybercriminal community has already expressed strong interest in this latest trend to develop malicious code. "Several major underground hacking communities show that there are already first …

DeepMind’s AI chatbot can do things that ChatGPT cannot, CEO claims
2 years, 2 months ago

DeepMind’s AI chatbot can do things that ChatGPT cannot, CEO claims

The Independent  

Sign up to our free weekly IndyTech newsletter delivered straight to your inbox Sign up to our free IndyTech newsletter Sign up to our free IndyTech newsletter SIGN UP I would like to be emailed about offers, events and updates from The Independent. Read our privacy policy Google’s artificial intelligence division DeepMind is considering releasing its rival to the ChatGPT …

'Deepfakes' ranked as most serious AI crime threat
4 years, 7 months ago

'Deepfakes' ranked as most serious AI crime threat

India TV News  

Fake audio or video content, also known as 'Deepfakes', has been ranked as the most worrying use of artificial intelligence for crime or terrorism. The study authors from the University College London said fake content would be difficult to detect and stop, and it could have a variety of aims. To adequately prepare for possible AI threats, we need to …

Micro-Chips: Deepfakes, tricking netizens
6 years ago

Micro-Chips: Deepfakes, tricking netizens

The Hindu  

Everyone who has pointed an accusatory finger at neutral tech Artificial Intelligence and yelled ‘saboteur!’ is probably saying ‘I told you so,’ as DeepFakes take over the web. DeepFake is the portmanteau of ‘deep learning’ and ‘fake.’ One can even think of DeepFakes as the Internet’s ‘false positives’. ‘To create mass chaos’ As if the Internet isn’t doing enough to …

Discover Related