a16z - How to use DeepSeek safely
Deep Seek, a reasoning model from China, has garnered attention due to its open-source nature and the influence of the Chinese government on its development. The model is particularly susceptible to jailbreaks, making it less secure compared to other models like GPT. It performs about 20% worse than GPT in benchmarks, and its infrastructure is considered insecure. The model also has hard limits on politically sensitive topics, especially those related to China, which are heavily censored. This raises concerns about potential unknown manipulations or backdoors that could be present.
For enterprises considering using Deep Seek, the recommendation is to avoid using the China-hosted model due to security concerns and instead opt for US-hosted versions or wait for more stable open-source alternatives. The model's current state makes it unsuitable for end-user-facing applications due to its vulnerabilities. Enterprises are advised to wait for a more trusted source to produce a similar model that can be run locally, as Deep Seek is not considered a reliable daily driver due to its slow performance and occasional errors in output.
Key Points:
- Deep Seek is open-source and influenced by the Chinese government, raising security concerns.
- The model is highly susceptible to jailbreaks, performing 20% worse than GPT in security benchmarks.
- Deep Seek has hard limits on politically sensitive topics, especially those related to China.
- Enterprises should avoid using the China-hosted model and consider US-hosted versions or wait for alternatives.
- Deep Seek is not recommended for end-user applications due to its vulnerabilities and performance issues.
Details:
1. 🔍 Enterprise Caution with Deep Seek
1.1. Deployment Challenges
1.2. Security Concerns
1.3. Potential Solutions and Alternatives
2. 📰 Recent Buzz Around Deep Seek
2.1. Deep Seek and New Reasoning Models
2.2. Opportunities and Economic Implications
2.3. Risks and Security Concerns
3. 🌐 Open Source Models from China: Potential and Concerns
- DeepSeek is a sophisticated open source model developed in China, notable for its reasoning capabilities.
- The model's development is influenced by the Chinese government, affecting its openness and alignment with state policies.
- Researchers tested DeepSeek's response to adversarial techniques, such as prompt injections and jailbreaks, revealing sophisticated speech limitations.
- On politically sensitive topics like Taiwan or Tiananmen Square, DeepSeek often refuses to answer or aligns with the CCP Party Line, indicating a separate system from typical model guardrails.
4. 🔒 Deep Seek's Security Challenges and Censorship
4.1. Security Vulnerabilities in Deep Seek
4.2. Performance Issues Impacting Deep Seek
5. 🛡️ Comparing Security and Censorship Across Models
- Both China-hosted and open source models exhibit similar censorship levels, but the China-hosted version includes a client-side guard rail for additional control.
- Censorship persists even when models are hosted locally or through US providers, indicating the presence of intrinsic 'hard guard rails.'
- Operating models locally ensures data privacy, preventing inclusion in training datasets or transfer to China, thus addressing privacy concerns.
- A benchmark on Chinese politically sensitive topics revealed that around 85% were hard-censored, aligning responses with the Chinese Communist Party's stance.
6. 🚦 Censorship and Sensitivity in AI Models
- Deep Seek exhibits heavy-handed censorship, leading to concerns about potential control and influence, with an 85% censorship rate in specific tests.
- Western models like those trained in the U.S. filter sensitive topics, such as hate speech, with varying levels of maturity in their censorship controls.
- Anthropic's CLA has censorship levels on Chinese-related topics comparable to Deep Seek, highlighting similar control measures.
- GPT models exhibit less censorship, with a 40% freedom to respond compared to Deep Seek.
- Google's Gemini outperforms GPT in terms of less restrictive responses.
- Grok from xAI demonstrates the least censorship among the models, especially on sensitive Chinese political topics, showing more freedom in response.