AMD’s Radeon RX 7900 XTX has set a high bar by outperforming the NVIDIA GeForce RTX 4090 in inference benchmarks, especially when running the DeepSeek R1 AI model.
Swift Support for DeepSeek’s R1 Models with Impressive Performance by AMD
DeepSeek’s latest AI model is creating quite a buzz across the tech industry. While many in the field speculate about the hefty computational power required for training such a model, AMD’s "RDNA 3" Radeon RX 7900 XTX GPU proves to be up to the task for the everyday user. AMD has shared benchmark comparisons, spotlighting their RX 7900 series GPU leading over NVIDIA’s offerings, across a variety of models.
Here’s what AMD’s David McAfee tweeted about the situation: DeepSeek is excelling on the Radeon 7900 XTX, and there’s guidance available on using Radeon GPUs and Ryzen AI APUs. The tweet includes a link to more resources.
For many, using consumer GPUs for AI tasks has become a practical solution, offering good performance at a relatively low cost compared to specialized AI hardware. Moreover, executing models locally means your data privacy stands intact—a crucial point with DeepSeek’s AI innovations. AMD has thoughtfully published a detailed guide on running DeepSeek R1 distillations on their GPUs, and here’s a quick rundown of that process:
- Ensure your PC is using the 25.1.1 Optional or newer Adrenalin driver.
- Head to lmstudio.ai/ryzenai to download LM Studio version 0.3.8 or later.
- Install LM Studio and bypass the onboarding screen.
- Navigate to the Discover tab.
- Select your preferred DeepSeek R1 Distill. For starters, the Qwen 1.5B is a great pick, delivering speed and decent capabilities. Larger distills provide better reasoning abilities.
- On the right, select “Q4 K M” quantization and click "Download."
- post-download, return to the chat tab, pick the DeepSeek R1 distill from the menu, and tick “manually select parameters.”
- Slide the GPU offload layers to the maximum setting.
- Boot up the model load.
- Enjoy interacting with a reasoning model powered by your local AMD setup!
If you hit a snag with these steps, fret not—AMD has released a YouTube tutorial that breaks down each stage clearly. Watching that will help ensure that you can run DeepSeek’s LLMs on your AMD hardware, keeping your data secure. As we look forward to future GPUs from NVIDIA and AMD, we can anticipate a substantial leap in inference capabilities, thanks to dedicated AI engines tailored for such demanding tasks.
Take advantage of these advancements and experiment with cutting-edge AI technology right from your desktop.
