Run Local LLMs on an AMD Ryzen AI Max 395 Mini PC

An AMD Ryzen AI Max+ 395 mini PC can run language models locally, keeping prompts and responses on your machine and avoiding per-request cloud use. This guide covers a practical setup with Windows and a local model runner, plus what to check if you use Linux. You’ll need a model file or an internet connection to download one, and enough memory and disk space for the model you choose.

What You’ll Need

A mini PC with the Ryzen AI Max+ 395, updated firmware, and its supplied power adapter.
Windows 11 or a supported Linux distribution, with current system updates and AMD graphics drivers where applicable.
At least several gigabytes of free disk space; larger models can require tens of gigabytes. Leave additional space for downloads and caches.
Internet access for downloading software and models. Once downloaded, many local runners can work offline.
Optional: a chat interface or coding tool that can connect to a local model server.

Step-by-Step

Check the exact hardware and memory. Open Windows Settings > System > About, or check your system information in Linux. Confirm the processor is Ryzen AI Max+ 395 and note installed RAM. Integrated graphics use system memory, so the amount available to a model is less than the installed total: the operating system, display, and other applications need memory too. Close demanding applications before testing a large model.
Update the operating system and drivers. Install pending system updates and restart. On Windows, use the computer maker’s support page for firmware and platform drivers, and AMD’s official support resources for compatible graphics drivers. On Linux, follow the distribution’s documented AMD graphics and ROCm support instructions. Don’t assume that a driver or AI runtime for a different Ryzen AI processor will support this system.
Choose a runner that supports your operating system and hardware. For a straightforward first test, install a maintained local model runner that offers a Windows or Linux build and supports CPU inference. If its current release explicitly supports AMD GPU acceleration on your operating system, you can try that path later. Download installers only from the project’s official site or repository, and check its current hardware and operating-system requirements before installing.
Install and verify the runner. Follow the installer prompts, then open the application or its documented command-line interface. If the runner provides a built-in model catalogue, browse it there. If it provides a local service, check its documentation for the launch command and local address; don’t guess ports or command flags. Confirm the runner starts successfully before downloading a large model.
Select a suitable model. Start with a small instruct or chat model and a quantized version if available. Quantization reduces memory use, often with some quality trade-off. Check the model’s listed file size and memory guidance, and leave room for the context window, runner, and operating system. A model advertised as fitting in memory may still fail when you set a long context or run other applications.
Download the model through the runner. Use its model browser or documented download method. If you have a compatible model file already, follow the runner’s instructions for importing it and check the model’s license and format compatibility. Wait for the download to finish, then verify that the model appears as available. Avoid interrupting a download; partial files may need to be removed or downloaded again.
Run a short test conversation. Load the model, use a modest context length, and ask a simple question. The first load can take longer as files are read and memory is allocated. Watch memory use in Task Manager on Windows or a system monitor on Linux. If generation works, test a longer prompt gradually rather than immediately choosing the maximum context.
Connect an optional interface. Many runners provide a local API or a built-in chat window. If connecting another app, use the runner’s documented local endpoint and API settings. Keep the service bound to the local machine unless you understand how to secure network access; a local model does not automatically mean a network-exposed service is private.
Try acceleration only after the basic test works. Check the runner’s current documentation for the supported AMD backend, driver, runtime, and installation steps for your exact OS. Change one setting at a time, restart the runner, and compare speed and stability with CPU inference. The NPU is not automatically used by every text-generation program; use it only when a particular runner and model explicitly support it.

Common Mistakes

Choosing by parameter count alone: model size, quantization, context length, and runtime overhead all affect memory use and speed.
Assuming all memory is available to the model: integrated graphics and the operating system share system RAM. Leave headroom instead of targeting the full installed capacity.
Mixing model formats and runners: a downloaded file is not necessarily compatible. Follow the runner’s supported formats and import steps.
Installing an acceleration stack from an unrelated guide: AMD GPU and NPU support varies by OS, driver, and application. Verify exact compatibility before changing runtimes.
Expecting cloud-model performance or answers: local speed depends on the model, settings, cooling, and software support; smaller models may be less capable.

Tips

Keep a small known-good model installed as a baseline for checking updates or configuration changes.
Start with a short context and increase it only when needed. Longer context consumes more memory.
Use the system’s performance mode and provide clear airflow, but avoid blocking vents or placing the mini PC on soft material.
Record the runner version, model name, quantization, context setting, and driver version when comparing results.
For repeatable offline use, download the model first and test it with the network disconnected. Some applications still need internet for updates or optional features.

Troubleshooting

The runner does not start: Restart after updates, check its operating-system requirements, and install only the drivers it documents. Review the application’s error message or logs. If it recently stopped working after a driver change, revert to a known-supported driver rather than adding several runtimes at once.

The model fails to load or the PC becomes unresponsive: Choose a smaller or more heavily quantized model, reduce context length, close other applications, and check free memory and disk space. Restart before retrying if memory was exhausted.

Generation is unusually slow: First confirm that the model is actually generating and that the PC is not busy with a download or update. Check the runner’s logs to see whether it is using CPU or a supported AMD backend. Compare with a smaller model; do not assume that enabling GPU or NPU options helps unless the backend is supported.

A model download is missing or corrupt: Check the download completed, verify available disk space, and use the runner’s own download or verification feature if provided. Remove only the incomplete model file or cache identified by the application, then download again.

Another app cannot connect: Confirm the local runner is running and consult its documentation for the correct endpoint and API format. Check that both programs are on the same machine and that local firewall settings are not blocking the connection. Do not expose the service to the internet as a troubleshooting shortcut.

Frequently Asked Questions

Can the Ryzen AI Max+ 395 run LLMs without internet?
Yes. After you download a compatible model and runner, basic inference can be offline. Updates and optional online features may still require internet.

How much RAM do I need?
It depends on model size, quantization, context length, and other running software. Check the model’s memory guidance and leave space for the operating system and shared graphics memory.

Does every local LLM use the NPU?
No. A runner must explicitly support the NPU and the selected model. Many general-purpose local text-generation setups use the CPU or GPU instead.

Should I use CPU or AMD GPU acceleration?
Start with the runner’s simplest supported configuration. Try GPU acceleration only when its documentation lists support for your operating system, driver, and hardware; compare stability and speed on your own workload.

Can I use a model from another source?
Often, if the runner supports its file format and architecture. Check compatibility and the model’s license before downloading or using it.

Will local models keep my data private?
Inference can stay on the PC, but privacy also depends on the runner, connected interface, extensions, and network settings. Check those components and avoid exposing a local service unintentionally.

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →