Can a Thinvent PC run a local LLM?
Yes, a local large language model (LLM) runs on the CPU and system memory of a computer. Thinvent systems with 16GB or more of DDR4 RAM and a 12th or 14th generation Intel Core processor can run small quantized models on the CPU. Larger models need more memory than these systems provide.
The main limit on local LLMs is memory. In general, a Q4 quantized model needs about 0.5 GB of memory for each billion parameters. A 7B to 8B model therefore needs about 4 to 5 GB, and a 13B to 14B model needs about 7 to 8 GB. Context length adds to these figures. A longer context window uses more memory. For this reason, a system needs more RAM than the model file alone.
Thinvent systems for local LLMs
The Thinvent range has no discrete GPU in any listed model. All inference runs on the processor and system RAM. The 64GB Industrial PC IPC5 with Intel Core i7-1255U gives the most memory on this page, and it suits quantized models that need more than 32GB.
| Model | Processor | Cores | Max Frequency | RAM | Storage |
|---|---|---|---|---|---|
| Industrial PC IPC5 (64GB) | Intel Core i7-1255U | 10 | 4.7 GHz | 64 GB DDR4 | 1 TB SSD |
| Industrial PC IPC5 (32GB) | Intel Core i7-1255U | 10 | 4.7 GHz | 32 GB DDR4 | 1 TB SSD |
| Industrial PC IPC5 (16GB) | Intel Core i7-1255U | 10 | 4.7 GHz | 16 GB DDR4 | 512 GB SSD |
| Aero Mini PC | Intel Core 5 120U | 10 | 5.0 GHz | 16 GB DDR4 | 512 GB SSD |
| Treo Mini PC | AMD Ryzen 5 7430U | 6 | 4.3 GHz | 16 GB DDR4 | 512 GB SSD |
The Industrial PC IPC5 with 1TB SSD and 64GB RAM suits users who need the most memory on this page. The Aero Mini PC with the Intel Core 5 120U has the highest maximum frequency in the range at 5.0 GHz.
Industrial PC IPC5 for local inference
The Industrial PC IPC5 is a fanless-class industrial computer for continuous operation. It includes dual LAN ports, and it is available with Windows 11 Pro or Ubuntu Linux. Ubuntu is the usual choice for running open-source LLM tools such as llama.cpp and Ollama.
Thinvent systems with 16GB RAM
The Aero Mini PC and the Treo Mini PC each have 16GB of DDR4 RAM. They suit small quantized models of 7B to 8B parameters. Both are sold with Windows 11 Pro.
