What Hardware Do You Need to Run Large Language Models Locally?
· By Michel Valenzuela, Founder of Daryl
The hardware you need depends on the model, the application, and the workload. Check processor compatibility, available memory, storage, and supported acceleration before buying anything.
For a personal assistant, start by checking whether your existing computer can run a suitable model comfortably. Serving several users or running larger models requires a separate assessment.
Daryl’s final Mac requirements have not yet been announced. General local-AI guidance should not be treated as a Daryl compatibility statement.
Are you running a model or training one?
These are different workloads.
Running an existing model to generate responses is called inference. Training changes the model through a learning process and can require substantially different resources.
If you want a local assistant for conversation, you are generally looking at inference requirements.
Make sure the hardware advice you follow addresses that task. A workstation recommended for model training may be unnecessary for the application you intend to use.
How much memory do I need?
There is no single memory requirement for all local language models.
The model needs memory while running. The conversation context, application, and other components also consume resources.
Your operating system and everyday applications need room, too.
As one product-specific example, LM Studio’s published Mac guidance recommends 16 GB or more of memory. It notes that 8 GB Macs may still work with smaller models and modest context sizes.
That is guidance for LM Studio. It does not establish a universal minimum or guarantee that a particular model will fit.
Check both the application requirements and the selected model’s requirements.
Why does conversation length matter?
A model needs resources to work with the context available during an interaction.
Long conversations and large documents can increase memory demands. Depending on the implementation, they can also affect response time.
That means a short test question may work well while a more substantial workflow struggles.
Test the amount of information you actually expect to provide. If you need to discuss long documents, include representative documents in the evaluation.
Use only material you are authorised to provide.
Do I need a dedicated graphics card?
Not always.
Some local-AI applications can run supported models using the CPU. Others support GPU acceleration, which can improve performance for compatible workloads.
On supported Macs, applications may use Apple Silicon’s processing capabilities and unified memory. On other computers, a supported dedicated GPU may have its own video memory.
Software compatibility matters as much as the presence of a GPU.
Before purchasing hardware, confirm that the application supports the processor, graphics hardware, and model format you intend to use.
What is the difference between RAM and video memory?
RAM is memory available to the computer’s system and applications.
A dedicated GPU typically has its own video memory, often called VRAM. Models running through the GPU may need to fit within that memory or use a supported arrangement that distributes the workload.
Apple Silicon uses a unified memory architecture shared by its processing components.
These arrangements are different, so memory figures should be interpreted in the context of the hardware and software.
Avoid assuming that two computers with the same advertised memory capacity will deliver the same local-AI experience.
Does quantisation reduce hardware requirements?
It can reduce the memory needed to store a model’s weights.
Quantisation represents model values with lower numerical precision. This can make a model more practical to run on limited hardware.
The result depends on the model, quantisation method, and runtime. It can also affect output quality.
A smaller download is not proof that the model will perform your task well. Evaluate its responses alongside memory use and speed.
For nontechnical users, a well-designed application should explain which supported configuration fits their device.
How much storage should I allow?
Account for the complete installation:
- The application.
- Model files.
- Any additional voice resources.
- Saved conversations or supported documents.
- Space needed during updates.
If you download several models, their storage requirements accumulate.
Check the actual download sizes and installation guidance. The size of the application alone may represent only a small part of the setup.
Can I use a laptop?
Yes, when the application and model support its hardware.
A laptop can be practical for personal conversation or a focused local workflow. Its suitability depends on memory, processing performance, thermal behaviour, and battery demands.
Test it alongside the applications you normally use.
An assistant that works comfortably in isolation may behave differently during a video meeting or while editing a large creative project.
Assess the working environment you need, rather than relying only on a demonstration.
What changes when several people need access?
A personal setup and a shared service are different deployments.
Serving multiple users introduces simultaneous requests, access controls, network configuration, and maintenance requirements.
You need to establish the expected workload and acceptable response times before choosing hardware.
A computer that serves one person well should not be assumed to support a team at the same level of performance.
Should I upgrade my computer for local AI?
Test first, then identify the actual limitation.
Your existing hardware may support a smaller model that handles your intended task. If performance is inadequate, determine whether the issue is memory, processing speed, the application configuration, or the model’s capabilities.
More hardware will not automatically correct poor answers or an unsuitable workflow.
A purchase makes sense when you can connect the upgrade to a verified need.
What hardware will Daryl require?
Daryl is being developed as a personal AI for Mac, centred on local conversation and support for decisions.
Its requirements will need to reflect the complete release experience, including the local AI components and supported voice workflow.
Final guidance should identify supported Macs, macOS versions, memory, storage, and any meaningful limitations.
Until that information is verified, I would not recommend buying a computer specifically for Daryl.
The goal is to make useful local intelligence approachable on supported hardware, with requirements people can understand before committing.
Get the Daryl launch announcement