Local LLMs can be utilized in two exceptional methods on Windows 11 through WSL (Windows Subsystem for Linux), as highlighted by Ollama.

Local LLMs can be utilized in two exceptional methods on Windows 11 through WSL (Windows Subsystem for Linux), as highlighted by Ollama.

Ollama, a local language model, can now be used on Windows PCs, offering two options: running the Windows app natively or using the Linux version through Windows Subsystem for Linux (WSL). In this article, we'll guide you through the process of installing and using Ollama on WSL.

Prerequisites and Recommendations

To use Ollama on WSL, you'll need an up-to-date NVIDIA driver for Windows and a WSL-specific CUDA toolkit. The best Linux distribution for this purpose is Ubuntu, preferably version 22.04 or later, which is the default and recommended distribution when setting up WSL for Ollama. Ubuntu is developed and maintained by Canonical Ltd.

Installation Process

The installation process involves downloading the necessary bits and running an installation script. During the installation, the script should automatically detect the NVIDIA GPU if set up correctly. Microsoft and NVIDIA's documentation are recommended for guidance throughout the installation process.

Performance and System Resources

The performance of Ollama in WSL is excellent, especially with an NVIDIA GPU. It's comparable to that on Windows 11, with minor fluctuations. However, it's important to note that running WSL uses some of the overall system resources.

Using Ollama on WSL

Using Ollama on WSL is similar to using it on Windows, minus the GUI application. If a model doesn't fit entirely into VRAM, it may use regular system memory and the CPU. None of the tested models exceeded the dedicated VRAM, so WSL's activity had no impact on the model's performance.

Managing WSL Resources

Users can adjust the amount of RAM and CPU threads for WSL in the WSL settings app. This can help manage system resources when running Ollama on WSL.

Alternative Experiences

For an alternative experience, the Page Assist browser extension can be used with Ollama on Windows 11.

WSL vs Windows App

The Windows app method is easier, as it doesn't require WSL installation and has a simpler setup process. However, running Ollama on WSL is a suitable option for developers with necessary WSL workflows, as it provides full use of the NVIDIA GPU without performance loss.

Recognition of Models

If running WSL in a different tab, Ollama in Windows will only recognize models installed on WSL as active.

Conclusion

Using Ollama on WSL, specifically Ubuntu, can offer a magical experience with full functionality of the NVIDIA GPU. While it may be unnecessary for casual users, it's a valuable tool for education and understanding the software's workings, especially for developers with necessary WSL workflows.

Read also:

• Understanding Hemorrhagic Gastroenteritis: Key Facts
• Expanded Community Health Involvement by CK Birla Hospitals, Jaipur, Maintained Through Consistent Outreach Programs Across Rajasthan
• Abdominal Fat Accumulation: Causes and Strategies for Reduction
• Deepwater Horizon Oil Spill of 2010 Declared Cleansed in 2024?