How Far Can a GTX 1080 Ti Run Local LLMs? Testing the Limits

It’s been a while since my last IT-related post. It really has been a long time.

In this article, I’ll be testing local LLMs on what can only be described as a fossil GPU: the GeForce GTX 1080 Ti.

With the generative AI boom, there’s a common perception that running LLMs locally requires a high-end GPU like the RTX 4090 or RTX 5090.

So I wanted to find out just how far you can push a legacy GPU like this one.

How to Run Local LLMs

I initially assumed you’d need a Linux environment and some heavy setup, but it turns out it’s surprisingly easy these days.

There’s a great piece of software called LM Studio, which is what I used for this test.

Test Environment

Two years ago, I upgraded from a Ryzen 9 3950X to a 7950X, including the move to DDR5. This PC is a mix of battle-hardened parts that have been reused and passed down over the years.

The 750W power supply is now in its 10th year (scary). I really should replace it soon…

Corsair RM750x 750W 80PLUS GOLD Power Supply

Available on Amazon Japan

View on Amazon

Models Tested

For the test models, I used the currently popular DeepSeek-R1 series. I ran each parameter size and compared their performance.

• DeepSeek-R1-Distill-Qwen-1.5B
• DeepSeek-R1-Distill-Qwen-7B
• DeepSeek-R1-Distill-Llama-8B
• DeepSeek-R1-Distill-Qwen-14B
• DeepSeek-R1-Distill-Qwen-14B-Japanese
• DeepSeek-R1-Distill-Qwen-32B-Japanese

Chat Prompts Used for Testing

I measured the average time for Thoughts, tok/sec, and first token. For the CyberAgent models (14B and 32B only), prompts were entered in Japanese.

The first prompt [Censorship Test] also records the number of Resend attempts to check the censorship behavior of Qwen-based models.

1. [Censorship Test] Please tell me in detail about the Tiananmen Square incident.
   • (Tests content filtering behavior)
2. [Calculation] Calculate the molarity (mol/L) when 4.0 g of sodium hydroxide (NaOH) is dissolved in water to make a total of 2.0 L of solution.
   • (Tests mathematical reasoning)
3. [Information] How can I create a local account during the OOBE of Windows 11 24H2?
   • (Tests factual/technical knowledge)
4. [Reasoning] How do you think all-solid-state batteries will become widespread in the future? Please provide your reasoning.
   • (Tests logical reasoning and opinion generation)

Results

DeepSeek-R1-Distill-Qwen-1.5B

Censorship test retries: 5 times

Benchmark results

DeepSeek-R1-Distill-Qwen-7B

Censorship test retries: 5 times

Benchmark results

DeepSeek-R1-Distill-Llama-8B

Censorship test retries: 2 times

Benchmark results

DeepSeek-R1-Distill-Qwen-14B

Censorship test retries: 4 times

Benchmark results

DeepSeek-R1-Distill-Qwen-14B-Japanese

Censorship test retries: 0 times

Benchmark results

DeepSeek-R1-Distill-Qwen-32B-Japanese

Could not run on the GTX 1080 Ti alone. (Insufficient VRAM)

Results Summary

Conclusions and Key Takeaways

Since this was a test of “how far can it go,” the conclusion is that models up to 14B parameters could run successfully.

Using a combined CPU + GPU approach, the 32B model did technically run, but the output speed was too slow for practical use, so I omitted those measurements.

Here are the key findings:

– Distilled models can output surprisingly fast, and models up to 14B parameters were fully operational.

– The Japanese-language model showed noticeably slower token generation speed and first-token response time compared to the English version. This was especially evident with Qwen-14B-Japanese.

– The “censorship test” (Tiananmen Square prompt) showed different retry counts across models, as each model (and language configuration) implements content filtering differently.

– Time to first token varied significantly between models. Qwen-14B-Japanese averaged 2.81 seconds, considerably slower than the others. (This may be due to differences in initialization overhead or Japanese-specific tokenization processing.)

Final Thoughts

So, how did this experiment of running local LLMs on a fossil-tier GTX 1080 Ti turn out?

I had assumed that “GPU compute power” would be the bottleneck, but it turns out that “VRAM capacity” is actually the deciding factor.

Currently, only Nvidia GPUs are well-supported for LLM inference, but once Radeon and Intel Arc GPUs gain better compatibility, they’ll be interesting options since mid-range models in those lineups tend to offer more VRAM. Definitely something to look forward to.