Can GTX 1080 Ti Run Modern AI? Qwen 3.5 Local LLM Benchmark Results

Hello, this is Shichinomiya (@shichinomiya_s).

This time, I ran a full benchmark to see how far the latest local LLM “Qwen 3.5 9B” (released in 2026) can go on the legendary 2017 GPU: the NVIDIA GeForce GTX 1080 Ti.

If you think “there’s no way an old GPU can handle generative AI,” you might be surprised. Spoiler: it works surprisingly well.

Why Test Local LLM on an Old GPU?

With the recent AI boom, more and more people want to run generative AI locally. But the latest GPUs like the RTX 4090 or RTX 5090 are expensive. Spending over $700 on a GPU just for AI is a tough sell for many.

That’s where the used GTX 1080 Ti comes in. It packs 11GB of VRAM and can be found on the used market for under $100. Since VRAM capacity is the most important factor for local LLMs, an “old but VRAM-rich” GTX 1080 Ti is actually a pretty interesting option.

Honestly, I didn’t expect it to work this well. To give you the conclusion upfront: it’s perfectly practical for everyday use.

Test Environment

By the way, my GTX 1080 Ti was purchased used. You can still find used ones on Amazon, making it a surprisingly viable entry point for local AI experimentation.

What is Qwen 3.5?

Qwen 3.5 is a large language model developed by Alibaba Cloud. The 9B parameter model is relatively lightweight yet delivers strong performance in Japanese, code generation, and reasoning tasks.

With Ollama, you can install it with a single command:

ollama pull qwen3.5:9b
ollama pull qwen3.5:9b-q8_0

For this benchmark, I compared two quantization levels: Q4_K_M (4-bit quantization) and Q8_0 (8-bit quantization). Quantization is a technique that compresses model data to make it lighter — lower bit counts mean smaller size but slightly reduced accuracy.

If you want to run local LLMs comfortably, memory matters too. Having 64GB or more of RAM makes model loading and multitasking much smoother.

Q4_K_M (4-bit Quantization) Results

Let’s start with the lightweight Q4_K_M (4-bit quantization). VRAM usage was approximately 6.6GB (60%), leaving plenty of headroom on the 11GB GTX 1080 Ti.

Average speed: 34.4 tok/s. In practice, that’s over 30 characters per second streaming smoothly — actually a bit faster than ChatGPT’s streaming display. The quality of Japanese responses was also impressively high. I was honestly surprised.

Q8_0 (8-bit Quantization) Results

Next up, the higher-precision Q8_0 (8-bit quantization). VRAM usage was approximately 10.0GB (91%), pushing the 11GB limit to its edge.

Average speed: 14.3 tok/s. That’s about 58% slower than Q4_K_M. You can definitely feel the difference — long text generation can take over 8 minutes. That said, 14 tokens per second is still fast enough to read along in real-time as it generates.

Quantization Comparison

Here’s a head-to-head comparison of Q4_K_M vs Q8_0:

The verdict: Q4_K_M is the clear winner on GTX 1080 Ti. The speed difference is 2.4x, while the quality difference is barely noticeable in practice. Q8_0 only makes sense if you absolutely need maximum precision.

With Q4_K_M’s lower VRAM footprint, you can comfortably run other applications alongside it. On the other hand, Q8_0 occupies 91% of VRAM, making multitasking (like watching YouTube while running inference) quite difficult.

Thinking Mode Comparison

Qwen 3.5 features a “Thinking” mode that runs an internal reasoning process before generating the final answer. I tested this with a FizzBuzz code generation task.

With Q4_K_M, enabling Thinking mode increases token count by about 1.5x and takes 1.5x longer. However, the generation speed itself stays nearly the same. For simple tasks, Think OFF is fine, but for complex reasoning, turning it on is worthwhile.

An interesting finding with Q8_0: Think ON actually produced fewer tokens and finished faster. This suggests that the internal reasoning process can lead to more concise final output in some cases.

VRAM Usage — Running on 11GB by a Thread

Here’s the VRAM usage breakdown on the GTX 1080 Ti’s 11GB:

• Q4_K_M: ~6.6GB used (60%) — Plenty of headroom
• Q8_0: ~10.0GB used (91%) — Barely fits

Q4_K_M leaves about 4.4GB free, so you can run other GPU tasks in parallel. Q8_0, on the other hand, uses 10 out of 11GB, putting you right on the edge of OOM (Out of Memory) errors. During my tests, I had to close all other applications when running Q8_0.

If your budget allows, the RTX 4060 Ti 16GB offers the best value in the current generation. With 16GB of VRAM, you can comfortably run even larger models.

Practical Evaluation — Can It Replace ChatGPT?

Based on all these results, here’s my practical evaluation of GTX 1080 Ti + Qwen 3.5 9B (Q4_K_M).

Good for:

• Programming assistance (code generation, bug fixing)
• Japanese text writing and summarization
• Translation (Japanese-English, English-Japanese)
• Logical question answering
• Processing sensitive data locally (everything stays on your machine)

Not ideal for:

• Questions requiring the latest information (depends on training data)
• Image generation or speech recognition (requires separate models)
• Processing extremely long texts in one go (context length limitations)

My personal take: for everyday questions and coding assistance, it can absolutely serve as a ChatGPT replacement. At 34 tok/s, there’s no frustrating wait, and best of all — you can use it as much as you want without worrying about API costs.

Canceling your $20/month ChatGPT Plus subscription and switching to a used GTX 1080 Ti for under $100 is a seriously viable option.

Context Length Limits — How Far Can 11GB VRAM Go?

For practical LLM usage, context length — how much text the model can process at once — is crucial. Qwen 3.5 9B supports up to 256K tokens on paper, but with only 11GB VRAM on the GTX 1080 Ti, we needed to find the real-world limit.

I tested with Q4_K_M (4-bit quantization), progressively increasing the num_ctx parameter:

The interesting finding: generation speed remains nearly constant (32-33 tok/s) regardless of context length. Increasing context doesn’t slow down inference.

However, beyond 24,576 (24K), you’ll hit timeouts or OOM errors. At 32K, the GPU runs completely out of VRAM.

For daily use, 16,384 (16K) is the sweet spot. It’s enough for summarizing papers or analyzing long code files, with enough headroom for stability. 24K works too, but 16K gives you more breathing room for multitasking.

Note: the first load (2,048) takes 56 seconds because it includes loading the model into VRAM. Subsequent loads take only about 3 seconds.

If you’re looking to get started with local LLM, here are our recommended parts.

MSI GeForce RTX 4060 Ti GAMING X 16G Graphics Card

16GB GDDR6 / Best current-gen GPU for local LLM

View on Amazon

* Prices may change. Please check Amazon for the latest pricing.

AMD Ryzen 9 9950X W/O Cooler (16C/32T, 4.3GHz, 170W)

¥
93,852

Price as of 2026/03/18

View on Amazon

* Prices may change. Please check Amazon for the latest pricing.

Conclusion

This benchmark proved that even a GTX 1080 Ti can run the latest local LLMs at a perfectly practical level.

Key takeaways:

• Q4_K_M (4-bit) is the sweet spot — averaging 34.4 tok/s for comfortable performance
• Q8_0 struggles with speed — 14.3 tok/s, 58% slower
• 11GB VRAM is perfect for 9B models — Q4 runs with headroom, Q8 barely fits
• Japanese quality is excellent — Qwen 3.5 excels at Japanese tasks
• Best bang for your buck — a used GPU under $100 gives you a practical local AI setup

If you’re interested in local LLMs but can’t justify an expensive GPU, I’d recommend starting with a GTX 1080 Ti + Ollama. I think you’ll be surprised at what an “old” GPU can still do.

While pairing it with a powerful CPU like the Ryzen 9 9950X helps, remember that for local LLMs, GPU VRAM capacity is by far the most important factor.

Related Posts

Check out these other articles on local AI and PC builds:

• GTX 1080 Ti Local LLM Benchmark — How Far Can It Go?
• Claude Code Beginner’s Guide for Windows
• Streamlit Complete Guide: Python Web App Development