Higher Quants or More Params? Benchmarking Local LLMs for AI Marketing
— 11 min read
Higher Quants or More Params? Benchmarking Local LLMs for AI Marketing
Short answer: neither. The long answer involves 74 benchmark queries, four quantization strategies, one overheating desk, and a 120-billion-parameter model getting beaten by something less than a quarter its size.
The Local LLM Tradeoff: Params vs Context vs Speed
You already know why you want to run a local LLM instead of renting one. No per-token bills on AI marketing work that involves a hundred drafts. No confidential client brief taking a scenic tour through someone else's data center. And unlimited retries at 9 PM when the account manager decides the campaign is "more aspirational now."
So you saved up, you built the box, and then you hit the question that eats an entire weekend of forum-reading. Most people phrase it as bigger model or better quantization? — which is the wrong frame, because it treats VRAM as if it only buys one thing.
Here's the real question:
Do you spend your VRAM on parameters, or on everything else?
Parameters are not free. You pay for them in the two properties you actually notice every working day:
- Context window. The 120B model in this test runs at 16K context. The 35B runs at 131K — eight times the room, on the same GPU. One of those can hold your brand bible, the research deck, and last quarter's retrospective. The other can hold the brand bible.
- Speed. The 120B produces 15–20 tokens per second. The 35B does 95–105. That's the difference between requesting twenty headline variants and getting them, versus requesting twenty and going to make coffee.
So the tradeoff isn't quantization quality versus model size. It's this: more parameters buy you a deeper model that thinks in a smaller window, slowly. Fewer parameters buy you a shallower model with room to read everything, fast. Which one wins depends entirely on whether your work is bottlenecked on reasoning or on context and iteration.
That's the question this benchmark was built to answer.
The rig: Core i5-13500, 64 GB RAM, an RTX 5070 Ti alongside an RTX 4060. Sixteen and eight gigs of VRAM, a generous pool of system memory, and a CPU whose main job during all this is to pass the popcorn.
Four candidates, four completely different bets on how to spend those resources.
The Contenders: Four Local LLM Quantization Strategies
| # | Model (quant) | Context | Speed | Vision |
|---|---|---|---|---|
| 1 | qwen3.6-35b-a3b-i1 (Q4_K_S) | 131,072 | 95–105 tps | No |
| 2 | qwen3.6-27b-mtp (Q4_K_M) | 60,000 | 28–40 tps | Yes |
| 3 | gemma-4-31b-it-qat (Q4_K_XL) | 16,384 | 25–35 tps | Yes |
| 4 | gpt-oss-120b (Q8_0) | 16,384 | 15–20 tps | No |
LM Studio config screenshots (click a thumbnail to view full size):
All four are GGUF builds served through LM Studio. A word on each, because the hardware story here is half the fun.
gpt-oss-120b at Q8_0 has no business running on this machine. A hundred and twenty billion parameters at eight-bit precision is roughly 120 GB of weights, and there are 24 GB of VRAM in the building. It works anyway, because it's a mixture-of-experts model: most of those expert layers doze in system RAM, and only the active ones get called up to the 5070 Ti. The price of the trick is 15–20 tokens per second. Think of it as a very well-read sloth.
qwen3.6-35b-a3b-i1 at Q4_K_S pulls the opposite stunt. Thirty-five billion total parameters, but only about 3B active per token, which is how it reaches 95–105 tokens per second — call it seven times the giant's pace. It also carries a 131K context window, enough to swallow an entire brand bible without chewing. The catch: it's text-only.
qwen3.6-27b-mtp at Q4_K_M is the boring one on paper. Twenty-seven billion, dense, a standard middle-of-the-road quantization. It also happens to see images, hold a 60K context window, and run at a perfectly livable 28–40 tps.
gemma-4-31b-it-qat at Q4_K_XL was quantization-aware trained, meaning it was born compressed rather than squeezed after the fact — in theory the cleanest small model of the bunch. It sees images, runs 25–35 tps, and is stuck with a 16K context window.
The Benchmark: WritingBench for AI Marketing Copy
One blog post and a vibe check isn't a benchmark. This was WritingBench, filtered to the English-only Advertising & Marketing subset: 74 real queries across 11 writing tasks that marketing people genuinely produce every week.
Brand Story. Travel Guide. Social Media Content. Multimedia Script. Personal Blog. Promotional Voiceover. Marketing Commentary. Product Description. Sales Letter. Promotional Copy. Slogans.
If you are evaluating a local LLM for AI marketing work, this is a far better proxy than a leaderboard average, because it scores the specific jobs rather than general capability. Every response gets scored out of 10 against per-query criteria — relevance, creativity, persuasiveness, tone — plus separate style, format, and length compliance subsets. That last part matters more than it sounds like it should, because it measures not just whether the writing is good but whether the model did what it was told. The judge doesn't know who wrote what. No brand loyalty, no benefit of the doubt.
Results: The Smallest Local LLM Won
| Rank | Model | Overall |
|---|---|---|
| 1 | qwen3.6-27b-mtp (Q4_K_M) | 7.31 |
| 2 | qwen3.6-35b-a3b-i1 (Q4_K_S) | 7.13 |
| 3 | gpt-oss-120b (Q8_0) | 7.07 |
| 4 | gemma-4-31b-it-qat (Q4_K_XL) | 6.75 |
The 120B at Q8_0 — the most parameters and the highest precision in the lineup, the configuration every "just buy more VRAM" thread on Reddit is implicitly recommending — came third.
The smallest model in the field won.
If more parameters and higher precision were the answer, that table would be upside down. It isn't, so something else is doing the work.
Why the 27B Won: Vision, Context Window, Instruction Following
It can see. Advertising runs on images: product shots, brand assets, mood boards, whatever the competitor just posted. The 27b-mtp looks at them. The a3b and the 120B are working blind from text descriptions, which in this industry is like art-directing over the phone.
It can hold the brief. Campaign inputs are long — brand guidelines, research decks, last quarter's retrospective. The winner has a 60K context window. The two models it beat are capped at 16K, and that cap is exactly what those extra parameters cost them. (The a3b's 131K is the biggest of all, which makes its second-place finish its own kind of story. More on that shortly.)
It follows instructions. This is the unglamorous one, and it decides more real work than anything above it. Marketing briefs are full of "match the tone deck" and "two to three sentences, maximum."
| Metric | 27b-mtp | 35b-a3b | 120b | gemma-4 |
|---|---|---|---|---|
| Format compliance | 7.84 | 7.68 | 7.84 | 7.24 |
| Length compliance | 7.16 | 6.32 | 6.72 | 6.08 |
The 120B ties the winner on format. Nobody comes close to it on length. And the speed demon, for all its 131K of context, is the worst offender in the field at 6.32 — which in practice is the model that answers "give me 300 words" with 900 words and an air of having done you a favor.
Task by Task: No Single Local LLM Sweeps
Zoom into the individual tasks and the picture stops being a leaderboard and starts being a team roster.
| Task | Winner | Score | Runner-up |
|---|---|---|---|
| Personal Blog | 27b-mtp | 7.83 | 120b · 7.20 |
| Social Media Content | 27b-mtp | 7.80 | 120b · 7.57 |
| Travel Guide | 27b-mtp | 7.73 | 120b · 7.67 |
| Multimedia Script | 27b-mtp | 7.70 | 35b-a3b · 7.67 |
| Brand Story | 27b-mtp | 7.68 | 120b · 7.64 |
| Promotional Voiceover | 27b-mtp | 7.35 | 35b-a3b · 7.32 |
| Slogans | 27b-mtp | 6.83 | 35b-a3b · 6.63 |
| Marketing Commentary | 120b | 7.47 | 27b-mtp · 7.33 |
| Product Description | 120b | 7.09 | 35b-a3b · 6.93 |
| Sales Letter | 35b-a3b | 7.15 | 27b-mtp · 6.95 |
| Promotional Copy | 35b-a3b | 7.04 | 27b-mtp · 7.00 |
Seven of eleven go to the 27B. The giant takes two — and which two is the interesting part. Marketing Commentary and Product Description: the analytical work, the explain-what-this-is-and-why-it-matters work. That's exactly where a big model's knowledge and structure earn their keep.
Now look at where it lands hardest. Slogans, 6.48. Promotional Copy, 6.44. Both last or near it. The 120B is a superb analyst who cannot write a jingle to save its life, and no amount of Q8_0 precision fixes that, because precision was never the missing ingredient.
The a3b takes the two short-punch persuasion tasks, Sales Letter and Promotional Copy — though it wins the latter by 0.04 over the 27B, which is a rounding error wearing a medal. Still, a model running at 100 tokens per second is a model you can ask for twenty variants and actually get twenty variants before you lose interest. That is the fast-and-shallow side of the tradeoff paying off exactly where you'd expect: volume work, where the win comes from trying more things rather than thinking harder about one. Its only real sin remains ignoring the word count.
And gemma-4? It won nothing. Last on the overall score, last on format compliance, last on length compliance, last on style relevance at 6.70. Quantization-aware training is a genuinely good idea, and this particular model still finished fourth in a field of four. Its one respectable showing is style consistency at 7.25, where it edges out the 120B. That's the whole highlight reel.
The Verdict: How to Choose a Local LLM for Marketing Work
Stop asking bigger or more precise. Start asking what your actual week looks like — and which side of the params-versus-context-and-speed tradeoff it lands on.
| If you mostly… | Run | Because |
|---|---|---|
| Work with visuals — product photos, brand assets, references | qwen3.6-27b-mtp (Q4_K_M) | The only top-scorer with eyes, best compliance in the field, wins 7 of 11 tasks |
| Brainstorm and iterate fast over long campaign docs | qwen3.6-35b-a3b-i1 (Q4_K_S) | 95–105 tps and 131K context; best sales letters and promo copy; budget a "stick to the word count" line in every prompt |
| Analyze, describe, commentate | gpt-oss-120b (Q8_0) | It wins those two tasks outright — bring patience for 15–20 tps |
| Are already running gemma and wondering | gemma-4-31b-it-qat (Q4_K_XL) | Decent style consistency, but it lost every task in this subset; the others are a free upgrade |
The real answer is that this was never a one-model decision. All four GGUF files fit on the same drive, and LM Studio will swap between them in about the time it takes to reread the brief. Use the 120B for strategy and analysis, the 27b-mtp for anything with an image attached, and the a3b for the 9 PM "we need a new slogan in five minutes" call.
Bottom Line: Quantization Quality Isn't Your Bottleneck
Q8_0 did not beat Q4_K_M. 120B did not beat 27B. Once you're at Q4, the quantization gap is background noise next to the things that actually decide whether a local LLM is useful for marketing work:
- Can it see the assets?
- Can it hold the whole brief in its context window?
- Does it do what the brief says?
- Is it fast enough that you'll actually use it?
Three of those four are things you buy by not spending your VRAM on parameters. That's the whole finding. More parameters bought a brilliant analyst who can't write a slogan, in a 16K window, at reading speed. More precision bought accuracy that couldn't outrank a Q4. The upgrade path isn't more — it's matched.
And yes, running a 120B model by stashing it in system RAM is a party trick. It's also two gold medals. Just don't ask it for a jingle.
Method & Caveats
- Benchmark: WritingBench, English-only Advertising & Marketing subset — 74 queries, 11 writing task types, scored out of 10 by an LLM judge against per-query criteria, plus style/format/length compliance subsets.
- Hardware: Core i5-13500, 64 GB RAM, RTX 5070 Ti + RTX 4060 (16 + 8 GB VRAM), served through LM Studio.
- Configs: qwen3.6-27b-mtp (Q4_K_M, 60K ctx, vision), qwen3.6-35b-a3b-i1 (Q4_K_S, 131K ctx), gemma-4-31b-it-qat (Q4_K_XL, 16K ctx, vision), gpt-oss-120b (Q8_0, 16K ctx, MoE layers offloaded to RAM).
- Caveats: English only, a single judge, one sampling run per query. These are directional numbers, not gospel — different GPUs, quants, or prompting styles will move them. The gaps at the top are also tight: 0.24 covers all three leading models, and only 0.06 separates second from third. Gemma's 0.32 deficit to third is the one clearly separated result. Treat the task-level splits as the real signal, not the overall ranking.
