The real upgrade for local LLMs lives in the settings panel
Fine-tuning local LLM settings means adjusting generation controls like repeat penalty and presence penalty so the model reuses words and ideas in a more controlled way, reducing loops, circular reasoning, and filler while keeping outputs coherent and relevant for practical day-to-day use cases rather than benchmark-style prompts.
If your local model keeps rephrasing the same intro, circling a point, or drowning you in word salad, the problem is rarely the model alone. People swap from one model family to another, or from small to large, and end up with the same spirals. That is because most local LLM settings are tuned for leaderboard evaluations, not for real conversations, everyday instructions, or short prompts. On top of that, your runner—whether it is LM Studio, Ollama, or llama.cpp—adds its own default knobs over the model’s built-in configuration, so you are using someone else’s guess about what “good enough” looks like. If you care about LLM response quality on your own machine, you need to stop treating those defaults as sacred and start treating repeat and presence penalties as your main model tuning configuration tools.
Tip 1: Use repeat penalty to stop word-level loops, not to censor language
Repeat penalty is your first line of defense against local LLM responses that open with the same sentence three times or repeat phrases in every paragraph. Technically, it is a token-level control that penalizes specific tokens which have already appeared in the output. At its neutral value of 1.0, the model feels no extra pressure to avoid repetition; nudge it to 1.05 or 1.1 and you usually see a clear reduction in those word-level loops without much downside. In OpenAI-style APIs this behaves like frequency_penalty, while llama.cpp calls it repeat_penalty; they all aim at the same underlying mechanism.
The key is moderation. Push repeat penalty too high—around 1.2 and beyond—and the model starts dodging common, harmless words like “the,” because the math tells it those tokens are dangerous. That is how you end up with broken grammar and strange, contorted phrasing. The misconception is to treat repeat penalty as a blunt anti-rambling tool; in reality, it is a scalpel. A small change cleans up loops, but a big one starves the model of natural language and lowers LLM response quality.
Tip 2: Use presence penalty to push ideas forward, not sideways
Presence penalty looks similar on paper but solves a different problem: idea-level circling. It also operates at the token level, but instead of scaling the penalty as a word is reused over and over, it applies a flat penalty the moment a token has appeared once. Its neutral point is 0, and typical values range from -2.0 to 2.0. That means the second time the model reaches for the same word, it is already nudged away. Different words often drag the model to different topics, which is why presence penalty is what keeps it from returning to the same point four different ways.
When tuned well, presence penalty reshapes the structure of an answer. In one test, a balanced configuration—with repeat at 1.1 and presence at 0.6—produced a clean three-part explanation where each audience segment got a distinct, useful angle from the same prompt. By contrast, when repeat was cranked to 1.3 and presence to 1.8, the reply started okay and then collapsed into a list of disconnected geography terms that had to be stopped manually. That is what overcorrection looks like: the penalties starve the model of high-probability tokens until it grabs whatever is left, even if it no longer forms sentences.
Where these settings shine—and where they should stay near neutral
Here is the most common misconception about local LLM settings: people treat repeat penalty and presence penalty as interchangeable sliders that both “fight repetition,” and that is wrong. Repeat penalty handles exact words; presence penalty steers the ideas behind those words. That difference matters for where you should use each. Coding work, for example, thrives on repetition—function names, brackets, imports—so repeat penalty should stay near neutral because code is supposed to reuse syntax. Reasoning models, on the other hand, must revisit earlier ideas to reach a conclusion, so they tend to need lower presence penalty; if you push it high, you punish the model for reminding you of important steps.
Quantization level also changes the safe range. Smaller quants like Q3 or Q4 are already more fragile because precision has been traded for memory footprint and speed, and repetition and looping are among the first failure modes when precision drops. That is why they need lighter penalties in general. When your model is already working with rougher numbers, aggressive penalties make it more likely to break into nonsense. The takeaway: match your model tuning configuration to the task and quant, not to a one-size-fits-all blog recipe.
Tuning that travels with you: from runners to hardware limits
The beauty of this kind of inference optimization is that it works across different runners and hardware setups. LM Studio, Ollama, and llama.cpp all sit on the same core engine but ship with their own default local LLM settings layered over the model’s properties, and those defaults differ. You are never dealing with a pure, universal configuration; you are dealing with what the runner author thought would be acceptable for most people. My values for a Qwen 9B model in LM Studio were a helpful starting point, but they are not a universal rule of thumb. The only sound recommendation is to avoid leaving repeat and presence penalties at their defaults and avoid cranking them to extremes; find the middle sweet spot for your model, quant, temperature, prompts, and task.
Hardware also shapes what is possible, but tuning still matters once the model fits. Quantization, such as Q4_K_M, can shrink a model like Ornith 9B to around 5.6GB so it fits neatly inside a 16GB RAM budget while leaving room for the OS and context window. On dedicated GPUs, VRAM—not total system RAM—governs speed, and if the quant does not fit in VRAM, runners like Ollama offload parts to the CPU, which slows generation. In that world, tuning penalties is not a luxury; it is how you keep a quantized model from falling into loops made more likely by reduced precision. Settings-level optimization does not replace good hardware or good models, but it unlocks the performance you already paid for.
