What Intelligent Terminal Does and Why Local LLMs Matter
Windows Terminal local LLM integration in Microsoft's experimental Intelligent Terminal lets you dock an AI agent directly in your command line, using open models that run on your own hardware instead of relying on cloud-based Copilot or other remote providers, giving developers and power users private, configurable assistance for shell workflows and scripting while keeping sensitive data on their machines.
Think of Intelligent Terminal as Windows Terminal with a built‑in AI pane that can explain errors, suggest commands, and keep session context as you work. It is an experimental fork of Windows Terminal that installs alongside your existing terminal, so it does not replace or modify your current setup. The key design choice is Agent Client Protocol (ACP), an open standard that lets any compatible AI agent connect as a subprocess over stdio instead of forcing you into one vendor. In practice, that means the default agent is GitHub Copilot, but you can swap in local AI command line tools backed by models running through llama.cpp or Ollama. If you care about privacy or you often work with confidential code and infrastructure, this ACP-based approach gives you an AI helper without sending terminal context to a cloud service.
Prerequisites: What You Need Before You Start
Before you start your Intelligent Terminal setup, it helps to know what pieces you are wiring together and why. Intelligent Terminal itself is a separate app that sits next to classic Windows Terminal, so you can install it safely without affecting your current shell profile or configuration. Under the hood, it expects an ACP‑compatible agent: that is where tools like Pi, an open‑source, MIT‑licensed coding agent that works with both hosted and local models, come into play. For a local LLM, you need a model server such as llama.cpp or Ollama and at least one model downloaded and running on your machine or home server.
| Prerequisite | Why You Need It | Notes |
|---|---|---|
| Intelligent Terminal installed | Provides the AI-enabled Windows Terminal fork | Installs as a separate app and does not touch your existing terminal. |
| ACP-compatible agent (e.g., Pi) | Bridges the terminal to your model | Pi works with providers from Claude and GPT to local models through llama.cpp or Ollama. |
| Local model server (llama.cpp / Ollama) | Runs your chosen LLM on your own hardware | Allows a local Qwen or similar model to back your agent without cloud dependency. |
| Basic comfort editing JSON configs | Needed to set default models and agent options | Intelligent Terminal uses settings and models.json files for configuration. |
The good news: you do not need to write any glue code or plugins for Windows Terminal configuration here. The terminal only “cares that something on the other end of the ACP pipe can receive shell context and send back responses.” Once you have a local model process that can speak ACP, the rest is point‑and‑click in the settings UI plus one models.json edit. That is why someone was able to get a local Qwen 3.6 27B model working with Intelligent Terminal in about five minutes.
Step‑by‑Step: Wiring a Local LLM Into Intelligent Terminal
Here is the practical walkthrough, like showing a friend how to get an AI pane running beside their shell without handing everything to Copilot. We will focus on Pi as the ACP agent and a local model served by llama.cpp, but the pattern is the same for other agents and providers. Be aware that Intelligent Terminal is a version 0.1 release, so the UI feels basic and a bit rough. That simplicity is part of why setup is quick: there are only a handful of moving parts, and each one does a narrow job.
- Install Intelligent Terminal and open it. Because it is an experimental fork, it installs next to your existing Windows Terminal instead of replacing it, so your current shell remains untouched.
- Optionally accept the prompt to install GitHub Copilot via WinGet if you want a quick way to compare cloud and local agents; the default agent slot is configured for Copilot out of the box.
- Install the Pi ACP extension, which provides an ACP‑compatible CLI for the Pi coding agent. This is the bridge that lets Intelligent Terminal talk to a model through ACP instead of a hard‑wired Copilot integration.
- Open Intelligent Terminal’s settings and switch the active agent from the default Copilot entry to the pi-acp agent in the agent dropdown. The settings menu is designed to accept any ACP-compatible CLI, not only Microsoft’s own services.
- Edit the Pi configuration file, models.json, and set your default Pi model to a local Qwen instance (for example, Qwen 3.6 27B) running on your home server via llama.cpp.
- Restart or reload Intelligent Terminal, then run a few commands and watch the dockable agent pane. The terminal should display a persistent agent status bar at the bottom and allow you to toggle the AI pane with Ctrl+Shift+. to see suggestions, explanations, and error diagnoses driven by your local model.
Once you complete those steps, “you have a Microsoft terminal talking to a community‑built ACP bridge, driving an open‑source coding agent, backed by a model from Alibaba, working alongside your terminal.” It feels like a native terminal feature, not an LLM jammed awkwardly into a UI, because the agent runs as a subprocess and shares shell context over ACP. The main gotcha to watch for is version maturity: at 0.1, some edges are sharp, and you may run into occasional quirks or missing polish. Still, for a five‑minute setup, you end up with a Windows Terminal local LLM that is fully under your control.
Using AI Command Line Tools and Advanced Terminal Features
Once your local model is wired in, the fun begins: you can treat the agent pane as a smart companion for shell and scripting tasks. Intelligent Terminal’s interface adds a persistent agent status bar at the bottom of the window, a dockable agent pane you can position on any side with Ctrl+Shift+., and an error detection indicator that lights up whenever a command fails. That error signal is where AI command line tools shine: the agent can automatically examine failed commands, explain what went wrong, and suggest corrected invocations based on your current context.
For more professional‑grade Windows Terminal configuration, you get an agent management panel (Ctrl+Shift+/) that tracks active and past sessions across tabs so you can resume earlier workflows without losing AI context. The Command Palette gains a prefix mode that lets you send prompts straight to the agent and even spin up work in background tabs so your foreground shell keeps running. Under ACP, this all works regardless of whether the agent is backed by Copilot or by a local GPU model. The terminal does not care which model you pick: it cares that the ACP endpoint can receive shell state and respond. If you want a local model running on your own GPU with an open‑source agent, that is fully supported and keeps sensitive data on your hardware.
Is Running Local LLMs in Windows Terminal Worth It?
If you are a developer or power user who spends much of the day in a shell, adding a local LLM to Windows Terminal via Intelligent Terminal is worth the short setup time. The architecture choices indicate that local AI is treated as a first‑class option, even though the default path is Copilot. Between RTX Spark‑style hardware, sandboxes for local agents, and a terminal that treats your choice of model and agent as configuration instead of lock‑in, the direction points toward choice and privacy.
The main thing to watch for is the 0.1 release feel: the feature set is basic and the experience may show rough edges. But in exchange, you gain a private AI‑assisted command line that can help with debugging, scripting, and system tasks without sending shell history to a cloud service. Once configured, it “feels like a native terminal feature rather than an LLM jammed awkwardly into a UI,” and it delivers more than you might expect from a new terminal experiment. If that trade‑off sounds good to you, start with Pi and a local Qwen model and treat this as a living part of your daily workflow rather than a novelty.
