ChatMol

Natural-language control of PyMOL. Describe the scene, get the structure.

A natural-language chat frontend for PyMOL (open-source, 3.1.0), powered
by a local Ollama model (default: qwen3.6:27b).

Why it matters: PyMOL is the lingua franca of molecular visualisation, but producing a polished figure often means stacking a dozen exact commands for selections, representations, colouring, and views. ChatMol layers a local LLM directly onto PyMOL’s command line so plain-English requests (“colour by chain and show the ligand as sticks”) become correct PyMOL syntax instantly, while any valid PyMOL you type still runs untouched. It is useful for structural biologists, crystallographers, and cryo-EM scientists who want to move faster from raw coordinates to publication-quality scenes, lower the barrier for students learning PyMOL, and keep every byte of structural data local.

Type plain English into PyMOL’s normal command line:

PyMOL> fetch 1abc and show it as a cartoon coloured by chain, then show the ligand as sticks
[ChatMol] > fetch 1abc, async=0
[ChatMol] > hide everything
[ChatMol] > show cartoon, polymer
[ChatMol] > util.cbc polymer
[ChatMol] > show sticks, organic

Anything that already looks like valid PyMOL syntax is run unchanged, so
your existing scripts and muscle memory keep working. Anything that looks
like English is sent to a local LLM, translated into PyMOL commands, echoed
to the console, and executed. Everything runs locally — no data leaves
your machine.

1. Prerequisites

You need two things installed before the plugin itself: Ollama (with a
model pulled) and open-source PyMOL 3.1.0.

1.1 Ollama + model

1. Install Ollama:
2. macOS: download the app from https://ollama.com/download, or
   brew install ollama
3. Linux: curl -fsSL https://ollama.com/install.sh | sh

4. Windows: download the installer from https://ollama.com/download

5. Make sure the server is running. The desktop app starts it
   automatically; otherwise start it manually and leave it running:

sh
ollama serve

1. Pull a Qwen3.6 model:

sh
ollama pull qwen3.6:27b

This is the default ChatMol uses, and gives the best translation
quality. It’s a ~17 GB download and needs roughly 20 GB+ of free
RAM/VRAM to run comfortably. If your machine can’t handle that:

sh
# Much smaller/faster, good for trying ChatMol out, lower translation quality:
ollama pull qwen3:0.6b

Then point ChatMol at whichever tag you pulled (see
Configuration) — just make sure the model setting
matches a tag from ollama list exactly.

1. Sanity check the server is reachable and the model responds:

sh
curl http://localhost:11434/api/chat -d '{
"model": "qwen3.6:27b",
"messages": [{"role": "user", "content": "say hi"}],
"stream": false
}'

You should get back a JSON response with a message.content field. If
this fails, fix it before installing the plugin — see
Troubleshooting.

If you run Ollama on a different host/port, configure that too (see
Configuration).

1.2 PyMOL (open-source, 3.1.0)

Pick whichever matches how you normally manage Python environments:

• conda / mamba (any OS):

sh
conda create -n pymol -c conda-forge -c schrodinger pymol-open-source=3.1.0
conda activate pymol

• Homebrew (macOS):

sh
brew install pymol

• Linux package managers also commonly provide pymol (e.g.
  apt install pymol), though the bundled version may differ from 3.1.0.

Launch it once (pymol) to confirm it starts normally and note where its
Python lives — that’s the interpreter ChatMol will run under. ChatMol only
uses the Python standard library, so no extra pip install is required.

2. Installing the plugin

ChatMol is a plain Python package (the chatmol/ directory). There are two
ways to load it; pick one.

Option A — ~/.pymolrc.py (recommended, most reliable)

This runs ChatMol from the source directory directly — easiest to update
(just edit the files and restart PyMOL).

Create or edit ~/.pymolrc.py (in your home directory) and add:

import sys
sys.path.insert(0, "/Users/dellboy/Documents/Vibe Coding/chatmol")
import chatmol

(Adjust the path if you move the chatmol project folder.)

Restart PyMOL. You should see the [ChatMol] AI chat interface loaded
banner in the console — that confirms it’s active.

Option B — Plugin Manager (chatmol.zip)

A ready-to-install zip is included at the project root:
chatmol.zip (contains chatmol/__init__.py etc. at its root, as the
Plugin Manager expects).

1. In PyMOL: Plugin > Plugin Manager > Install New Plugin > Choose
   file… and select chatmol.zip.
2. Restart PyMOL.

This also adds a Plugin > ChatMol Settings menu item with a small GUI
for configuration.

If you edit the source after installing this way, rebuild the zip from the
project root:

cd "/Users/dellboy/Documents/Vibe Coding/chatmol"
rm -f chatmol.zip
zip -r chatmol.zip chatmol -x '*.pyc' -x '*__pycache__*' -x '*.DS_Store'

and reinstall it.

Verifying the install

Either way, after restarting PyMOL you should see:

[ChatMol] AI chat interface loaded.
[ChatMol]   - Type plain English into the PyMOL command line, e.g.
[ChatMol]       fetch 1abc and show it as a cartoon coloured by chain
[ChatMol]   - Or use the explicit command:  ai <request>
[ChatMol]   - Settings:  chatmol_config [key [value]]
[ChatMol]   - Forget conversation context:  chatmol_reset

Then try it:

PyMOL> fetch 1ubq, async=0
PyMOL> colour the protein by chain and show it as cartoon

3. Usage

Implicit (command-line box)

Just type natural language into PyMOL’s command line and press Enter:

PyMOL> load the protein in ~/Downloads/kinase.pdb and colour it by chain
PyMOL> zoom in on the active site, residues 45 to 60 of chain A
PyMOL> show the ligand as sticks coloured by element with green carbons
PyMOL> make a high resolution image and save it to ~/Desktop/render.png
PyMOL> mutate residue 100 of chain A to alanine

Valid PyMOL/Python lines (e.g. color red, chain A, zoom, @script.pml,
util.cbc, lines starting with #, /, @) are always passed straight
through to PyMOL, untouched.

Explicit

If auto-detection is off, or you just prefer to be explicit:

PyMOL> ai colour everything by secondary structure
PyMOL> chatmol show a surface around the binding pocket, transparent

Multi-turn context

ChatMol keeps a short rolling history of your requests and the commands it
generated, so follow-ups like “now make it bigger” or “undo the surface,
show sticks instead” have context. Clear it with:

PyMOL> chatmol_reset

4. Configuration

All settings are stored in ~/.chatmol/config.json and can be viewed/changed
with chatmol_config, or via Plugin > ChatMol Settings if installed
through the Plugin Manager.

PyMOL> chatmol_config                       # list everything
PyMOL> chatmol_config model                 # show one value
PyMOL> chatmol_config model, qwen3:0.6b     # change the model
PyMOL> chatmol_config host, http://localhost:11434
PyMOL> chatmol_config temperature, 0.1
PyMOL> chatmol_config auto, off             # disable command-line auto-translation
PyMOL> chatmol_config debug, on             # print raw LLM output (troubleshooting)

5. How it works

chatmol/
├── __init__.py        plugin entry point: registers commands, installs hook
├── config.py          JSON-backed settings (~/.chatmol/config.json)
├── ollama_client.py   minimal urllib client for POST /api/chat
├── pymol_reference.py system prompt + curated PyMOL command cheat-sheet
├── nl_router.py       PyMOL-vs-English heuristic, cmd.do hook, translation
├── commands.py        `ai`, `chatmol_config`, `chatmol_reset` commands
└── gui.py             Plugin-menu settings dialog (Qt)

1. Hooking the command line. PyMOL’s GUI command-line box (and
   @script.pml, menu actions, etc.) all funnel through cmd.do(line).
   ChatMol saves the original cmd.do and replaces it with a wrapper.

2. Deciding what’s “PyMOL” vs “English”. nl_router.looks_like_pymol()
   checks whether the first token of the line is a known PyMOL command (in
   cmd.keyword, which covers both built-ins and anything registered via
   cmd.extend, e.g. ChatMol’s own chatmol_config), a known namespace
   (util, pymol, _self, …), or a Python keyword. If the word
   following the command looks like an English article/preposition/etc.
   (e.g. “show the ligand…”) or the line contains connector words like
   “except”/”but”/”without”, it’s treated as English instead. As a second
   safety net, if a line doesn’t look like PyMOL or English, it’s passed
   through unchanged so odd internal strings aren’t sent to the LLM.

3. Translation. Anything left over is sent to Ollama
   (ollama_client.chat) with:

4. a system prompt describing the required output format and a curated
   PyMOL command reference (pymol_reference.PYMOL_CHEAT_SHEET),
5. a snapshot of the current session (cmd.get_names, chains per
   object), so “the protein” / “it” / “chain A” resolve correctly,
6. the last few turns of conversation for follow-up requests.

The model is instructed to output only PyMOL command lines (or a
# comment if the request can’t be done), one per line, and to pass
already-valid PyMOL syntax through unchanged — a safety net for cases
the heuristic misclassifies. Generation is capped (num_predict) and any
response that comes back too long or repetitive is discarded with a
friendly message rather than executed or remembered, so a bad response
from a small model can’t snowball into garbage for every later request.

1. Execution. Each returned line is echoed as [ChatMol] > ... and run
   through the original cmd.do, so results are indistinguishable from
   typing the command yourself (including normal PyMOL error messages if a
   translated command is wrong).

6. Safety notes

• ChatMol executes whatever the model returns. A bad translation could run
  a destructive command (e.g. delete all, overwriting a file with save).
  Review the echoed [ChatMol] > ... lines, especially for destructive
  requests, and use chatmol_config auto, off + the explicit ai command
  if you want a manual checkpoint before every translation.
• Everything runs locally against your Ollama server — no data leaves your
  machine.
• chatmol_config debug, on is useful when a translation looks wrong: it
  prints the model’s raw output (including any stripped <think> blocks
  or code fences) so you can see what was sent to PyMOL.

7. Troubleshooting

Could not reach Ollama at http://localhost:11434 ...
Start Ollama (ollama serve or open the desktop app) and confirm the model
is pulled (ollama pull qwen3.6:27b, check with ollama list).

Ollama at ... did not respond to model '...' within 120s
The model took too long to respond — this is caught gracefully and won’t
crash PyMOL. Common causes: the model is still loading into memory (try
again), the machine is under heavy load (e.g. another ollama pull running
in the background), or the model is just slow for your hardware. Increase
chatmol_config timeout, 240 or switch to a smaller/faster model.

The model's response looked malformed or repetitive ... and was
discarded
Small models occasionally loop (e.g. repeating enable all; enable all; ...)
instead of producing a short translation. ChatMol detects this, discards the
response, and — importantly — does not add it to conversation history (so
it can’t poison later requests). Just try again, rephrase, or switch to a
larger model (chatmol_config model, qwen3.6:27b).

Nothing happens / commands look wrong
Turn on debug mode (chatmol_config debug, on) and re-run the request to
see the model’s raw output. Try a larger model for better translation
quality, or lower temperature for more deterministic output.

A plain command got sent to the LLM by mistake
The heuristic in nl_router.looks_like_pymol() is conservative but not
perfect. The system prompt asks the model to pass valid PyMOL syntax
through unchanged, so this is usually harmless — but if you hit a
recurring case, add the command/namespace to _EXTRA_KEYWORDS in
nl_router.py.

Disable everything temporarily
chatmol_config auto, off stops the command line from auto-translating;
ai <request> / chatmol <request> still work explicitly.

8. Extending

• More commands / better translations: extend
  pymol_reference.PYMOL_CHEAT_SHEET with examples for operations you use
  often (e.g. project-specific selection macros).
• Session-aware context: pymol_reference.build_context() is the place
  to add more state (current view, sequence, B-factor ranges, etc.).
• Confirmation step: wrap nl_router.run_natural_language‘s execution
  loop with a prompt/dialog if you want to approve translated commands
  before they run.
• Streaming / richer UI: a dedicated chat panel could be added in
  gui.py using pymol.Qt for a conversational view alongside the command
  line.

👤 Author

Marc C. Deller, D.Phil.
Structural biologist & drug discovery scientist