ChemSage: Vibe-Coding a Chemically Aware Drug-Discovery LLM on Local Hardware
ChemSage: Vibe-Coding a Chemically Aware Drug-Discovery LLM on Local Hardware
- Hybrid by design: Retrieval for facts, fine-tuned weights supply behavior, and tools including PyMOL, PDBTOOLS, Biopython, ChEMBL, PubChem, and RDKit ensure chemical truth.
- 78 Data Classes: Covering structural biology, MedChem, computational chemistry, protein families, and drug-target knowledge.
- 13 Metric evaluation script: Grounded truths beat guessing, teaching model to run tools lifted Round 5 to 95% overall against Round 1’s 72%, with tool executability climbing from 0% to 95%.
- Unified memory removes the wall: Apple Silicon fine-tunes a 32B model on old M1 MacBook Pro, no discrete GPU required.
- Data quality beats compute: Validated training examples drove validation loss from 0.389 down to 0.041.
- Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU: Apple Machine Learning Research, 2025
- MLX: An Array Framework for Apple Silicon: Apple ml-explore, 2026
- MLX-LM: LoRA, Fuse and Serve on Apple Silicon: ml-explore, GitHub, 2026
- QLoRA: Efficient Finetuning of Quantized LLMs: Dettmers et al., NeurIPS, 2023
- LoRA: Low-Rank Adaptation of Large Language Models: Hu et al., arXiv, 2021
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: Lewis et al., NeurIPS, 2020
- ChemSage: A Hybrid, Chemically Aware LLM for Drug Discovery (Project Plan): Deller, GitHub, 2026
- ChemSage: Multi-Round Comparative Evaluation: Deller, marcdeller.com, 2026
How a structural biologist fine-tuned, grounded, and evaluated a 32-billion-parameter chemistry assistant entirely on Apple Silicon
Why build it?
Hosted frontier models are convenient until you meet their ceilings: token caps, document-upload limits, and the simple fact that they cannot see your private SAR data. The starting question for ChemSage was a practical one: if a model has to run somewhere, why not run it on the Mac already sitting on the desk? The spark came from @ai.christianson, whose TikTok video of MLX-LM fine-tuning on Apple hardware showed the path.
The rationale is about correctness, not just cost. Off-the-shelf models invent chemistry: a confidently wrong logP, a malformed SMILES, a property quoted from memory rather than computed. ChemSage answers that by separating three concerns that are usually conflated. Volatile knowledge (your assays, dossiers, papers) lives in retrieval; stable behavior (when and how to reach for a tool, house output style) lives in the fine-tuned weights; and deterministic chemical facts are never guessed at all. They are computed live by RDKit and PyMOL, the same tools a medicinal chemist already trusts.
“Chemical correctness is delegated to deterministic tools that already exist and are trusted.” – ChemSage Project Plan, 2026
A note on provenance, since it matters here: I am a structural biologist, not a professional software engineer. ChemSage is a vibe-coding exercise, written to find out how far a subject-matter expert can get with their own data, moderate desktop hardware, and very little money. The chemistry and biology are still under validation, so treat it as a proof of concept rather than a finished product. The full build log, with the warts left in, sits on the making-of page.
How it works
Two failure modes kill most custom-chemistry-LLM projects, and ChemSage is designed around both. The first is the model inventing numbers: the fix is not to fine-tune harder but to make the model emit tool calls and reason over the returned values. The second is treating facts and behavior as the same thing. Keeping facts in retrieval and behavior in the weights means you re-tune only when you hit a behavioral gap that retrieval cannot fill. New papers and SAR tables just get ingested.
The request loop is easy to read even if you do not write code. Retrieval supplies the facts, the tuned model supplies the behavior and any tool code, and a sandboxed shim supplies the deterministic chemical truth before anything reaches you.
on user_query:
context = rag.retrieve(query) # facts from your own corpus
response = model.generate(query, context) # tuned behaviour; may emit tool code
for block in response.tool_calls: # RDKit / PyMOL / docking
result = sandbox.run(block) # deterministic, validated
response = model.ground(response, result)
return response # grounded, numerically correct
The architecture (from the project plan) wires those steps together as a single local pipeline, served behind an OpenAI-compatible endpoint so any GUI can talk to it.
user +---------------------------------------------+
query->| GUI front end (AnythingLLM / Open WebUI) |
+----------------------+----------------------+
v
+---------------------------------------------+
| RAG retriever (Chroma + embeddings) | <-- ingested corpus (papers, SAR, dossiers)
+----------------------+----------------------+
v query + retrieved context
+---------------------------------------------+
| mlx_lm.server -> fused QLoRA model (MLX)|
+----------------------+----------------------+
v may emit a tool call (RDKit / PyMOL / dock)
+---------------------------------------------+
| tool-exec shim (sandboxed) |
+----------------------+----------------------+
v validated results (canonical SMILES, props, RMSD)
grounded answer
The stack, and why MLX-LM?
The whole thing is pure Python on Apple Silicon. The dependency list is short, which is part of the point.
# Fine-tune + serve: MLX-LM (Apple Silicon / Metal)
mlx-lm # Apple Silicon-native LLM toolkit: LoRA fine-tune, fuse adapters, and run local server
# Core / shared
rdkit # Cheminformatics toolkit for molecular representations, descriptors, and chemistry ground truth
transformers>=4.46 # Hugging Face model + tokenizer stack for model zoo access and config/weights handling
pandas # Tabular data handling, corpus metadata, logs, and evaluation outputs
numpy # Array math backbone for tensors, preprocessing, and numerical utilities
pyyaml # Human-readable config management for experiments, models, and pipelines
# RAG layer
chromadb # Local/remote embedding database for vector storage, filtering, and similarity search
sentence-transformers # Local embedding models for document/query vectors feeding the RAG index
pypdf # PDF ingestion: split pages, extract text, and build structured document chunks for RAG
# Chat CLI + eval client
rich # Structured, colorful terminal output for chat UI, logs, and evaluation reports
prompt_toolkit # Interactive command-line interface: input editing, history, and keybindings for the chat client
requests # HTTP client for talking to remote MLX-LM servers, RAG APIs, or evaluation services
tqdm # Progress bars for training loops, corpus ingestion, and batch evaluation runs
psutil # System resource monitoring to keep Apple Silicon runs and RAG indexing within hardware limits
A handful of scripts carry the load. ingest_rag.py chunks, embeds and stores the corpus; build_dataset.py generates and validates the training data; train_qlora.py wraps mlx_lm.lora; merge_export.py fuses the adapter and serves it; tool_exec.py is the sandboxed RDKit and PyMOL runner; and eval_chem.py scores the result. The engine underneath all of it is MLX, Apple’s array framework, and its MLX-LM companion for language models.
MLX matters because of how Apple Silicon handles memory. A discrete NVIDIA card forces a hard VRAM wall and constant copying between CPU and GPU memory. Apple’s unified-memory architecture removes both: the CPU and GPU share one pool, so there are no memory copies and no separate VRAM budget. In practice, that means a 16 GB Mac comfortably tunes an 8B model, a 32 GB machine handles roughly a 14B model, and a 64 GB machine reaches a full 32B model. The trade-off against NVIDIA is wall-clock speed, not capability, and because the base model is already a 4-bit checkpoint, the fine-tune runs automatically as QLoRA. Apple’s own write-up on MLX and the M5 neural accelerators sets out the same case in more detail.
The training corpus
The dataset is where this project succeeds or fails, so the rules are strict. Every numeric answer is generated by first running RDKit to compute the real value, then templating the question and the worked solution around it, never by hand-authoring a property. Each example is validated on write: if its code block fails to run or its SMILES will not canonicalize, it is rejected. SMILES are kept inside fenced code blocks so the model treats them as literals it passes to a tool, and the splits are an 80/10/10 train/validation/frozen-test partition. The supervised set grew across rounds from 1,500 examples to 20,000, spanning 78 behavior classes, alongside a 55,742-chunk retrieval corpus drawn from ChEMBL, PubChem and the PDB. The data layout keeps the rounds separate and the test set untouched.
data/
|-- corpus/ # RAG sources: PDFs, SAR CSVs, dossiers (gitignored)
|-- sft/ # Rounds 1-4 SFT data
| |-- train.jsonl # 6,400 examples (Round 4)
| |-- valid.jsonl # 800 examples
| '-- test.jsonl # 800 examples, frozen, eval only
'-- sft_r5/ # Round 5 SFT data
|-- train.jsonl # 16,000 examples
|-- valid.jsonl # 2,000 examples
'-- test.jsonl # 2,000 examples, frozen, eval only
Five rounds, and how they were judged
ChemSage went through five training rounds. Round 1 was a 7B proof of concept; Round 2 moved to a 32B base; Rounds 3 to 5 grew the dataset and the behavior coverage, with Round 5 doubling the LoRA rank for more adapter capacity. The five fine-tuned models are published on Hugging Face. Full stats are available from the HTML output of eval_compare.py.
| Round | Base model | Trainable params | Train examples | Behaviour classes |
|---|---|---|---|---|
| 1 | Qwen2.5-7B-Instruct-4bit | 26 M (0.34%) | 1,200 | 8 |
| 2 | Qwen2.5-32B-Instruct-4bit | 134 M (0.41%) | 1,200 | 8 |
| 3 | Qwen2.5-32B-Instruct-4bit | 134 M (0.41%) | 4,000 | 19 |
| 4 | Qwen2.5-32B-Instruct-4bit | 134 M (0.41%) | 6,400 | 59 |
| 5 | Qwen2.5-32B-Instruct-4bit | 268 M (0.82%) | 16,000 | 78 |
Table 1. Model lineage. Each 32B round trains less than 1% of the parameters via QLoRA on a 4-bit base.
Validation loss tells the training story. Better data, not more compute, drove the gains: Round 3’s RDKit-validated, code-heavy examples cut the best validation loss to 0.054, an order of magnitude below the early rounds, and Round 4 reached 0.041. Round 5 settled slightly higher at 0.055, which is the expected cost of a much harder task (78 behavior classes against 59). Every model fits and trains inside the unified memory of a single 64 GB machine.
| Round | Best val loss | Iters (actual) | Peak train memory | Fused size |
|---|---|---|---|---|
| 1 | 0.389 @ iter 675 | 750 | 12.0 GB | 4.2 GB |
| 2 | 0.347 @ iter 675 | 750 | 26.1 GB | 17.0 GB |
| 3 | 0.054 @ iter 625 | 625 | 25.2 GB | 17.0 GB |
| 4 | 0.041 @ iter 950 | 1,500 | 25.5 GB | 17.2 GB |
| 5 | 0.055 @ iter 1,600 | 2,000 | 30.3 GB | 17.2 GB |
Table 2. Training outcomes. Rounds 3 and 5 stopped early once validation loss plateaued.
Generic LLM benchmarks miss what matters here, so ChemSage is scored with a chemistry-aware evaluation harness that checks whether emitted SMILES parse, whether tool code actually runs, and whether stated property values match an RDKit recompute. The comparison below runs all five rounds on the same 100 held-out test examples, scoring each example (an example passes a metric only if every instance in the response is correct). Round 5 leads at 95% overall, with the largest gains on the things that matter most: tool executability, numerical fidelity, and code-then-quote transcription.
| Metric (pass rate) | R1 | R2 | R3 | R4 | R5 | Δ vs R1 |
|---|---|---|---|---|---|---|
| SMILES validity | 100% | 100% | 100% | 100% | 100% | +0% |
| SMARTS validity | n/a | 0% | 100% | 55% | 100% | n/a |
| Tool executability | 0% | 28% | 79% | 71% | 95% | +95% |
| Code attempted | 14% | 41% | 100% | 97% | 100% | +86% |
| Python exec (extended) | 36% | 34% | 78% | 69% | 99% | +63% |
| Code-to-quote fidelity | n/a | 0% | 47% | 19% | 61% | n/a |
| Numerical fidelity | n/a | 18% | 57% | 47% | 89% | n/a |
| Rounding precision | 100% | 100% | 98% | 98% | 99% | -1% |
| Refusal accuracy | 98% | 98% | 97% | 98% | 100% | +2% |
| QED range | 100% | 100% | 100% | 100% | 100% | +0% |
| PDB ID validity | 100% | 100% | 100% | 100% | 100% | +0% |
| PyMOL syntax | 77% | 97% | 89% | 89% | 90% | +13% |
| Degeneration-free | 93% | 96% | 91% | 98% | 100% | +7% |
| Overall | 72% | 63% | 87% | 80% | 95% | +23% |
Table 3. Comparative evaluation across all 13 metrics, per-example pass rate over 100 shared test examples. The Δ column is Round 5 against the Round 1 baseline. Metrics show n/a where Round 1 never triggered that check, so no baseline exists.
Is the AI bubble coming?
Here is the part that nags at me. The economics of hosted frontier models do not obviously add up. The subscription you pay and the compute you consume are increasingly different numbers. A recent Claude Code session of mine is a clear illustration: a $ 20-per-month plan against roughly $57 in metered API-equivalent usage consumed in about 4 hours of work.
Multiply that across millions of users, and the question writes itself. When the headline subscription sits well below the marginal cost of serving heavy users, the gap is being filled by investment rather than revenue, and that is the shape of a bubble. None of this means the frontier labs vanish: the hardest reasoning, the largest contexts, and the newest capabilities will keep living in big data centers, and for plenty of work, that is exactly where they belong. The more interesting shift is at the other end. A great deal of routine, sensitive, or latency-critical work does not need a frontier model at all, and it has good reasons to come home: cost you control, data that never leaves the building, and no rate limit between you and your own files.
That is the quiet argument ChemSage is really making. A capable 32B model that trains, serves, and reasons on a desktop, grounded in your own corpus and checked by your own tools, is no longer exotic. With Apple now putting neural accelerators into consumer chips, the on-device floor keeps rising. The future is unlikely to be all-cloud or all-local. It looks more like a split: frontier models in the cloud for the genuinely hard problems, and a growing tier of specialist, device-embedded models doing the daily work, privately and at the cost of electricity.
How to run…
Run these 2 commands in separate terminals, first start the model server, then run chat.py.
Note: the –model flag on chat.py sets the model name shown in /info but the server is what actually runs inference — make sure the server is fully loaded (you’ll see “HTTP server listening”) before starting chat. The server defaults to port 8080 so no extra flags needed on the chat side.
#Terminal 1 — start the model server:
cd /Users/dellboy/Documents/Vibe_Coding/chem_sage
source .venv/bin/activate
mlx_lm.server --model models/chem_sage_32b_v5 --port 8080
#Terminal 2 — run chat.py:
cd /Users/dellboy/Documents/Vibe_Coding/chem_sage
source .venv/bin/activate
python scripts/chat.py --model models/chem_sage_32b_v5
What comes next?
Two threads are open. The first is getting ChemSage in front of people: wrapping the chat CLI (chat.py) in a Gradio interface hosted on marcdeller.com, standing up a lightweight Hugging Face Spaces demo, exposing a local endpoint for live talks through a Cloudflare Tunnel, and packaging the client as a pipx-installable tool so anyone with a local Qwen 32B can point it at their own server. The second is a planned Round 6 aimed squarely at the residual weak spots. Distribution shift, not raw example count, caused Round 4’s regressions, so the priority is weighted class-balanced sampling across all 78 classes to lock the fragile metrics in place. Beyond that: triple the code-to-quote drill (still the weakest metric), add a dedicated PyMOL syntax generator, try a two-phase curriculum that follows a full run with a short fidelity-focused continuation, raise the LoRA rank to 96, and run a per-class evaluation on a mid-training checkpoint so class-level dilution shows up after an hour rather than after forty.
To Do List
Lots to learn with some ideas below, and in all honesty, I don’t fully understand what all these mean yet! The joys of Vibe Coding! Build quick, learn later!
- Host on my lightweight DigitalOcean Droplet at mdeller.com?
- Serve via Ollama or Web-UI?
- HuggingFace Spaces hosting through Cloudflare tunnel?
- Build a single-file package installer?
- Round 6 refined training?
- Weighted class-balanced sampling
- Improve code-to-quote corpus
- Pymol syntax generator
- Raise to 96 layers
- Two-phase curriculum
- Mid-training checkpoints