LispLoom

LispLoom is a small, educational decoder-only LLM inference engine written in Common Lisp. Its ASDF system and command-line interface are named cl-llm. It loads selected LLaMA/Qwen-style GGUF models, keeps quantized weights memory-mapped, runs CPU inference through portable Lisp or native AVX2/OpenMP kernels, and exposes a persistent local assistant in the terminal.

Release status: v0.1.0-alpha. The supported path is SBCL on x86-64 Windows or Linux. This is an inference project, not a replacement for llama.cpp, and its model/quantization compatibility is intentionally narrow.

What Is Included

• Decoder-only transformer inference with RoPE, GQA, SwiGLU, RMSNorm, and KV cache
• Native GPT-2/Qwen2 BPE and SentencePiece-style tokenization from GGUF metadata
• Direct Q8_0, Q4_K, and Q6_K matrix operations without full dequantization
• Memory-mapped GGUF tensors on Windows and Linux
• AVX2/FMA/F16C kernels, OpenMP row parallelism, and batched prompt prefill
• Streaming ChatML REPL with 2K context management and reusable system-prefix KV
• Saved sessions, local BM25-style document retrieval, and chat commands
• Optional grounded web retrieval with bounded evidence and structured weather data
• Model downloader with HTTPS-only redirects, resume, size checks, and SHA-256
• Portable Common Lisp reference math and a cross-platform test suite

The repository does not contain SBCL, a standalone executable, or model weights. You install and configure SBCL yourself.

Requirements

• 64-bit SBCL. Versions 2.0.0 (Windows) and 2.2.9 (Linux) are tested.
• x86-64 CPU with AVX2, FMA, and F16C for the optimized backend
• curl
• GCC with OpenMP support to build the native kernels
• certutil on Windows or sha256sum/shasum on Linux
• About 4 GiB free RAM for Fast, 5 GiB for Quality, plus model storage

The portable backend documents and tests the math, but it is too slow for a useful interactive chat with the bundled profiles. The CLI therefore requires the native backend for chat.

Quick Start: Windows

Install and configure SBCL, GCC, and curl first. SBCL may be on PATH, or its executable may be selected with CL_LLM_SBCL. Configure SBCL_HOME yourself if your SBCL distribution requires it.

git clone https://github.com/akluth/LispLoom.git
cd LispLoom

# Optional when sbcl.exe is not on PATH:
$env:CL_LLM_SBCL = 'C:\path\to\sbcl.exe'

.\cl-llm.ps1 doctor
.\cl-llm.ps1 setup
.\cl-llm.ps1 chat

setup compiles native/cl_llm_kernels.dll, downloads the Fast model directly from its publisher, resumes interrupted transfers, and verifies its exact size and SHA-256 hash.

If local PowerShell policy blocks scripts, invoke the wrapper explicitly:

powershell -ExecutionPolicy Bypass -File .\cl-llm.ps1 doctor

Quick Start: Linux

Install and configure SBCL, GCC/OpenMP, curl, and sha256sum first.

git clone https://github.com/akluth/LispLoom.git
cd LispLoom
chmod +x cl-llm tools/build-native.sh

# Optional when sbcl is not on PATH:
export CL_LLM_SBCL=/path/to/sbcl

./cl-llm doctor
./cl-llm setup
./cl-llm chat

setup builds native/libcl_llm_kernels.so and installs the Fast model under the current user's data directory.

Both wrappers are thin launchers. The equivalent direct invocation is:

sbcl --dynamic-space-size 4096 --script run.lisp doctor

Model Profiles

Models are downloaded directly from Hugging Face and are never redistributed inside source or release archives.

Profile	Model	File size	License	Default
fast	Qwen2.5-1.5B-Instruct Q8_0	1.76 GiB	Apache-2.0	Yes
quality	Qwen2.5-3B-Instruct Q4_K_M	1.96 GiB	Qwen Research License, non-commercial	No

The Quality profile is deliberately opt-in. Read its upstream license before installing it, then explicitly record acceptance:

.\cl-llm.ps1 model install quality --accept-license
.\cl-llm.ps1 chat --profile quality

./cl-llm model install quality --accept-license
./cl-llm chat --profile quality

Acceptance is stored as a small receipt in the user data directory. It does not change the model's terms. See Model policy and third-party notices.

CLI

cl-llm doctor [--deep]
cl-llm setup [--profile fast|quality] [--accept-license]
             [--no-native] [--no-model] [--force]
cl-llm model list
cl-llm model install fast|quality [--accept-license] [--force]
cl-llm model verify fast|quality
cl-llm model path fast|quality
cl-llm chat [--profile fast|quality] [--model PATH]
            [--context N] [--threads N] [--temperature F]
            [--top-k N] [--max-tokens N] [--accept-license]
cl-llm test
cl-llm version

doctor --deep additionally hashes installed multi-gigabyte model files. --force restarts a model download instead of resuming its .part file.

Assistant Commands

Inside chat:

/help                 show commands
/stats                context, retrieval, and timing statistics
/reset                clear chat and KV state
/save [PATH]          save conversation and settings
/load PATH            load a saved session
/system [TEXT]        show or replace the system prompt
/docs add PATH        index a local text file or directory
/docs list            list indexed files
/docs clear           clear the document index
/web status           show web provider and mode
/web auto|always|off  change web retrieval behavior
/web test QUERY       inspect normalized search results
/web evidence QUERY   inspect the bounded evidence pack
/web cache-clear      clear the persistent web cache
/temperature N        set sampling temperature
/top-k N              set top-k sampling
/tokens N             set the answer token limit
/quit                 close the model and exit

With no path, /save writes last.clchat under the platform session directory. Session files contain text and settings, not model weights, raw KV arrays, or native pointers.

Data Locations

Default writable data:

• Windows: %LOCALAPPDATA%\cl-llm\
• Linux: $XDG_DATA_HOME/cl-llm/ or ~/.local/share/cl-llm/

Environment overrides:

Variable	Purpose
CL_LLM_SBCL	SBCL executable used by the wrappers
SBCL_HOME	SBCL core/contrib directory, when required by your installation
CL_LLM_DYNAMIC_SPACE_MB	SBCL dynamic-space size; default 4096
CL_LLM_HOME	Root for writable cl-llm data
CL_LLM_MODEL_DIR	Model directory only
CL_LLM_THREADS	Native OpenMP thread count; default is capped at 6
CL_LLM_CC	C compiler used by the native build scripts
CL_LLM_CURL	curl executable override
CL_LLM_NATIVE_LIBRARY	Native DLL/shared-object override

Optional Web Retrieval

Network access is off when no provider is configured. A model can never enable it by itself. Configure a provider before starting chat.

Brave Search:

export CL_LLM_WEB_PROVIDER=brave
export BRAVE_SEARCH_API_KEY='...'

SearXNG:

export CL_LLM_WEB_PROVIDER=searxng
export CL_LLM_SEARXNG_URL='https://search.example.org/search'

Use $env:NAME = 'value' for the same variables in PowerShell. Set CL_LLM_WEB_PROVIDER=off to fail closed even when credentials exist. The web cache can be relocated with CL_LLM_WEB_CACHE or disabled by setting it to off.

Retrieved pages are treated as untrusted data. The runtime restricts protocols, ports, resolved addresses, redirects, response size, content type, and evidence length. Current-data failures are surfaced instead of silently falling back to an unsupported model guess. Details are in Architecture and Security.

Performance

On an Intel Core i7-1255U with six native threads and a 2K context, prior local measurements produced approximately:

Profile	Prompt prefill	Decode	Peak working set
Fast	14.3 tokens/s	7.5 steps/s	1.82 GiB
Quality	6.7 tokens/s	3.8 steps/s	2.11 GiB

These are workload-specific measurements, not guarantees. Reproduce them with the scripts described in Benchmarks.

Compatibility and Limits

• Supported release targets: SBCL, x86-64 Windows, and x86-64 Linux
• Optimized quantization: Q8_0, Q4_K, and Q6_K
• Tested chat architectures: Qwen2 and LLaMA/SmolLM-style models mapped by the loader
• No CUDA, Vulkan, Metal, distributed inference, training runtime, or generic GGUF promise
• 2K is the recommended assistant context for the included profiles
• Small local models can still make factual or reasoning errors; retrieval reduces unsupported current claims but does not make generation infallible
• Arbitrary URLs are not a public fetch API; web retrieval accepts search results through a constrained, validated path

Development

Run the full suite after building native kernels:

.\cl-llm.ps1 setup --no-model
.\cl-llm.ps1 test

./cl-llm setup --no-model
./cl-llm test

The test suite uses generated tiny tensors and does not download external model weights. See Contributing, Architecture, and Benchmarks.

License

Project source is available under the MIT License. Model weights and services have their own terms. See THIRD_PARTY_NOTICES.md.