glassllmDocumentation
Open lab
DocumentationLibrary reference

Library reference

API map and conventions

Imports, types, dimensions and contracts of the NumPy glassllm engine.

glassllm 0.1.03 min read · estimate

Choose an entry level

The glassllm package exports 12 names: Config, GenStep, GlassLM, LANGUAGES, STEPS, StepRecord, Tensor, Tokenizer, Tracer, load_pretrained, no_grad and __version__. They cover configuration, generation, loading and model inspection. Mathematical operations, layers, optimizers and training helpers are imported from their submodules.

import numpy as np
from glassllm import Config, GlassLM, Tensor, Tracer, no_grad

cfg = Config(vocab_size=8, d_model=8, n_layers=1,
             n_heads=2, n_kv_heads=1, max_len=4)
model = GlassLM(cfg, seed=0)
trace = Tracer()
with no_grad():
    logits = model([1, 2, 3], trace=trace)
assert logits.shape == (1, 3, 8)
assert not logits.requires_grad
np.testing.assert_allclose(
    trace.get("attn.softmax", layer=0, head=0).value.sum(-1), 1.0,
    rtol=1e-5,
)

This example uses random weights to check contracts. To generate with trained weights, use model, tokenizer = load_pretrained("en"). Library publication remains deferred, so installation uses the checkout or a local wheel.

Import paths

Module Symbols and purpose
glassllm.config Config: dimensions, limits and validation
glassllm.model GlassLM, GenStep, next_token_probs: logits, loss and generation
glassllm.tokenizer Tokenizer, pre_tokenize, SPACE, SPECIALS, UNK_TEXT: custom BPE
glassllm.trace Tracer, StepRecord, record: copied snapshots
glassllm.steps StepInfo, STEPS, LAYER_STEPS, HEAD_STEPS, scan_markers, load_step_index: equations and source locations
glassllm.autograd Tensor, no_grad and differentiable operations: reverse graph
glassllm.nn Module, RMSNorm, Attention, SwiGLU, Block, KVCache, parameter, mask and angle helpers
glassllm.optim AdamW, clip_grad_norm, lr_schedule: parameter updates
glassllm.train get_batch, evaluate, train: windows, evaluation and training
glassllm.cli main, build_parser, format_matrix, cmd_* handlers: terminal

Names starting with _ are internal helpers. This is not the PyTorch module interface: Module has no .train(), .eval(), .to() or .state_dict(). The engine runs on NumPy; the browser runs the same Python through Pyodide.

Shape and indexing conventions

Symbol Meaning Included model default
B batch 1 in the interface and trace
n positions received by this call up to 128
m key positions, including cache up to 128
V vocabulary size 1024
d model dimension 128
Hq, Hkv query and key/value heads 4 and 2
dk = d / Hq head dimension 32

IDs and targets are integer indices; positions, layers and heads are zero based. forward accepts (n,) or (B,n) IDs and returns a (B,n,V) Tensor. The batch axis remains even when the input was a list. The tracer removes a unit batch axis: a full attention record is (Hq,n,m), or (n,m) after head selection. The tokenizer does not automatically add BOS/EOS.

Types, state and errors

Tensor.data is a numpy.ndarray; Tensor.grad starts as None. Parameters usually use float32; derivative examples may use float64. There is no GPU device, symbolic type system or second-order differentiation. Direct NumPy operations on .data are outside the graph.

forward, loss and generate do not update weights. train and AdamW.step modify the existing object; forward also fills a supplied cache. generate owns its cache and creates a fresh tracer for each token when trace=True.

Explicit validation primarily raises ValueError, TypeError, KeyError and FileNotFoundError. Incompatible shapes, out-of-range IDs and malformed files can also raise NumPy or JSON errors. The library does not wrap every internal exception in one error class. Because generate is a generator, its validation runs only when iteration advances.

Continue with Configuration and loading, GlassLM or Tensor and autograd.

Signatures in the code

12 root exports; 64 class/function declarations in 11 modules. Open a module to inspect methods, operators and properties.

glassllm1

src/glassllm/__init__.py

  • load_pretrained(lang: str, model_dir: str | Path | None=None) -> tuple[GlassLM, Tokenizer]
glassllm.autograd28

src/glassllm/autograd.py

  • no_grad()
  • Tensor
    • __init__(self, data, parents=(), backward=None, requires_grad=False)
    • shape(self)
    • ndim(self)
    • backward(self, grad=None)
    • __add__(self, other)
    • __radd__ = __add__
    • __sub__(self, other)
    • __rsub__(self, other)
    • __mul__(self, other)
    • __rmul__ = __mul__
    • __truediv__(self, other)
    • __rtruediv__(self, other)
    • __neg__(self)
    • __pow__(self, exponent)
    • __matmul__(self, other)
    • __getitem__(self, index)
    • sum(self, axis=None, keepdims=False)
    • mean(self, axis=None, keepdims=False)
    • max(self, axis=None, keepdims=False)
    • reshape(self, *shape)
    • transpose(self, *axes)
  • add(a, b)
  • neg(a)
  • mul(a, b)
  • div(a, b)
  • power(a, exponent)
  • matmul(a, b)
  • exp(a)
  • log(a)
  • tsum(a, axis=None, keepdims=False)
  • mean(a, axis=None, keepdims=False)
  • max(a, axis=None, keepdims=False)
  • reshape(a, shape)
  • transpose(a, axes)
  • concat(tensors, axis=0)
  • where(condition, a, b)
  • embedding(weight, ids)
  • masked_fill(a, mask, value)
  • softmax_np(x, axis=-1)
  • softmax(a, axis=-1)
  • log_softmax(a, axis=-1)
  • sigmoid(a)
  • silu(a)
  • cross_entropy_grad(logits, targets)
  • cross_entropy(logits, targets)
  • rope(x, cos, sin)
  • repeat_kv(x, repeats)
glassllm.cli8

src/glassllm/cli.py

  • format_matrix(values: np.ndarray, row_labels: list[str], col_labels: list[str] | None=None) -> str
  • cmd_tokenize(args, model: GlassLM | None, tok: Tokenizer) -> None
  • cmd_generate(args, model: GlassLM, tok: Tokenizer) -> None
  • cmd_trace(args, model: GlassLM, tok: Tokenizer) -> None
  • cmd_steps(args, model, tok) -> None
  • cmd_train(args, model: GlassLM, tok: Tokenizer) -> None
  • build_parser() -> argparse.ArgumentParser
  • main(argv: list[str] | None=None) -> None
glassllm.config1

src/glassllm/config.py

  • Config
    • vocab_size: int
    • d_model: int
    • n_layers: int
    • n_heads: int
    • n_kv_heads: int
    • max_len: int
    • rope_base: float
    • eps: float
    • init_std: float
glassllm.model3

src/glassllm/model.py

  • GenStep
    • token: int
    • probs: np.ndarray
    • trace: Tracer | None
  • next_token_probs(logits: np.ndarray, temperature: float) -> np.ndarray
  • GlassLM
    • __init__(self, cfg: Config, seed: int=0, dtype=np.float32)
    • new_cache(self) -> list[KVCache]
    • forward(self, ids, trace: Tracer | None=None, cache: list[KVCache] | None=None, start: int=0) -> Tensor
    • __call__ = forward
    • loss(self, ids, targets, trace: Tracer | None=None) -> Tensor
    • generate(self, ids, n_new: int, temperature: float=1.0, greedy: bool=False, seed: int | None=None, trace: bool=False) -> Iterator[GenStep]
    • save(self, path: str | Path, fp16: bool=True) -> None
    • load(cls, path: str | Path, dtype=np.float32) -> GlassLM
glassllm.nn9

src/glassllm/nn.py

  • Module
    • named_parameters(self, prefix: str='') -> dict[str, Tensor]
    • parameters(self) -> list[Tensor]
  • init_param(rng: np.random.Generator, shape: tuple[int, ...], std: float, dtype) -> Tensor
  • RMSNorm
    • __init__(self, d: int, eps: float, dtype)
    • __call__(self, x: Tensor) -> Tensor
  • rope_angles(d_k: int, max_len: int, base: float) -> tuple[np.ndarray, np.ndarray]
  • causal_mask(n_queries: int, n_keys: int, start: int) -> np.ndarray
  • KVCache
    • __init__(self)
  • Attention
    • __init__(self, d: int, n_heads: int, n_kv_heads: int, max_len: int, rope_base: float, rng: np.random.Generator, std: float, dtype)
    • __call__(self, x: Tensor, layer: int, trace: Tracer | None=None, cache: KVCache | None=None, start: int=0) -> Tensor
  • SwiGLU
    • __init__(self, d: int, rng: np.random.Generator, std: float, dtype)
    • __call__(self, x: Tensor, layer: int, trace: Tracer | None=None) -> Tensor
  • Block
    • __init__(self, cfg, rng: np.random.Generator, dtype)
    • __call__(self, x: Tensor, layer: int, trace: Tracer | None=None, cache: KVCache | None=None, start: int=0) -> Tensor
glassllm.optim3

src/glassllm/optim.py

  • AdamW
    • __init__(self, params: list[Tensor], lr: float=0.003, betas=(0.9, 0.95), eps: float=1e-08, weight_decay: float=0.1)
    • zero_grad(self) -> None
    • step(self, lr: float | None=None) -> None
  • clip_grad_norm(params: list[Tensor], max_norm: float) -> float
  • lr_schedule(step: int, total_steps: int, base_lr: float, warmup: int, min_ratio: float=0.1) -> float
glassllm.steps3

src/glassllm/steps.py

  • StepInfo
    • latex: str
    • plain: str
  • scan_markers(package_dir: Path=PACKAGE_DIR) -> dict[str, dict]
  • load_step_index() -> dict[str, dict]
glassllm.tokenizer2

src/glassllm/tokenizer.py

  • pre_tokenize(text: str) -> list[str]
  • Tokenizer
    • __init__(self, alphabet: list[str], merges: list[tuple[str, str]])
    • vocab_size(self) -> int
    • train(cls, text: str, vocab_size: int) -> Tokenizer
    • encode(self, text: str) -> list[int]
    • encode_verbose(self, text: str) -> list[dict]
    • tokens(self, ids: list[int]) -> list[str]
    • decode(self, ids: list[int]) -> str
    • save(self, path: str | Path) -> None
    • load(cls, path: str | Path) -> Tokenizer
glassllm.trace3

src/glassllm/trace.py

  • StepRecord
    • id: str
    • layer: int | None
    • value: np.ndarray
    • order: int
    • head: int | None
    • shape(self) -> tuple[int, ...]
    • equation(self) -> str
    • latex(self) -> str
  • Tracer
    • __init__(self)
    • record(self, step_id: str, value, layer: int | None=None, batched: bool=True, *, head: int | None=None) -> None
    • get(self, step_id: str, layer: int | None=None, head: int | None=None) -> StepRecord
    • step_ids(self) -> list[str]
  • record(trace: Tracer | None, step_id: str, value, layer: int | None=None, batched: bool=True, *, head: int | None=None) -> None
glassllm.train3

src/glassllm/train.py

  • get_batch(data: np.ndarray, batch_size: int, seq_len: int, rng: np.random.Generator) -> tuple[np.ndarray, np.ndarray]
  • evaluate(model: GlassLM, data: np.ndarray, batch_size: int, seq_len: int, n_batches: int=10, seed: int=1234) -> float
  • train(model: GlassLM, data: np.ndarray, steps: int, batch_size: int=16, seq_len: int=128, lr: float=0.003, warmup: int=100, seed: int=0, val_data: np.ndarray | None=None, eval_every: int=100, keep_best: bool=False, log: Callable[[str], None] | None=print) -> list[dict]

Project sources

src/glassllm/__init__.pysrc/glassllm/