glassllmDocumentation
Open lab

DOCUMENTATION · TRANSFORMER IN NUMPY

See the calculation.
Understand the model.

From the first token to the gradient: explore the network, follow the values and open the code behind each operation.

Start an experimentOpen the lab
4Transformer blocks77calculations per predictionNumPyinspectable engine
A prediction's path through the TransformerTokens enter as vectors. Four blocks combine attention and SwiGLU. The output supplies the next-token distribution. This is a schematic drawing.INSIDE ONE PREDICTIONtextastokensE[ids]vectors · 128 dimensionsTransformer block × 4QKVattention → residuals → SwiGLURMSNorm → Eᵀ1024 logitssoftmax(z / T)next tokenback into context
Schematic model flow. Explore actual values in the lab.

CHOOSE WHERE TO START

Explore. Understand. Experiment.

  1. 01

    Explore with one token

    Generate a prediction, read the tokens and explore the lab controls.

    Interface guideSlow motion
  2. 02

    Understand each calculation

    Connect vectors, attention and rotations to equations and actual values.

    Tokens and BPEPosition and RoPE
  3. 03

    Program with the engine

    Use the same Transformer in Python: inference, snapshots, autograd and training.

    GlassLMTensor and autograd

FOLLOW THE DATA

One model, two paths

Each step opens its explanation

FROM DIAGRAM TO VALUES

Make time for each step.

Enable slow motion, pause at Q, K or V and compare the equation with the computed matrix. The inspector shows dimensions, statistics and the Python line that produced the result.

  • 01Generate an actual token.
  • 02Pause at the calculation you want to study.
  • 03Read the explanation, values and source.
Slow-motion guide
Actual causal-mask inspector: future positions marked −∞, matrix and source line
The causal mask blocks access to future positions. Inspect its actual values.

ONE LIBRARY, TWO WAYS TO EXPLORE

The site uses the same Python engine.

Lab

Controls, diagram and matrices

TypeScript

Web Worker

Python in the browser

Pyodide · wheel

glassllm

Model, autograd and snapshots

Python · NumPy

Generation runs locally after loading the runtime and weights. The interface receives results and replays the path; training runs through the Python library.

Understand the integration

LOOK IT UP WHEN YOU NEED IT

All documentation

35 pages per language

Usage guides, foundations, function reference and papers organized around the path you are studying.

01

Getting started

3

Understand the project and run a first experiment.

  1. What glassllm is
  2. Local installation and execution
  3. First experiment: site and Python
02

Interface guide

8

Components, screenshots and lab workflows.

  1. Component map
  2. Text, generation and tokens
  3. Temperature and token selection
  4. Reading attention and probabilities
  5. Exploring the complete architecture
  6. Studying in slow motion
  7. Inspecting tensors and code
  8. States, limits and troubleshooting
03

Transformer foundations

9

Intuition, equations, dimensions and links to the code.

  1. Tokens and BPE
  2. Embedding and shared weights
  3. Causal attention: Q, K and V
  4. Position and RoPE
  5. Heads, GQA and K/V cache
  6. RMSNorm, pre-norm and residuals
  7. FFN and SwiGLU
  8. Logits, temperature and generation
  9. Loss and backpropagation
04

Library reference

10

Contracts, functions, examples and NumPy engine behavior.

  1. API map and conventions
  2. Configuration and pretrained models
  3. Tokenizer: complete reference
  4. GlassLM and GenStep
  5. Tracer, records and step catalog
  6. Tensor and autograd engine
  7. Differentiable operations
  8. Layers and cache
  9. Training and optimization
  10. CLI, files and scripts
05

Project and references

5

Integration, data, validation, glossary and papers.

  1. How the interface uses the engine
  2. Data, models and reproducibility
  3. Validation and contribution
  4. Glossary and notation
  5. Papers and reference map