zxell.aiPhase reports
zxell.ai builds a language model from scratch on a news corpus and distributes the training across ordinary low-spec machines. These are the project's internal reports for each roadmap phase, published as they were written.
This is an internal project report edited for publication, translated from the Japanese original. Server operations and security details, and details of the corpus, are omitted; omissions are marked in the text.
Phase 0: Quantitative understanding and legal groundwork
zxell.ai Roadmap (clean copy v2)
Project goals and success criteria, the decision log, model sizes and compute estimates based on measured data, the distribution scheme, the phase plan, and risks
Phase 0: Data Inventory Report
Results of the corpus inventory: article counts, per-language distribution and quality, duplication rate, how many images actually exist, and the resulting token estimate (about 6.5 billion)
Phase 1: Single-machine baseline
zxell.ai Tokenizer Comparison Report (regenerated)
Comparison of SentencePiece BPE vocabularies of 48k and 64k; 48k was adopted. Compression efficiency by language, what the vocabulary contains, and segmentation examples
Framework methods used in phase 1 — those directly involved in LLM training (by module)
Which SentencePiece and PyTorch methods the phase-1 code relies on, and where the line runs between what was written in-house and what was left to the framework in the model itself
Phase 1: File layout and data formats on the client (the RTX 3060 machine)
Layout and formats of the files placed on the client machine by the S baseline run: shards, metadata, training log, checkpoints