Residual Modpack (26.1.2) – Build and Train Real Language Models with In-Game Machines

Residual Modpack lets you build and train real language models using machines inside Minecraft. Each machine does one real math job, like a table lookup, a matrix multiplication, a softmax, or a full transformer layer. The math follows what PyTorch does, all the way down to the optimizer. This pack is for players who want to learn how neural networks and transformers work by building and testing things in the game.

Features:

• Bench Wiring: You wire modules on a Bench from basic operations like add, multiply, exp, log, sum, maximum, and transpose.
• Spec Tests: Specs check each module against exact test cases, including tricky ones like softmax overflowing on large scores.
• Photomasks and Fabricator: Working designs get recorded onto photomasks, and a Fabricator builds machines from them.
• Model Core Training: A Model Core trains cabled machines as one model on up to four kinds of practice at once.
• Training Visuals: Pulses show the forward pass and the returning gradient, holograms show each machine’s output, and attention heads show where tokens look.
• Old photomasks found in ruins give you correct designs that are less efficient.
• Specs build on each other, from matrix multiply and softmax up to attention and full transformer blocks.
• Adds variant parts like alternative norms, gated layers, and rotating positions.
• The Model Core shows learning curves, output shapes, weights, and gradients.
• Models get tested on data they never saw during practice.
• You can change learning rate schedules, gradient clipping, number rounding (quantization), and how text is split.
• A Core can retrain a design from five fresh seeds to check if a change matters beyond random noise.
• Server leaderboards list saved models by score and size.
• A Forecaster uses a trained model to run nearby furnaces ahead of time.
• Agents run player-built programs that use models where learning beats a fixed rule.

How to Play:

1. Wire modules on a Bench using basic operations like add, multiply, and sum.
2. Pass the Spec tests to prove your module gives the right answers.
3. Record your working design onto a photomask, then use a Fabricator to make the machine.
4. Move up through the Specs, from matrix multiply and softmax to attention and full transformer blocks.
5. Cable your machines to a Model Core and train them as one model.
6. Put your trained model to work with a Forecaster or an agent.

Screenshots:

Residual Modpack Download Links