docs: Add a docs folder on building, the code's structure, the search and trainingHEADv0.1.0master

Covers building and dependencies, the architecture and game loop, the board
representation, the Player and MinimaxPlayer traits, the minimax search,
the network and self-play training, benchmarking, and how to add a player.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

1 files changed, 260 insertions, 0 deletions

diff --git a/docs/training.md b/docs/training.md
new file mode 100644
index 0000000..79da8f2
--- /dev/null
+++ b/docs/training.md
@@ -0,0 +1,260 @@
+# The neural network and training
+
+`NeuralPlayer` uses the same [minimax search](search.md) as `ClaudePlayer`, but scores positions with
+a small neural network instead of hand-written rules. The network starts out random and learns only
+by playing against itself. No human idea of a good position goes into it, and `ClaudePlayer` is
+only ever used to measure progress, never as something to learn from.
+
+The network is in [`src/network.rs`](../src/network.rs) and the training loop in
+[`src/train.rs`](../src/train.rs). Neither uses a machine-learning library: the forward pass, the
+backpropagation and the optimiser are all written by hand.
+
+## The network
+
+A multilayer perceptron:
+
+```text
+52 inputs ──▶ 64 ReLU ──▶ 32 ReLU ──▶ 1 tanh
+```
+
+The output is in `-1..1` and, like every score, positive favours the tigers.
+
+### Inputs
+
+Only the raw position, nothing derived from it:
+
+| Inputs | Meaning |
+|--------|---------|
+| 0..25 | 1 if a tiger is on that point (bit `x + 5y`) |
+| 25..50 | 1 if a sheep is on that point |
+| 50 (`HAND`) | Sheep in hand / 20 |
+| 51 (`TIGER_TO_MOVE`) | 1 if tigers are to move |
+
+The network has to work out for itself that dead sheep are bad for the sheep, that a sheep with an
+empty point behind it can be jumped, and so on.
+
+### Parameters
+
+All the weights and biases live in one flat `Vec<f32>` (`params`), at fixed offsets `W1`, `B1`,
+`W2`, `B2`, `W3`, `B3`, `PARAMS` in all (about 5,500). That lets the optimiser, the gradient and the
+file format treat them as one list of numbers.
+
+- `W1` is **input-major**: input `i`'s weights to the 64 hidden units are contiguous, at
+  `W1 + i*64 .. W1 + (i+1)*64`.
+- `W2` is **unit-major**: hidden-2 unit `k`'s 64 incoming weights are contiguous.
+
+### Forward pass
+
+Most inputs are 0: at most 4 tiger bits and 20 sheep bits are set out of 50. So instead of a full
+matrix multiply, `forward_full` starts layer 1 at its biases and **adds the weight row of each set
+input**, then adds `hand × W1[HAND]` and applies ReLU. That's why `W1` is input-major. `for_active`
+walks the set bits of the board masks with `trailing_zeros`. Layers 2 and 3 are ordinary dot
+products, then `tanh`.
+
+`forward_matches_a_plain_matrix_multiply` checks this against the textbook calculation.
+
+### Initialisation
+
+`Network::random(seed)` uses He initialisation: normal weights scaled by `sqrt(2 / fan_in)`, with
+zero biases. For layer 1 the fan-in is taken as 20 rather than 52, since only about a third of the
+inputs are set at once.
+
+### Backpropagation
+
+The loss for one sample is the squared error `(out - target)² / 2`. `add_gradient` runs the forward
+pass, keeping each layer's activations, then works backwards:
+
+1. Through `tanh`: `d_out = (out - target) × (1 - out²)`.
+2. Layer 3's weight gradients are `d_out × h2`. The gradient flowing into each hidden-2 unit is
+   `d_out × W3[k]`, or 0 where that ReLU was off.
+3. The same again through layer 2 into hidden layer 1, skipping units with no gradient.
+4. Layer 1: only the set inputs have non-zero gradients, so again only their rows are touched.
+
+`gradient_matches_finite_differences` checks every layer's gradients against numerical estimates.
+
+### Parallel and deterministic
+
+`gradient(batch)` splits a batch into chunks of `GRAD_CHUNK` = 32 samples, computes each chunk's
+gradient on rayon's thread pool, then **adds the chunks up in order**. `loss` does the same in chunks
+of `LOSS_CHUNK` = 1024. Floating-point addition isn't associative, so summing in a fixed order is
+what makes the result identical whatever the number of threads
+(`training_is_the_same_on_any_number_of_threads`). Training is therefore reproducible: the same
+seed gives the same network on any machine with any `RAYON_NUM_THREADS`.
+
+### The optimiser: Adam
+
+`train_batch` averages the gradient over the batch and takes one `Adam` step. Adam keeps running
+averages of each parameter's gradient (`m`, decay `β1` = 0.9) and squared gradient (`v`, decay
+`β2` = 0.999), corrects their bias towards their zero start, and moves each parameter by
+`learning_rate × m̂ / (√v̂ + ε)`. In effect every parameter gets its own step size, which suits a
+network where some inputs (a sheep on a rarely used point) are seen far less often than others.
+
+### File format
+
+`save` and `load` use a small binary format, all little-endian:
+
+| Bytes | Contents |
+|-------|----------|
+| 0..4 | `TSNN` |
+| 4..8 | version (`u32`, currently 1) |
+| 8..20 | inputs, hidden 1, hidden 2 (`u32` each) |
+| 20.. | every parameter as `f32`, in `params` order |
+
+`load` rejects files of the wrong size, magic, version or layer sizes with `InvalidData`. So
+changing the layer sizes makes old weights fail to load, and changing what an input *means* without
+changing the sizes calls for bumping `VERSION`.
+
+## Self-play training
+
+```bash
+cargo run --release -- train [generations] [games] [out-dir] [--from <weights>] [options]
+```
+
+Training runs in **generations**. Each one plays a batch of games, turns them into training samples,
+trains the network on them, and saves it.
+
+### 1. Starting point
+
+`run` starts from `--from` if given, otherwise the newest `gen-NNN.bin` in the out dir, otherwise
+random weights (seed `0x5EED`). Generation numbers carry on from the newest file in the out dir
+either way, so a run can be stopped and restarted. The Adam state and the sample buffer aren't saved,
+so they start empty on a restart.
+
+### 2. Self-play
+
+Each generation plays `games_per_generation` (500) games of the network against itself, in parallel
+across all cores (`parallel`, on rayon's pool). Each game:
+
+- uses a `NeuralPlayer` that searches to exactly `depth` (3) plies. A depth limit rather than a time
+  limit keeps runs reproducible, and the game's seed is derived from the generation and game number;
+- plays random moves for the first `random_plies` (10) plies, so games start from different places;
+- after that, plays a random move instead of the searched one with probability `epsilon` (0.05),
+  to explore positions the network would never choose;
+- is a draw on a threefold repetition (`play` passes the earlier boards to the search, so the
+  search sees repetitions too), or after `ply_cap` (200) plies, a backstop for games that wander
+  without repeating.
+
+Every position is recorded as a `Ply` with the search's score for it (clamped to `-1..1`, so forced
+wins become ±1), and whether a random move was played there.
+
+### 3. Labelling positions: TD(λ)
+
+Each position needs a target, the score the network should learn to give it. Two obvious choices
+both have problems:
+
+- **The game's result** (+1, -1 or 0) is the truth, but noisy: a position early in the game had many
+  moves, some of them random, between it and the result.
+- **The search score** of the next position is less noisy but biased, since it comes from the
+  network itself.
+
+TD(λ) blends them, working backwards from the end of the game:
+
+```text
+target[last] = result
+target[t]    = (1 - λ) × score[t+1] + λ × target[t+1]
+```
+
+With `λ = 1` every position is labelled with the result; with `λ = 0` each is labelled with the next
+position's search score. The default `lambda` is 0.7.
+
+**Random moves cut the chain.** A position where a random move was played is labelled with its own
+search score instead, and the positions before it then build on that, so they don't learn from the
+random move's consequences. This matters because random moves aren't equally bad for both sides: a
+random sheep move usually leaves a sheep to be jumped, while a random tiger move costs little.
+Without the cut, the network learnt that tigers nearly always win.
+
+### 4. The sample buffer
+
+The samples of each generation are kept, split by outcome (tiger win, sheep win, draw), and the
+network trains on the last `buffer_generations` (5) generations together. Training on more than the
+latest games smooths out the network's swings from one generation to the next.
+
+### 5. Balancing outcomes
+
+Self-play ends mostly in tiger wins, largely because of those random sheep moves. Trained on
+everything, the network would mostly learn what tiger wins look like. So with `balance_outcomes`
+(on by default; `--no-balance` turns it off), `balanced` builds each epoch from:
+
+- as many positions from sheep-win games as from tiger-win games: their mean, so the epoch keeps the
+  same size. The bigger pool gives a random subset (every position at most once), and the smaller one
+  gives every position plus random repeats;
+- every position from drawn games, once.
+
+If either kind of win has no positions at all there is nothing to balance, and everything is used.
+
+### 6. Training
+
+For each of `epochs` (2) epochs: build the epoch's samples, shuffle them, show each in a **random one
+of the 8 symmetric orientations** (`GameBoard::transformed`), and train in batches of `batch_size`
+(256) with Adam at `learning_rate` (1e-3). The random orientations are data augmentation: they teach
+the network that symmetric positions are equally good, and multiply the variety of positions it sees.
+
+### 7. Matches
+
+On every generation whose number is a multiple of `match_every` (10), the network plays two matches
+of `match_games` (100) games each, alternating sides, at the same depth:
+
+- against the generation that last played matches (N − 10), loaded from its `gen-NNN.bin`, or the
+  network the run started from if that file is missing;
+- against `ClaudePlayer`.
+
+Each pair of games (one as each side) shares a seed, so both sides get the same 4 random opening
+moves. The score is the share of points won, with a draw worth half. Matches are slow and only
+measure progress, so other generations skip them.
+
+### 8. Saving
+
+Each generation is saved as `out_dir/gen-NNN.bin` and copied to `out_dir/latest.bin`.
+
+## Reading the output
+
+```text
+ gen trained tigers  sheep  draws T before S before  T after  S after vs prev vs Claude train s  time s
+ 499  128657    460     22     18   0.0161   0.0211   0.0194   0.0165       -         -    0.26     1.3
+ 500  128335    461     17     22   0.0198   0.0185   0.0161   0.0229     45%       82%    0.28     2.2
+```
+
+| Column | Meaning |
+|--------|---------|
+| `gen` | Generation number |
+| `trained` | Positions in each epoch |
+| `tigers`, `sheep`, `draws` | How this generation's self-play games ended |
+| `T before`, `S before` | Mean error on positions from tiger-win and sheep-win games in the buffer, before training |
+| `T after`, `S after` | The same after training |
+| `vs prev`, `vs Claude` | Match scores (only every `match_every` generations; `-` otherwise) |
+| `train s`, `time s` | Seconds spent training, and for the whole generation |
+
+**Judge progress by `vs Claude`.** Self-play results say little, since the random moves skew them
+towards tigers, and the errors measure how well the network fits its own targets, not how well it
+plays. `vs prev` above 50% means the network beats its older self.
+
+## Settings
+
+Every `train::Config` field is a command-line option (`train --help`):
+
+| Option | Default | Meaning |
+|--------|---------|---------|
+| `generations` (positional) | 50 | Generations to train |
+| `games` (positional) | 500 | Self-play games per generation |
+| `out_dir` (positional) | `weights` | Where weights are saved and resumed from |
+| `--from` | – | Start from this weights file |
+| `--depth` | 3 | Search depth in self-play and matches |
+| `--ply-cap` | 200 | A game this long is a draw |
+| `--random-plies` | 10 | Random opening plies in self-play |
+| `--epsilon` | 0.05 | Chance of a random move after that |
+| `--lambda` | 0.7 | TD(λ) blend |
+| `--buffer-generations` | 5 | Generations of samples to train on |
+| `--no-balance` | off | Train on all positions without balancing outcomes |
+| `--epochs` | 2 | Passes over the samples per generation |
+| `--batch-size` | 256 | Positions per optimiser step |
+| `--learning-rate` | 1e-3 | Adam's learning rate |
+| `--match-games` | 100 | Games per match |
+| `--match-every` | 10 | Play matches every this many generations |
+
+`RAYON_NUM_THREADS` limits the number of threads; the results don't depend on it.
+
+Then play the result:
+
+```bash
+cargo run --release -- -S human -T weights/latest.bin
+```