Four trucks, thirty customers, one depot (blue square). A 5k-parameter policy network learns the skill of routing by building millions of routes on RANDOM maps and being rewarded for shorter total distance. The maps below are all the SAME fixed test map — one it never trains on. Benchmarks on this map: nearest-neighbour heuristic 6.97, with 2-opt polishing 6.53. Act IV-B: a professional skill, learned from the rules alone.
| epoch | distance | vs 2-opt benchmark |
|---|---|---|
| 0 | 7.056 | +8.0% |
| 3 | 7.165 | +9.7% |
| 8 | 7.141 | +9.3% |
| 15 | 7.281 | +11.5% |
| 30 | 6.807 | +4.2% |
| 60 | 6.648 | +1.8% |
| 100 | 6.662 | +2.0% |
| 150 | 6.857 | +5.0% |
| 225 | 6.684 | +2.3% |
| 325 | 6.684 | +2.3% |
| 450 | 6.684 | +2.3% |
| 600 | 6.591 | +0.9% |
| 800 | 6.466 | -1.0% |
| 1000 | 6.484 | -0.7% |