Learning Pokémon TCG from Scratch with Pure Self-Play

Author: Soichiro Nishimori and Eason Yu

Date:

Pokémon is loved by a lot of people around the world, us included! The Pokémon Trading Card Game is one of its spin-off games, and it is played in two stages. First, you build a deck by picking 60 cards from a pool of about 1,200, made up of Pokémon and the Trainer and Energy cards that back them up. Then the actual battle starts, turn by turn, each player using the deck they built.

What hooked us is the first stage. There are more than \(10^{100}\) possible decks, and everyone has their own take on what the best one is. A great deck is creative. Someone spots a combination nobody tried before, and the cards click in a way that is hard to predict until you see it play out. So when the Pokémon TCG AI Battle Challenge came along, we saw a good chance to study the game.

Honestly, neither of us knew the game that well going in. That made it more interesting: how far could a general-purpose learning system go without expert card knowledge, hand-written strategy, or demonstrations?

Our answer was to learn through self-play. We built a fast simulator, represented every decision through one shared interface, and trained a deck-agnostic policy with NashPG [1].

So, did we solve it? Short answer: not yet, at least not with this version of the algorithm. The decks our agent built for itself trailed human-designed ones. But we did see signs of life. The final system placed 36th out of 6,807 teams [5], and from nothing but wins and losses, the deck builder discovered recognizable Pokémon cores on its own.

One policy for a very complicated game

Pokémon TCG is a two-player, zero-sum game with hidden information and a large, structured action space. A player may need to choose a card, a target, an attack, or several cards in sequence. The legal choices change with the current prompt, and a useful policy must also understand the cards, Pokémon in play, and both players' states.

We converted each decision into fixed-shape tensors: card slots, Pokémon slots, game state, the current prompt, and up to 63 legal options plus a stop action. The first 60 card slots always contain the player's own deck, so the same policy can play any deck. Unavailable options are masked, leaving one common 64-way interface for every decision type.

Perceiver IO actor-critic architecture for the Pokémon TCG policy

Our 9.36-million-parameter actor-critic. Perceiver IO compresses 208 observation tokens into 64 latents, then produces scores for the legal options and a value estimate.

Full self-attention over every token was too expensive at our simulation scale, so we used Perceiver IO [3]. Sixty-four latent tokens attend to the observation once, interact through six self-attention layers, and are queried by the option tokens. This kept the policy expressive while making large-scale self-play practical.

Self-play needs a fast world

Pure self-play is sample-hungry. The competition simulator reached only 77 games per second in one process because its Python bindings called the C++ battle engine at every decision. We integrated the engine and our observation encoder into EnvPool [2]. The result was 2.0× faster on one thread and 7.3× faster across 20 threads, with 1,024 games running concurrently per GPU.

We trained with NashPG [1], which regularizes the current policy toward a reference policy and periodically moves that reference forward. Inside each reference period, ordinary PPO [4] updates the policy from self-play experience. Across four stages, we trained for 30,900 iterations, used about 1,017 GPU-hours, and evaluated 6.5 million games.

Self-play policy strength over stages three and four

Later self-play checkpoints increasingly beat the frozen end-of-stage-2 policy when both sides used the same deck. Stars mark the two submitted policies.

Learning to play was only half of the decision

We first tried to learn deck construction and gameplay together. It was the cleanest version of our idea, but it was not strong enough for the final submission. We therefore collected 113 deck lists from high-ranked agents and fixed the deck during each game. The gameplay policy still learned entirely through self-play, without imitation-learning pretraining.

Choosing the submitted deck required care. A deck's public leaderboard win rate reflects real opponents, but it is generated by other agents. Its self-play win rate reflects our own policy, but only inside our learned model of the field. We combined both signals and also considered sample size. The selected list had been played in 861 games by 10 teams; several apparently stronger alternatives had only one or two teams behind them.

Competition and self-play win rates for candidate deck lists

Each point is a captured deck list. The selected list performed well both in competition games and under our policy, with substantially more evidence than the other high-scoring candidates.

This compromise worked. Our stage-4 policy and data-selected deck placed 36th overall. More importantly for us, the result showed that a fairly general recipe—fast simulation, a shared policy architecture, and pure self-play—could become competitive in a game we initially knew little about.

What the failed version taught us

The deck-learning experiment was not our best submission, but it may be the most interesting part of the project. A causal Transformer selected 60 legal cards autoregressively, and NashPG supplied only the win or loss of the resulting self-play game. After roughly 20,000 updates, recognizable structures emerged: 95.3% of sampled Fighting decks shared one Pokémon core, while the dominant Water and Grass cores appeared in 98.7% and 97.3% of samples.

Dominant learned deck cores and uncertainty during one autoregressive deck draft

Left: one Pokémon core came to dominate each color. Right: a Lightning draft remained uncertain through a broad Trainer shell before committing to an attacker line and Energy near the end.

Yet these decks still trailed human-designed leaderboard lists. We suspect exploration was the bottleneck. Strong combinations often require several individually unhelpful choices: Dragapult needs an evolution line and two Energy types, while Ogerpon benefits from Meganium support. Partial combinations look weak, so self-play rarely reinforces the whole package. For us, this turns Pokémon TCG into more than a competition environment: it is a useful benchmark for coordinated exploration in equilibrium finding.

References

  1. Eason Yu, Tzu Hao Liu, Clément L. Canonne, Yunke Wang, Chang Xu, Nguyen H. Tran, and Stefano V. Albrecht. "NashPG: A Policy Gradient Method with Iteratively Refined Regularization for Finding Nash Equilibria." Transactions on Machine Learning Research, 2026.
  2. Jiayi Weng et al. "EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine." Advances in Neural Information Processing Systems, 2022.
  3. Andrew Jaegle et al. "Perceiver IO: A General Architecture for Structured Inputs & Outputs." International Conference on Learning Representations, 2022.
  4. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. "Proximal Policy Optimization Algorithms." arXiv:1707.06347, 2017.
  5. Soichiro Nishimori and Eason Yu. "Pure Self-Play Reinforcement Learning for PTCG." Pokémon TCG AI Battle Challenge Strategy write-up, Kaggle, 2026.