Bots in Diplicity #907
Replies: 5 comments 4 replies
Proposed path forward: walking skeleton firstBefore refining bot behaviour, I want to get the whole system connected end to end with the dumbest possible bot — a tracer bullet. The milestone is "a bot completes a real game," even if its moves are terrible. Refinement (real order reasoning, negotiation, personas, memory) comes only after the parts are wired together and proven. Walking-skeleton scopeIn:
Deferred to refinement (explicitly out of the skeleton):
Sequencing (small, independently shippable PRs)To the skeleton:
Refinement afterwards (each its own increment): swap first-legal for a single constrained-selection LLM call (keeping the deterministic path as the fallback) + thin per-decision logging → provisional plan step → negotiation (chat facade + batched window) → structured commitments + memory → personas → order/chat budgets → observability. Testing & access
Acceptance check for the skeletonA maintainer creates an Italy-vs-Germany game with the bot, plays the human side to completion, and the bot submits legal orders every phase — including retreat and build phases — without a single NMR, with no manual intervention. |
|
Some random comments: 1. Bots are added through an invite-player flow (search by username, like inviting someone to a group chat); the bot auto-accepts. The same flow handles drop-outs: when a player goes into civil disorder, the game master invites a bot to take over the vacated nation.
2. vent-driven; the engine stays bot-agnostic. The bot app listens for game events (phase change, new message, all-human-orders-in) and decides whether to act. The game app emits these events but never calls the bot app. 3. pass through a "does this even need a reply?" gate so the bot can stay silent on throwaway messages — silence reads as more human than dutifully answering everything 4. (the latter feeds future trust and lets a bot remember a betrayal twenty turns later). 5. In-turn negotiations The risk is that during negotiation the plans change - that is the nature of the game. So the risk is that in four different conversations it starts to make 4 different plans. At the end, it consolidates everything, but sees it promised four different alliances, and will need to break 3. That will make the conversation very random - Ideally, you want the bot to make a simple call 'did anything fundamental change' and act on that. This is a tricky one if we want to keep token levels low. 6. Board/Game state I think this is a critical one to make the bots make sense of orders. I think some kind of non-LLM tactical layer that decides on moves procedurally (no LLM/GenAI, just some logical algorithm that defines what a next SC could be - attack/defend/support and key SCs it wants the next turns) would go a very long way. But it's complex. 7. Bot-to-bot loops. Left unchecked, two bots can talk to each other indefinitely. The rule: bots initiate only to humans, and a bot replies to another bot at most once per phase — enough for a bot pair to coordinate, bounded against runaway cost. 8. Deferred: Negotiation / chat / structured commitments / memory / consolidate step Great work! Final pet grievance - while I agree with you that it is very nice for the bots to have personality (aggressive, cautious, trusting) and different language (short 'brb', 'cu' language vs eloquent sentences) - I saw somewhere a glint of caricature historical people. I would very very strongly advice against this. I think the goal is to have the bots feel 'human' rather than 'bot' - both for the way it feels to play them, as well as your professional development. |
|
Thanks for the thorough review, Joren — a lot of this is really useful. Let me go through each point. 1. UX for bot addition Agreed that the UX needs more thought and I'm leaving it a little open for now. The guiding principle is: bots and humans should share the same underlying endpoints for joining games. At the data model level, a bot is simply a 2. Bot-aware engine You're right that this is a deviation from the "bot knows the same as the user" principle — but I think it's an acceptable one, and here's why: the information will be visible to humans too. Rather than a new phase status, I'm thinking we add an 3. LLM chattiness Agreed — this is a prompt/instruction tuning problem. Bots should be told explicitly when a message doesn't warrant a reply. Easy to build into the system prompt from the start. 4. Trust registry Agree on the concept — bots absolutely need some internal state tracking broken promises, betrayals, etc. My gentle pushback is on pre-specifying the shape of this structure. We don't know yet what information the bot will actually need to track once it's playing real games. I'd rather defer the trust registry until we can observe what gaps emerge through testing, then design it around real evidence rather than assumptions. 5. In-turn negotiation Good point. The ideal is that the bot re-evaluates its plan whenever critical new information arrives in chat — a sudden alliance offer or a betrayal should be able to shift its orders or what it ties to achieve through negotiation. Whether this is feasible at reasonable cost we'll need to validate in practice. For now I'm treating it as a goal rather than a given. 6. Board state and tactical layer I think the deterministic tactics layer is a good idea. I want to treat it like the trust registry. It's something I'm confident we will need, but we should let the evidence reveal that as we go. 7. Bot-to-bot communication I'd actually push back a little here. I think bots should communicate freely with each other — including being able to deceive one another — rather than treating inter-bot messages as honest contracts. This creates more interesting emergent scenarios (e.g. a bot revealing to a human what another bot told it in confidence). Loop prevention is the real concern, and I think that's handled by: a character limit per message (to be calibrated against historical message data), a cap on exchanges per channel per phase (something around 20, again to be validated against data), and bots naturally knowing when a message doesn't require a response. 8. Personas This is the one point where I genuinely disagree. Caricatures are consistent with the culture and style of the game — and could have real viral potential, with players sharing funny moments from their games. On a practical level, building a bot that would pass the Turing test is significantly harder. Most importantly: we have no evidence either way about what players actually prefer. Rather than pre-committing to either direction, I'd rather build in the flexibility for different persona styles and let player response tell us what works. |
|
Super cool ideas! 5. In-turn negotiation 7. Bot-to-bot communication In any case it might make sense to prompt the bots that they are playing against only human players (they can't see if they're talking to a bot or a human), so that it should in principle cooperate/collude/backstab at a similar rate. I think it will be really interesting to see if this actually plays out!
|
Diplicity LLM-Bot Planning — Discussion SummaryA series of five voice notes between John (project owner / backend) and Tim (ML & evals, previously built an LLM "color trails" game), with Joren (product) referenced second-hand. The thread is a design discussion about how to build an LLM-powered Diplomacy bot for Diplicity. A crude bot already exists; the conversation is about how to proceed from here. Areas of agreement
Points of interest (open questions & tensions)
The throughlineThe conversation keeps returning to one tension: the problem space is genuinely unknown, so the agreed move is to build crudely and learn empirically — but the thing that's hardest to resolve by building, and remains unsettled, is how to measure quality ("a metric on good"). Most of the open questions (procedural layer, graph reasoning, enhanced context, Cicero/Dumbot comparison) are really facets of that same measurement problem. |
Research summary: prior art on the bot questions
A useful framing that held up throughout: almost all prior work optimises for the bot being strong. Our aim is for the bot to be fun to play against. That difference matters — a lot of the prior art is reusable as machinery, but the goal it was built for is not our goal. 1. How do we measure quality of play?We can measure strength cheaply by having our bot play against DumbBot (a simple, free-to-run, 20-year-old rule-based bot). Two setups: one of our bots vs. six DumbBots, and six of ours vs. one DumbBot. The key refinement: don't measure wins — outright wins are rare in Diplomacy. Instead measure the average share of the board (supply centres) our bot ends up with, over many games. If all seven players were equal, each would average ~14%. So above 14% means our bot is genuinely better than DumbBot; below means it's worse than a two-decade-old script. Two caveats: (a) our bot calls an LLM every turn, so running enough games to get a stable number costs money — we'll likely want short/capped games for eval runs; (b) this only measures competence, not fun. But it's still worth having as a regression test: when we change the bot's strategy based on feedback, the DumbBot score tells us if we broke its basic ability to play. 2. How does a phase work? (what Cicero does)Cicero decides a concrete draft set of moves up front — not vague goals, but an actual list of intended orders, including its best guess at what everyone else will do. Its messages are then written in service of that plan. So yes: provisional orders + an agenda before it talks. But Cicero doesn't lock the plan and then just chat — it re-plans continuously, every time it's about to send a message and after it hears something new, then computes its final orders at the deadline. There's no fixed number of revisions; it's tied to how much conversation happens. This continuous re-planning is a big part of why Cicero is expensive. Our decision here feels premature. We may take a simpler route: set a negotiation agenda at the start of the phase, negotiate as much as we can, and only determine orders at the end. Or re-plan only when something important changes (the "did the fundamentals change?" gate). We don't need to decide yet. 3. How do we represent the board, and how much do we help the LLM?The tension Joren flagged. Two ends of a spectrum:
This is unresolved because Diplomacy is a map game and LLMs are historically weak at spatial/graph reasoning. The trap: the more map-specific tactical computation we bake in, the more we risk a bot that only works on the classical map — which kills our variant-agnostic goal. Encouragingly, recent work (Every's project and the "Democratizing Diplomacy" harness) shows off-the-shelf LLMs can play full games from a good textual board description with only light, generic hints — no map-specific training. Recommendation: start by trusting the LLM, measure against DumbBot, and let the eval tell us if it needs help. Add the cheapest, most variant-agnostic hints only if it plays nonsense. 4. When and how should the bot talk?When: over-talking is the default LLM failure mode. The fix is to make "should I speak at all?" a separate decision from "what would I say," with:
How:
Much of this serves fun directly: a bot that knows when to stay quiet feels human; per-player deception is the source of Diplomacy's actual drama. 5. How does the bot remember betrayal? (the "journal")A game runs many years; a 1903 betrayal should still matter in 1908. We can't re-feed the entire history every turn (too long, too expensive), so the practical approach is a short running journal the bot updates each phase and re-reads instead of the raw history. (Cicero actually did re-read everything each turn — fine with Meta's compute, not with ours.) Remembering betrayal is mechanically simple: compare what each player promised to what they actually did. Log promises received, check them against revealed moves, tick trust up or down. Every's project does exactly this with a per-player enemy↔ally trust level. What goes in the journal:
To stop it growing forever, summarise periodically (Every compresses roughly once per in-game year) — this is our "Consolidate" phase, and it's reassuring an independent project landed on the same idea. Two notes: (a) the Avalon research found strong agents build trust early specifically to spend it on a late betrayal — which is also the dramatic arc of a good game; (b) this strongly supports the build-crudely-and-grow instinct — start with just trust level + betrayal record, read traces, add fields only when their absence visibly hurts. A bot that remembers — colder to you because you stabbed it — feels like it holds a grudge. That persistent narrative is a huge part of what makes the game fun. Reusable resources (each worth a deeper look)
|
|
Sorry for this comment being a bit LLM-infused. I'm just trying to capture some thoughts from discussions that I've been having with Claude. I will probably transfer this whole discussion into a new document soon. The framing we keep landing on: almost all the prior art is chasing a bot that plays strongly. We want one that's fun to play against. That's a genuinely different goal, and it's quietly behind most of the calls below — cheap, characterful opponents over a strategic monster. Architecture. The game service knows nothing about bots. Bot logic lives in its own app with a one-way dependency (game never imports bot), and the bot reaches the game through the same player-facing facade a human client uses — get_board, get_options, submit_orders, send_message. So it structurally can't see more than a human in that seat, and there's no separate bot code path to maintain. This also answers JorenC's point about the engine signalling bot activity: we add an The turn loop. Plan → negotiate → commit, each phase. Plan is persona-free analysis that spits out provisional orders, which double as the fallback if anything downstream breaks. Negotiate happens in batched windows, not real-time ping-pong. Commit picks final orders from the engine's enumerated legal list, so an illegal order is impossible by construction. A fourth consolidate/summarise step comes later. I don't think this is the right structure yet — it's just a sane starting point. Plan stability — the thing JorenC and timwyse both flagged. The risk is a bot promising four different things in four channels and breaking three. For v1 we keep it dumb: the plan is frozen during negotiation and only reconciled at commit. A mid-negotiation re-plan is tempting but I don't know if it's affordable, so it's explicitly out of v1. Board understanding. Start by trusting plain LLM reasoning over an enriched text description, no bespoke tactical layer. The "Democratizing Diplomacy" harness shows off-the-shelf models can play full-press from a decent textual board with no fine-tuning, so this isn't a leap of faith. Board representation is basically a solved problem in the prior art — we lift it rather than design our own. Negotiation. Over-talking is the default LLM failure, so "say nothing" is a first-class choice, the bar to start a conversation is higher than the bar to reply, and there's a cap per channel per phase. Deception is a deliberate dial: the bot holds what it's telling someone separately from what it actually intends, and messages get reality-checked before they go out. Bots can talk to and lie to each other freely — a polite bot cartel is far less interesting than letting them deceive one another. (This is where I land differently from JorenC's honest-contract idea for bot-to-bot.) Memory. Re-feeding the whole history every phase is too expensive, so the bot keeps a short journal it rewrites each phase: a per-player scorecard (trust, promises kept/broken, apparent aims), its own strategy notes, and a running game summary. Trust starts crude — a level plus a betrayal record — and we only add fields when their absence visibly hurts. A bot that remembers being stabbed and goes cold on that player is a big part of the fun, not just a cost trick. Personas shape how a bot writes, not how well it plays (plan stays persona-free), so they're orthogonal to the competence work and don't block it. Human-like vs. caricature stays open — we decide from playtesting, and timwyse's "let the user pick" may well just be the answer. UX. Add-a-bot at game creation, a GM-only equivalent on staging games, and automatic takeover when a seat falls into civil disorder. Because bots are ordinary users on the same endpoints, none of this needs a parallel flow. Telling if it's any good. Two tracks. Competence is measured against DumbBot — six DumbBots vs. one of ours, and the reverse — on average share of supply centres (wins are too rare to measure). Clearing ~14% (one in seven) means we genuinely beat the baseline. But DumbBot doesn't negotiate, so that number is blind to the entire social layer; that half we judge by hand, reading traces. The DumbBot number measures competence only, never fun. Cost. Aim for €100–200/month steady-state, eval bursts kept separate. Usage shape is genuinely unknown, so we ship, watch, and measure — which only works if per-phase and per-game cost is instrumented from day one. Prompt caching helps a lot: persona + board + journal are a constant prefix, so negotiation and commit only pay for the delta. Fallback chain is re-plan → provisional orders → hold, so a bot never NMRs. Build wide and crude across all the interaction points, then improve from real behaviour and from actually playing against the thing — not from heavy up-front design. First target is classical 7-player, since that's where the interesting problems live and where the DumbBot baseline applies. Still open:
|
|
"Personas shape how a bot writes, not how well it plays (plan stays persona-free)." My hunch is that a better user experience is if different bots have different playing styles (more aggressive, deceptive, etc.), rather than just different writing styles. I don't really see the point of different personas if the way they all try to play the game is the same anyway? One thing to flag is that we need to put in specific guardrails against jailbreaking. I imagine a weak model could be tricked into playing the move a user tells it to (eg many-shot) unless we warn specifically against it. Decoupling the messages from the decisions/replies might be enough . |
|
"Usage shape is genuinely unknown, so we ship, watch, and measure" - I'm
wondering whether we should *plan* for a BYOK feature.
Not necessarily build it out - but have some issues defined that we can
easily pull and push into a PR.
That way, if the feature is genuinely used way above budget, we don't have
to pull the plug, but we can 'deviate' traffic into a funnel that still
works.
Any traction we got (which I believe we will get - the first 'live bots')
on fora, discussions, reddit etc will then not be void.
Otherwise the user flow is 'Look at diplicity - awesome new feature > link
feature doesn't work > trust broken'.
Then it would be 'Look at diplicity - awesome new feature > link > BYOK
instruction > BYOK > playable OR No BYOK > It was a user decision not to
BYOK".
…---> This is a bit like a smoke screen test - if you handle it correctly,
users don't even realise it's a smoke screen ("click here to cancel your
flight > group or individual > group > "Please call this number" - that's a
really weird flow but actually a smoke screen that most users don't even
realise).
On Mon, Jun 29, 2026 at 9:56 PM Tim Wyse ***@***.***> wrote:
"Personas shape how a bot writes, not how well it plays (plan stays
persona-free)." My hunch is that a better user experience is if different
bots have different playing styles (more aggressive, deceptive, etc.),
rather than just different writing styles. I don't really see the point of
different personas if the way they all try to play the game is the same
anyway?
One thing to flag is that we need to put in specific guardrails against
jailbreaking. I imagine a weak model could be tricked into playing the move
a user tells it to (eg many-shot
<https://www.anthropic.com/research/many-shot-jailbreaking>) unless we
warn specifically against it. Decoupling the messages from the
decisions/replies might be enough .
—
Reply to this email directly, view it on GitHub
<#907?email_source=notifications&email_token=ABXURGMV72U36C256JJEM7D5CLCVTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZUG42TSMJTUZZGKYLTN5XKOY3PNVWWK3TUUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17475913>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABXURGMDOBDMVCACOWQHHOD5CLCVTAVCNFSNUABIKJSXA33TNF2G64TZHM4TAMRZGA4DMMRQHNCGS43DOVZXG2LPNY5TCMBTGI4TQMZTUF3AE>
.
You are receiving this because you commented.Message ID:
***@***.***
com>
|
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Diplicity Bots — Design
LLM-controlled players that fill empty or vacated seats. A bot reads the game, negotiates in chat, and submits legal orders each turn. The bar is cheap + safe + responsive + reasonable + in-character — a credible opponent with a temperament, not a strategic master.
Strategic mastery is explicitly out of scope. The target is an opponent that is never obviously stupid or suicidal toward a human, and that negotiates like a thinking actor — not one that plays optimally.
Product
A bot is a normal Diplicity user with a
[bot]badge. It holds a nation, sends messages, and submits orders like any other player. Surfacing bots as ordinary players (rather than a separate concept) means the entire existing game UI, permissions model, and engine work on them unchanged — the bot-ness is just a flag.Personas. Each bot has a distinct persona so a board of bots reads as a cast, not one opponent copied five times. A persona is a disposition (how it plays and negotiates) plus a voice (how it writes). For example:
How bots enter a game. Bots are added through an invite-player flow (search by username, like inviting someone to a group chat); the bot auto-accepts. The same flow handles drop-outs: when a player goes into civil disorder, the game master invites a bot to take over the vacated nation. This makes bots a rescue mechanism for games wrecked by flaky humans — arguably their highest-value use — not just a way to fill empty seats at creation.
Architecture
The guiding principle is that the game service should know almost nothing about bots. All bot intelligence lives in one place, and the game engine stays as simple as it is today.
One service, two apps. A new
botDjango app lives inside the existing service, with its own models (decisions, summaries, commitments) in the same database.One-way dependency. The game app never imports the bot app. The bot reaches game state only through a narrow internal game-client facade —
get_board(),get_options(),submit_orders(),send_message(). That facade calls the existing player-facing views as the bot user, in-process, with forced authentication.Same information as a human. Reaching the game through the player-facing API means the bot structurally sees only what a human in that seat sees. No hidden information, no cheating — and a bounded, well-understood surface.
Event-driven; the engine stays bot-agnostic. The bot app listens for game events (phase change, new message, all-human-orders-in) and decides whether to act. The game app emits these events but never calls the bot app. The engine resolves phases strictly on the deadline and knows nothing about bots beyond an
is_botflag (used only for the badge and to skip push notifications). This is what keeps "wait for the bots" from ever leaking into the engine.The per-phase loop
Each phase, a bot runs four steps. The split exists so that the expensive, reliability-critical work (orders) is isolated from the cheap, variable work (chat), and so that strategic reasoning is never contaminated by roleplay.
1. Plan — one strategic call at phase start. The board is serialised. The call reasons on top of that serialisation and emits a strategic read, a provisional set of orders, and a negotiation agenda:
This step is persona-free — pure analysis. The provisional orders are written to the engine immediately as a floor (orders are secret until resolution, so this leaks nothing), guaranteeing there is always a sensible order on file no matter what fails later.
2. Negotiate — the bot opens with the players on its agenda and replies to incoming messages. Replies are batched over a short window rather than answered one-at-a-time (cheaper, more coherent, and far less robotic than instant ping-pong), and pass through a "does this even need a reply?" gate so the bot can stay silent on throwaway messages — silence reads as more human than dutifully answering everything. Deals struck here can revise the intended orders and are recorded as structured commitments, not buried in prose:
3. Finalise orders — one call, triggered by all humans having submitted or the deadline approaching (whichever comes first; the deadline fallback is essential so a single non-submitting human can't freeze the bot). The bot selects from the engine's enumerated legal options rather than writing order strings — so an illegal or malformed order is impossible by construction. The selection is informed by the plan and the commitments made in step 2, and overwrites the provisional floor.
4. Consolidate — at phase end, the phase's chat is compressed into a carry-forward summary, and commitments are reconciled: which did the bot keep or break, and which did others keep or break (the latter feeds future trust and lets a bot remember a betrayal twenty turns later). The raw messages are then dropped. Reconciliation is what protects persona coherence across a long game — character and trust live in durable structured memory, not in a transcript that gets compressed away.
Cost mechanics. Within a phase, the persona, the serialised board, and standing memory are a constant prefix — prompt-cached once, so the negotiation and finalise calls pay only for the small delta (the incoming message, the options list). The fixed cost is therefore roughly three calls per phase plus bounded negotiation.
Bot-to-bot loops. Left unchecked, two bots can talk to each other indefinitely. The rule: bots initiate only to humans, and a bot replies to another bot at most once per phase — enough for a bot pair to coordinate, bounded against runaway cost.
Mid-game entry. A bot taking over a vacated nation has no accumulated memory, so its first act is to bootstrap an initial summary from the existing game record. From then on it runs the normal loop.
Cross-cutting guarantees
Legality. Constrained selection from the engine's legal options makes illegal orders structurally impossible. Hold exists only as a last-resort fallback and should almost never fire.
Cost — separate budgets. Order-planning and chat have independent budgets. The order budget is a small guaranteed reserve that chat spending can never touch; chat draws from a separate, capped pool. They protect different things — orders protect legality (non-negotiable), chat protects experience (degradable) — so a chat-heavy phase can never starve the bot's ability to play. The payoff is a clean property: a player who spams a bot to run up cost can make it go quiet, but can never make it play badly.
Failure / degradation — a bot never stalls a game. If thinking fails for any reason (LLM outage, exhausted budget, a crash), the bot falls back to the provisional plan from step 1; only if even that is missing does it hold. The engine never waits for a bot — it resolves on the deadline exactly as it does for a human who didn't submit.
Safety. A bot's entire action space is "submit a legal order" and "send a message," so even a fully prompt-injected bot has no blast radius beyond the game — no tools, no data to exfiltrate, no spend beyond a message it was already going to send. Player messages are fed to the model as untrusted data, clearly delimited, never as instructions.
All reactions