The chess game makes it clear to me that out-of-the-box Astra is not going to be a competitive player. Chess is almost certainly “easier” for an LLM than Warlight and yet it’s straight up hanging minor pieces.

As I said, Astra could one shot a chess computer though, so.... Sports is different from the real world in that there is a sense of fairness.

I am curious what level of play we get to from a training set of ~10,000 Fogless Fighting games. It’s probably still too small to make a big impact but I’m trying to be pragmatic and recognize that no one is going to fund the tokens for millions of self-play games.

You would really need to define what a "fair" playing ground is. I could BTW, fund millions of tokens (edit, just saw that you wrote tokens for millions of self play games), costs me nothing, all part of my 200 dollar subscription. But I can not train an AI. Once deployed out to the end users, AIs are unable to learn. What is possible is that Astra writes its own strategy guide which it updates from experience but that is all. But again, I do not know your chess Elo but I am very sure that if it is not at some IM level, it would take Astra 15 or so minutes max to build a chess computer which beats you.

Edited 2026-09-11 16:44:11

I might even be able to prove this. lichess allows for bot accounts. But IDK the details. The caveat for IM was BTW just for my 15 minute mark, at the end of the day it should not matter if you are Magnus Carlsen, will just need a couple more minutes.

Edited 2026-09-11 16:52:51

Ah, at some point the goalposts moved from LLMs playing chess to LLMs building engines to play chess. I completely agree I get thrashed by an LLM-built engine at my lowly 1900 FIDE (or even at 2800 FIDE as you say).

But when Astra built a machine to beat you in War.app, you would accept that? Because I have developed a couple War.app bots in the past. They could surprise even strong players if they were too confident but at the end of the day they were far from good human strength apart from a few embarassing upsets where the players got too cocky. So building a War.app bot to play at e.g. top 100 ladder rating is currently unsolved. If I could build such a bot with Astras help, I would be very happy. If Astra could build such a bot without my domain expertise guiding it, that would be a huge upset in the community of War.app bot creators (consisting of only myself).

Edited 2026-09-11 19:42:55

Yea goalposts get constantly moved here.
Nobody was arguing that an LLM could built an AI able to beat anyone.. infact, coding is something ITS TRAINED ON.... its text, its like the one thing it can do, predict the next word. Like thats trivially done nowadays. Nobody was arguing that.
What this was about, atleast certainly on my end is that ASTRA, so the LLM is unable to ever play chess or Warzone, which you made it seem like. It just doesnt understand it. And you can give it mountains of text input information, it will always come out with trash.. confidently, but trash. If you do not hand hold it by f.e. only giving it legal moves, or wahtever other handholding things you add, it is unable to play... because it simply is incapeable of understanding it.. as it was never trained on it.
And LLM will simply never be able to play a game on any decent level. WIll it come up with some bullshit and create some bullshit explanation? yea, its quite literally trained to say something that sounds impressive or informative.

Saying 'Astra could make a bot that crushes you in 15 minutes.' is quite literally a non-statement as.. LLM's could built you that already for years. There is quite literally nothing impressive by that statement, literally none. And to praise an LLM like the media like this. This whole praising of these LLM's for the past years is exhausting and to see it being now here aswell. God do i hate it.
Its an amazing tool, but stop fucking giving it credit to things just because the media calls it so great. It will never reach AGI, the fundamental framework of an LLM, could simply never produce it. Its exhausting.

I don’t personally make a distinction between someone building a very good Warlight bot by hand versus someone building a very good Warlight bot using modern techniques.

Either way they would be standing on the shoulders of giants to create something which doesn’t exist today. As long as it’s clearly flagged as a bot account, I would love to play a few games against it to see how it plays.

Bonus points if it actually resigns in a lost position because the bot account that plays CW games is annoying as hell.

@Roi Joleil: I literally have a video of Astra beating me in chess. Not "great" play, I should have won. No giving nudges to the LLM. The LLM played like a human. If someone makes a setup where the LLM has to keep track during all its moves itself, that is highly unfair, then the human opponent should also play blind chess. I did not give "valid moves" to the LLM, the LLM just plaid chess like you do on chess.com. That is all. You are also just allowed to perform valid moves.

And in the opening post I have a video of it beating the default AI in 20 turns. 15 turns is the gold star threshold.


So I fail to see a single valid point you are making. I also am not praising LLMs because of any media attention. I am using 200 dollar LLM subs for over 1 year, I have been one of the very first in my profession to understand that my profession is as people say "cooked". The issue with "coding" is that when you can "code" you have the key to do everything on the computer.

Edited 2026-09-11 20:00:53

Saying that I shift the goalpost when it comes to AI is also quite funny.The goalpoast for machine intelligence has always been the Turing test: https://en.wikipedia.org/wiki/Turing_test . I never remembered any other goalpoast when I was at university. This has long been reached, nobody cares. The goalpoast indeeds keeps shifting but not by the "AI bros" but by the naysayers.

Recently AI solved a math millenium problem with 1 million dollar prize money. But if AI can do that, where do you want to shift the goalpoast next?

Edited 2026-09-11 20:15:26

You state yo do not see my point while then quite literally making my point.
It can only '''play''' when all valid moves or options are legal. congratulaiton, you know, a random move generator can also play if you only feed it legal moves, i doubt that is any intelligent now, now is it???
The point is, which you keep ingoring is, that the LLM, i mean THE LLM does not understand chess AT ALL. Thats the claim, but you keep twisting it.
Yes, ofcourse it can play if it is enforced by some structure to only play legal moves, and if its by the framework the LLM built with your local chess board. It is limiting the choices to only valid options...

You have shown nothing that shows Astra understands chess. (because there is nothing that could show that)
Just accept the simple fact, that it doesnt understand it. And if you disagree with this, then i guess we should hail a random number generator (only allowing answers tho that are legal ofcourse) as a new marvellous Artifical Intelligence. Im done with this thread.

You have shown nothing that shows Astra understands chess. (because there is nothing that could show that)

https://www.youtube.com/watch?v=TxOUh991K6E

Here is the video in case you have missed it. You can even see the thoughts of Astra while playing.

Yes, ofcourse it can play if it is enforced by some structure to only play legal moves, and if its by the framework the LLM built with your local chess board. It is limiting the choices to only valid options...

You are free to state that my gameplay was aweful, that is true. But my moves were better than throwing a dice, apparently roughly 1k Elo according to Stockfish. But you questioning the setup is something which I find highly nonsensical. It is just a random chessboard which you have on chess.com. Really nothing else.

Edited 2026-09-11 20:21:30

just one last time, because i found it hilllarious.

you bringing up the Turing test which is obsolete for i dont know how long.
Nobody in the field considers that a real meassurement anymore.

it is genuinely funny. Good job, proud of you.

@Roi Joleil: You are a troll. The turing test became obsolete as LLMs have blown past it and not one second earlier (in case someone is interested). That is the definition of a moved goalpost.

Edit: https://arxiv.org/abs/2503.23674

Edited 2026-09-11 20:35:29

Personification and “understanding” (whatever this means) feel like weird things to get hung up on. My microwave doesn’t understand physics yet it can heat up water.

LLMs are a tool, just like my microwave. If Norman uses Astra to make a Warzone bot that can beat me then I’ll respect the accomplishment. It’s irrelevant to me whether Astra or the bot (or even Norman) know what a bonus is or recognize that they’re playing a board game.

Likewise, if Norman uses a microwave to make me a warm cup of tea I’ll drink it and be thankful. I’d be very curious why he used a microwave - but realistically I probably wouldn’t know unless he told me.

I also don’t really understand the point Roi Joleil is making about the chesscom interface only allowing legal moves

I can’t think of any video game in any genre that allows you to submit illegal moves which break the rules of the game. None of the chess platforms do. Warlight doesn’t either. When I was a kid playing Street Fighter we’d just mash all the buttons and let CAPCOM figure out what to do with the inputs. That’s… extremely normal?

Love the clash of great minds in here.

I am not very technically familiar with the structure of recent AI progress, but my intuition says that treating LLMs just as “prediction of the next token” either is outdated or underestimated view.
When complexity rises who knows how it affects the very limitations. The frontier is exciting.

@Norman actually, modern LLMs isn't the first instance of Turing test being passed, it's a computer program called Eugene Goostman in 2014.

I'd expect you, who's clearly in field, to know better than that.

@Norman actually, modern LLMs isn't the first instance of Turing test being passed, it's a computer program called Eugene Goostman in 2014.

I'd expect you, who's clearly in field, to know better than that.


Lol, without looking it up, it most definitely it has only been passed in a sense that the human judges did not know how to trick the counterpart ;) 2014 was when I was fresh out of university and AI and the chatbots there were complex structures with keyword matches. The way to trick them was to add... how is the term... backreferences into it. So you avoid saying the keywords but you say e.g. "the thing we talked about earlier". AI was one of the topics I went deeper at university into partly because it was interesting and also honestly because the particular prof gave easy grades. But I remember quite well the criticism back then that it all was just "lets optimize some numbers", it never was anything "real".

Let me check with AI help:
-----
Eugene Goostman (2014): 10 of 30 judges were fooled—33%.
https://archive.reading.ac.uk/news-events/2014/June/pr583836.html

GPT‑4.5 with the persona prompt (2025): 185 of 254 judgments favored the AI as human—73%. These count test rounds, not unique judges.
https://arxiv.org/html/2503.23674v1

Per aspera ad Joi

LLMs have incredible abilities but also serious limitations. They do all their real learning in their training phase and can't adapt to new circumstances that they haven't been trained to handle. As top War.app play is not going to be in an LLM's training set I can't see it being very good at this game.

I'm certain with the right resources you could develop an AI that would play this game better than any human, but no one with those resources are spending them on this game.

I'm certain with the right resources you could develop an AI that would play this game better than any human, but no one with those resources are spending them on this game.

Game on!