Writing strategy game AI requires a complex set of algorithms. Too easy and players will discard the game, too hard and neophyte gamers may give up.
A recent AI algorithm called Cicero ranked among the top 10% in an online tournament of the strategy game Diplomacy without most human players realising they were playing with an artificial intelligence. This new approach uses reinforcement learning.
How to improve strategy game AI
The complexity of strategic games requires an AI to be able to make complex and long-term decisions over the course of multiple turns. This type of cognitive challenge is a common theme in scientific artificial intelligence research and is one of the reasons why real-time strategy (RTS) video games have become such popular and challenging testbeds for this type of research (Buro 2003; Ontanon et al. 2013).
To create a bespoke plan for each game, the AI-bot uses its case-base to retrieve previous plan-game pair matches that are similar to the current situation, and then extracts relevant information from these plans via decision tree learning. A new plan is then generated by choosing branches of this decision tree using fitness-proportionate search (as described in Section 4) to determine an optimal path to the desired outcome.
The AI must also be able to spatially analyze the map in order to adequately maneuver military units, as well as to decide where to place new buildings such as airbases and silos. These structures must be positioned to optimize bombing runs from the player’s cities and to defend against enemy missile strikes and planes. To do this, the AI-bot uses a high-level strategy to dictate initial placement and metafleet movement/attack strategies which are then implemented at game time.
What are key factors in strategy game AI
A key challenge in strategy games is ensuring that the AI provides a suitable level of challenge. Too easy, and experienced players will quickly grow bored. Too hard, and neophyte players may give up in frustration. AI can help to overcome this challenge by analyzing the game data and identifying potential balance issues that would be difficult for humans to spot.
This is accomplished by using AI to analyze the player’s current state and predict future actions. This can be done using a variety of techniques, including FSMs and Behaviour Trees. The AI can then use this information to determine the best course of action and optimize its behaviour.
Another way that AI is transforming strategy games is by improving the quality of non-player characters (NPCs). Traditionally, NPCs in digital strategy games have been limited to pre-programmed sequences of movements and a limited repertoire of spoken responses. However, thanks to advances in AI technology, this is changing. AI algorithms can now write basic code, meaning that it is possible that in the future a creative teenager might develop the next Command & Conquer from their bedroom in their parents’ house. This will create a more immersive and engaging gaming experience, as well as making the games more realistic.
Can AI in strategy games learn
It’s a big challenge for strategy games to teach AI to understand the game and make good decisions. This is because the game involves a lot of moving pieces and many different possible outcomes, making it difficult for an AI to pre-program every move. That’s why it is important for AI to learn as it plays and adapt its strategies.
In fact, the recent success of AI in strategy games has proven that it can indeed learn. For example, the AI Hiro was able to beat Freeciv’s built-in computerized opponents by using a rules-based approach combined with machine learning. The AI was taught by human players to play the game, and it learned from its mistakes by trying out different tactics and evaluating each one.
However, if we want the AI to win in the long run, it needs to be able to deal with the complexities of imperfect information and multivariable planning. For this, it is necessary to employ machine learning and develop a model of the game. The models can then be used to train the AI to find optimal strategy choices in real time. This could be a key area for future research in this field.
Benefits of enhancing strategy game AI
Developing AI systems for strategy games is an expensive endeavor. Having an AI system that can learn on its own is even more cost-effective. Moreover, AI can help in finding potential game balance issues that might not have been identified by human developers.
Traditionally, deterministic AI for video games is based on finite state machines. This makes the challenge level of the AI hard to modify and limits it to a fixed set of actions. However, ML can help in overcoming this limit and create more challenging AI opponents.
For instance, a ML system could analyze the influence maps and identify which factions are at risk of being attacked or overrun. Then, it could send units to reinforce these parties. Moreover, it can also use the data from the influence map to predict which units are under threat of being destroyed.
Another example of an improved strategy game AI is Cicero, a Meta-designed AI that recently beat humans in an online tournament of the board game Diplomacy. This is significant because unlike chess and Go, the game involves more complex interactions between players, including forming alliances and recognizing when someone is bluffing. Moreover, it is an important step toward AI that can help solve real-world problems that require compromise and tradeoffs.
How to make strategy game AI challenging
In order to make strategy game AI challenging, developers must define the decision-making algorithms and learning mechanisms that will allow it to interact with the player and the game environment. This will also involve defining the communication protocols that the AI will use to communicate with the players.
Developing strategy game AI is one of the most complex tasks in the field of AI. This is because these games require a combination of analytical thinking, creativity, and adaptability, traits that have traditionally been reserved for human brains. Moreover, strategy games often have long time horizons, meaning that the payoff of a decision can take a significant amount of time to appear.
For example, if a player builds a city, they must wait for that city to become fully developed before they can attack it. This is a significant delay that can be difficult for AI to handle.
The best way to make strategy game AI challenging is to implement a robust pathfinding algorithm that can deal with long time horizons. A good example of this is the Heroes III AI, which used a priority queue to limit the number of paths evaluated. It worked well and was a great step forward for the genre.
What role does AI play in strategy games
While the development of AI technology has revolutionized digital games, a strategy game still requires critical thinking and adaptability–traits that are best left to human brains. However, as AI algorithms get smarter and more powerful, they can be used to create more challenging and realistic game AI.
The success of AlphaGo has led to increased interest in using AI to play real-time strategy (RTS) video games. RTS games are complex, demanding that the AI make decisions under uncertainty and navigate a large action space.
One of the challenges with creating an AI to play strategy games is that it can be difficult to pre-program every possible move the AI might make. This is particularly the case when a game uses imperfect information, like the board game Stratego, where the player cannot observe their opponent’s pieces. This means that AI methods that worked well on Chess or Go are not easily transferred to Stratego.
To address these challenges, Arago developed the free Civilization VI AI Hiro, which it trained using Freeciv players from the gaming community. It combines a reinforcement learning model to figure out what moves to make with a large language model that negotiates with other players. This approach allows the AI to learn from its mistakes and improve over time.
How to balance strategy game AI
The balancing of strategy game AI is a complex problem. There is no one size fits all solution, but it can involve using reinforcement learning and explaining artificial intelligence (XAI). Reinforcement learning allows the AI to learn over time which battles are worth fighting by analyzing its own previous win rates. Then, it can use this information to select units to send to a battle. XAI allows the AI to evaluate its decisions by predicting their consequences and then comparing those predictions to actual outcomes.
Aside from these tools, it is also important to consider the player experience when balancing strategy game AI. If a match-up feels unfair to both sides, then it is not balanced. This can be due to issues such as the aforementioned cheesefest or if a game revolves around timing attacks and turns into a rock-paper-scissors situation.
Another key consideration is the ability of AI to make free decisions in a game. If the AI is only following a precoded set of rules, then it will not feel like an intelligent player. In addition, it may be too easy or hard for advanced players, and neophytes will lose interest quickly. This is why games have different difficulty levels.