An overview of current and potential techniques for squad based tactics in games.

Bryan Auslander

[email protected]

    I am originally from New Jersey.

Fourth year CSB student.

Planning to be here a fifth year obtain a masters in computer science.

Spent the last 2 semesters working on a case based reasoning system (RetaliCase) for Unreal Tournament bots. (More on this later.)

     Introduction to squad tactics.

Brief overview of waypoints and squad tactics derived directly from these.

Decentralized Team AI based on emergent behavior.

Planned squad tactics through centralized squad leader/system.

RetaliCase development and overview.

Halo 3 examples from developers and in game footage will be used throughout presentation.

 Deviation between true AI systems and the illusion of intelligence is most evident.

 Difficulty to program.

  CPU time.

Unpredictable behavior.

Chris Butcher and Jamie Griesemer (designers on Halo) comment: “If your looking for tips about how to make the enemies in your game intelligent we don’t have any for you we don’t know how to do that, but it sounds really hard.”



The Madden NFL is a widely praised learning squad AI.

  

11 players on each team to coordinate.

 

Each player has different attributes.

Coordinate blocking to allow running plays.

Learns how to predict and counter players play choices.

Can choose a wide variety of offensive plays based on situation.

Video: http://www.youtube.com/v/qNoxHk53uNw&rel=1 Top: Madden 07 Bottom: Madden 08

  Accused of cheating:  Game more difficult than previous ones.

 AI seems to make miraculous comebacks.

Other ways Madden makes AI seem better.

 Higher difficulties increase AI player attributes while decreasing player attributes.

  Large area of focus for squad tactics.

 Military applications.

 Used to simulate Large dynamic and uncertain environment.

  Limited information.

Imperfect communication.

Popular gaming category.

   Halo Rainbow 6 Bio Shock.

 Waypoints are commonly used in many games.

 This is going to be covered extensively in another class     NPCs use them to navigate the environment.

Points are nodes in a node graph which is a representation of all paths NPC can travel.

 Greatly simplifies AI movement.

Use path finding like A* to get around.

Nodes can be used for other applications besides just path finding.

  Store precalculated visibility information.

Tactical value can be derived.

Supreme Commander

 For entire node graph store visibility and connectivity as a bit string.

    Used to find safe nodes. Good attack positions.

Multiple team mates sharing information increases knowledge of good and bad nodes.

Coordinate Flanking.

  In FPS need to anticipate opponent.

Pinch Points are places to launch an ambush.

 Single exit from a room.

  End of hallway.

Coordinated effort in a multiple exit room

 Node graphs can become large so may need to only store part of it at a time.

 Processing issues.

  Memory issues More complex.

 Requires proper node locations.

 Less of a problem with experience.

 No good automatic node placement algorithms available yet.

logic every frame then you will bring down the whole frame rate. Raycasts account for 60% of the AI computation budget.”

    No squad Leader.

Each squad member can have its own unique AI.

Team members communicate with each other.

Squad tactics are an emergent behavior.

Decentralized Halo video.

    Easy extension to individual AI.

Can handle a wide variety of situations.

Can effectively use each team members capabilities.

Easy to combine with scripted action’s.

   Squad members move and announce movements and actions to squad.

When enemy found decides whether to engage threat based on:  Number of squad members engaging.

  Number of squad members moving.

Engaging an enemy too long.

Results in Fire and Maneuver behavior:      Squad members are able engage multiple enemies effectively.

Don’t stand still to long.

Can switch targets easily.

Allows for individual member specialties to be used.

Can easily handle squad member deaths.

Halo vs swarm

 Each squad member chooses next position.

   Based off individual preferences Intentions of other team members Can make heavy use of waypoints as described before for faster planning.

   Wait till enemy arrives in kill zone.

Engage them then pull back to rally point.

If discovered return fire and head back to rally zone.

   Difficult for squad to decide on action since no leader. Would have to put squad to sleep till enemy in kill zone.

Squad can not react if the enemy decides to pull back before reaching kill zone.

Squad would never create L shaped ambush on its own.

  Advantages:  Does not require major addition to individual AI.

  Can make team members act to take advantage of their specialties.

Easy to Combine scripting to create custom scenarios.

Disadvantages:  Can not easily obtain unanimous decisions.

  Movement can become difficult when avoiding each other paths.

Can be memory intensive for each member to maintain its own state.

Halo: [Maintaining full state]too complicated – O(n 2 ) for n=25. instead actors only track characters that are important to them, n around 3 for friends, 5 for enemies

" There is nothing inevitable about military victory, even for forces of apparently overwhelming strength. The Greeks at Marathon, Alexander against the Persian Empire, the success of the colonists against the British in the American Revolution, Napoleon over the Austrians in Italy... all offer dramatic evidence to the contrary. In the absence of inspired military leadership... the more powerful side wears down the weaker."

- Bevin Alexander

  As is true in the military squad AI can not assess its situation and decide the best course of action without a leader and predefined courses of action.

Adding a centralized planning element does not replace the individual AI, but extends it.

   Authoritarian     Squad members always obey and execute commands.

Yields tightly coordinate maneuvers.

Allows squad member sacrifices for overall squad.

Ineffective when squad member view conflicts with squad level.  Ex: Sniper visible to squad member, but conflicts with orders Coaching    Squad AI assigns tasks to members.

Squad members tries to execute it and reports when it can’t so it can be reassigned.

Yields looser coordination due to members being easily distracted.

By mixing command styles advantages from both can be obtained.

  Authoritarian  When command is issued unit must perform.

Coaching  Command is issued then up to NPCs to decide how to proceed.

  Situations can be expressed in abstract features.

 Features can be used to choose best maneuver.

Situations features can be represented through influence map as above.

 Choosing a Maneuver  AI keeps track of permissible maneuvers.   Evaluates each ones fitness.

Chooses maneuver with highest fitness.

 Fitness can be represented with fuzzy rules that express situations the maneuver is applicable in.

  Situations can have multiple rules to represent different situations Closely resemble our own rules of thumb.

Halo: Behaviors are made obvious:“Initially nobody noticed so we had to keep adding clues to make it more obvious. By the time we shipped we had made it so every single grunt runs away every single time an Elite is killed.”

 Pull back while maintaining cover fire.

   Members in line provide suppressive fire.

Lead member runs to back.

Repeats until they have reached destination.

   Path used to pullback requires finding a path that   Has short travel times Provides concealment from enemies  Gives sufficient space to leapfrog teammates.

Searching for this path is CPU intensive so squad level decision so save CPU time.

A maneuver is performed in several steps   Preparation.

Execution  Finalization

  Advantages  Good for synchronization and coordination   Resembles human tactic and rules of thumbs.

Can include create many premade plans.

Disadvantages  Can not easily adopt to small variations.

   Ignoring a sniper because it goes against plans.

Can sometimes ignore squad member strengths.

Requires a leader to analyze the situation  Requires lots of communication and processing to keep state current.

Halo: Hide weaknesses with design.

• Design levels to showcase AI.

• Design behaviors to cover up weaknesses • Ex: AI gets more accurate the longer a player is out of cover.

  Thank you to Jim Blaney and Tabin Ahmad for assisting me in creating these clips.

 Clip 1   Clip 2 Clip 3 Halo AI problems.

CONSIDER MOVING CLIPS T APPROPRIATE PLACES IN THE PRESENTATION

  Case based reasoning system applied to a reinforcement learning algorithm to control a team of unreal tournament bots.

Retaliate originally created by Megan Vasta  Was presented in class before   Bots would learn strategies during game based of current state and score (Reinforcement Learning).

System proved very effective, capable of defeating the previous research project “HTN bots” in only two games.

 Retaliate’s biggest weakness is maintaining memory between different opponents.

 Retaliate must unlearn a case to learn a new one.

  Can not identify previous opponents.

Can take a while to learn a strategy.

60 50 40 30 20 10 -10 0 RETALIATE HTNbots

Time

20 10 -10 0 60 50 40 30 RETALIATE HTNbots

Time

  Retaliate could be improved to respond to changing opponents.

  Could store previous knowledge (qtables) for future use.

Could be more competitive against new opponents by finding a similar situation.

Case-Based reasoning is ideally suited for this task.

 Store plan information.

  Can retrieve data based on in game situation.

Could be built right on top of Retaliate.

   Uses 18 calculated features about current state to save and retrieve cases  Location of Bots compared to domination locations.

  Average time a domination point is owned by Retaliate or Enemy Team score and enemy score differences.

These features are stored in the case based system for each case.

A case about the current situation is also maintained and continuously updated.

Loads case into retaliate Maintains Retaliate Case Evaluates Case RetaliCase No (do nothing) Load Case No Save Case Yes (winning) Yes (Losing) Retreive and Load Case Retrieves Case Store Case Save Case Case List

  Cases are saved during every successful game.

At the start of every game cases are read into memory.

  Each case takes up very little memory and comparisons are fast.

On larger projects however may need a database to speed up case retrieval and minimize memory usage

 Results so far  Loading good cases has shown increases in performance.

  Retalicase seems to be performing better then Retaliate in start of game.

Discovered the need to make sure cases are mature and have had a chance to have been modified before saving/loading.

   Poor cases were being saved.

Bad case loading loops were occuring.

Hindered learning.

 Possible demonstration.

     Squad tactics are starting to move toward true AI.

Squad tactics are applicable to variety of genres and applications.

Makes heavy use of other technologies   Waypoints Scripting Languages Increasing research and development going into organizing and controlling squads.

  Centralized squads Decentralized squads.

True learning AI systems being created and tested all the time (ex. RetaliCase).