Ms. Pac-man for Autonomous Networks?

Published December 07, 2021

What do Ms. Pac-Man and Autonomous Networks have in common?  Admittedly, not very much at a glance.  But they do have something in common. Both require autonomous agents to work, and these agents must balance conflicting priorities.   

In my work on Ms. Pac-Man, I wrote and Ensemble agent to play the game.  An Ensemble agent is comprised of multiple Voices, where each Voice in the Ensemble is tasked with generating opinions about a single facet of the game, ignoring everything else.  In my research, I am exploring the use of Ensemble systems for Autonomous Networks.  But before we get into that, let’s talk about Ms. Pac-Man! 

Ms. Pac-Man 

Ms. Pac-Man only really has one metric for success – high score – and the gameplay consists of four things that need to be considered: 

Don’t get killed 
Eat ghosts when energized 
Eat bonus fruits 
Clear the maze 
 
That’s really all there is to it.  My Ensemble agent is composed of four Voices – one for each of the above.  Decomposing the problem like this enables the individual Voice to focus on its own task, completely disregarding everything else.  This in turn allows the algorithms used to be very efficient. 

How the Ensemble system uses the results from the voices has a marked effect on the behavior of the agent.  An example of conflict can be seen in Fig. 1.  I looked at simple voting (democracy) and task prioritization, but these did not produce good agents. 

 
Fig. 1: Conflicting opinions
 

The approach I found that worked best was to have each Voice give its opinion about all available moves, not just the preferred moves.  This, combined with weights, enables each Voice to contribute to each decision made, becoming a true Ensemble decision system. 

But what about Autonomous Networks? 

When we say “Autonomous Networks” we are talking about handing over control of networks to autonomous agents.  Just like playing Ms. Pac-Man, running a network requires the balancing of multiple features, such as cost, bandwidth, latency, and reliability.  In other words, its complicated.    

One approach is to use an Ensemble system, which allows us to strip away layers of complexity and deal with features individually, using weights to manipulate balance and achieve the desired behavior.  The weights can also be continuously evolving to ensure that the agent is behaving optimally, for a given measure of optimality. 

Voices can be added or removed at any time. Furthermore, weight sets and contingency plans can be made to instantly switch to a known good configuration if something catastrophic happens, such as a major outage.  All this can be done automatically at runtime, without the need for recompilation or even restarting.  

This isn’t just cool: it’s necessary! 

In summary

Ensemble systems can be used for more than just playing Ms. Pac-Man.  The individual voices may be tailored to a specific task, but the approach has general applicability.  Moreover, Ensemble systems can be found at the core of many general AI agents.  Any problem that can be partitioned into sub-problems, or where the problem is highly dynamic, is a candidate for an Ensemble solution. 

About the author

Philip Rodgers attained a PhD in Artificial Intelligence from the University of Strathclyde.  His research is on agent-based AI and Machine Learning.  He is a father of two, so has no time to play video games and must write AI to play them for him. 

Still interested? 

If you want to know more about how an Ensemble system plays Ms. Pac-Man, the full paper can be found hereA video of the agent in action can be found here