Why to use Unity ML-Agents to create intelligent gaming experience?

What is a Unity ML-Agent?

For those of you who are not well acquainted with the term “Unity Machine Learning Agents”, it is an open source Unity plugin with a license of Apache 2.0. that allows developers to train, modify, and embed intelligent agents to create an intelligent gaming experience by creating smart behaviors and reasonable virtual characters. A Unity ML-Agent allows programmers to not just code but teach these agents and make them intelligent using imitation learning, neuro evaluation, deep reinforcement learning, and other ML techniques through a simple Python API that contains the RL Algorithm for interacting and manipulating a learning environment. These Unity ML-Agents learn from the AI environments that contribute towards enhancing gameplay and giving players an immersive gaming experience. 

In September 2017, Unity released its first version of Machine Learning Agent Toolkit, the v.10 Unity package which was verified for the 2020.2 Editor release. Then in May 2021, they released the v2.0 version of the ML-Agent Unity package. The primary benefits that it gave the developers was to use Unity to train and integrate intelligent agents with API stability, easy installation, and a verified package. Over the years, these agents have improved and evolved in the gaming industry by addition of new training options like curriculum learning, the curiosity module and by improving the speed and efficiency, self play for multi-agent scenarios, and providing native, cross-platform inference support for all Unity ML-Agents. This toolkit has been developed by Unity through the years and there continues to be work done to make it even better in its performance and efficacy.

You may also read: Unity or unreal for game development

Machine Learning in video game creation is leveraged by developers across the globe to adjust game balance by training models. This helps in creating games that are entertaining, balanced and aligned with the creator’s vision. However, Unity brings these amazing innovations and upgrades that contribute towards the expansion and transformation of the gaming industry.

1. Grid sensors in Unity ML-Agents were developed and restructured by Eidos and added to the extensions package with changes to both sensor interfaces and behaviors.

2. It has cross-platform inference support which means that the developers that use Unity inference engine can deploy their ML-Agents on any platform that is supported in Unity.

3. It offers 18+ example Unity starter environments which contain the Unity scenes and all the game characters. This environment is a scene that has one or more ML-agents and other GameObjects that the agents can interact with. The Unity game developers can use these environments to get started on their projects by just opening an example environment and training the agent in it. Having said that, the agents have a certain behavior which can be Heuristic (this is used to test the environment), Training (where the AI trains), or Inference (where the learned model is used to make decisions) which determines how the agent will make decisions.

4. A Unity ML-agents Extensions package contains optional add ons to the C# SDK for the Unity ML-agents Toolkit. Its structure includes documentation, Editor, runtime, and tests where the runtime includes Physics-based sensors, input system package integration, custom grid-based sensors.

5. The Match 3 Unity environment was introduced in Release 10 and new features were added to the extensions package.

6. Data-Oriented Technology Stack (DOTS) is being used to build Unity’s core that lets builds be smaller in size and run faster boosting its overall performance. This also lets developers train agents in large and complex Unity environments.

7. NVIDIA PhysX 4.0 works great for robotic simulation and thus Unity 2020.1 has a new articulation joint system powered by NVIDIA PhysX 4.1 which has improved in robotic arms and joints simulation. It works on a featherstone’s algorithm, a non-linear iterative solver, and a reduced coordinate representation which helps in shrinking the unwanted stretch on the joints and helps accomplish stable movement.

8. There are several training options that are environment agnostic and environment specific that it offers the users. Curiosity for sparse-reward environments, RND for Sparse-reward Environments, Self-Play and MA-POCA for multi-agent scenarios, and few others are the varied training options.

9. The trained intelligent agents help in automated testing of the game builds, controlling NPC behaviour,  and assessing numerous game design decisions previous to its release. 

10. Its progressive algorithms are based on TensorFlow that lets Unity 3D game developers train brainy agents in different Unity environments with ease.

The working of an ML-Agent

The process involves Integration, training and Embedding. It starts by consolidating and connecting the Unity ML-Agent toolkit package. Then the agents are trained as per the desired behavior. Once the training ends, the trained agent has to be embedded back to the Unity project. The setup is quick and easy and requires minimum coding. 

From v1.0 to v2.0

In the past years there have been fifteen key updates bringing new training algorithms, improvements in workflow, and promoting performance. Cooperative Push and Dungeon Escape are two new sample environments that have also been developed. Both the environments highlight tasks that require one agent (in Dungeon Escape) and also multiple agents (in Cooperative Push) to complete. Then another feature update that the toolkit offered is to enable game characters’ reactions to various entities. The memory allocation during interference has been more enhanced. The deprecated APIs are removed in the newer version and along with that the API footprint is also reduced. The three core developments in this version have been the ability to train corporate behaviors, support training multiple tasks, and enable agents to observe various entities in their environment. 

The newer 2.0 version was about algorithm improvements and enhanced approaches like Grid Sensors component, Match 3 game boards, ML-Agents Cloud offering, exciting new game environments and with every release there were many more incredible features that were added to the list.

You may also like to read about ML-Agents v2.0 release

The update with ML-Agent Cloud gave the programmers the ability to start, pause, resume, and stop training experiments whenever they like. It lets them upload game builds with ML-Agents implemented. Multiple experiments can be launched at the same time with cutting-edge machines and also results can be downloaded from multiple training experiments. Other than this, there is continuous work being done to develop ML-Agents Cloud and to add great functionalities to it.

The Closing

The Unity Machine Learning Agents Toolkit comprises of a C# software development kit and an advanced library to train AI agents for 2D, 3D, and VR and AR environments. The advancements in ML in Game AI development and the tools, features, and programmes in this tool-kit allows developers and artists to create real-time solutions for different industries. 

This plugin in the Unity game engine lets game developers and AI researchers innovate & create and its pre-made environments already contain the Unity scene, game characters, and the environment elements where the agents observe, act, and learn. The process allows you to create intelligent agents and integrate them into your own Unity games to impart compelling and intelligent experiences. 

Logic Simplified is one such game development company based out of India that works towards giving the gamers around the world an enhanced gaming experience with its games that are built using advanced technologies and techniques to keep up with the emerging industry. Over the years, our team of game designers and developers have learnt a lot and their curiosity and zeal towards their work pushes them to explore and to be creative and innovative with their approach and it wouldn't be wrong to say that all of this reflects in the amazing projects and work that our company has done so far. To know more about us and our work, just contact us here or drop us an email at enquiry@logicsimplified.com