Under the Weather Software

Video games these days usually contain some form of inventory, or list of objects. Just like on Amazon, the end-user can have a better experience perusing through pictures of items rather than a list of text items. Here’s an example from The Sims 4.

Generating this group of small images, or thumbnails, can be time-consuming. While you’ll most likely have better looking results if you stage and render each thumbnail yourself, if you have hundreds or thousands of items, this can be prohibitive if not impossible. This especially becomes true when you consider that in game development, every part of the game is repeatedly re-worked. This constant iteration to get things just right means that any changes you make to an object, need to also be propagated to any other processing it needs, such as thumbnail renders, adding hours or even days of extra processing time, and you can basically torpedo your launch date, because your game would pretty much be sunk.

The solution to saving you massive amounts of time from manually creating object thumbnail renders is obviously to have this done automatically. Before we begin, if you just want to see the source itself, just go straight to github here. Otherwise….

Let’s dive in.

The Concept

Suppose you have a group of 3D objects in your Unity project, say in the FBX format.

To create a thumbnail image of the object, a camera in the scene renders its view to a texture, called a RenderTexture. The graphics technique itself is called “Render-to-Texture”, or RTT.

Once you have a Texture object that contains the view of the camera, that Texture can be written out to an image file; in this case, we’ll write out a PNG file.

Then, later on, we’ll cover automating the process of repeating this concept for a list of objects.

Run-Time Generated Thumbnails for Custom Objects

I should mention a quick caveat about the solution below. The design of this solution involves running a Unity scene to generate thumbnails. This technique allows thumbnails to be generated based on any customizations the player may have introduced, such as custom colors or textures, prior to jumping into the game world. If you need to generate thumbnails during design-time (i.e. outside of Play mode of a Unity scene), you may consider automatically generating thumbnails from your 3D Modeling Tool instead.

Manual Thumbnail Generation

Setting this up manually in Unity is pretty straightforward. You can skip this section if you already know how to render a camera view to a texture and write it out to a PNG file. For everyone else, we’ll go step-by-step below.

Unity Setup

1. Create a new Scene.
2. Create an Empty Object (this will hold the script), rename it “ThumbGen”.
3. Create a Camera and parent it to “ThumbGen”.
4. Create a RenderTexture, assign it to the camera as TargetTexture.
5. Create an object (say, a Cube) in the scene, in view of the camera. Click on the camera to check that it’s actually looking at the object, and continue positioning the camera and the object until it’s in view.

That’s it as far as the scene setup goes. Now, let’s write the script.

ThumbnailGenerator Script

Create a new script called ThumbnailGenerator.cs and attach it to the “ThumbGen” object. Implement it as follows:

using System.Collections;
using UnityEngine;

namespace ThumbGenExamples
{
    public class ThumbnailGenerator1 : MonoBehaviour
    {
        public RenderTexture TargetRenderTexture;

        public Camera ThumbnailCamera;

        public Transform ObjectPosition;

        public int ThumbnailWidth;

        public int ThumbnailHeight;

        public Texture2D TextureResult { get; private set; }

        /// <summary>
        /// If this is not null or empty, the texture is exported as a png to the file system.
        /// </summary>
        public string ExportFilePath;

        void Start()
        {
            ThumbnailWidth = 256;
            ThumbnailHeight = 256;
            
            Render("render_manual");
        }

        private void AssignRenderTextureToCamera()
        {
            if (ThumbnailCamera != null && TargetRenderTexture != null)
            {
                ThumbnailCamera.targetTexture = TargetRenderTexture;
            }
            else if (ThumbnailCamera.targetTexture != null)
            {
                TargetRenderTexture = ThumbnailCamera.targetTexture;
            }
        }

        private void Render(string filename)
        {
            StartCoroutine(DoRender(filename));
        }

        IEnumerator DoRender(string filename)
        {
            yield return new WaitForEndOfFrame();

            ExecuteRender(filename);
        }

        private void ExecuteRender(string filename)
        {
            if (ThumbnailCamera == null)
            {
                throw new System.InvalidOperationException("ThumbnailCamera not found. Please assign one to the ThumbnailGenerator.");
            }

            if (TargetRenderTexture == null && ThumbnailCamera.targetTexture == null)
            {
                throw new System.InvalidOperationException("RenderTexture not found. Please assign one to the ThumbnailGenerator.");
            }

            AssignRenderTextureToCamera();

            Texture2D tex = null;

            {   // Create the texture from the RenderTexture

                RenderTexture.active = TargetRenderTexture;

                tex = new Texture2D(ThumbnailWidth, ThumbnailHeight);

                tex.ReadPixels(new Rect(0, 0, ThumbnailWidth, ThumbnailHeight), 0, 0);
                tex.Apply();

                TextureResult = tex;
            }

            // Export to the file system, if ExportFilePath is specified.
            if (tex != null && !string.IsNullOrWhiteSpace(ExportFilePath) && !string.IsNullOrWhiteSpace(filename))
            {
                string dir = System.IO.Path.GetDirectoryName(ExportFilePath);
                if (!System.IO.Directory.Exists(dir))
                {
                    System.IO.Directory.CreateDirectory(dir);
                }

                foreach (char c in System.IO.Path.GetInvalidFileNameChars())
                {
                    filename = filename.Replace(c, '_');
                }
                
                string finalPath = string.Format("{0}/{1}.png", ExportFilePath, filename);

                byte[] bytes = tex.EncodeToPNG();
                System.IO.File.WriteAllBytes(ExportFilePath + "/" + filename + ".png", bytes);
            }
        }
    }
}

Finally, assign the Camera you created to the ThumbnailCamera property in the Inspector for the ThumbnailGenerator1 script, and specify the ExportFilePath as “./test”, without the quotes. This will output the rendered image to the test directory.

Run the scene. A thumbnail image is written out to the file system.

Automatic Thumbnail Generation

Now that we have a script to render out a thumbnail, we still need to automate object creation and placement into the scene. Basically, Step (5) above can be automated so that an object can be created, snapshotted, and then destroyed.

Modify the ThumbnailGenerator script

Update the ThumbnailGenerator script.

1. Make Render() public
2. Remove the call to Render() in the Start() function.

With this, we still have a ThumbnailGenerator, but Render() must be called from somewhere else to actually render the view. The script is ThumbnailGenerator2.cs in the git example.

Add an EmptyObject called “Stage”, parent it to the “ThumbGen”. When the object is instantiated, it will be placed at the position of the Stage object. It serves as the camera’s “look at” point. Since we’ll be automatically creating the object, we don’t need the cube from before.

Delete the Cube.

(Also note that from here on out, you’ll need the FBX objects from the Resources folder contained in github.)

Create an Object Loader

The first thing we’ll need for auto-generating an object’s thumbnail is an object loader. In this example, I’m simply calling Resources.Load() on one of the objects in our list, using the path to the model resource in the /Resources folder as an identifier. You may consider any other sort of identifier (i.e. unique name or inventoryId, etc.) to load your models, but that’s beyond the scope of this article.

Also keep in mind that Resources.Load() loads the prefab of a GameObject from the asset database. It doesn’t create an instance to be used in the game. Technically, we *can* use the loaded GameObject prefab, but it’s typically dangerous to do this because you can potentially modify all subsequent instances of the GameObject. It’s safer to call GameObject.Instantiate() on the loaded GameObject instead.

We’ll use the following class below as our ObjectLoader.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

namespace ThumbGenExamples
{
    public class ObjectThumbnailActivity
    {
        private GameObject _object;
        private ThumbnailGenerator2 _thumbGen;

        private string _resourceName;

        public ObjectThumbnailActivity(ThumbnailGenerator2 thumbGen, string resourceName)
        {
            _thumbGen = thumbGen;
            _resourceName = resourceName;
        }
        
        public bool Setup()
        {
            if (string.IsNullOrWhiteSpace(_resourceName))
                return false;

            GameObject prefab = Resources.Load<GameObject>(_resourceName);

            GameObject stage = GameObject.Find("Stage");

            _object = GameObject.Instantiate(prefab, stage?.transform);

            Camera cam = _thumbGen.ThumbnailCamera.GetComponent<Camera>();

            cam.transform.LookAt(stage.transform);
            
            return true;
        }

        public bool CanProcess()
        {
            return true;
        }

        public void Process()
        {
            if (_thumbGen == null)
                return;

            string filename = string.Format("render_objectloader_{0}", _resourceName);
            _thumbGen.Render(filename);
        }

        public void Cleanup()
        {
            if (_object != null)
            {
                GameObject.Destroy(_object);

                _object = null;
            }
        }
    }
}

Most of the implementation for object loading will occur in Setup() and Cleanup(), but let’s start with a constructor.

private ThumbnailGenerator2 _thumbGen;
private string _resourceName;

public ObjecctThumbnailActivity(ThumbnailGenerator2 thumbGen, string resourceName)
{
     _thumbGen = thumbGen;
     _resourceName = resourceName;
}

We’re simply saving the resourcePath and thumbGen to private members of the ObjectThumbnailActivity, so that the Setup() method can use them. You may be asking yourself, why don’t we just pass in the resourcePath to Setup()? We’ll cover that in the Optional Improvements section below.

Now we can implement Setup() and Cleanup().

Setup() instantiates a GameObject based on the specified resourcePath, and then positions the object at the “Stage” position.

    public bool Setup()
    {
        if (string.IsNullOrWhiteSpace(_resourceName))
            return false;

        GameObject prefab = Resources.Load<GameObject>(_resourceName);

        GameObject stage = GameObject.Find("Stage");

        _object = GameObject.Instantiate(prefab, stage?.transform);

        Camera cam = _thumbGen.ThumbnailCamera.GetComponent<Camera>();

        cam.transform.LookAt(stage.transform);
            
        return true;
    }

Cleanup() simply destroys the instantiated GameObject.

        public void Cleanup()
        {
            if (_object != null)
            {
                GameObject.Destroy(_object);

                _object = null;
            }
        }

Now we can implement CanProcess() and Process() as follows:

        public bool CanProcess()
        {
            return true;
        }

For now, CanProcess() will simply return true.

        public void Process()
        {
            if (_thumbGen == null)
                return;

            string filename = string.Format("render_objectloader_{0}", _resourceName);
            _thumbGen.Render(filename);
        }

Now that we have our ObjectThumbnailActivity, the MonoBehaviour class below uses the activity to execute all its methods.

ObjectProcessor.cs script

    public class ObjectProcessor : MonoBehaviour
    {
        private ObjectThumbnailActivity _activity;

        void Start()
        {

        }

        void Awake()
        {
            ThumbnailGenerator2 thumbGen = GetComponent<ThumbnailGenerator2>();


            _activity = new ObjectThumbnailActivity(thumbGen, "objects/Cube");
            StartCoroutine(DoProcess());
        }

        IEnumerator DoProcess()
        {
            yield return new WaitForEndOfFrame();

            _activity.Setup();

            if(!_activity.CanProcess())
    
            yield return null;

            _activity.Process();

            StartCoroutine(DoCleanup());
        }

        IEnumerator DoCleanup()
        {
            yield return new WaitForEndOfFrame();

            _activity.Cleanup();
        }
    }

Things to notice here, we use coroutines for calling DoProcess() and DoCleanup(). For DoProcess(), we allow the Setup() method to do a lot of the work, so yielding for WaitForEndOfFrame() allows Setup() to finish doing any initialization needed for the GameObject instantiation. The coroutine that calls DoCleanup() also serves the same purpose. The coroutine allows the Process() method to fully run before cleaning up the GameObject.

Attach this MonoBehaviour to the root “ThumbGen” object, run it, and the object’s thumbnail will be generated.

You may need to place the “Stage” object in a position that would result in a more pleasing thumbnail, but that’s on you.

When you ran the scene, you may have caught a glimpse of the instantiated object before it gets destroyed by the activity.

After running the scene and getting a thumbnail generated, you’re probably saying, “But that’s exactly what the manual scene setup does, and even with far less code”. While this is true, the ObjectThumbnailActivity sets us up to repeatedly load objects without us having to delete the current object and place a new one. We’re getting to that next.

Multiple Object Loading

Now that we have a way to load/unload our objects, we still need a way to process all of them automatically. There are a few ways we can do this, and we’ll go over them below.

• First, is simply loading the objects in a loop. This is simple to implement, but it can pause your game for a significant amount of time while it loads all the objects in the list. This is obviously dependent on the number of items we plan to take thumbnail images of.

• Second, is multi-threading. This is the ideal solution for this situation, as you can have thumbnails load in the background while your game is loading, and the game would show no signs of slowing down. It’s possible to do this in Unity, but this requires using the newer Jobs system, and the solution is far more complicated. This article does not cover that.

• Third, is using Unity’s coroutine system. This solution is kind of an “in between” the two solutions above. Thumbnail generation would occur on the same thread as our game, but it splits up the workload at a specified time interval such that it seems like the game is playable while there is some work being done intermittently. This is the solution that we’ll go with in this article, as it strikes a good balance between implementation complexity and end-user experience.

Per-Frame Processor

Lucky for us, we already have an ObjectProcessor using coroutines, from the section above.

This means we can use that class as the basis for our new class. We just need to add a way to repeatedly process a list of objects.

Here’s the implementation of the per-frame processor, called the MultiObjectProcessor:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

namespace ThumbGenExamples
{
    public class MultiObjectProcessor : MonoBehaviour
    {
        private List<ObjectThumbnailActivity> _activities;

        private ObjectThumbnailActivity _curActivity;

        void Start()
        {
        }

        void Awake()
        {
            _curActivity = null;
            _activities = new List<ObjectThumbnailActivity>();

            ThumbnailGenerator2 thumbGen = GetComponent<ThumbnailGenerator2>();

            string[] objectResourceNames =
            {
                "objects/Cube",
                "objects/Cylinder",
                "objects/Capsule",
                "objects/Sphere"
            };

            foreach (var name in objectResourceNames)
            {
                var thumbActivity = new ObjectThumbnailActivity(thumbGen, name);
                _activities.Add(thumbActivity);
            }
        }

        void Update()
        {
            if (_curActivity == null)
            {
                if (_activities.Count > 0)
                {
                    _curActivity = _activities[0];
                    
                    _curActivity.Setup();

                    StartCoroutine("DoProcess");

                    _activities.RemoveAt(0);
                }
            }
        }

        IEnumerator DoProcess()
        {
            yield return new WaitForEndOfFrame();

            if (_curActivity == null)
                yield return null;

            if(!_curActivity.CanProcess())
                yield return null;

            _curActivity.Process();

            StartCoroutine(DoCleanup());
        }

        IEnumerator DoCleanup()
        {
            yield return new WaitForEndOfFrame();

            if (_curActivity != null)
            {
                _curActivity.Cleanup();
                _curActivity = null;
            }
        }
    }
}

First thing to note, is a quick reminder that this is a MonoBehaviour. This is needed so that we can leverage the Coroutine system.

Awake() sets up the list of thumbnail activities to process, and initializes the current activity to null.

The Update() function then checks if the current activity is null. If it is, that means the processor is not doing anything, so the function proceeds to initialize the next activity. We check if the activities list is not empty, set the first activity as the current activity, remove it from the list, and call StartCoroutine() to execute the activity’s Setup() and Process() functions. Once the Process() function has completed, the current activity is set to null, so that the Update() function can pick up the next activity in the list.

On the “ThumbGen” object in the Unity Inspector, remove the ObjectProcessor component, and add the MultiObjectProcessor component.

Run the scene. This time, you may notice that the multiple objects in our list appear and disappear in a split second, similar to our original ObjectProcessor.

Check the output directory, and notice the four thumbnails that have been created.

That’s it! That’s pretty much the basics of the thumbnail object processing functionality. Below is a discussion of the design, as well as a few more improvements that you may want to make to the code.

Design

All the work above may seem like some over-engineering. To an extent, it is. It really depends on what level of flexibility you need for your own project. If you need to simply set up an object, snapshot it, and clean it up, why not just run through a for loop or something? Well, the way it’s designed above enforces a lot of principles known as SOLID in computer science.

Okay, here is what the code would look like if it was all thrown into a single class or method:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

namespace ThumbGenExamples
{
    public class ThumbGenProcessor : MonoBehaviour
    {
        private List<string> _objectResourceNames;

        private GameObject _curObject;

        public RenderTexture TargetRenderTexture;

        public Camera ThumbnailCamera;

        public int ThumbnailWidth;

        public int ThumbnailHeight;

        public Texture2D TextureResult { get; private set; }

        /// <summary>
        /// If this is not null or empty, the texture is exported as a png to the file system.
        /// </summary>
        public string ExportFilePath;

        public ThumbGenProcessor()
        {
            ThumbnailWidth = 256;
            ThumbnailHeight = 256;
        }

        void Start()
        {
        }

        void Awake()
        {
            _curObject = null;
            
            _objectResourceNames = new List<string>()
            {
                "objects/Cube",
                "objects/Cylinder",
                "objects/Capsule",
                "objects/Sphere"
            };
        }

        void Update()
        {
            if (_curObject == null)
            {
                if (_objectResourceNames.Count > 0)
                {
                    string resourceName = _objectResourceNames[0];
                    if (!string.IsNullOrWhiteSpace(resourceName))
                    {
                        GameObject prefab = Resources.Load<GameObject>(resourceName);

                        GameObject stage = GameObject.Find("Stage");

                        _curObject = GameObject.Instantiate(prefab, stage?.transform);

                        Camera cam = ThumbnailCamera.GetComponent<Camera>();

                        cam.transform.LookAt(stage.transform);

                        StartCoroutine(DoProcess(resourceName));
                    }
                    _objectResourceNames.RemoveAt(0);
                }
            }
        }

        IEnumerator DoProcess(string resourceName)
        {
            yield return new WaitForEndOfFrame();

            if (_curObject == null)
                yield return null;

            if (ThumbnailCamera == null)
                yield return null;

            string filename = string.Format("render_objectloader_{0}", resourceName);
            Render(filename);

            StartCoroutine(DoCleanup());
        }

        IEnumerator DoCleanup()
        {
            yield return new WaitForEndOfFrame();

            if (_curObject != null)
            {
                GameObject.Destroy(_curObject);

                _curObject = null;
            }
        }

        private void AssignRenderTextureToCamera()
        {
            if (ThumbnailCamera != null)
            {
                ThumbnailCamera.targetTexture = TargetRenderTexture;
            }
        }

        public void Render(string filename)
        {
            StartCoroutine(DoRender(filename));
        }

        IEnumerator DoRender(string filename)
        {
            yield return new WaitForEndOfFrame();

            ExecuteRender(filename);
        }

        private void ExecuteRender(string filename)
        {
            if (ThumbnailCamera == null)
            {
                throw new System.InvalidOperationException("ThumbnailCamera not found. Please assign one to the ThumbnailGenerator.");
            }

            if (TargetRenderTexture == null)
            {
                throw new System.InvalidOperationException("RenderTexture not found. Please assign one to the ThumbnailGenerator.");
            }

            AssignRenderTextureToCamera();

            Texture2D tex = null;

            {   // Create the texture from the RenderTexture

                RenderTexture.active = TargetRenderTexture;

                tex = new Texture2D(ThumbnailWidth, ThumbnailHeight);

                tex.ReadPixels(new Rect(0, 0, ThumbnailWidth, ThumbnailHeight), 0, 0);
                tex.Apply();

                TextureResult = tex;
            }

            // Export to the file system, if ExportFilePath is specified.
            if (tex != null && !string.IsNullOrWhiteSpace(ExportFilePath) && !string.IsNullOrWhiteSpace(filename))
            {
                string dir = System.IO.Path.GetDirectoryName(ExportFilePath);
                if (!System.IO.Directory.Exists(dir))
                {
                    System.IO.Directory.CreateDirectory(dir);
                }

                foreach (char c in System.IO.Path.GetInvalidFileNameChars())
                {
                    filename = filename.Replace(c, '_');
                }


                string finalPath = string.Format("{0}/{1}.png", ExportFilePath, filename);

                byte[] bytes = tex.EncodeToPNG();
                System.IO.File.WriteAllBytes(ExportFilePath + "/" + filename + ".png", bytes);
            }
        }
    }
}

There’s a lot going on here, and sure, it works, but reading it takes a little longer to pick it apart and figure out what’s doing what. It is more difficult to test and debug just the thumbnail code, just the object creation code, or just the multiple-object handling code.

The classes we implemented above separate the concerns of our final goal. One class is solely responsible for taking a snapshot of an object, one class is solely responsible for creating and destroying an object, and one class is solely responsible for looping through a list of objects.

In the section below, we’ll go over a way to extend the MultiObjectLoader (or per-frame processor) to do more than just take thumbnail images of objects. The generic IGameObjectActivity interface makes this architecture a lot more reusable, extensible, flexible, and testable.

Optional Improvements

Taking it a step further…

Focus Point

I have to mention “focus point”. This is optional, but can help improve the resulting thumbnail image. It’s very possible, and maybe very likely, that the focus point of an object for the purpose of a thumbnail image is not the same location as the object’s position.

Here’s an example from one of my game’s models:

The artillery unit’s origin is at the base of the model, so that it can interact with the terrain in the game without having to artificially move the origin during runtime, which can be unnecessary transform calculations. This is a convention that I decided on, and may not be what your game is using. But this has the adverse effect that, for thumbnail generation, the visual center point is too low at the object’s origin. Ideally, the visual center point would be at the base of the turret instead.

If this is the case, there are a couple of techniques we can use. I’ll only mention a few of them, and we’ll use one of them. 

• One technique is to use the average point of all vertices in the 3D model. This can be time-consuming to calculate, and may not necessarily produce the “aesthetic center point” of the model.
• Another technique is to take the average point of the object’s bounding box, but this also presents the same problem of possibly resulting in a center point that’s not aesthetically pleasing.
• Lastly, we can use a focus point object. This is an Empty Object, which just contains a Transform for the sole purpose of informing our ThumbnailGenerator where to look. This is the technique we’ll use. 

Parent an Empty GameObject to each of the target objects to serve as the focus point for the camera, and name it “ThumbFocus”.

Add following block of code to the Setup() function in ObjectThumbnailActivity.cs. Basically, replace the existing camera LookAt() function call with this:

Transform camTarget = stage.transform;
            
List<Transform> transforms = new List<Transform>();
GameUtils.GetTransformsWithNameInHierarchy(_object.transform, "thumbfocus", ref transforms);
if (!transforms.IsEmpty())
{
     Transform first = transforms.FirstOrDefault();
     if (first != null)
     {
         camTarget = first;
         break;
     }
}


cam.transform.LookAt(camTarget);

You’ll need the following utility function below:

public static void GetTransformsWithNameInHierarchy(Transform root, string name, ref List<Transform> transforms)
{
     if (root == null)
          return;

     foreach (Transform t in root)
     {
          if (string.Compare(t.name, name, true) == 0)
              transforms.Add(t);

          GetTransformsWithNameInHierarchy(t, name, ref transforms);
     }
}

This searches for an EmptyObject named “ThumbFocus”, and if it can’t find it, it just resorts to the object’s position.

Refactor ObjectThumbnailActivity to IGameObjectActivity

In the Design section above, I mentioned that the implementation we went over was over-engineered. It certainly is for the immediate use-case, but let’s look at how that architecture works for us and pays itself off with more complex use-cases.

In fact, I had such a use-case, and it’s the very reason why I wanted to write this article. For my turn-based strategy game, I needed thumbnails for not only the units (tanks, planes, ships), but for the structures (factories, seaports, airports), and also for the terrain (field, forest, road, mountain, beach, water, and bridge).

Each of these different types (units, structures, and terrain) all have specific requirements for setting up the object before taking the thumbnail snapshot of it. There are different reasons for this.

For one, the terrain types generate their object geometry in different ways. For example, the forest tile type is basically the field type with some tree geometry, and the beach and bridge types require the water model to indicate its relation to the water.

if (terrainType == TerrainType.Field)
{
    RenderField();
}
else if (terrainType == TerrainType.Forest)
{
    RenderField();
    RenderTrees();
}

...

if (terrainType == TerrainType.Water)
{
    RenderWater();
}
else if (terrainType == TerrainType.Bridge)
{
    RenderWater();
    RenderBridge();
}
else if (terrainType == TerrainType.Shore)
{
    RenderShore();
    RenderWater();
}

For another, my seaport requires additional logic because I have two variants of it. A straight and a diagonal, but I only want to show the straight model, so I need to hide the diagonal.

if (model.Name.Contains("diagonal")
{
    HideModel(model);
}

RenderSeaportModel();

The above is just pseudocode for how all these issues were resolved, but I wanted to point out that there are indeed uses for supporting this architecture.

The best part about this is, you can take advantage of this pattern for other things that need per frame processing without creating a separate thread. You would write a new concrete class that implements the IGameObjectActivity interface, instantiate this new class and add it to the queue. Here is IGameObjectActivity.cs:

    /// <summary>
    /// Interface for any activity to perform
    /// </summary>
    public interface IGameObjectActivity
    {
        /// <summary>
        /// Run any setup, or bail if setup fails
        /// </summary>
        /// <returns></returns>
        bool Setup();

        /// <summary>
        /// Allow the runtime actvity to check
        /// if the process can be performed
        /// </summary>
        /// <returns></returns>
        bool CanProcess();

        /// <summary>
        /// Actually perform the activity.
        /// </summary>
        void Process();

        /// <summary>
        /// Run any cleanup necessary to advance to a next activity
        /// </summary>
        void Cleanup();
    }

So, for example, for the pseudocode above, I would have three separate activities:

public class UnitThumbActivity : IGameObjectActivity
{
    ...  // Implements unit-specific thumbnail behavior.
}

public class StructureThumbActivity : IGameObjectActivity
{
    ... // Implements structure-specific thumbnail behavior.
}

public class TerrainThumbActivity : IGameObjectActivity
{
    ... // Implements terrain-specific thumbnail behavior.
}

Similarly, we can create a new class ThumbnailActivity, which is EXACTLY the same as ObjectThumbnailActivity, but now implements IGameObjectActivity. It’s exactly the same because I had already named the functions to implement the interface.

/// <summary>
/// This class is exactly like ObjectThumbnailActivity.cs, but 
/// this now implements the IGameObjectActivity interface.
/// </summary>
public class ThumbnailActivity : IGameObjectActivity
{
    ... // Implementation is exactly the same as the existing ObjectThumbnailActivity.
}

And now in the MultiObjectProcessor, the _activities list contains objects that implement the IGameObjectActivity interface:

private List<IGameObjectActivity> _activities;

...

_activities = new List<IGameObjectActivity>();

...

Our per-frame processor now operates on any object that implements the interface IGameObjectActivity. It is no longer limited to just rendering thumbnails, or one type of thumbnail, for example.

Conclusion

And there you have it. The entirety of the project is available on github. While you can probably just take that code wholesale, I took the opportunity in this article to start from manual thumbnail creation in Unity to automating the entire process, with a little taste of Dependency Injection (DI) and good ol’ software engineering practices along the way.

Hope you learned something, but either way…

Make it fun!

For the game that I’m currently working on, I decided to implement support for Unity’s AssetBundles. Right out of the gate, I jumped into the API and started implementing it in my game, since it seemed so straightforward. Lo and behold, I hit the wall of failure. The concept is clear, but getting it all implemented such that it didn’t slow down my day-to-day development flow was a challenge. Let me explain how I got through things. But first…

What are AssetBundles?

Go ahead and skip this section if you already know what they are, but for the rest of us, here is a brief explanation.

An AssetBundle is a library of assets that can be stored as a file that is hosted separately from the game build. Before AssetBundles were more accessible for developers to work with, any assets that you used in your game (i.e. images, meshes, materials, animations, audio, etc.) needed to be included in the project as part of the game build. That’s fine and all, but as the number of assets in the game grew, it also meant the amount of possibly unused assets were unnecessarily loaded into memory, wasting a lot of system resources, thereby bogging down your game build, and not to mention your entire system.

To alleviate this problem, developers would use the Resources folder to manage their own asset loading, as Unity treats this folder differently from the rest of the assets. This is more of a traditional convention with resource management, in that developers can load their assets “on-demand”, and thereby be a little bit more picky about what assets are loaded into RAM, at any moment in time during the game’s run. This also meant that developers are also responsible for unloading the assets from RAM.

Gamedevs still use the Resources folder (I’ve seen the files in a few popular shipped Unity games), but Unity Technologies has since identified several problems with using the Resources folder for a production release, that can be found here.

So Unity Technologies recommends that AssetBundles are the way to go for packaging content, especially if you plan on delivering that content alongside your game build, or some time in the future with patches and content updates. There’s plenty more information in Unity’s documentation on AssetBundles, so I’m just going to move on now to the more “in-practice” side of things; areas that don’t really seem to be covered very well in Unity documentation.

Multiple Sources

The challenge with AssetBundles isn’t so much the actual authoring and loading of them. It’s that, every time you make a change to an asset in the AssetBundle, you have to rebuild. This completely kills your iteration time on editing those assets. Is there no way around this!? Well… There are ways to expedite the testing process of assets that you have included in an AssetBundle, but it really depends on what stage of implementation you are on your game. This determines which method you should use for testing your assets.

You could be testing the asset:

1. In the Editor, as a loose asset. This tests the asset itself, but not the AssetBundle that contains it.
2. In the Editor, as an asset in an AssetBundle. This tests the asset in the AssetBundle, but in the context of the Editor. Typically, the players of your game will only have a standalone build. Not something tied to the editor.
3. In a published Development Build, from an AssetBundle. This tests the asset in the AssetBundle in a standalone build, with some extra debug tools, to track down any errors that may not show up when testing in the Editor.
4. In a published Release Build, from an AssetBundle. This tests the asset in the AssetBundle in a standalone build which would be released to players, with absolutely no extra debug help, other than maybe some output logs. If a bug happens here that cannot be replicated in one of the previous 3 cases, it can be difficult to fix.

These four ways to test an asset and its AssetBundle can get confusing really quickly. To alleviate this confusion, Unity had provided the AssetBundleManager, which is a set of scripts that wrap up the AssetBundle API into a nicer high-level API that abstracts away the various methods an asset can be loaded, from an AssetBundle. Well, that’s nice!….. If it worked.

AssetBundleManager

“If it worked? What do you mean?”

Heh. I eventually got it working, and you can scroll down to see how if you want to skip this rant.

I’m running Unity 5.6.3f, which was released in August 2017. That was a couple of years ago, as of this writing. Within that time, AssetBundleManager has been marked as deprecated by Unity, yet there is no other alternative* in its place (other than dealing with the AssetBundle API directly), which is quite confusing to us users who are just starting out with it. Straight off of bitbucket, the AssetBundleManager will not work, without a few changes and configurations. This puts developers who would like to develop the “right” way by using AssetBundles in a sort of limbo, and I wouldn’t blame any developer who feels like they got shortchanged by this sort of messaging from Unity’s documentation.

So, the only course of action is to fix the AssetBundleManager to my needs, as suggested by Unity, but is by no means anything that I had planned on spending my time on.

* There is a new Addressable Asset System which provides far more than what AssetBundleManager gives. It has recently gotten out of “preview” status, so adoption of this new system is still happening. 

Okay, end rant.

Back to Basics

I certainly want to utilize the benefits of Unity’s AssetBundleManager. It has everything I need to alleviate the inefficiencies of testing AssetBundle builds. I made the mistake of immediately integrating the AssetBundleManager into my game, and that seriously set me back, because I started to get so confused with the various asynchronous load handlers. After much failure, I had to get back to basics, so I downloaded Unity’s AssetBundleDemo and started from there.

What follows is pretty much a walkthrough of the demo.

Get the AssetBundleDemo from bitbucket here. It contains the AssetBundleManager.

Create a new project, and put the AssetBundleDemo contents into the project’s Assets folder.

Also, get the AssetBundle Browser from github as recommended on this Unity page. Drop this in your project’s Assets folder too.

The scene of interest is AssetLoader.unity. Load that up, and we’ll step through what to do.

First, this is what the AssetBundleManager menu looks like.

The three we’ll be using are:

• Build Assets

• Simulation Mode

• Local AssetBundle Server
  

I don’t think I had a need for “Build AssetBundles from Selection”, so I have not tried it. And I have no idea what “Build Player (for use with engine code stripping)” is. I tried it, and did not get any good results, and I found it unnecessary.

This is what the AssetBundle Browser UI looks like:

Follow Along

This assumes you are developing on Windows, so, sorry in advance to all you other development platforms.

In the following tests, we should see a Unity cube appear on the screen. If it doesn’t, then the AssetBundles failed to load.

The asset of interest is called MyCube, and it is assigned the “cube-bundle” AssetBundle group.

In the scene, the Loader object has a LoadAssets.cs script, which sets up the AssetBundleManager and attempts to load the AssetBundle and the cube object from that AssetBundle. 

The Unity Editor

Editor with No Changes

The very basic test you can do is in the Editor. Press Play. Notice that no cube shows up. 

This is expected. Since we have not built an AssetBundle yet, there is nothing to load.

Editor with Simulation Mode

The next test you can do is activate Simulation Mode, and press Play. The cube shows up.

Simulation Mode is great for very quick tests of the assets that are labeled for an AssetBundle. Use this mode when iterating on edits of an Asset. But in reality, this mode doesn’t even check for the AssetBundle itself, bypassing that entire system. It simply loads the asset directly from your Assets tree. This isn’t a very comprehensive test for a final build. So, be warned.

Editor with the Local Server

Next test, turn off Simulation Mode, and turn on the Local AssetBundle Server.

Go into Windows Processes, under Unity processes, notice mono.exe. That’s the Local Assetbundle server.

Play the game, and notice that nothing loads. See the error message. The game is connected to the local server, but it fails to download the AssetBundles.

This is because we still have not built any AssetBundles, so we’ll do that next. Build AssetBundles, and keep the Local Server enabled.

Running the game in the Editor now shows the Cube.

Great! So we can now load the cube using the AssetBundleManager in the Editor, using either Simulation Mode or the Local Server when AssetBundles are built.

Standalone Builds

Now let’s publish some builds. For now, disable the Local Server from the Editor menu.

For simplicity’s sake, we are going to use Unity’s StreamingAssets folder to automatically have the AssetBundles copied to the build when the Player is built.

For convenience, the AssetBundle Browser has a check box for pushing to StreamingAssets, so we’ll use that to build our AssetBundles instead of the AssetBundleManager menu.

Change the output path from AssetBundles/WindowsStandalone to AssetBundles/Windows. The LoadAssets script is expecting this path, simply due to the way AssetBundleManager has been written.

All of this special-case treatment of the AssetBundleManager versus AssetBundle Browser, in regards to StreamingAssets and output paths, was definitely part of my confusion. It’s as if Unity Technologies just stopped what they were doing and checked it in without a second thought about how the output will be affected. Although there’s adequate warning about the AssetBundleManager being deprecated, I’m sure it’s led to a lot of frustration for anyone running this demo.

Development Build with StreamingAssets

Click on Build in the AssetBundle Browser. This copies the AssetBundles to the StreamingAssets folder. (As a side note, notice that the StreamingAssets output is not contained in a /Windows subfolder. This is something that you would also have to change in the AssetBundle Browser scripts if you want to keep the platform-specific folder organization.)

Okay, make a standalone build, with the Development Build Option on (I like to publish my builds to an “/out” directory”). 

Verify that the build has written out the AssetBundles in the StreamingAssets folder.

Run the build. Notice no cube appears.

Also notice the debug error log at the lower left. This is very helpful in debugging why assetbundles don’t load, among other errors.

This is the same error as the one we got when running the game in the Editor.

Development Build with the Local Server

Activate the Local Server from the Unity Editor, and then re-run the standalone Development Build.

Now, the cube loads. This development build is communicating with the Local AssetBundle Server. The content that is being served is contained in the AssetBundles directory in the root of your project, and NOT the StreamingAssets folder. This is the snippet of LoadAssets.cs that uses the Local Server:

    // Initialize the downloading URL.
    // eg. Development server / iOS ODR / web URL
    void InitializeSourceURL()
    {
        // ...
#if DEVELOPMENT_BUILD || UNITY_EDITOR
        // With this code, when in-editor or using a development builds: Always use the AssetBundle Server
        // (This is very dependent on the production workflow of the project.
        //      Another approach would be to make this configurable in the standalone player.)
        AssetBundleManager.SetDevelopmentAssetBundleServer();
        return;
#else
        // ...
#endif
}

Convenient for testing, but not at all reflective of a completely independent standalone build.

Release Build

And finally, create a full Release build (Disable the Development Build option, and rebuild to a Release executable).

The contents of this build should be pretty much the same as the Development Build. The main difference now is that we are now looking at the StreamingAssets folder, regardless of if we have the Local AssetBundle Server running.

Run the build. Notice no cube shows.

Open the output_log.txt, notice the error.

It couldn’t load the manifest. Whoops, I just threw in new vocabulary. The manifest AssetBundle is the entry point into loading the rest of the assets in that AssetBundle. In this case, it holds the name of the platform: “Windows”.

The Fix

This brings us back to modifying the LoadAssets script. Open it up in an IDE (You’re using Visual Studio, right? Because you should be…).

Find this block of code in InitializeSourceURL():

AssetBundleManager.SetSourceAssetBundleURL(Application.dataPath + "/"); 

Comment this out, and change it to this instead:

AssetBundleManager.SetSourceAssetBundleDirectory("/"); 

What this does is set the AssetBundle path to the root of the StreamingAssets folder.

Before rebuilding the release build with the above script change, be sure to close output_log.txt if you have it open. Unity can’t build to the same output directory if any file in that path is open.

Re-build the release build, and run the executable. The cube shows up!

The Unity documentation for the AssetBundle Browser states that the StreamingAssets path is useful for testing, but should not be used in production. I’m not sure why this would be the case though; I plan on using StreamingAssets for my project.

What I described above is only one solution to getting this to work, so you may have to modify the solution for your own build pipeline if you choose to use these scripts as a base.

Review

Here are some guidelines for when to use what mode of testing:

1. Test in Simulation Mode when testing loose assets marked as AssetBundle files in the editor
2. Test with the Local Server to test published AssetBundle files (they must be published to the AssetBundles folder at the root of your project).
3. Test with the Local Server (or published AssetBundles in StreamingAssets) also with published Development Builds.
4. Test the Final Build and use the output_log.txt to debug any problems at this point. But hopefully, all the previous steps helped that this stage is not a problem. But, since loading goes through completely different paths, there is still a chance that the AssetBundles do not load at any stage of this development.

Going through all these modes should give you a clue on how and when to use these different techniques. Throughout the development of your game and as you create more content, you’ll want to test on the simplest level, via Simulation Mode, all the way to the published non-development Release Build, with the output_log.txt file.

As you can see from the entire process above, it adds a significant amount of overhead to your development workflow. You can always build tools to automate some things, like auto-assigning AssetBundle ids to certain assets, but ultimately, you (or someone else) will still have to test the assets on every stage of development, from inside the Editor to a full-blown Release build. And this essentially introduces a lot more chances for bugs to show up, and I hate bugs which show up in a Release build, that cannot be reproduced in a Debug build. Ugh!

NOTE: This article only covers the very basic loading method for AssetBundles that get locally published alongside the game build, using StreamingAssets. AssetBundles can also be loaded from other sources, like a web server, for example.

The Future of AssetBundles

In recent months, Unity Technologies has spent a large amount of effort to get the new Addressable Asset System to a solid, production-ready state. The Addressables system is meant to alleviate all the issues that AssetBundles have. It’s meant to replace the AssetBundleManager and Browser and add a lot more robustness to the inherent difficulties of dealing with resource management. Some developers have had success with the current state of Addressables, and others have not.

All in the name of preparing your game for a “commercial” release. Of course, if you’re not looking to commercially release your game, I probably could not recommend using AssetBundles. It’s just a large amount of effort that takes away from where your greatest effort should be: making the actual game. In this case, the Resources folder and direct references are still suitable options.

So, whether or not you choose to integrate AssetBundle support in your game, remember to…

Make it fun!

Aside from working on a side project to my side project, which I recently finished, I’ve gotten back to my original side project (my next game) and found myself researching on some tech for a large feature. Talk about “writing in code”! Spit it out! That tech I’ve been researching on is multiplayer networking solutions for Unity. There. Like the title didn’t give it away.

For the uninitiated, multiplayer games are games that can be played with other human beings across the Internet. Yes, this includes social games like those popular Facebook games of old*, and games like Pokemon GO, all the way to games like Call of Duty and World of Warcraft. Some of these games have competitive play and others have cooperative play, and sometimes there’s a combination of both. This, of course then, is different from single player games, like Crossy Road or Flappy Bird, which don’t involve multiple people playing together. Some games actually have both singleplayer and multiplayer modes, like the aforementioned Call of Duty. Nonetheless, I’ve already established that I’m writing about multiplayer here.

Game developers know full well that one of the most difficult systems to build is the multiplayer networking system of a game. I’m not even going to go into how difficult it is to build a Massively-Multiplayer Online (MMO) game, as that’s way out of scope of this article. If you’re still thinking about it, I’d highly recommend reconsidering your life goals.

But enough of the intro.

Unity Networking, Photon SDK, and PUN

For the past several days, I’ve been looking at potential network solutions for Unity, namely Unity Networking (Unity’s first-party solution) and Photon by Exit Games. From what I gathered, Unity Networking still needs some time to mature, whereas Photon is a more established networking package, with a long running track record. I decided to go with Photon for this reason, as well as for its healthy amount of documentation, so I’ll go into more detail about it here.

As of this writing, Photon has a sort of mixed personality. For Unity, it comes in two flavors. Photon SDK for Unity, and Photon Unity Networking. Whazzawha? Right. Photon SDK for Unity is available on the Exit Games website. Whereas Photon Unity Networking (PUN) is available as a downloadable asset package from the Unity asset store. The difference is that PUN is a wrapper around the underlying Photon SDK, to help you save time with the lower level groundwork code already implemented for you. PUN is also supposed to emulate the API of Unity Networking, so that it is more familiar to the Unity network application programmer.

So, I chose to use the Photon SDK for Unity. Why would that be if PUN already provides a lot of the underlying work for me? Well, there’s one feature that is currently not supported by PUN that I need for my next game: Persistence of asynchronous game sessions. I’ll go into more detail about what that is in a moment.

To make things more complicated, doing any Google searches for Photon Networking will inevitably yield results on “Realtime” vs “Turnbased” Photon. The hope for Exit Games was that Photon can be marketed for developing games that are fast-paced, and played simultaneously, or synchronously, between gamers (Realtime), and for games that are slower-paced, and played by one player at a time, or asynchronously, like Chess (Turn-Based). As of May 2016, Exit Games discontinued Photon Turnbased, which by itself sounds as if Photon would no longer support turn-based games. In reality, the Turnbased features have been merged into Photon Realtime. So that means, I’m using Photon RealTime for a turn-based game. How confusing! At the same time that Exit Games was attempting to simplify the delivery of their product(s) to its customers, it managed to confuse even newer customers like me.

Asynchronous Gameplay and Persistence

To support my next game, which is a turn-based game, I need to implement asynchronous gameplay and persistence via something called Webhooks/WebRPCs.

“Asynchronous gameplay” means that the players do not play simultaneously, together on the same turn. From a network programming perspective, this is far less difficult to implement than synchronous gameplay because with synchronous, realtime gameplay, there is no concept of “turns”. With synchronous games, developers have to implement the system such that every player connected to the game can give input and receive a result immediately, so as to produce the illusion that all players “exist” in the same virtual world. In reality, it takes fractions of a second, or even whole seconds to transmit that data across to all other connected players. Network programmers have to implement numerous tricks and techniques like dead reckoning and client prediction to generate the illusion that a player has moved, without any lag in time. Otherwise, if the game goes out of sync, the illusion is completely broken.

For each turn that a player takes in my turn-based game, the current state of the game has to be saved so that the next player can continue from that state. This is where persistence comes in.

So for persistence, or storage, of game sessions in progress, the Photon SDK provides Webhooks and WebRPCs to load and save game data; something that PUN does not provide.

Webhooks and WebRPCs go hand-in-hand for game clients to communicate with web servers. WebRPCs (Remote Procedure Calls) are used by clients to send messages across the network, so that web hooks on a web server can receive those messages and process them in a secure, authenticated environment. Note that web servers are your typical type of server for hosting web pages. So while in this case, a web server is being used as a game server, it could also be the host for web pages to convey that data, say, for players to check their progress, or the states of their game sessions. Also note that web servers are not used as game servers for real-time games. Servers for real-time games require support for higher loads and bandwidth, because they generally process a lot more network traffic than for turn-based games, to maintain synchronicity across all connected clients, as mentioned above. This is why web servers are more suitable for turn-based games.

And with all of that knowledge, I discovered that Photon does not provide web servers, only the hooks! Bah! Before learning about it, I thought Photon was going to be my one-stop shop for all my network and multiplayer needs for my game. At least, that’s what their marketing made me feel like what it was. In reality, a more seasoned network gameplay engineer probably would have been able to see past the bullet points and tell me that I still needed another component to get Photon working the way I wanted for my turn-based game.

Luckily, through Photon documentation (or somewhere on the Photon or Unity forums), I found out that PlayFab is the recommended solution that provides that web server that I’m looking for. Not only that, it provides some other good stuff that I’d need that Photon doesn’t provide, like user accounts, leaderboards, etc. Also, GameSparks is a competitor which provides pretty much the same thing, but I admit that I didn’t do much research into which would be better for me. Since PlayFab has a partnership with Photon, I decided to go with that.

PlayFab

Once I found out that I had to use yet another system besides Photon, my heart sunk. I was already feeling pretty overwhelmed with having to learn the new Photon API. I was reluctant to learn even more than I was planning to. But we’re game developers, right? We keep learning and work through it! That’s what we do!

Turns out PlayFab, after learning what they offer and how to navigate their browser-based Game Manager, provides quite a bit for the entry price of free. I like how it provides user accounts, tracks when and where they’ve logged on, and other analytics, such as segmentation, which is a business term that is beyond the scope of this post. They have this system called PlayStream which is intended for logging, but I haven’t figured this one out yet. What I was most interested in was, of course, how to implement web hooks for the Photon service.

PlayFab has a proprietary scripting system called CloudScript, found in the Automation section of the GameManager, which is essentially Javascript functions on the web server that handle the events sent via Photon from your game client. I find it a bit awkward and time consuming to upload and deploy cloudscript to PlayFab, but since it’s free, and they’re providing the web servers for me, I’m not complaining all that much. At least not yet… Well (that didn’t take long), trudging through more documentation on PlayFab, I discovered that the free tier of the service is severely limiting in terms of how much data you can save to their web server. It’s understandable, as it’s free, but it’s a fair warning to anyone who wants to try it out, that once you roll a game into production and release, you’ll most likely need to choose a non-free tier to get all your data persisted on their servers. 

Keep On Keepin’ On

At the time I am writing this, I’m still in the midst of working through Photon and PlayFab so that I can get a working game flow for the player. While there’s a wealth of information on both services, it’s very fragmented and difficult to piece everything together. In future posts, I’d like to go more in-depth with my Photon and PlayFab setup, because I’m finding it a pain to get through right now. Perhaps others planning to take this route will benefit from my findings. In the mean time…

Make it fun!

 

 

* Of course Facebook is not that old. But in “technology-and-video-game” years, 4-5 years is like 4-5 decades…. Is Farmville even still a thing?