Fog Of War in Unity 3D Part One of Three

The Research

So the first implementation followed this guy's tutorial, he's kinda hard to understand but I just looked at his code. Before getting into the problem of height, his solution has a hard coded entity limit. This is a big deal for me, and anyone making an RTS game where you potentially have hundreds to almost one thousand entities on screen.

So I ended up implementing something like this for Unreal Engine 4 only in Unity. Which uses a texture to create the fog of war effect by modifying the pixels to "reveal" an area.

The Scene

Going to explain the scene setup. This is being run in Unity 5.1, so open a new scene, you should have a main camera, a directional light, and a skybox. The first thing we want to do is make terrain, go to GameObject -> 3D Object -> Terrain. You can paint it however you like using the built in Environment asset package in Unity. I positioned the terrain so the center is at (0,0,0) but you can do whatever you want. Here is what we have so far:

Now you should position and rotate your camera so its "RTS" like (above the terrain looking down), my position is (0, 15, 0) and rotation is (50, 0, 0) on the x, y, z respectively. Now lets add a capsule, GameObject -> 3D Object -> Capsule. If you imported unity's standard assets (I'm not sure what one exactly) but you might have an FramerateCounter in Standard Assets -> Utility -> Prefabs, add that to the scene if you want. Our game view now looks like this:

OK now we are going to create our Fog Of War. Create a quad by going to GameObject -> 3D Objects -> Quad. Rename the quad to "FogOfWar" or something similar, you also need to rotate the quad so its parallel with your terrain (90, 0, 0). Then scale the quad to cover your terrain, this should actually be the maximum size your camera can move around the scene but we won't worry about that stuff in this post. Now with unity's new global illumination, it will consider the fog of war for its calculations but we don't want that! So on the Mesh Renderer of the FogOfWar make sure it does not cast shadows, it doesn't receive shadows and it doesn't use light probes. Should look like this:

Now, this is the last thing before we get to code, while the FogOfWar object is selected drop down the Layer setting (on the top of the Inspector). Then select add layer and create a FogOfWar layer, then assign the object with your new FogOfWar layer.

The Shader

First we are going to write a basic shader, I'm not sure if this is necessary now that I think about it but it's really simple and should run faster than the default shaders. So right click in your project pane in Unity and go to Create -> Shader, I named mine FogOFWarCG. Open that shader up!

Shader "Custom/FogOfWarCG" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
	}
	SubShader {
		Tags { "RenderType" = "Transparent" "Queue" = "Transparent" }
		Blend SrcAlpha OneMinusSrcAlpha
		Cull Off
		LOD 200
		Lighting off
		
		pass {
			CGPROGRAM
			#pragma vertex vert_img
			#pragma fragment frag

			#include "UnityCG.cginc"
	        
			uniform sampler2D _MainTex;
			
			fixed4 frag(v2f_img i) : SV_Target {
				return tex2D(_MainTex, i.uv);
			}
			ENDCG
		}
	}
	FallBack "Diffuse"
}

Nothing too complicated right? If you said yes skip this paragraph. We just have a Texture in the properties, we define the RenderType and Queue in the subshader to be transparent as this object will be transparent. Then we set our Blend, Cull, LOD, and make sure Lighting is off. Next we make a pass and put our cgprogram in it. We use a built in vertex function because we don't need to do anything special there, then we define our own fragment function and include Unity's cg library. Then we grab the texture in cg and then in the frag function we return the color of the texture at the given UV coordinates. If your confused I'm sorry, the focus is not going to be on this shader. All you need to know is that lighting won't effect this object, we can give it a texture, and it's transparent.

OK after you have saved your shader make a material by right clicking in your project pane again and selecting Create -> Material. Name it FogOfWar and attach the shader you just wrote to it, then assign the material to the FogOfWar mesh. Cool, now we can get into some code!

The Code

OK so we are going to have two scripts FogOfWarManager.cs and Revealer.cs, lets start by creating both of those. Right click on the project pane again and go to Create -> C# Script name this one FogOfWarManager, repeat the process but name this one Revealer.

Open FogOfWarManager.cs at the top of the file add "using System.Collections.Generic" and at the top of the class add these variables:

#region Private
/// 

/// The size of the texture in BOTH x and y.
/// Should be a power of 2.
/// 
[SerializeField]
private int _textureSize = 256;
[SerializeField]
private Color _fogOfWarColor;
[SerializeField]
private LayerMask _fogOfWarLayer;

private Texture2D _texture;
private Color[] _pixels;
private List _revealers;
private int _pixelsPerUnit;
private Vector2 _centerPixel;

private static FogOfWarManager _instance;
#endregion

#region Public
/// 

/// Note this is NOT a singleton!
/// This just needs to be globally accessable AND still be a MonoBehaviour.
/// 
public static FogOfWarManager Instance
{
	get
	{
		return _instance;
	}
}
#endregion

OK the top most variables (with the [SerializeField] tag) can be changed in the inspector. The _textureSize defaults to 256 (I like 512, 1024 is really nice and performs fine on my machine 2048 starts to run into performance problems). The _fogOfWarColor is the unrevealed color. And _fogOfWarLayer is the layer that the fog of war object is on.

Next we have our _texture that we create and the _pixels it holds. We have a list of _revealers to iterate over. There's a _pixelsPerUnit, we will go into how that is calculated in a sec. Then there is the _centerPixel, this is pixel location (x, y) that is located in the center of our fog of war mesh. This will be used for a reference point. Then we have a global reference for our Revealer class (Note that this not a singletone, just a global reference, might also be a code smell...).

Time for some meat!

private void Awake()
{
	_instance = this;

		
	var renderer = GetComponent();
	Material fogOfWarMat = null;
	if (renderer != null)
	{
		fogOfWarMat = renderer.material;
	}

	if (fogOfWarMat == null)
	{
		Debug.LogError("Material for Fog Of War not found!");
		return;
	}

	_texture = new Texture2D(_textureSize, _textureSize, TextureFormat.RGBA32, false);
	_texture.wrapMode = TextureWrapMode.Clamp;

	_pixels = _texture.GetPixels();
	ClearPixels();

	fogOfWarMat.mainTexture = _texture;

	_revealers = new List();
		
	_pixelsPerUnit = Mathf.RoundToInt(_textureSize / transform.lossyScale.x);

	_centerPixel = new Vector2(_textureSize * 0.5f, _textureSize * 0.5f);
}

In the Awake function we first assign our _instance to this so the Revealers can access this class. Then we grab the renderer we are attached to (this will be attached to our FogOfWar game object) and grab the material from that renderer. Then we do some error handling on the material. We create our _textrue with a new Texture2D that is the size of the _textureSize in X and Y, we also set the format to RGBA32 and we do not enable mip maps. Next we get the _pixels from the _texture, the ClearPixels method sets all the pixels to _fogOfWarColor (I'll show that in a second). Next we assign the fogOfWarMat's texture to the texture we just created. After initializing the revealers list, we calculate the _pixelsPerUnit by dividing the _textureSize and this transforms scale on the X axis. This will tell us how many pixels are in one Unity unit on the fog of war mesh.

public void RegisterRevealer(Revealer revealer)
{
	_revealers.Add(revealer);
}

private void ClearPixels()
{
	for (var i = 0; i < _pixels.Length; i++)
	{
		_pixels[i] = _fogOfWarColor;
	}
}

Now we have a method to register a revealer, note that there is no UnregisterRevealer method it can easily be added I just haven't needed it yet. Then we have the ClearPixels method that simply changes all the values in _pixels to the _fogOfWarColor.

/// 

/// Sets the pixels in _pixels to clear a circle.
/// 
/// in pixels
/// in pixels
/// in unity units
private void CreateCircle(int originX, int originY, int radius)
{
	for (var y = -radius * _pixelsPerUnit; y <= radius * _pixelsPerUnit; ++y)
	{
		for (var x = -radius * _pixelsPerUnit; x <= radius * _pixelsPerUnit; ++x)
		{
			if (x * x + y * y <= (radius * _pixelsPerUnit) * (radius * _pixelsPerUnit))
			{
				_pixels[(originY + y) * _textureSize + originX + x] = new Color(0, 0, 0, 0);
			}
		}
	}
}

This will clear a (crappy) circle given the origin x and y pixel and the radius. In this method the radius gets converted to pixels. Essentially you go through every pixel in a square (that has a size of -radius to radius on the X and Y) and checks if the pixel point (x and y) is in the circle then we set that pixel in our _pixels array to transparent. This method was found here, I might end up implementing the midpoint circle formula as this does not look that good.

private void Update()
{
	ClearPixels();

	foreach (var revealer in _revealers)
	{
		// should do a raycast from the revealer to the camera.
		var screenPoint = Camera.main.WorldToScreenPoint(revealer.transform.position);
		var ray = Camera.main.ScreenPointToRay(screenPoint);
		RaycastHit hit;
		if (Physics.Raycast(ray, out hit, 1000, _fogOfWarLayer.value))
		{
			// Translates the revealer to the center of the fog of war.
			// This way the position lines up with the center pixel and can be converted easier.
			var translatedPos = hit.point - transform.position;

			var pixelPosX = Mathf.RoundToInt(translatedPos.x * _pixelsPerUnit + _centerPixel.x);
			var pixelPosY = Mathf.RoundToInt(translatedPos.z * _pixelsPerUnit + _centerPixel.y);

			CreateCircle(pixelPosX, pixelPosY, revealer.radius);
		}
	}

	_texture.SetPixels(_pixels);
	_texture.Apply(false);
}

To finish up the FogOfWarManager class we have the Update where all the magic happens! First we clear the pixels, in the future we can have a list of "revealed" pixels and color them differently, but for now just set them to _fogOfWarColor. Next we iterate through all our revealers, we convert their world position to screen position and cast a ray from the camera to the entity. This is done so the fog of war is always revealed in relation to the camera. After we get a successful hit, we translate the hit position with the FogOfWar's world position so we can place the FogOfWar anywhere in the scene and have proper calculations. Using the translated position we find the closest pixel position on the X and Y on our texture, this is done by converting the translated position to pixels and offsetting it by the center pixel. We offset it by that center pixel because we already translated the hit.point to the center of the FogOfWar transform (which is where the center pixel is in world space). Now with our found X and Y pixels we call the CreateCircle method to clear the pixels that this revealer can see. Finally, after the loop finishes, we call SetPixels on the _texture with our updated pixels and then Apply (without calculating mip maps) the texture. We don't need to reassign the texture to the material because the material already has a handle to it (or we have a handle to it...). Here is the entire FogOfWarManager.cs file:

using UnityEngine;
using System.Collections;
using System.Collections.Generic;

/// 

/// Manages the fog of war.
/// Generates the texture with the alpha "holes" for visable units.
/// Will also disable other players unity that are no longer visable. (todo)
/// 
public class FogOfWarManager : MonoBehaviour
{
	#region Private
	/// 

	/// The size of the texture in BOTH x and y.
	/// Should be a power of 2.
	/// 
	[SerializeField]
	private int _textureSize = 256;
	[SerializeField]
	private Color _fogOfWarColor;
	[SerializeField]
	private LayerMask _fogOfWarLayer;

	private Texture2D _texture;
	private Color[] _pixels;
	private List _revealers;
	private int _pixelsPerUnit;
	private Vector2 _centerPixel;

	private static FogOfWarManager _instance;
	#endregion

	#region Public
	/// 

	/// Note this is NOT a singleton!
	/// This just needs to be globally accessable AND still be a MonoBehaviour.
	/// 
	public static FogOfWarManager Instance
	{
		get
		{
			return _instance;
		}
	}
	#endregion

	private void Awake()
	{
		_instance = this;

		if (!SystemInfo.supportsComputeShaders)
		{
			Debug.LogWarning("No Compute Shader support!");
		}

		var renderer = GetComponent();
		Material fogOfWarMat = null;
		if (renderer != null)
		{
			fogOfWarMat = renderer.material;
		}

		if (fogOfWarMat == null)
		{
			Debug.LogError("Material for Fog Of War not found!");
			return;
		}

		_texture = new Texture2D(_textureSize, _textureSize, TextureFormat.RGBA32, false);
		_texture.wrapMode = TextureWrapMode.Clamp;

		_pixels = _texture.GetPixels();
		ClearPixels();

		fogOfWarMat.mainTexture = _texture;

		_revealers = new List();
		
		_pixelsPerUnit = Mathf.RoundToInt(_textureSize / transform.lossyScale.x);

		_centerPixel = new Vector2(_textureSize * 0.5f, _textureSize * 0.5f);
	}

	public void RegisterRevealer(Revealer revealer)
	{
		_revealers.Add(revealer);
	}

	private void ClearPixels()
	{
		for (var i = 0; i < _pixels.Length; i++)
		{
			_pixels[i] = _fogOfWarColor;
		}
	}

	/// 

	/// Sets the pixels in _pixels to clear a circle.
	/// 
	/// in pixels
	/// in pixels
	/// in unity units
	private void CreateCircle(int originX, int originY, int radius)
	{
		for (var y = -radius * _pixelsPerUnit; y <= radius * _pixelsPerUnit; ++y)
		{
			for (var x = -radius * _pixelsPerUnit; x <= radius * _pixelsPerUnit; ++x)
			{
				if (x * x + y * y <= (radius * _pixelsPerUnit) * (radius * _pixelsPerUnit))
				{
					_pixels[(originY + y) * _textureSize + originX + x] = new Color(0, 0, 0, 0);
				}
			}
		}
	}

	private void Update()
	{
		ClearPixels();

		foreach (var revealer in _revealers)
		{
			// should do a raycast from the revealer to the camera.
			var screenPoint = Camera.main.WorldToScreenPoint(revealer.transform.position);
			var ray = Camera.main.ScreenPointToRay(screenPoint);
			RaycastHit hit;
			if (Physics.Raycast(ray, out hit, 1000, _fogOfWarLayer.value))
			{
				// Translates the revealer to the center of the fog of war.
				// This way the position lines up with the center pixel and can be converted easier.
				var translatedPos = hit.point - transform.position;

				var pixelPosX = Mathf.RoundToInt(translatedPos.x * _pixelsPerUnit + _centerPixel.x);
				var pixelPosY = Mathf.RoundToInt(translatedPos.z * _pixelsPerUnit + _centerPixel.y);

				CreateCircle(pixelPosX, pixelPosY, revealer.radius);
			}
		}

		_texture.SetPixels(_pixels);
		_texture.Apply(false);
	}
}

Don't run it quite yet, we still need to finish the Revealer class. Open that up and lets get started!

public class Revealer : MonoBehaviour
{
	public int radius;

	private void Start()
	{
		FogOfWarManager.Instance.RegisterRevealer(this);
	}
}

Oh good, small class. We have a public int for our radius and in the Start we inform the FogOfWarManager that we are a revealer. You could create an OnDestroy() method and call an UnregisterRevealer method if you created that/need it.

Alright now go back to Unity and attach the Revealer to the capsule and the FogOfWarManager to the quad we made earlier, you should be able to run that and move your capsule around (in the scene or you can add controls) and the fog of war will be revealed. If you don't see anything make sure your settings are correct. FogOfWar layer and FogOfWarManager's _fogOfWarLayer need to match, be sure to set the _fogOfWarColor. And on the revealer be sure to set the radius or nothing will be revealed ~_^. Here is what my game view looks like:

What's Next

So this is a decent starting point, a lot better than my last attempts. If you wanted to duplicate the capsule you can see that multiple (I've only gone up to 20) Revealers work fine, well at least for my PC it runs fine. Optimization is something that is bothering me, I feel like this should be calculated on the GPU. I feel like multipass shaders would be the solution, but conceptually I am having problems with it. Another possible solution would be to use a compute shader to generate the texture.

Aside from optimization, I still need to add a height map so the height of the unit will effect what it can see (can't see up a hill, flying units are not effected by this). Also the circle algorithm (as you can see) doesn't look good, that could use some improvements, also blurring the edge of the circle would help it visually.