Shared Unity Code: The Implementation

Introduction

If you haven’t already read it, take a look at the first in this two parter. It covers the journey we took and why we decided to give Nuget a shot (Shared Unity Code: The Journey).

How we store NuGet packages

Our server team use a packaging system called Maven.

We actually experimented with sharing code using Maven too, but it was overly complex for our needs and didn’t quite fit what we were looking for.

We have a lot of shared server code, and this code lives in a Maven repository hosted on a service called Artifactory. As it turns out Artifactory supported Nuget too, so we were able to use this to host our private Nuget repository!

For testing purposes you can easily create a private repo on an internal server, an AWS node or even just a new folder on someone’s computer on a shared network – if you do this, just be sure that its backed up! All of these options are covered in this article.

Setting up NuGet package

Once you’ve got somewhere to store your packages, it’s time to get people uploading and downloading them. You need to first install Nuget on your system.

  • On Windows I’d recommend using Chocolatey
    choco install nuget.commandline
  • On MacOS I’d use Homebrew
    brew install nuget

You’re also going to let Nuget know about your private server, you can do this by firing up the terminal/command prompt and running the following commands:

nuget sources Add -Name MyNugetServer -Source https://mynugetserver.com/api/nuget/nuget

If your using a folder on your local machine, you’d do something like this:

nuget sources Add -Name MyNugetServer -Source /Users/myname/NugetPackages

If you have setup a username, password and/or API key to your server you will also need to let Nuget know about this:

nuget sources update -Name MyNugetServer -UserName [username] -Password [password]
nuget setapikey [username]:[apikey] -Source MyNugetServer

Consuming Packages

Once you’ve done this you can start downloading packages from your private server! Doing this is simple:

nuget install MyCompany.MyPackage

However this isn’t ideal on a project with multiple developers.

Each package is versioned, and packages have dependencies on other packages that are also versioned. Therefore we use a configuration file, per project, to specify what packages we are going to use. Here is an example packages.config file:

<?xml version="1.0" encoding="utf-8"?>
<packages>
 <package id="SpaceApe.Logger" version="1.0.0" target="net35"/>
 <package id="SpaceApe.Common" version="1.0.8" target="net35"/>
</packages>

Each entry in this file specifies a package name, a version number and a target (the .Net framework version). You can find all ther reference material for the config file here. When we have a packages.config file we can then use it to install our packages:

nuget install packages.config

This will go through every package in your config file and download it to your computer.

Releasing Packages

In order to release a package you have to create a Nuspec file. This file will define all the information about your package, including:

  • Package Name
  • Developer
  • URL
  • Package Version
  • Dependencies
  • Files

It’s pretty important to keep this file up to date, if you change a dependency or update one of the dependencies to a newer release, update your Nuspec file!

A Nuspec file is actually just another XML document, here is an example of one of ours:

<?xml version="1.0"?>
<package>
 <metadata>
    <!-- The identifier that must be unique within the hosting gallery -->
   <id>SpaceApe.PerformanceTesting</id>
    <!-- The package version number that is used when resolving dependencies -->
   <version>1.0.3</version>
    <!-- Authors contain text that appears directly on the gallery -->
   <authors>SpaceApeGames</authors>
    <!-- Owners are typically nuget.org identities that allow gallery users to earily find other packages by the same owners.  -->
   <owners>mshort</owners>
    <!-- License and project URLs provide links for the gallery -->
   <projectUrl>https://github.com/spaceapegames/performance-testing</projectUrl>
    <!-- If true, this value prompts the user to accept the license when installing the package. -->
   <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <!-- Any details about this particular release -->
    <releaseNotes>Adding timers</releaseNotes>
    <!-- The description can be used in package manager UI. Note that the nuget.org gallery uses information you add in the portal. -->
    <description>Unity performance recording</description>
    <!-- Copyright information -->
    <copyright>Copyright ©2017 Space Ape Games</copyright>
    <!-- Tags appear in the gallery and can be used for tag searches -->
    <tags>unity dll c# performance test testing</tags>
    <dependencies>
    </dependencies>
 </metadata>
 <files>
    <file src="Nuget/bin/Unity54/SpaceApe.PerformanceTesting.dll" target="lib/unity54" />
    <file src="Nuget/bin/Unity54/SpaceApe.PerformanceTesting.dll.*db" target="lib/unity54" />
    <file src="Nuget/bin/Unity56/SpaceApe.PerformanceTesting.dll" target="lib/unity56" />
    <file src="Nuget/bin/Unity56/SpaceApe.PerformanceTesting.dll.*db" target="lib/unity56" />
    <file src="Nuget/bin/Unity2017/SpaceApe.PerformanceTesting.dll" target="lib/unity2017" />
    <file src="Nuget/bin/Unity2017/SpaceApe.PerformanceTesting.dll.*db" target="lib/unity2017" />
    <file src="Nuget/bin/Unity2017/Editor/SpaceApe.PerformanceTestingEditor.dll" target="lib/unity2017/Editor" />
    <file src="Nuget/bin/Unity2017/Editor/SpaceApe.PerformanceTestingEditor.dll.*db" target="lib/unity2017/Editor" />
 </files>
</package>

As you can see there’s quite a lot of information in there, but once you’ve set this file up, it wont change much. For all the details on Nuspec files, you can head over to the reference page. So, now you have your project set up, its building, the tests are passing and you have your Nuspec file ready to go. Jump back into the terminal and run the following commands:

nuget pack MyCompany.MyPackage
nuget push MyCompany.MyPackage.1.0.0 -source MyNugetServer

The first command will package up all the files into a nupkg (which is actually just a .zip file, you can change the extension to .zip and unzip it to see what’s inside it) file and the second will push it up to your repository.

People can now use your NuGet package!

We actually build and release all of our modules via Jenkins.

For the entire Nuget Command Line Interface reference, head over to here.

Multiple Teams, Multiple Versions of Unity

You may have noticed that we hijack the target field to distribute different DLL’s compiled against different versions of Unity in a single package. This is quite unconventional and is something we’ve only recently started trying, but it seems to be working ok so far.

It means a single package can contain a DLL compiled against Unity 5.6 and Unity 2017, but using the target field we specify that our game is only interested in the Unity 2017 DLL.

This allows us to easily support multiple teams with the same shared modules.

Building Multiple DLLs

In order to build our modules against different versions of Unity we change the C# project file so that it has multiple configurations – one for each version of Unity that we are building against.

Each configuration then has the required preprocessor defines assigned to them (e.g. UNITY_5_6_OR_NEWER, UNITY_2017_1_OR_NEWER etc). And each configuration links to the correct Unity DLL’s.

We can then use the exact same source code in a test Unity project to quickly track down any bugs or issues we might encounter.

Unity and NuGet

When you create prefabs in Unity and assign a custom script to that prefab, the GUID of that script is stored within the prefab. So if the GUID of the script ever changes, your prefab will lose the script reference.

So next time you release your package, and you’re super excited to get everyone to update, they do so, and all their prefabs break. Why?

Nuget places the version number in the name of the DLL (eg SpaceApe.PerformanceTesting.1.0.0.dll)

When you update your package, the version number will change (which changes the filename too!)

Unity calculates the GUID of a script within a DLL by taking

  • the name of the DLL
  • the name of the script
  • And then hashing them together

Therefore when we install packages we use a special –excludeversion option that doesn’t put the package version number onto the DLL and we don’t delete the DLL’s meta file, eg:

nuget install packages.config --excludeversion

Where we are now

So as it stands everything is working really well for us.

  • We can share code between teams
  • We can support multiple versions of Unity
  • Our code is shipping with more and more tests
  • A lot of our modules are well documented and the documentation is automatically updated on each release

This all leads to one thing, our dev teams are able to get on with what matters – making that genre defining hit.

Sharing Unity Code: The Journey

Introduction

At Space Ape we have a lot of shared code which we use across all of our projects. As of now, we have 57 shared code modules; these range from messaging systems, logging and our in-game console, to crowd rendering and asset processing, .

Why Share Code?

  • We don’t need to spend time reinventing the wheel.
  • We have the flexibility to take the shared modules we require.
  • We’re confident that this code comes with a suite of unit and integration tests.
  • We know it’s been proven in practical use across previous projects.
  • We are all familiar with the code, making project start-up times much faster.

Sharing code allows developers to focus more on the final product, in our case the games we create, because they spend less time worrying about low-level implementation details and non functional requirements.

The Journey

All of this shared code didn’t just happen overnight. We’ve tried and failed quite a few times with our release process, distribution and collaboration. As a result of failing and learning from these failures, we’re now able to easily share our code, and developers from various projects are all contributing to it.

The process is by no means perfect – we’re still working out the kinks – but it’s improved our workflow a lot, and maybe it will work for you too.

At the Beginning, there was only Ctrl+C, Ctrl+V

When Space Ape first started we were all focused on one project, so there was no need to share code. We were a start-up and our primary objective was to ship a product.

When the time came to start our second project, some of the code was copied over from the first project and we remove any project dependencies, but we had a code base to start with. Our first title was still being developed, bugs were being fixed, and features were being added.

Whilst all of this was happening the two code bases diverged quite a bit. If a bug was fixed in one of our games, there’s a good chance that fix didn’t make it into the other game – the same goes for features and improvements. As each project went on, the ‘shared’ code was modified to fit each project, and in the end sharing code between projects was more hassle than it’s worth.

Context of the Problem

Roll forward a few years. We now have a few established games, Samurai Siege, Rival Kingdoms, Transformers: Earth Wars and Fastlane, and we’ve entered into a partnership with Supercell.

Unsurprisingly, our company goals have changed a bit. We’re no longer a start-up in the true sense of the phrase. Our goal is to create a genre defining mobile hit, and in doing so we are moving away from our build and battle heritage.

We are branching out into many new genres, so we need to iterate quickly. We can’t predict that every new game idea will be a success. We need to try new things rapidly, and learn as quickly as possible.

Having a solid foundation of shared code would help us to iterate faster. In order to do this we would have to look at how we could share code between projects, with as little pain and slowdown as possible. When it comes to sharing code, the biggest obstacle is not writing the code itself, it’s the tooling and practices around releasing and distributing it.

Enter Git Submodules

Git Submodules are like a repository within a repository. You can continue to work on your code base and once you’ve finished a feature or fixed a bug, you can check it in. You just push your shared code up to one repository, and your project’s code to another.

This seemed ideal at first! We were already using Git across our studio so everyone was familiar with it. But we soon ran into problems.

As the source code is there for you to edit freely, teams would obviously change shared code, check it in and then when the other team pulled changes, their code wouldn’t compile! This sounds a little lazy and reckless, but this issue stems from the fact that there is no boundary between what is shared code and what isn’t. From a team’s point-of-view, they are just changing code in one big solution. The ideal solution here is to expose a simple yet well-defined API to the game teams.

So once this became an issue, each team decided to branch the shared modules off the master branch, and we were back to square one. Two diverging code paths, never merged together.

Further to this, we found that anyone who’s not a developer (artists, animators etc) can have quite a hard time using submodules. The tooling around submodules isn’t straight forward. Often we would update a submodule but someone wouldn’t pull changes for that submodule, so project and shared code would get out of sync.

Maven

Our server developers use Maven to manage and release packages. Maven is a tool developed for the Java ecosystem. When you are ready to release your project, Maven will take all of the information within a pom file and then package up your code so that it can be shared with others.

Because of all the features offered by Maven, and the fact that it’s not a native .Net tool chain, it often felt more complicated than it needed to be. Out of the box it comes with things like build life-cycle management. But at the end of the day all we were really interested in was dependency management, versioning and packaging; and that came with a lot of overhead. We ended up creating custom build steps to install our packages which made our build and release process even more complicated. As it wasn’t natively supported (or developed for) either Unity or .Net we felt that there must be a better solution.

Unity Packages

Because we are using Unity, the next technology that came to mind was Unity Packages, just like you see on the Asset Store. It was really easy to integrate. However, the whole release process and package storage was quite unregulated. There’s no real package versioning support and no dependency management. You also need additional tooling to uninstall a package as there’s no defined package structure, so we would have to clean up the old package before installing the new one.

Finally, Unity packages traditionally contained source code. We wanted to stop teams making changes to source code within these shared modules and improve compile times. This meant we needed to use Dynamic Link Libraries. DLL’s also allow us to easily develop shared code modules that depend on other modules, without having to make sure that the source code for the dependency was the correct version and compiled in the first place. Whats more using DLL’s would also lead us to faster compile times.

So we looked elsewhere, and found:

NuGet

If you’ve not come across Nuget before, it’s a package management system designed specifically for the .Net framework and it supports dependency management. There are currently over 110,000 packages on the public repository, some of which we were already using. However this repository is public, and a lot of our code isn’t for public release, so we couldn’t just go ahead and push our packages up to this public repository.

Before we could make a start there was quite a bit of work involved in setting up a whole development and release process around Nuget, not to mention setting up our own Nuget package server and getting everything to work nicely with Unity. In my next blog post I’m going to take you through everything, from start to finish.