Generic Database in Unity using ScriptableObjects

This is a Tech post, it requires intermediate levels in Unity & C#

I strongly believe that what we used to call the Level Editor is now a Data Editor. They are many data types, and to manage them it’s quite useful to build a database system within Unity. The goal is not to replace the built-in Asset Database but to organize, manage, find and reference our data more easily. For my next project I started to build such a database using ScriptableObjects, and I will share some progress here.

Example of some data & features that could be in this database:

• Assets. Heavy assets (2D textures, meshes, sounds…) stored as strings or as Addressable

• Text. Includes translations in each language, dynamic text with variables, and if needed import / export to Excel or any other format used by translators

• Gameplay. List of all weapons, monsters, obstacles, bonuses, tiles types, projectiles, buffs, quests, achievements, distances, color palette, difficulties, etc.

• Engine. Settings for the engine, debug options, platform specifics, backward compatibility data, systems configs, etc.

• Code As Data. A layer on top of code to define important variables, events, states and flow. Think of this as light & simple visual programming but with data

• Project Info. List of all people who contributed (for credits), important milestones, link to important documents, etc.

For ease of use and genericity I decided to use classes (instead of struct) to hold the data. As a result the DB is not designed to hold large data made of a lot of small data structures. For example it is not meant to store something like a large map made of a grid. This is something I would do separately in a struct[].

Architecture

Overall it’s quite simple, they are two main objects: the database and the row, and the database holds an array of rows, all indexed by enums.

• The BaseDatabase provides data through an array of BaseRows (allRows) indexed by an Enum (PrimaryKeyEnum). The BaseRow class holds the generic data

• Everything is indexed by Enums. It’s lightweight, compiler friendly and strongly typed. It’s an int allowing us to access data quickly while still providing a humanly readable string. The only real drawback is that adding Enums requires modifying the code

• The database is a ScriptableObject that can be accessed from any scene while the BaseRow is a [Serializable] Class to be displayed in the inspector (of the ScriptableObject)

• The DBMaster is an array of DBs, and is the only DB needed to be referenced by the game / app. Through the Master Database one can access all DBs and all Rows, iterate on them, etc.

• EnumDB is an Enum holding a list of all the Database that we’re using. EnumExampleBonus is an enum used by the example database to list some bonuses and properties on them

• We have an AbstractDatabase because the RowDBMaster needs to be able to list all database, but it cannot know all their types in advance. It references them using this abstract type and perform casts for operations like GetRows

Base Database & Data Integrity

Note the CheckIntegrity() function. Here it does 2 generic checks, making sure that:

• We have as many elements in allRows as we have in the enum for this DB

• That each Row has the correct enum value

• Also takes care of checking the integrity of all rows

Then it’s up to each database’s row class to check its own integrity. For example your People DB might want to make sure that everyone has a role assigned (for credit display). Or your obstacle data might need to have a difficulty specified, etc.

As the game or app grows in complexity, more and more databases and rows of data will be created. With very refined integrity checks you can minimize bugs due to incorrect data setup. And every time a bug arise due to incorrect data setup, you can create a new CheckIntegrity() rule to prevent it from ever happening again.

Example DB

To create a new database we must:

• First create the Enum that will list our data

• Create the Row that will receive the data (here RowDBExample) using the Enum

• If needed override CheckIntegrity() to make sure the data is setup correctly

• Create the database (here DBExample) that will become another ScriptableObject

• Finally this new database needs to be linked to all other DBs through the DBMaster. We must add another element to the EnumDB and link the ScriptableObject in a new row of the DBMaster

Inside The Inspector

Accessing The Data

The interesting part is how I retrieve a specific row using GetRowFromDB(). It might look like a long line just to get a row, but it’s actually very easy to write thanks to the generics setup. Once you specified which Row type you want to get (here RowDBExample) no mistake is possible and most of the work is done by auto-completion. The second parameter (EnumExampleBonus) is type checked from the Row type who also has an enum. And then the function parameters are enums that are automatically filled by the auto-completion, and are easily searchable.

In term of performance GetRow() on the DBMaster does a generic cast (to get the generic database from the abstract database) so it’s better to retrieve all data rows at init instead of doing it each frame. It’s possible to avoid this casting by implementing AbstractDatabase / BaseDatabase a bit differently. Myself I prefer to have as much code as possible in the BaseDatabase because it has the type of Row & Enum so that everything can be strongly typed.

Using Enums

I want to finish with some notes on using enums for the indexing:

• Enums are ordered int so you cannot remove an element in the middle of the enum, because it will offset the entire list. At first this looks like a constraint, but it’s a good one because for backward compatibility reasons you don’t really want to delete data keys. When you ship a software version that deprecates some data you still need to convert the old data into the new one

• Enums are unique (enforced by the compiler) so it’s quite easy to build a unique identifier number for each DB and more importantly for each row. This can be very useful for light data reference or networking call

• Enums are int and make data access super quick

• Enums are easy to use in different systems, for example if you have a data to index all your visual fx, they can easily reference a sound using their enum

• Unity doesn’t display large enum very well by default, but they are many editor script that solve it, for example Searchable Enum from Ryan Hipple