Designing a data-driven engine, part 3: Blueprints

Designing a data-driven engine, part 3: Blueprints

In part 1 I described the reflection system which allows C++ types to be serialized and deserialized from storage formats such as XML. Today I will talk about the system built on top of the properties library which forms the framework used by the rest of the engine to construct XML-defined objects at runtime.

My goals for this engine are influenced heavily by my experiences working with the Starcraft 2 engine, which is impressively data-driven. In SC2, nearly every aspect of the game is defined using XML files (with the remainder largely handled by a custom scripting language), which allows for an enormous amount of flexibility. Here's an example of what an XML node defining a unit might look like in SC2*:

* This is simplified pseudo-code, not an actual example.

    <CUnit name="marine">
        <weapon>machine_gun</weapon>
        <hitPoints>45</hitPoints>
        <armor>1</armor>
    </CUnit>
    <CUnit name="tough_marine" parent="marine">
        <hitPoints>100</hitPoints>
        <armor>2</armor>
    </CUnit>
    <CRangedWeapon name="machine_gun">
        <damage>4</damage>
        <range>3</range>
    </CRangedWeapon>
 
A couple of observations:

• The node name appears to indicate the concrete C++ class to construct. For some entity types, there are actually multiple classes which implement the same type - for example, "CRangedWeapon" and "CMeleeWeapon" are both weapon definitions. How does the engine identify the entity type (i.e. "weapon")? In SC2, all entity definitions are located within a single file particular to that entity, i.e. Weapon.xml.
• The schema can be slightly different for each class. For example, CRangedWeapon can have a "range" property which CMeleeWeapon might not have. Both would have all of the properties of CWeapon.
• Entities can inherit from other entities. This is a really useful feature to have.

The most basic method for constructing an object at runtime would be to define a property mapping for it for it, store the parsed XML tree in memory, and then deserialize it after construction:

Unit* marine = new Unit();
XmlDocument.GetNodeNamed("unit", "marine") >> XmlSerializer(*marine);

There are a few problems with this approach:

• Naively, it does not work with multiple classes implementing a single entity type. What if "marine" should be implemented by InfantryUnit instead of Unit? This isn't too hard to overcome - you simply need to look at the XML first, check the node name, and then either use hardcoded known values or some sort of factory system to create the appropriate type.
• It's not very fast. Even if we store the parsed XML tree we still need to do things like string to int conversions. And while the properties library is fairly optimized, it still has overhead which should be limited where possible.
• It prevents data sharing. If the entity definition contains a string or collection, that would be duplicated in every object created. If we're creating thousand of entities, that could be a real source of inefficiency.
• Validation occurs when the object is constructed, instead of when the entity definitions are loaded. It's much easier to find bugs when they manifest as early as possible.

I went with a slightly more complex approach. For each entity, I define a Blueprint class, which contains the properties which need to be serialized. The Blueprint also contains the code to create the appropriate object, via a Construct method. Essentially, blueprints are implementations of the factory pattern, coupled with a reflection system.

Blueprints are managed by a global BlueprintService. This service is responsible for reading in XML and maintaining a repository of blueprints objects. Constructing an entity is reduced to:

Unit* marine = blueprintService.Construct<Unit>("marine");

There are a few nifty things about blueprints. You can have multiple blueprint types associated with a single C++ type - i.e., in the above sample, "marine" could actually be constructed by an InfantryUnitBlueprint. Because the property system also supports inheritance, you can have a UnitBlueprint class which defines properties common to all units, then multiple subclasses such as InfantryUnitBlueprint, VehicleUnitBlueprint, etc. Blueprints can inherit from each other at the data level as well, i.e. "marine_captain" inherits all the properties from "marine", but increases the hitPoints property. 

At some point in the future I might elaborate more on how blueprints are implemented, but for now here's some sample code from the unit tests to demonstrate how blueprints are defined and interact with other components:

 namespace  
 {  
   enum AnimalRole  
   {  
     AnimalRole_Wild,  
     AnimalRole_FoodSource,  
     AnimalRole_Pet,  
     AnimalRole_Intelligent  
   };  
   
   BEGIN_ENUM_LABELS(AnimalRole)  
     DEF_ENUM_LABEL("wild", AnimalRole_Wild)  
     DEF_ENUM_LABEL("foodSource", AnimalRole_FoodSource)  
     DEF_ENUM_LABEL("pet", AnimalRole_Pet)  
     DEF_ENUM_LABEL("intelligent", AnimalRole_Intelligent)  
   END_ENUM_LABELS()  
   
   class AnimalBlueprint : public BlueprintInterface<class Animal>  
   {  
   public:  
     std::string scientificName;  
     AnimalRole role;  
     BEGIN_HOST_PROPERTY_MAP()  
       DEF_PROPERTY(scientificName)  
       DEF_PROPERTY(role)  
     END_HOST_PROPERTY_MAP()  
   };  
   
   class Animal  
   {  
   public:  
     std::string scientificName;  
     AnimalRole role;  
   
     Animal(const class AnimalBlueprint& blueprint)  
     : scientificName(blueprint.scientificName),  
      role(blueprint.role)  
     {  
     }  
   
     virtual ~Animal() = 0 { };  
   };  
   
   class BirdBlueprint : public BasicBlueprint<BirdBlueprint, Animal, AnimalBlueprint>  
   {  
   public:  
     virtual std::unique_ptr<Animal> Construct() const;  
   
     bool canFly;  
     BEGIN_HOST_PROPERTY_MAP()  
       INHERIT_PROPERTIES(AnimalBlueprint)  
       DEF_PROPERTY(canFly)  
     END_HOST_PROPERTY_MAP()  
   };  
   
   class Bird : public Animal  
   {  
   public:  
     bool canFly;  
   
     Bird(const class BirdBlueprint& blueprint)  
     : Animal(blueprint), canFly(blueprint.canFly)  
     {  
     }  
   };  
   
   inline std::unique_ptr<Animal> BirdBlueprint::Construct() const  
   {  
     return std::unique_ptr<Animal>(new Bird(*this));  
   }  
   
   class MammalBlueprint : public BasicBlueprint<MammalBlueprint, Animal, AnimalBlueprint>  
   {  
   public:  
     virtual std::unique_ptr<Animal> Construct() const;  
   
     int legs;  
     BEGIN_HOST_PROPERTY_MAP()  
       INHERIT_PROPERTIES(AnimalBlueprint)  
       DEF_PROPERTY(legs)  
     END_HOST_PROPERTY_MAP()  
   };  
   
   class Mammal : public Animal  
   {  
   public:  
     int legs;  
   
     Mammal(const MammalBlueprint& blueprint)  
     : Animal(blueprint), legs(blueprint.legs)  
     {  
     }  
   };  
   
   inline std::unique_ptr<Animal> MammalBlueprint::Construct() const  
   {  
     return std::unique_ptr<Animal>(new Mammal(*this));  
   }  
 }  
   
 TEST(BlueprintTests, BlueprintsCanBeLoadedFromXml)  
 {  
   auto svc = GetFramework().Acquire<BlueprintServiceFactory>("DefaultBlueprintService").Construct();  
   svc->Register("mammal", std::unique_ptr<Blueprint>(new MammalBlueprint()));  
   svc->Register("bird", std::unique_ptr<Blueprint>(new BirdBlueprint()));  
     
   const char* xml =  
     "<animals>\n"  
     "  <mammal name=\"man\">\n"  
     "    <scientificName>Homo sapien</scientificName>\n"  
     "    <legs>2</legs>\n"  
     "    <role>intelligent</role>"  
     "  </mammal>\n"  
     "  <mammal name=\"dog\">\n"  
     "    <scientificName>Canis familiaris</scientificName>\n"  
     "    <legs>4</legs>\n"  
     "    <role>pet</role>"  
     "  </mammal>\n"  
     "  <bird name=\"raven\">\n"  
     "    <scientificName>Corvus corax</scientificName>\n"  
     "    <canFly>1</canFly>\n"  
     "    <role>wild</role>\n"  
     "  </bird>\n"  
     "  <bird name=\"chicken\">\n"  
     "    <scientificName>Gallus gallus domesticus</scientificName>\n"  
     "    <canFly>0</canFly>\n"  
     "    <role>foodSource</role>\n"  
     "  </bird>\n"  
     "</animals>\n";  
   std::vector<char> buf(xml, xml + strlen(xml) + 1);  
   svc->Parse(std::move(buf));  
   
   auto man = svc->Acquire<Animal>("man").Construct();  
   ASSERT_STREQ("Homo sapien", man->scientificName.c_str());  
   ASSERT_EQ(2, dynamic_cast<Mammal*>(man.get())->legs);  
   ASSERT_EQ(AnimalRole_Intelligent, man->role);  
   auto dog = svc->Acquire<Animal>("dog").Construct();  
   ASSERT_STREQ("Canis familiaris", dog->scientificName.c_str());  
   ASSERT_EQ(4, dynamic_cast<Mammal*>(dog.get())->legs);  
   ASSERT_EQ(AnimalRole_Pet, dog->role);  
   auto raven = svc->Acquire<Animal>("raven").Construct();  
   ASSERT_STREQ("Corvus corax", raven->scientificName.c_str());  
   ASSERT_TRUE(dynamic_cast<Bird*>(raven.get())->canFly);  
   ASSERT_EQ(AnimalRole_Wild, raven->role);  
   auto chicken = svc->Acquire<Animal>("chicken").Construct();  
   ASSERT_STREQ("Gallus gallus domesticus", chicken->scientificName.c_str());  
   ASSERT_FALSE(dynamic_cast<Bird*>(chicken.get())->canFly);  
   ASSERT_EQ(AnimalRole_FoodSource, chicken->role);  
 }