Neural network propery class
This is a work-in-progress to develop a CEL property class implementing a neural network for use in game AI.
Everything in this document is subject to change. It is meant only as an explanation of what we are doing for anyone who's interested. We will try to keep it up-to-date as it continues to be developed. A very basic knowledge of neural networks is assumed. The property class is designed to require the minimum of knowledge about neural networks.
This is being developed for use in the game Ecksdee, but it is intended that it will be generic enough for many different applications.
There is a companion class, PcEvolve, which will be used to train the neural network using a genetic evolutionary algorithm. Other property classes may be added in future, to implement different training strategies.
Contact: Mat Sutcliffe <oktal@…>
Properties
The property class has the following mutable properties:
- cel.property.inputs (long)
- The number of inputs to the neural network.
- cel.property.outputs (long)
- The number of outputs on the neural network.
- cel.property.layers (long)
- The number of hidden (intermediate) layers in the neural network.
- cel.property.dispatch (bool)
- If true, then the message pcneuralnet_outputs will be sent after every call to cel.action.Process. If false, the code will be expected to use iPcNeuralNet::GetOutput() directly.
Given inputs, outputs and layers, the property class will calculate a suitable number of nodes for each hidden layer.
Actions
The property class has the following actions:
- cel.action.SetActivationFunc (parameter: string func)
- Sets the activation function, e.g. cel.activationFunc.int32.log or cel.activationFunc.float.atan.
- cel.action.SetInputs (parameters: input0...inputN (type determined by activationFunc))
- Sets the input values of the neural network.
- cel.action.Process
- Runs one iteration through the neural network.
- cel.action.LoadCache (parameters: string scope, long id)
- Loads preset weightings for the neural network from a cache.
- cel.action.SaveCache (parameters: string scope, long id)
- Saves the weightings of the neural network to a cache.
Messages
The property class can send the following message to the behaviour:
- pcneuralnet_outputs (parameters: output0...outputN (type determined by activationFunc))
- If cel.property.dispatch is true, this message will be sent after every call to cel.action.Process.
iPcNeuralNet interface
This is the C++ API:
struct iPcNeuralNet : public virtual iBase
{
/**
* Set the number of inputs, outputs and hidden layers.
*/
virtual void SetSite(size_t inputs, size_t outputs, size_t layers) = 0;
/**
* Set the activation function.
* This will also specify the datatype used for the inputs and outputs.
*/
virtual void SetActivationFunc(celNNActivationFunc *func) = 0;
/**
* Ensures the property class is initialized, and returns false if
* it is not ready to be initialized.
*/
virtual bool Validate() = 0;
/**
* Set the value of one of the inputs.
*/
virtual void SetInput(size_t index, const celData &value) = 0;
/**
* Get the value of one of the outputs.
*/
virtual const celData& GetOutput(size_t index) const = 0;
/**
* Set the values of all the inputs.
*/
virtual void SetInputs(const csArray<celData> &values) = 0;
/**
* Get the values of all the outputs.
*/
virtual const csArray<celData>& GetOutputs() const = 0;
/**
* Returns a new empty genome structure of the correct size.
* Used by the training algorithm.
*/
virtual csPtr<iCelNNWeights> CreateEmptyWeights() const = 0;
/**
* Get the neural network's genome (does deep copy).
* Used by the training algorithm.
*/
virtual void GetWeights(iCelNNWeights *out) const = 0;
/**
* Set the neural network's genome (does deep copy).
* Used by the training algorithm.
*/
virtual bool SetWeights(const iCelNNWeights *in) = 0;
/**
* Caches the genome using the iCacheManager.
* Used by the training algorithm.
*/
virtual bool CacheWeights(const char *scope, uint32 id) const = 0;
/**
* Loads cached genome data using the iCacheManager.
*/
virtual bool LoadCachedWeights(const char *scope, uint32 id) = 0;
};
Activation Functions
A neural network's activation function is the simple mathematical operation performed by each of its nodes. The celNNActivationFunc abstract class referenced above is declared thusly:
struct celNNActivationFunc : public virtual csRefCount
{
/// Perform operation on data.
virtual void Function(celData &data) const = 0;
/// Returns the type of data upon which this function operates.
virtual celDataType GetDataType() const = 0;
protected:
/**
* Utility function for templated implementations of celActivationFunc
* to retrieve data from a celData struct.
* Specialised for T = { int8, int16, int32, uint8, uint16, uint32, float }.
*/
template <typename T>
static const T& GetFrom(const celData &input);
/**
* Utility function for templated implementations of celActivationFunc
* for GetDataType() to return the appropriate CEL_DATA constant.
* Specialised for T = { int8, int16, int32, uint8, uint16, uint32, float }.
*/
template <typename T>
static celDataType DataType();
};
And some example implemenations:
template <typename T>
class celLogActivationFunc : public celNNActivationFunc
{
virtual void Function(celData &data) const
{
const T &inval = GetFrom<T>(data);
T outval = (T) log((double) inval);
data.Set(outval);
}
virtual celDataType DataType() const { return DataType<T>(); }
};
template <typename T>
class celAtanActivationFunc : public celNNActivationFunc
{
virtual void Function(celData &data) const
{
const T &inval = GetFrom<T>(data);
T outval = (T) atan((double) inval);
data.Set(outval);
}
virtual celDataType DataType() const { return DataType<T>(); }
};
The activation function can be set in a script using the cel.action.SetActivationFunc action, which takes a single parameter which is a descriptive string e.g. cel.activationFunc.int32.log or cel.activationFunc.float.atan. Thus if you want to define your own activation function you must do a bit of C++ coding. You shouldn't need to though, because many commonly-used functions will be provided.
Training
Training is performed by a separate property class which holds pointers to the following interface:
struct iCelNNWeights : public virtual iBase
{
/// Access the genome data (non-const).
virtual csArray< csArray< csArray<float> > >& Data() = 0;
/// Access the genome data.
virtual const csArray< csArray< csArray<float> > >& Data() const = 0;
/// Returns a new genome structure which is a copy of this one.
virtual csPtr<iCelNNWeights> Clone() const = 0;
};
In our application we will initially be training the neural network with an evolutionary algorithm, and we anticipate this will be the most useful method of training throughout game AI. This will be implemented by PcEvolve. It will interface with the neuralnet property class through the GetWeights and SetWeights methods in iPcNeuralNet. Other training strategies could be added.
Application
To apply a neural network to game AI, you need to decide what it is that the inputs and outputs represent. Inputs are usually what the entity is able to "sense" in its world (e.g. enemy_x, enemy_y, friend_x, friend_y). Outputs usually correspond to actions which the entity is able to perform (e.g. forward, backward, strafe, turn, fire).
When using evolutionary techniques to train the network, you need to define some fitness function which will quantify how good a certain network is at doing its job. This might be something like "do as much damage to the player as possible, stay alive as long as possible." After training, the result should be a decision-making system which knows the best outputs to produce for any combination of inputs.
In the case of our AI for the game Ecksdee, the inputs will include a geometric description of the portion of the racetrack over which the entity is presently driving, and the fitness function will be "get around the track as fast as possible."
There are no strong rules about how many hidden layers there should be, or which activation function should be used. Choosing these things is an art, and only experience can inform these decisions. Or trial-and-error.
