KeLP adopts a simple and intuitive serialization/deserialization formalism for objects, such as kernels and algorithms, that is based on JSON.
JSON is the JavaScript Object Notation, a standard de facto when exchanging data in WEB, and its main characteristic is that it is easily readable by humans, and can be composed in an efficient way to represent object hierarchies. For example, a kernel function can be completely defined in a JSON representation without the need of programattically defining its characteristics.
We decided to provide a JSON interface to help those users that are not familiar with programming languages, or for those users that know other programming languages and are not familiar with Java. In fact, for the first class of users, JSON is easily readable and can be written without any programming skill. The second class of users can leverage on well-estabilished JSON library in their preferred language.
Storing/Loading JSON objects
Objects can be stored/loaded in the KeLP supported JSON format by using the Java class JacksonSerializerWrapper of the package it.uniroma2.sag.kelp.utils.
In general, given a KeLP serializable object, the methods writeValue and readValue will respectively store/load data to/from a JSON format.
For example, to store the JSON format of a LinearKernel:
1 2 3 4 |
JacksonSerializerWrapper serializer = new JacksonSerializerWrapper(); Kernel linearKernel = new LinearKernel("REPRENTATION_NAME"); String json = serializer.writeValueAsString(linearKernel); System.out.println(json); |
The previous snippet of code should print something similar to:
1 2 3 4 5 |
{ "kernelType" : "linear", "kernelID" : 1, "representation" : "REPRENTATION_NAME" } |
To load such kernel from its JSON representation stored in the json Java String object, it will be sufficient to do:
1 2 |
LinearKernel loadedKernel =(LinearKernel) serializer.readValue(json, Kernel.class); System.out.println(loadedKernel.getRepresentation()); |
JSON Representation of Kernel Functions
Definining a kernel function with JSON requires to insert in the function specification some parameters. A kernel must be defined with its name and the configuration of its parameters and, eventually, of the caches.
For example, if you want to define a kernel operating directly on a representation (DirectKernel) of type PartialTreeKernel operating over the representation named TREE, the associated JSON must contain at least:
1 2 3 4 5 |
{ "kernelType" : "ptk", "kernelID" : 1, "representation" : "TREE", } |
This will instantiate a PartialTreeKernel with default parameters by using its JSON name ptk. If one wants to customize a specific parameter, it can be done in the JSON representation. For example, to customize the terminalFactor value, the JSON should be modified as follows:
1 2 3 4 5 |
{ "kernelType" : "ptk", "representation" : "TREE", "terminalFactor" : 3 } |
The available kernel functions with their name and parameters that can be defined in JSON can be found on the Kernel documentation page.
JSON flexibility allows also the definition of complex kernel functions, such as composition or combination of them, that are characterized by complex object structures. Let us assume that we must define a kernel, that is the sum (KernelCombination) of a PolynomialKernel applied over a LinearKernel operating over the representation named VECTOR and a PartialTreeKernel operating over a representation named TREE. The following JSON code will define such kernel with a cache that will store the kernel computations of 1000 examples:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
{ "kernelType" : "linearComb", "toCombine" : [ { "kernelType" : "poly", "baseKernel" : { "kernelType" : "linear", "representation" : "VECTOR" }, "degree" : 2.0, "a" : 1.0, "b" : 0.0 }, { "kernelType" : "ptk", "representation" : "TREE", "mu" : 0.4, "lambda" : 0.1, "terminalFactor" : 3.0, } ], "weights" : [ 1.0, 1.0 ], "kernelCache" : { "cacheType" : "fixIndex", "examplesToStore" : 1000 } } |
Notice that the kernel so defined contains all the object hierarchy needed to rebuild at runtime the correct Kernel function.
JSON Representation of Algorithms
Learning algorithms can be represented in JSON as well as kernels. The JSON formalism allows to define full experiments configurations and to write only a general purpose algorithm that will read the learning algorithm specification from JSON and will produce its outputs over the provided datasets.
As for the kernel functions case, it is possibile to store/load such representation via simple calls to the class JacksonSerializerWrapper. For example, let us assume that we want to define a linear model over the representation REPRESENTATION_NAME through LibLinearLearningAlgorithm where the positive class is +1. This algorithm can be defined in JSON through the following code:
1 2 3 4 5 6 7 8 9 10 11 |
{ "algorithm" : "liblinear", "label" : { "labelType" : "StringLabel", "className" : "+1" }, "cp" : 1.0, "cn" : 1.0, "fairness" : false, "representation" : "REPRESENTATION_NAME" } |
The algorithm is defined by a full set of meta-parameters, such as cp, cn and fairness. This allows to define different parameterisation of the algorithms (for example in a parameter tuning phase) through different JSON specifications. Then, it can be possible to iterate over such specifications and use the same general code to perform different measures.
Kernel-based learning algorithms can be defined by defining in a single JSON representation both the learning algorithm and the kernel to be used. Again, it allows to fully specify an experiment via JSON, and to use a general purpose Java KeLP-based interpreter.
Let us assume, that we need to adopt the LinearKernelCombination kernel we defined above within an SVM for a binary classification task, i.e., a BinaryCSvmClassification.
The following JSON code will define such experiment:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
{ "algorithm" : "binaryCSvmClassification", "cp" : 1.0, "cn" : 1.0, "kernel" : { "kernelType" : "linearComb", "toCombine" : [ { "kernelType" : "poly", "baseKernel" : { "kernelType" : "linear", "representation" : "VECTOR" }, "degree" : 2.0, "a" : 1.0, "b" : 0.0 }, { "kernelType" : "ptk", "representation" : "TREE", "mu" : 0.4, "lambda" : 0.1, "terminalFactor" : 3.0, } ], "weights" : [ 1.0, 1.0 ], "kernelCache" : { "cacheType" : "fixIndex", "examplesToStore" : 1000 } }, "label" : { "labelType" : "StringLabel", "className" : "+1" }, "fairness" : false } |
Notice that the algorithms fully includes the set of its meta-parameters as well as the full kernel specification.
The list of available learning algorithms in KeLP with their JSON names can be found on the Learning Algorithms documentation page.
JSON Representation of Models
Learning processes produce the so-called models. These contain the outcome of the learning process, that is all the necessary information to provide future classifications. For example, a linear model will contain the values of the weights of the linear classification function, or a kernel-based model will contain the set of examples with their weights.
Models are meant to be produced during the learning phase, and then adopted during a separate classification phase. For this reason, a storing/loading mechanism is necessary: KeLP adopts, again, the JSON representation formalism to provide such functionality.
Notice that, differently from Kernel and Learning Algorithms, the JSON representation of models is not meant for a manual specification, but instead it is the result of an automatic process.
A model in KeLP is represented by a PredictionFunction object. This is obtained after the learning phase, by extracting it from the LearningAlgorithm object with a call of the method getPredictionFunction().
For example, let us assume we learn a binary linear model through a LibLinearLearningAlgorithm (similar to the one we defined above) over a dataset with positive class +1 over the first representation (0) with cp and cn values equal to 1. We can define such algorithm in JSON as
1 2 3 4 5 6 7 8 9 10 11 |
{ "algorithm" : "liblinear", "label" : { "labelType" : "StringLabel", "className" : "+1" }, "cp" : 1.0, "cn" : 1.0, "fairness" : false, "representation" : "0" } |
In order to produce a model we need to load a training dataset, load the JSON learning algorithm specification and call the learn method.
Let us assume our dataset is contained in the file train.klp and that the above JSON learning algorithm specification is contained in a file named learning_algo.klp.
1 2 3 4 5 6 |
JacksonSerializerWrapper serializer = new JacksonSerializerWrapper(); SimpleDataset trainingSet = new SimpleDataset(); trainingSet.populate("train.klp"); LearningAlgorithm binaryLearner = serializer.readValue(new File("learning_algo.klp"), LearningAlgorithm.class); binaryLearner.learn(trainingSet); |
Finally, the model (i.e., the PredictionFunction) can be obtained through:
1 |
PredictionFunction predictionFunction = binaryLearner.getPredictionFunction(); |
To save such model on a file named binary_model.klp, we can use again the JacksonSerializerWrapper with:
1 |
serializer.writeValueOnFile(predictionFunction, "binary_model.klp"); |
Notice that the Java code we just provided is general and it has no specific knowledge of the types of the experiment we are making. It means that it can be possible to write general purpose experimenter classes, and to customize its use via configuration files written in JSON.
The file binary_model.klp will contain the model specification, i.e., something similar to:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
{ "type" : "linearClassifier", "model" : { "type" : ".BinaryLinearModel", "bias" : 0.0, "hyperplane" : { "type" : "V", "content" : "__LIB_LINEAR_BIAS_:0.0 5045:-0.09723804 3804:-0.08888734 ... " }, "representation" : "0" }, "label" : { "labelType" : "StringLabel", "className" : "+1" } } |
The model contains some useful information: the type of the classifier (linear), its model (BinaryLinearModel) with its full set of weights.
The hyperplane field contain in fact the weights of the linear classification function. Notice how the JSON formalism allows to easily inspect such information.
Similarly, we can obtain a model of a kernel-based classifier. For example, if we adopt a 2-degree polynomial kernel over the first representation of the same dataset with a binary SVM, we can obtain a different model, made of support vectors, similar to:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
{ "type" : "kernelMachineClassifier", "model" : { "type" : ".BinaryKernelMachineModel", "bias" : 0.16920261, "kernel" : { "kernelType" : "poly", "baseKernel" : { "kernelType" : "linear", "representation" : "0" }, "degree" : 2.0, "a" : 1.0, "b" : 0.0 }, "supportVectors" : [ { "weight" : 0.6423006, "instance" : { "type" : "simple", "ID" : 3, "classificationLabels" : [ { "labelType" : "StringLabel", "className" : "+1" } ], "representations" : { "0" : { "type" : "V", "content" : "8311:0.21969146 3374:0.1667255 ... " } } } }, {}, ... ] }, "label" : { "labelType" : "StringLabel", "className" : "+1" } } |
Again, the model contains all the information necessary to provide future classification. In particular, this model contains the type of the model, the full specification of the kernel function that must be adopted, and the set of support vectors to be used when classifying a new instance.