it.uniroma2.sag.kelp.utils.evaluation

Class MulticlassClassificationEvaluator

java.lang.Object

it.uniroma2.sag.kelp.utils.evaluation.Evaluator

it.uniroma2.sag.kelp.utils.evaluation.MulticlassClassificationEvaluator

Direct Known Subclasses:

MulticlassSequenceClassificationEvaluator

public class MulticlassClassificationEvaluator
extends Evaluator

This is an instance of an Evaluator. It allows to compute the some common measure for classification tasks. It computes precision, recall, f1s for each class, a global accuracy, micro measures and macro measures. In case of multilabel classification, a prediction is considered correct (for computing the accuracy) if and only if all the predicted labels for a given instance are correct and no additional labels are predicted

Author:

Simone Filice, Giuseppe Castellucci

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	MulticlassClassificationEvaluator.ClassStats

Field Summary

Fields

Modifier and Type	Field and Description
protected HashMap<Label,MulticlassClassificationEvaluator.ClassStats>	classStats
protected int	correct
protected List<Label>	labels
protected int	total
protected int	totalFn
protected int	totalFp
protected int	totalTn
protected int	totalTp

Fields inherited from class it.uniroma2.sag.kelp.utils.evaluation.Evaluator

computed

Constructor Summary

Constructors

Constructor and Description
MulticlassClassificationEvaluator(List<Label> labels) Initialize a new F1Evaluator that will work on the specified classes

Method Summary

Methods

Modifier and Type	Method and Description
void	addCount(Example test, Prediction prediction) This method should be implemented in the subclasses to update counters useful to compute the performance measure
void	clear() Clear all the counters for a new processing.
protected void	compute() This method is intented to force the computation of the performance measure.
MulticlassClassificationEvaluator	duplicate() Returns a new instance of this Evaluator
float	getAccuracy() Returns the accuracy
float	getF1For(Label l) Return the f1 for the specified label
float	getFnFor(Label l) Return the false negatives for the specified label
float	getFpFor(Label l) Return the false positives for the specified label
float	getMacroF1() Return the macro-F1
float	getMacroPrecision() Return the macro-precision
float	getMacroRecall() Return the macro-recall
float	getMeanF1() Deprecated.
float	getMeanF1For(ArrayList<Label> ls) Return the mean of the F1 scores considering the specified labels only
float	getMicroF1() Returns the micro-f1
float	getMicroPrecision() Returns the micro-precision
float	getMicroRecall() Returns the micro-recall
float	getOverallF1() Deprecated.
float	getOverallPrecision() Deprecated.
float	getOverallRecall() Deprecated.
float	getPrecisionFor(Label l) Return the precision for the specified label
float	getRecallFor(Label l) Return the recall for the specified label
float	getTnFor(Label l) Return the true negatives for the specified label
float	getTpFor(Label l) Return the true positives for the specified label
void	printCounters(Label l) Print the counters of the specified Label l.
String	toString()

Methods inherited from class it.uniroma2.sag.kelp.utils.evaluation.Evaluator

getPerformanceMeasure

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

labels

protected List<Label> labels

classStats

protected HashMap<Label,MulticlassClassificationEvaluator.ClassStats> classStats

total

protected int total

correct

protected int correct

totalTp

protected int totalTp

totalTn

protected int totalTn

totalFp

protected int totalFp

totalFn

protected int totalFn

Constructor Detail

MulticlassClassificationEvaluator

public MulticlassClassificationEvaluator(List<Label> labels)

Initialize a new F1Evaluator that will work on the specified classes

Parameters:

labels -

Method Detail

addCount

public void addCount(Example test,
            Prediction prediction)

Description copied from class: Evaluator

This method should be implemented in the subclasses to update counters useful to compute the performance measure

Specified by:

addCount in class Evaluator

Parameters:

test - the test example

prediction - the prediction of the system

compute

protected void compute()

Description copied from class: Evaluator

This method is intented to force the computation of the performance measure.

Specified by:

compute in class Evaluator

getPrecisionFor

public float getPrecisionFor(Label l)

Return the precision for the specified label

Returns:

precision of Label l

getRecallFor

public float getRecallFor(Label l)

Return the recall for the specified label

Returns:

recall of Label l

getF1For

public float getF1For(Label l)

Return the f1 for the specified label

Returns:

f1 of Label l

getTpFor

public float getTpFor(Label l)

Return the true positives for the specified label

Returns:

tp of Label l

getTnFor

public float getTnFor(Label l)

Return the true negatives for the specified label

Returns:

tn of Label l

getFpFor

public float getFpFor(Label l)

Return the false positives for the specified label

Returns:

fp of Label l

getFnFor

public float getFnFor(Label l)

Return the false negatives for the specified label

Returns:

fn of Label l

getAccuracy

public float getAccuracy()

Returns the accuracy

Returns:

accuracy

getMicroPrecision

public float getMicroPrecision()

Returns the micro-precision

Returns:

micro-precision

getMicroRecall

public float getMicroRecall()

Returns the micro-recall

Returns:

micro-recall

getMicroF1

public float getMicroF1()

Returns the micro-f1

Returns:

micro-f1

getOverallPrecision

@Deprecated
public float getOverallPrecision()

Deprecated.

Return the precision considering all classes together

Returns:

precision This method is deprecated, use getMicroPrecision instead

getOverallRecall

@Deprecated
public float getOverallRecall()

Deprecated.

Return the recall considering all classes together

Returns:

recall This method is deprecated, use getMicroRecall instead

getOverallF1

@Deprecated
public float getOverallF1()

Deprecated.

Return the f1 considering all classes together

Returns:

f1 This method is deprecated, use getMicroF1 instead

getMeanF1

@Deprecated
public float getMeanF1()

Deprecated.

Return the mean of the F1 scores considering all the labels involved

Returns:

mean F1 of all Label{s} This method is deprecated, use getMacroF1 instead

getMacroPrecision

public float getMacroPrecision()

Return the macro-precision

Returns:

macro-precision

getMacroRecall

public float getMacroRecall()

Return the macro-recall

Returns:

macro-recall

getMacroF1

public float getMacroF1()

Return the macro-F1

Returns:

macro-F1

getMeanF1For

public float getMeanF1For(ArrayList<Label> ls)

Return the mean of the F1 scores considering the specified labels only

Returns:

mean F1 of specified Label{s} ls

clear

public void clear()

Clear all the counters for a new processing.

Specified by:

clear in class Evaluator

printCounters

public void printCounters(Label l)

Print the counters of the specified Label l.

Parameters:

l -

toString

public String toString()

Overrides:

toString in class Object

duplicate

public MulticlassClassificationEvaluator duplicate()

Description copied from class: Evaluator

Returns a new instance of this Evaluator

Specified by:

duplicate in class Evaluator

Returns:

a new instance of this Evaluator