Kernels on Pairs

They are implementations of the class KernelOnPairs and operate on instances of the class ExamplePair.

Preference Kernel

Java class: PreferenceKernel

Maven Project: kelp-core

JSON type: preference

Description: in the learning to rank scenario, the preference kernel (Shen and Joshi, 2003) compares two pairs of ordered objects $p_a = \langle a_1, a_2 \rangle$ and $p_b = \langle b_1, b_2 \rangle$ :

$K ( p_a, p_b ) = BK(a_1, b_1) + BK(a_2, b_2) - BK(a_1, b_2) - BK(a_2, b_1)$

where BK is a generic kernel operating on the elements of the pairs. The underlying idea is to evaluate whether the first pair $\langle a_1, a_2 \rangle$ aligns better to the second pair in its regular order $\langle b_1, b_2 \rangle$ rather than to its inverted order $\langle b_2, b_1 \rangle$ .

Parameters:

baseKernel: the base kernel BK applied over the elements of the pairs.

Uncrossed Pairwise Sum Kernel

Java class: UncrossedPairwiseSumKernel

Source code: UncrossedPairwiseSumKernel.java

Maven Project: kelp-core

JSON type: uncrossedPairwiseSum

Description: it compares two pairs of ordered objects $p_a = \langle a_1, a_2 \rangle$ and $p_b = \langle b_1, b_2 \rangle$ , summing the contributions of the single element similarities:

$K ( p_a, p_b ) = BK(a_1, b_1) + BK(a_2, b_2)$

where BK is a generic kernel operating on the elements of the pairs. It has been used in learning scenarios where the elements within a pair have different roles, such as text and hypothesis in Recognizing Textual Entailment (Filice et al., 2015), or question and answer in Question Answering (Filice et al., 2016).

Parameters:

baseKernel: the base kernel BK applied over the elements of the pairs.
intrapairSimProduct: whether adding or not to the kernel combination an extra term equivalent to the multiplication of the intra-pair similarities, i.e., $BK(a_1,a_2) \cdot BK(b_1, b_2)$ .

Uncrossed Pairwise Product Kernel

Java class: UncrossedPairwiseProductKernel

Source code: UncrossedPairwiseProductKernel.java

Maven Project: kelp-core

JSON type: uncrossedPairwiseProduct

Description: it compares two pairs of ordered objects $p_a = \langle a_1, a_2 \rangle$ and $p_b = \langle b_1, b_2 \rangle$ , multiplying the contributions of the single element similarities:

$K ( p_a, p_b ) = BK(a_1, b_1) \cdot BK(a_2, b_2)$

where BK is a generic kernel operating on the elements of the pairs. As for the UncrossedPairwiseSumKernel, it has been used in learning scenarios where the elements within a pair have different roles, such as text and hypothesis in Recognizing Textual Entailment (Filice et al., 2015), or question and answer in Question Answering (Filice et al., 2016). The product operation inherently applies a sort of logic and between the $BK(a_1, b_1)$ and $BK(a_2, b_2)$ .

Parameters:

baseKernel: the base kernel BK applied over the elements of the pairs.
intrapairSimProduct: whether adding or not to the kernel combination an extra term equivalent to the multiplication of the intra-pair similarities, i.e., $BK(a_1,a_2) \cdot BK(b_1, b_2)$ .

Pairwise Sum Kernel

Java class: PairwiseSumKernel

Source code: PairwiseSumKernel.java

Maven Project: kelp-core

JSON type: pairwiseSum

Description: it compares two pairs of objects $p_a = \langle a_1, a_2 \rangle$ and $p_b = \langle b_1, b_2 \rangle$ , summing the contributions of all pairwise similarities between the single elements:

$K ( p_a, p_b ) = BK(a_1, b_1) + BK(a_2, b_2) + BK(a_1, b_2) + BK(a_2, b_1)$

where BK is a generic kernel operating on the elements of the pairs. It has been used in symmetric tasks, such as Paraphrase Identification, where the elements within a pair are interchangeable (Filice et al., 2015).

Parameters:

baseKernel: the base kernel BK applied over the elements of the pairs.
intrapairSimProduct: whether adding or not to the kernel combination an extra term equivalent to the multiplication of the intra-pair similarities, i.e., $BK(a_1,a_2) \cdot BK(b_1, b_2)$ .

Pairwise Product Kernel

Java class: PairwiseProduct

Source code: PairwiseProduct.java

Maven Project: kelp-core

JSON type: pairwiseProduct

Description: it compares two pairs of objects $p_a = \langle a_1, a_2 \rangle$ and $p_b = \langle b_1, b_2 \rangle$ , summing the contributions of the two possible pairwise alignments:

$K ( p_a, p_b ) = BK(a_1, b_1) \cdot BK(a_2, b_2) + BK(a_1, b_2) \cdot BK(a_2, b_1)$

Parameters:

baseKernel: the base kernel BK applied over the elements of the pairs.
intrapairSimProduct: whether adding or not to the kernel combination an extra term equivalent to the multiplication of the intra-pair similarities, i.e., $BK(a_1,a_2) \cdot BK(b_1, b_2)$ .

Best Pairwise Alignment Kernel

Java class: BestPairwiseAlignmentKernel

Source code: BestPairwiseAlignmentKernel.java

Maven Project: kelp-core

JSON type: bestPairwiseAlignment

Description: it compares two pairs of objects $p_a = \langle a_1, a_2 \rangle$ and $p_b = \langle b_1, b_2 \rangle$ , evaluating the best pairwise alignment:

$K( p_a, p_b ) = softmax \Big ( BK(a_1,b_1) \cdot BK(a_2, b_2), BK(a_1,b_2) \cdot BK(a_2,b_1) \Big )$

where BK is a generic kernel operating on the elements of the pairs, and softmax is a function put in place of the max operation, which would cause K not to be a valid kernel function (i.e., the resulting Gram matrix can violate the Mercer’s conditions). In particular, $softmax$(x_1,x_2)= \frac{1}{c} \log(\exp(c x_1) + \exp(c x_2)$ (c=100 is accurate enough). The BestPairwiseAlignmentKernel has been used in symmetric tasks, such as Paraphrase Identification (Filice et al., 2015), where the elements within a pair are interchangeable.

Parameters:

baseKernel: the base kernel BK applied over the elements of the pairs.
intrapairSimProduct: whether adding or not to the kernel combination an extra term equivalent to the multiplication of the intra-pair similarities, i.e., $BK(a_1,a_2) \cdot BK(b_1, b_2)$ .

References

Simone Filice, Giovanni Da San Martino and Alessandro Moschitti. Relational Information for Learning from Structured Text Pairs. In Proceedings of the 53^rd Annual Meeting of the Association for Computational Linguistics, ACL 2015.

Simone Filice, Danilo Croce, Alessandro Moschitti and Roberto Basili. KeLP at SemEval-2016 Task 3: Learning Semantic Relations between Questions and Answers. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), Association for Computational Linguistics. (Best system @ SemEval-2016 task 3)

Libin Shen and Aravind K. Joshi. An svm based voting algorithm with application to parse reranking. In In Proc. of CoNLL 2003, pages 9–16, 2003