deepctr_torch.layers.interaction module

class deepctr_torch.layers.interaction.AFMLayer(in_features, attention_factor=4, l2_reg_w=0, dropout_rate=0, seed=1024, device='cpu')

Attentonal Factorization Machine models pairwise (order-2) feature interactions without linear term and bias.

Input shape

• A list of 3D tensor with shape: (batch_size,1,embedding_size).

Output shape

• 2D tensor with shape: (batch_size, 1).

Arguments

• in_features : Positive integer, dimensionality of input features.
• attention_factor : Positive integer, dimensionality of the

attention network output space.

• l2_reg_w : float between 0 and 1. L2 regularizer strength

applied to attention network.

• dropout_rate : float between in [0,1). Fraction of the attention net output units to dropout.
• seed : A Python integer to use as random seed.

References

• [Attentional Factorization Machines : Learning the Weight of Feature

Interactions via Attention Networks](https://arxiv.org/pdf/1708.04617.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.BiInteractionPooling

Bi-Interaction Layer used in Neural FM,compress the pairwise element-wise product of features into one single vector.

Input shape

• A 3D tensor with shape:(batch_size,field_size,embedding_size).

Output shape

• 3D tensor with shape: (batch_size,1,embedding_size).

References

• [He X, Chua T S. Neural factorization machines for sparse predictive analytics[C]//Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 2017: 355-364.](http://arxiv.org/abs/1708.05027)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.BilinearInteraction(filed_size, embedding_size, bilinear_type='interaction', seed=1024, device='cpu')

BilinearInteraction Layer used in FiBiNET.

Input shape

• A list of 3D tensor with shape: (batch_size,filed_size, embedding_size).

Output shape

• 3D tensor with shape: (batch_size,filed_size*(filed_size-1)/2, embedding_size).

Arguments

• filed_size : Positive integer, number of feature groups.
• embedding_size : Positive integer, embedding size of sparse features.
• bilinear_type : String, types of bilinear functions used in this layer.
• seed : A Python integer to use as random seed.

References

• [FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction

Tongwen](https://arxiv.org/pdf/1905.09433.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.CIN(field_size, layer_size=(128, 128), activation='relu', split_half=True, l2_reg=1e-05, seed=1024, device='cpu')

Compressed Interaction Network used in xDeepFM. Input shape

• 3D tensor with shape: (batch_size,field_size,embedding_size).

Output shape

• 2D tensor with shape: (batch_size, featuremap_num) featuremap_num =  sum(self.layer_size[:-1]) // 2 + self.layer_size[-1] if split_half=True,else sum(layer_size) .

Arguments

• filed_size : Positive integer, number of feature groups.
• layer_size : list of int.Feature maps in each layer.
• activation : activation function name used on feature maps.
• split_half : bool.if set to False, half of the feature maps in each hidden will connect to output unit.
• seed : A Python integer to use as random seed.

References

• [Lian J, Zhou X, Zhang F, et al. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems[J]. arXiv preprint arXiv:1803.05170, 2018.] (https://arxiv.org/pdf/1803.05170.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.ConvLayer(field_size, conv_kernel_width, conv_filters, device='cpu')

Conv Layer used in CCPM.

Input shape

• A list of N 3D tensor with shape: (batch_size,1,filed_size,embedding_size).

Output shape

• A list of N 3D tensor with shape: (batch_size,last_filters,pooling_size,embedding_size).

Arguments

• filed_size : Positive integer, number of feature groups.
• conv_kernel_width: list. list of positive integer or empty list,the width of filter in each conv layer.
• conv_filters: list. list of positive integer or empty list,the number of filters in each conv layer.

Reference:

• Liu Q, Yu F, Wu S, et al. A convolutional click prediction model[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 2015: 1743-1746.(http://ir.ia.ac.cn/bitstream/173211/12337/1/A%20Convolutional%20Click%20Prediction%20Model.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.CrossNet(in_features, layer_num=2, parameterization='vector', seed=1024, device='cpu')

The Cross Network part of Deep&Cross Network model, which leans both low and high degree cross feature.

Input shape

• 2D tensor with shape: (batch_size, units).

Output shape

• 2D tensor with shape: (batch_size, units).

Arguments

• in_features : Positive integer, dimensionality of input features.
• input_feature_num: Positive integer, shape(Input tensor)[-1]
• layer_num: Positive integer, the cross layer number
• parameterization: string, "vector" or "matrix" , way to parameterize the cross network.
• l2_reg: float between 0 and 1. L2 regularizer strength applied to the kernel weights matrix
• seed: A Python integer to use as random seed.

References

• [Wang R, Fu B, Fu G, et al. Deep & cross network for ad click predictions[C]//Proceedings of the ADKDD’17. ACM, 2017: 12.](https://arxiv.org/abs/1708.05123)
• [Wang R, Shivanna R, Cheng D Z, et al. DCN-M: Improved Deep & Cross Network for Feature Cross Learning in Web-scale Learning to Rank Systems[J]. 2020.](https://arxiv.org/abs/2008.13535)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.CrossNetMix(in_features, low_rank=32, num_experts=4, layer_num=2, device='cpu')

The Cross Network part of DCN-Mix model, which improves DCN-M by: 1 add MOE to learn feature interactions in different subspaces 2 add nonlinear transformations in low-dimensional space Input shape

• 2D tensor with shape: (batch_size, units).

Output shape

• 2D tensor with shape: (batch_size, units).

Arguments

• in_features : Positive integer, dimensionality of input features.
• low_rank : Positive integer, dimensionality of low-rank sapce.
• num_experts : Positive integer, number of experts.
• layer_num: Positive integer, the cross layer number
• device: str, e.g. "cpu" or "cuda:0"

References

• [Wang R, Shivanna R, Cheng D Z, et al. DCN-M: Improved Deep & Cross Network for Feature Cross Learning in Web-scale Learning to Rank Systems[J]. 2020.](https://arxiv.org/abs/2008.13535)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.FM

Factorization Machine models pairwise (order-2) feature interactions without linear term and bias.

Input shape

• 3D tensor with shape: (batch_size,field_size,embedding_size).

Output shape

• 2D tensor with shape: (batch_size, 1).

References

• [Factorization Machines](https://www.csie.ntu.edu.tw/~b97053/paper/Rendle2010FM.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.InnerProductLayer(reduce_sum=True, device='cpu')

InnerProduct Layer used in PNN that compute the element-wise product or inner product between feature vectors.

Input shape

• a list of 3D tensor with shape: (batch_size,1,embedding_size).

Output shape

• 3D tensor with shape: (batch_size, N*(N-1)/2 ,1) if use reduce_sum. or 3D tensor with shape:

(batch_size, N*(N-1)/2, embedding_size ) if not use reduce_sum.

Arguments

• reduce_sum: bool. Whether return inner product or element-wise product

References

• [Qu Y, Cai H, Ren K, et al. Product-based neural networks for user response prediction[C]//

Data Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, 2016: 1149-1154.] (https://arxiv.org/pdf/1611.00144.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.InteractingLayer(embedding_size, head_num=2, use_res=True, scaling=False, seed=1024, device='cpu')

A Layer used in AutoInt that model the correlations between different feature fields by multi-head self-attention mechanism. Input shape

• A 3D tensor with shape: (batch_size,field_size,embedding_size).

Output shape

• 3D tensor with shape:(batch_size,field_size,embedding_size).

Arguments

• in_features : Positive integer, dimensionality of input features.
• head_num: int.The head number in multi-head self-attention network.
• use_res: bool.Whether or not use standard residual connections before output.
• seed: A Python integer to use as random seed.

References

• [Song W, Shi C, Xiao Z, et al. AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks[J]. arXiv preprint arXiv:1810.11921, 2018.](https://arxiv.org/abs/1810.11921)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.LogTransformLayer(field_size, embedding_size, ltl_hidden_size)

Logarithmic Transformation Layer in Adaptive factorization network, which models arbitrary-order cross features.

Input shape

• 3D tensor with shape: (batch_size, field_size, embedding_size).

Output shape

• 2D tensor with shape: (batch_size, ltl_hidden_size*embedding_size).

Arguments

• field_size : positive integer, number of feature groups
• embedding_size : positive integer, embedding size of sparse features
• ltl_hidden_size : integer, the number of logarithmic neurons in AFN

References

• Cheng, W., Shen, Y. and Huang, L. 2020. Adaptive Factorization Network: Learning Adaptive-Order Feature

Interactions. Proceedings of the AAAI Conference on Artificial Intelligence. 34, 04 (Apr. 2020), 3609-3616.

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.OutterProductLayer(field_size, embedding_size, kernel_type='mat', seed=1024, device='cpu')

OutterProduct Layer used in PNN.This implemention is adapted from code that the author of the paper published on https://github.com/Atomu2014/product-nets.

Input shape

• A list of N 3D tensor with shape: (batch_size,1,embedding_size).

Output shape

• 2D tensor with shape:(batch_size,N*(N-1)/2 ).

Arguments

• filed_size : Positive integer, number of feature groups.
• kernel_type: str. The kernel weight matrix type to use,can be mat,vec or num
• seed: A Python integer to use as random seed.

References

• [Qu Y, Cai H, Ren K, et al. Product-based neural networks for user response prediction[C]//Data Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, 2016: 1149-1154.](https://arxiv.org/pdf/1611.00144.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class deepctr_torch.layers.interaction.SENETLayer(filed_size, reduction_ratio=3, seed=1024, device='cpu')

SENETLayer used in FiBiNET.

Input shape

• A list of 3D tensor with shape: (batch_size,filed_size,embedding_size).

Output shape

• A list of 3D tensor with shape: (batch_size,filed_size,embedding_size).

Arguments

• filed_size : Positive integer, number of feature groups.
• reduction_ratio : Positive integer, dimensionality of the

attention network output space.

• seed : A Python integer to use as random seed.

References

• [FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction

Tongwen](https://arxiv.org/pdf/1905.09433.pdf)

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.