DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
TLDR
Honestly this approach is very similar to the Wide & Deep Learning for Recommender Systems paper - basically, combining 'factorization machines' {linear models w/ feature interactions, lol} with some deep neural network layers to model more complex feature interactions. Looks like a slight difference is that DeepFM shares the feature embeddings between the FM and DNN components, which allows for a certain degree of interaction between low-order and high-order feature interactions. So basically, a learned embedding matrix was just inserted between the features and the linear regression and standard DNN components. Really seems like just a basic architecture change. Honestly, people just love to name things.
Summary
The paper "DeepFM: A Factorization-Machine based Neural Network for CTR Prediction" presents a model named DeepFM, which is designed for click-through rate {CTR} prediction tasks. DeepFM is a hybrid model that combines the strengths of Factorization Machines {FMs} for explicit high-order feature interactions and Deep Neural Networks {DNNs} for implicit high-order feature interactions.
In recommendation systems and other CTR prediction tasks, modeling feature interactions is critical. Linear models and FMs are good at capturing low-order feature interactions, but they can struggle with high-order feature interactions. DNNs, on the other hand, can capture high-order feature interactions but do so implicitly.
DeepFM addresses this issue by integrating an FM component and a DNN component into a unified model, trained end-to-end. The FM component is used to model the low-order feature interactions, and the DNN component is used to model the high-order feature interactions.
The architecture of the DeepFM model can be represented as follows:
[ \hat{y} = \sigma\left(y_{FM} + y_{DNN}\right), ]
where:
- (\hat{y}) is the predicted target variable.
- (\sigma) is the sigmoid function, which maps the output to the range [0, 1].
- (y_{FM}) is the output of the FM component, which models the second-order feature interactions.
- (y_{DNN}) is the output of the DNN component, which models the high-order feature interactions.
- the features being used here in this formula aren't the raw features, but the output features from a learned embedding.
The key innovation of DeepFM is that it shares the feature embeddings between the FM and DNN components, which allows for a certain degree of interaction between low-order and high-order feature interactions.
The authors evaluate the proposed DeepFM model on several real-world CTR prediction datasets and show that it outperforms several state-of-the-art models, demonstrating the effectiveness of combining explicit and implicit feature interaction modeling in a unified model.
The implications of this work are significant for the field of CTR prediction and recommender systems. The DeepFM model provides a flexible and powerful tool for modeling both low-order and high-order feature interactions, potentially leading to more accurate and personalized recommendations. It also opens up new opportunities for incorporating other types of information, such as textual or visual content, into the recommendation process.
Tags: recommendation, 2017