Wide & Deep Learning for Recommender Systems
TLDR
While the paper describes this approach as a combination of a deep neural network and a linear regression, it can also just be seen as a simple deep neural network, seeing as the formula for a single neuron is the same as a formula for a linear regression, just with an optional {e.g. RELU} non-linearity applied. Of course, linear regression fitting is much faster than using gradient descent, which is useful. Anyways, the approach is basically to have a logistic function {softmax neuron} that takes as input both a linear regression of your features {the 'wide' part of the model}, and a deep neural network's feature output {the 'deep' part of the model}.
Summary
This paper introduces the Wide & Deep learning model, a novel architecture designed to achieve both memorization and generalization in the context of recommender systems. The model is a hybrid of a linear model and a deep neural network, which are trained jointly to make predictions.
The "wide" part of the model refers to the linear model, which is designed to have a large number of sparse input features. This wide model component is capable of memorization, or learning the frequent co-occurrence of items or features. This can be particularly useful for recommender systems, where certain item pairs or feature combinations may be highly predictive of user behavior.
The "deep" part of the model refers to the deep neural network, which has multiple hidden layers of dense embeddings. This deep model component is capable of generalization, or learning abstract feature interactions. This can help capture user preferences based on less obvious patterns in the data, leading to more diverse and personalized recommendations.
The architecture of the Wide & Deep learning model can be represented as follows:
[ \hat{y} = \sigma(w_0 + \mathbf{w}^T \mathbf{x} + \mathbf{w}_d^T \mathbf{a}(\mathbf{x})) ]
where:
- (\hat{y}) is the predicted target variable.
- (\sigma) is the logistic function, which squashes the output between 0 and 1.
- (w_0) is the global bias.
- (\mathbf{w}) is the weight vector for the wide model.
- (\mathbf{x}) is the input feature vector.
- (\mathbf{w}_d) is the weight vector for the deep model.
- (\mathbf{a}(\mathbf{x})) is the output of the last hidden layer of the deep model, which is a function of the input feature vector.
The model is trained to minimize a regularized logistic loss function, which can be optimized using gradient-based methods.
The authors also discuss practical considerations for implementing and training the Wide & Deep model, such as the use of feature engineering to create cross-product feature transformations for the wide model, and the use of embeddings to represent categorical features in the deep model. They demonstrate the effectiveness of the model through a series of experiments on the Google Play app recommendation system, showing that the Wide & Deep model significantly improves app recommendation quality compared to a deep-only model.
The implications of this work are significant for the field of recommendation systems. The Wide & Deep learning model provides a flexible and powerful framework for building recommender systems that can both exploit known user-item interactions and explore new and unexpected recommendations. This can lead to improved user satisfaction and engagement, making the Wide & Deep model a valuable tool for many real-world applications.
Tags: recommendation, 2016