Neural Ranking at VK: How Replacing Gradient Boosting with Neural Networks Boosted Recommendations

Why Move Away from Gradient Boosting

For a long time, the final stage of content selection in recommendations was built on decision trees (GBDT). However, classic boosting reached the limits of the recommendation platform's growth potential due to two fundamental issues:

• Inflexible one-dimensional score: The final number cannot be dynamically disassembled or adjusted on the fly for different scenarios.
• Complex manual feature engineering: Developers had to manually compute cross-metrics, combine tabular indicators, and merge them with vector embeddings.

The VK AI team's solution was to transition to an end-to-end multi-task neural network model that autonomously discovers hidden patterns in raw logs.

Key Engineering Solutions of the Neural Ranker

Piecewise Linear Feature Encoding

Continuous numerical features with heavy-tailed distributions are poorly handled by neural networks directly. To solve this problem, the value range is divided into fixed buckets. The model determines the feature's position on the scale and interpolates it between boundaries, generating an informative trainable vector without sacrificing accuracy.

DCDN Architecture with Built-in Division

The classic Deep & Cross Network (DCN) efficiently multiplies features but cannot handle negative polynomial degrees. Because of this, relative metrics (such as CTR — the ratio of clicks to impressions) required many redundant parameters to approximate. In the updated DCDN (Deep Cross-Division Network) architecture, the division operation was integrated directly into the base layers. Now the network computes relative metrics and completion rates automatically during gradient descent.

Multi-Task Learning and Heterogeneous Attention

After the DCDN core, data passes through a Heterogeneous Attention mechanism and is routed into three independent predictive heads:

• Like probability estimation.
• Dislike or post-hide risk prediction.
• Expected watch time calculation.

To prevent task learning from conflicting, gradients between the heads are isolated using stop_gradient. A compact Learning to Rank (LTR) neural network operates on top of the generated scores, balancing the signals and determining the final feed order in runtime.

Probabilistic Watch Time Prediction (HEGM)

Attempting to predict watch time using a standard MSE loss function yielded averaged results due to the bimodal nature of audience behavior: users either swipe past a video quickly (skipping within 1–3 seconds) or get drawn into extended viewing. An average value failed to reflect the actual scenario.

Engineers applied a hierarchical mixture of distributions called HEGM (Hierarchical Exponential-Gaussian Mixture):

1. An exponential distribution estimates the probability of an immediate swipe.
2. A mixture of Gaussians predicts the stages of sustained, engaged viewing.

Optimization is based on Maximum Likelihood Estimation (MLE). To prevent Gaussian variances from collapsing into delta functions, they applied uniform initialization of expectations across the timeline and added regularization penalties for divergence.

Implementation Results

Migrating the ranking system to the new neural network architecture yielded noticeable product improvements in VK Clips:

• Total Watch Time increased by 5.5%.
• The number of likes grew by 5%.
• The number of shares surged by 15%.

The Discovery platform enables scaling this model and dynamically integrating new ecosystem products without having to rewrite recommendation algorithms from scratch.