LinkedIn Recommender System Predictive ML vs LLMs

Key takeaways

LLMs can automatically interpret user signals, reducing the need for detailed feature engineering in recommendation systems.
The move towards prompt engineering allows for more flexible and efficient inputs into recommender systems.
Latency remains a key challenge for LLMs, necessitating innovations in model size and performance.
Evaluating LLM-generated recommendations requires comparing user engagement metrics to those from traditional systems.
Understanding user clusters and optimizing content timing are crucial for maximizing user interaction with recommendations.
Future developments in recommender systems should consider ethical implications and the alignment of AI with human values.

Summary

Revolutionizing Recommendations with LLMs

In this episode, Demetrios and Arpita Vats delve into the transformative impact of Large Language Models (LLMs) on recommender systems. Unlike traditional systems, which rely heavily on hand-crafted features and predefined user clusters, LLMs possess the ability to interpret user behavior and content in a more nuanced, human-like manner. Arpita explains that this shift allows for smarter insights and less manual setup, ultimately enhancing the user experience by making recommendations more aligned with user interests.

The Shift from Feature Engineering to Prompt Engineering

A significant change highlighted in the podcast is the transition from traditional feature engineering to prompt engineering with the use of LLMs. Arpita emphasizes that while traditional systems required detailed user signals to be fed into the model, LLMs can operate effectively with broader inputs. This change reduces the complexity involved in preparing data for recommendations, as LLMs are capable of understanding the context from simpler prompts.

Latency and Infrastructure Challenges

Despite the advantages of LLMs, the podcast addresses several challenges, particularly relating to latency and infrastructure costs. As LLMs are larger and more complex than traditional models, they require more computational resources, leading to potential slowdowns in recommendation systems. Arpita discusses solutions, such as developing lightweight models and using knowledge distillation to enhance efficiency during inference.

Evaluating Recommendations: Traditional vs LLMs

Demetrios and Arpita also explore how to assess the effectiveness of recommendations from LLMs compared to traditional methods. The discussion revolves around evaluation criteria and outcomes, revealing that while LLMs can streamline processes and provide enhanced recommendations, it is crucial to measure user engagement and satisfaction against established benchmarks.

Understanding User Clusters and Content Timing

The conversation shifts to how user behavior influences content recommendations, and the importance of timing in delivering relevant content. Arpita explains that user clusters, formed by analyzing past interactions, are essential for tailoring recommendations, and the timing of content delivery can significantly influence user engagement.

The Future of Recommendations and Ethical Considerations

Looking ahead, Arpita shares insights into potential advancements in recommender systems, notably the integration of LLMs with existing models to create hybrid approaches. She emphasizes the importance of ethical AI and the ongoing efforts to align AI models with user values and interests.

Show notes

Demetrios chats with Arpita Vats about how LLMs are shaking up recommender systems. Instead of relying on hand-crafted features and rigid user clusters, LLMs can read between the lines—spotting patterns in user behavior and content like a human would. They cover the perks (less manual setup, smarter insights) and the pain points (latency, high costs), plus how mixing models might be the sweet spot. From timing content perfectly to knowing when traditional methods still win, this episode pulls back the curtain on the future of recommendations.

// Bio

Arpita Vats is a passionate and accomplished researcher in the field of Artificial Intelligence, with a focus on Natural Language Processing, Recommender Systems, and Multimodal AI. With a strong academic foundation and hands-on experience at leading tech companies such as LinkedIn, Meta, and Staples, Arpita has contributed to cutting-edge projects spanning large language models (LLMs), privacy-aware AI, and video content understanding.

She has published impactful research at premier venues and actively serves as a reviewer for top-tier conferences like CVPR, ICLR, and KDD. Arpita’s work bridges academic innovation with industry-scale deployment, making her a sought-after collaborator in the AI research community.

Currently, she is engaged in exploring the alignment and safety of language models, developing robust metrics like the Alignment Quality Index (AQI), and optimizing model behavior across diverse input domains. Her dedication to advancing ethical and scalable AI is reflected both in her academic pursuits and professional contributions.

// Related Links

#recommendersystems #LLMs #linkedin

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Arpita on LinkedIn: /arpita-v-0a14a422/

Timestamps:

[00:00] Smarter Content Recommendations

[05:19] LLMs: Next-Gen Recommendations

[09:37] Judging LLM Suggestions

[11:38] Old vs New Recommenders

[14:11] Why LLMs Get Stuck

[16:52] When Old Models Win

[22:39] After-Booking Rec Magic

[23:26] One LLM to Rule Models

[29:14] Personalization That Evolves

[32:39] SIM Beats Transformers in QA

[35:35] Agents Writing Research Papers

[37:12] Big-Company Agent Failures

[41:47] LinkedIn Posts Fade Faster

[46:04] Clustering Shifts Social Feeds

[47:01] Vanishing Posts, Replay Mode