Unleashing Unconstrained News Knowledge Graphs to Combat Misinformation // Robert Caulk // #279

Key takeaways

Knowledge graphs can dynamically represent relationships, leading to a nuanced understanding of complex narratives.
An ontology-free approach to knowledge graph construction offers flexibility at the cost of potential query complexity.
Misinformation can be combated through diverse and balanced news representation in knowledge graphs.
Human oversight remains crucial in data curation to prevent the dissemination of false information.
The integration of AI and knowledge graphs is paving the way for innovative solutions in real-time data analysis.

Summary

Introduction to Knowledge Graphs

In this episode, Robert Caulk introduces the concept of knowledge graphs, particularly focused on 'unconstrained' knowledge graphs that allow for more flexible and expansive modeling of relationships between data points. He elaborates on the importance of ingesting large volumes of real-time news data and how these graphs can give insights and context to complex stories.

Creating the News Knowledge Graph

Robert dives into the intricate process of building a massive news knowledge graph. The discussion highlights his team's techniques for extracting high-fidelity information from news articles and leveraging AI tools to create connections. He emphasizes the goal of enabling flexible user interactions with the graph, such as dynamically generating mini-graphs based on user queries.

Ontology-Free Approach

Robert contrasts traditional ontology-based methods with the ontology-free approach his team adopts. He explains that while traditional ontologies allow for structured and definable relationships, the ontology-free method offers high-resolution data that can dynamically express relationships without the restrictions of a predefined structure. This has opened new possibilities in how knowledge graphs can be utilized.

Dealing with Misinformation

A significant theme is how these knowledge graphs can combat misinformation. Robert shares how his team clusters various news sources to create a balanced view and present both sides of any story. He stresses the value of allowing users to filter sources based on various metrics, enhancing their ability to discern fact from speculation.

Future Prospects and Challenges

The conversation shifts to the future of knowledge graphs and the challenges faced, including legal implications of data sourcing, the potential volatility of AI model costs, and the necessity to balance speed and accuracy in news retrieval. Robert expresses optimism about the flexibility of their system and encourages further exploration into diverse data sources.

Show notes

Robert Caulk is responsible for directing software development, enabling research, coordinating company projects, quality control, proposing external collaborations, and securing funding. He believes firmly in open-source, having spent 12 years accruing over 1000 academic citations building open-source software in domains such as machine learning, image analysis, and coupled physical processes. He received his Ph.D. from Université Grenoble Alpes, France, in computational mechanics.

Unleashing Unconstrained News Knowledge Graphs to Combat Misinformation // MLOps Podcast #279 with Robert Caulk, Founder of Emergent Methods.

// Abstract

Indexing hundreds of thousands of news articles per day into a knowledge graph (KG) was previously impossible due to the strict requirement that high-level reasoning, general world knowledge, and full-text context *must* be present for proper KG construction.

The latest tools now enable such general world knowledge and reasoning to be applied cost-effectively to high volumes of news articles. Beyond the low cost of processing these news articles, these tools are also opening up a new, controversial approach to KG building - unconstrained KGs. We discuss the construction and exploration of the largest news-knowledge-graph on the planet - hosted on an endpoint at AskNews.app. During the talk, we aim to highlight some of the sacrifices and benefits that go hand-in-hand with using the infamous unconstrained KG approach.

We conclude the talk by explaining how knowledge graphs like these help to mitigate misinformation. We provide some examples of how our clients are using this graph, such as generating sports forecasts, generating better social media posts, generating regional security alerts, and combating human trafficking.

// Bio

Robert is the founder of Emergent Methods, where he directs research and software development for large-scale applications. He is currently overseeing the structuring of hundreds of thousands of news articles per day in order to build the best news retrieval API in the world: https://asknews.app.

// MLOps Swag/Merch

https://shop.mlops.community/

// Related Links

Website: https://emergentmethods.ai

News Retrieval API: https://asknews.app

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Rob on LinkedIn: https://www.linkedin.com/in/rcaulk/

Timestamps:

[00:00] Rob's preferred coffee

[00:05] Takeaways

[00:55] Please like, share, leave a review, and subscribe to our MLOps channels!

[01:00] Join our Local Organizer Carousel!

[02:15] Knowledge Graphs and ontology

[07:43] Ontology vs Noun Approach

[12:46] Ephemeral tools for efficiency

[17:26] Oracle to PostgreSQL migration

[22:20] MEM Graph life cycle

[29:14] Knowledge Graph Investigation Insights

[33:37] Fine-tuning and distillation of LLMs

[39:28] DAG workflow and quality control

[46:23] Crawling nodes with Phi 3 Llama

[50:05] AI pricing risks and strategies

[56:14] Data labeling and poisoning

[58:34] API costs vs News latency

[1:02:10] Product focus and value

[1:04:52] Ensuring reliable information

[1:11:01] Podcast transcripts as News

[1:13:08] Ontology trade-offs explained

[1:15:00] Wrap up