Infinite, shareable volume storage with Hunter Leath, Archil CEO

Key takeaways

Archil aims to be a universal storage engine that integrates seamlessly with S3 to improve performance for various applications.
The platform helps developers tackle challenges associated with state management and performance optimization.
Custom protocols are essential for achieving the performance goals needed for modern application demands.
Archil’s pricing model is user-friendly, charging only for actively used data, creating cost savings versus traditional options.
Understanding use cases is critical; Archil is not suitable for all workloads but excels in many scenarios, particularly those requiring quick file access and AI workflows.

Summary

Introduction to Archil Data

Hunter Leath, CEO of Archil, introduces the concept of a universal storage engine that transforms S3 buckets into a fast, POSIX-compatible disk. This service makes storage infinitely accessible for applications, containers, and servers while enhancing performance and usability.

The Problem with Current Storage Solutions

The discussion highlights the difficulties developers face with state management when using S3 versus traditional block storage. Hunter shares insights on bridging the gap between slow shared S3 storage and fast, expensive block storage options.

Transforming SQLite on S3

Hunter explains how Archil allows SQLite to function on S3, detailing the architecture and technology stack that enables this capability while maintaining performance standards versus traditional file storage methods.

Performance Improvements with SSD-Backed Clusters

A key aspect of Archil's offering includes introducing durable SSD-backed clusters. These clusters act as a caching layer between applications and S3, ensuring high-speed access to frequently used data.

Challenges with Small Files

The conversation addresses common issues related to handling small files, particularly when using systems like Git. Hunter discusses how Archil's technology mitigates the performance challenges typically associated with small file requests to S3.

Custom Protocol Development

Hunter elaborates on the decision to create a custom storage protocol optimized for the unique needs of applications using Archil, contrasting it with existing protocols like NFS and Luster to emphasize performance and compatibility.

Handling S3 Limits and Performance Throttling

The talk shifts to how Archil manages S3-related limitations and what strategies are in place to reduce throttling risks while maintaining an efficient data handling system.

Multi-Cloud Strategy and Regional Computation

Hunter shares insights into Archil's multi-cloud capabilities and why having compute resources in the same geographic region as data storage is critical for performance.

Innovative Pricing Model

Archil’s pricing model is designed so users only pay for data that is actively used. Hunter explains how this approach provides a financially attractive alternative to traditional storage solutions.

Trade-offs in Using Archil for Storage

Hunter outlines the scenarios where Archil’s services may not be the best fit, particularly for workloads requiring ultra-low latency or completely random read patterns.

Opportunities in AI and Agent-Based Workflows

Hunter discusses the emerging needs around AI-driven applications and how Archil’s technology provides a solution for agents needing to access file systems in a cloud-based architecture.

Marketing Challenges and Growth Plans

With only a year since launch, Hunter reflects on Archil’s marketing challenges and the iterative process of communicating the company's offerings as they relate to various industry needs.

Company Status and Future Directions

Currently, Archil consists of seven employees, focusing primarily on engineering and planning to expand significantly over the next year. Hunter expresses aspirations for Archil to become a leading storage solution.

Show notes

Hunter Leath, CEO of Archil, explains how they’re building a “universal storage engine” that sits between your apps and S3—making an S3 bucket behave like a fast, POSIX-compatible disk for containers, servers, and even Lambda. Along the way, we dig into how their SSD-backed clusters and custom protocol avoid the usual small-file pain and where this approach shines (and where it doesn’t).

Follow Hunter:
Twitter/X: https://twitter.com/jhleath
Archil Twitter/X: https://twitter.com/archildata
Archil: https://archil.com/

Follow Aaron:
Twitter/X: https://twitter.com/aarondfrancis
Database School: https://databaseschool.com
Database School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g (Subscribe today)
LinkedIn: https://www.linkedin.com/in/aarondfrancis
Website: https://aaronfrancis.com - find articles, podcasts, courses, and more.

Chapters:
00:00 - Intro: Archil Data and “S3 as a disk”
01:05 - Hunter’s background and the core pitch
02:32 - The real problem: state management (S3 vs block storage)
05:02 - SQLite on S3: what the stack looks like
07:13 - The missing layer: durable SSD-backed clusters
10:14 - Who uses this: unstructured data, CI/CD, Git, agents
12:15 - Small files + Git performance and avoiding S3 request explosion
16:22 - Why they built a new protocol (NFS vs Luster)
20:00 - What gets written to S3: real files in your bucket
22:29 - S3 limits, throttling, and the “keep it on SSD” escape hatch
25:32 - Multi-cloud + R2, and why regions/latency matter
32:10 - Pricing model: “pay only when data is active”
34:41 - Tradeoffs: random reads and ultra-low-latency metal
37:19 - Storage/compute separation and AI/agent-native workflows
43:21 - YC timeline + the marketing challenge of a “universal layer”
47:34 - Single-tenant clusters for enterprises and why it’s hard
50:27 - Where the company is now, hiring, and how to try it (disk.new)