Leaving Databricks to Build an LLM-First Company

Originally posted 2025-05-05

Obligatory disclaimer: all opinions are mine and not of my employer

A year ago, Databricks acquired Lilac, the startup where I had been first engineer. I thought there might be some years of stability after two years of tumult, but strangely enough, I found myself longing for the startup days at Lilac. I’d only been at Lilac for six months, and we were acquired pre-seed – a very early stage to be acquired – and felt like I’d missed out on some part of the journey. So I’m leaving to do a startup. This time, though - I’m putting myself in the driver’s seat as the founder, rather than as an employee.

My year at Databricks

Databricks’s core business is data management, with origins in providing managed Spark pipelines for data scientists and engineers. In this business, Databricks excels as a second-mover, with product development/release processes and customer support expectations measured in years and not months. A “public preview” implies that Databricks has thought through issues like API stability and compatibility, object lifecycle management, data ownership and ACLs, compliance, and multi-cloud support, while “general availability” adds full platform support for specialized customer classes (gov, on-prem, healthcare, financial, etc.) and stricter reliability/uptime. Databricks is quite successful in this market, rapidly converting customers from other data management vendors.

From the inside, Databricks has some of the strongest colleagues I’ve ever worked with. The environment is intense, fast-paced, and the AI division works extremely hard to try and keep up with the fast-changing world of LLMs. Leadership is incredibly involved, far-sighted, and exudes founder-in-charge energy. The organization is dynamic and core developer infrastructure is being reinvented monthly as Databricks hits the scaling point where it makes sense to dedicate engineers purely to internal productivity.

Databricks is moving into the “Data + AI” space - a natural fit, given that AI thrives when built atop the right business context and data. When Lilac was acquired, Daniel and Nikhil (the Lilac founders) and I joined the Agent Quality team to help Databricks customers iterate on and improve their agents. Unfortunately, our team struggled to balance enterprise product quality and a quickly evolving competitive landscape (foundation model/finetuning -> RAG -> tool/function calling -> MCP -> ???). We thrashed, trying to reach Public Preview readiness while not quite sure that we’d found product-market fit. Much work was wasted productionizing half-baked ideas that were discarded in the final release, and much work continues to be wasted maintaining backwards-compatibility with outdated approaches to working with LLMs.

To their credit, once the Databricks leadership’s Eye of Sauron turned to this problem, a new Beta process was rapidly created to address these issues. But process was only part of the mismatch. We also struggled to find team-product fit: the type of infrastructure our team specialized at building was a category error with respect to the product that we needed to build. We eventually found a good fit - our team most recently released a Beta for trace logging and agent monitoring, which I think is the right project for the Agent Quality team, for Databricks, for the LLM space.

Leaving Databricks

My 1-year cliff makes for a natural point to ask whether I should stay at Databricks. It finally feels like the team is hitting its stride, after a year of seeking company-team-product-market fit. Still, the year has been emotionally expensive for me. Frequent changes in product direction, developer infrastructure, combined with cultural mismatch have prevented me from performing at my best. I recently read an essay on skunkwork teams and projects which describes a team culture that I would strongly appreciate - and yet, the polar opposite of the culture I experienced at Databricks. As a Databricks shareholder, I remain extremely bullish on its prospects, but as a Databricks employee, I felt like I was trying to cast myself into an employee mold that I didn’t fit into.

The opportunity cost is high – there is very obviously a revolution happening with LLMs, and in the same way that software-first companies ate human-first companies, I anticipate that LLM-first companies will eat software-first companies. The AI quality infrastructure I was building at Databricks is important, but in the end it’s just software.

Towards an LLM-first future

Over the next 3-6 months, I’m going to prototype, release, and see if I can find some sort of product-market fit with an LLM wrapper product. I believe there are profitable niches that won’t be swallowed by the rising tide of foundation model capabilities, as discussed in a recent essay on LLM value chains, and I’ll see if I can claim one of those for myself.

Finding a profitable LLM-wrapper niche is my primary goal. Along the way, I hope to explore LLM augmentation and delegation patterns - most obviously in coding, but also in functions like marketing, sales, support, and management. For example, I’m tinkering around with a startup advisory board consisting of LLMs, to which I will submit weekly progress updates for feedback, mentorship, and new ideas. I’m heavily inspired by Manfred’s glasses from the now nearly 20-year old Accelerando.

Even if I fail at building a company, I anticipate that what I learn about LLM-first companies will prepare me well for the coming decade. The world is more fragile today, and nobody is safe from change. An essay I wrote about homeostasis and volatility four years ago now feels oddly prescient. In that essay, I mention stress-induced mutagenesis as an adaptation to volatile times. Success requires embracing risk, not just adapting to changes.

Stay in touch

I have a private friends-and-family mailing list for weekly updates; if you haven’t already gotten an invitation from me and are interested in weekly updates, send me an email. I also plan to condense the weekly updates into monthly updates for public consumption on my blog.