Posts

Yesterday at TechCrunch Disrupt, Harrison Chase, founder of LangChain, Ashe Magalhaes founder of Hearth, & Henry Scott-Green, founder of Context.ai, & I discussed the future of building LLM-enabled applications.

We assembled the panel as a three layer cake : Hearth, the application ; Langchain, the infrastructure ; Context.ai, the product analytics. Here are my takeaways from the conversation.

First, it’s very early in LLM application development in every sense of the word. Few applications are managing significant user volumes. Many remain in testing & are working to develop quality scores for LLM performance before launch. The state of the art is using “vibes” ; how much better did the model feel?

Earlier today, Cisco announced its intention to acquire Splunk for $28b, a 30% premium to the closing price.

Reviewing the financials, we see Splunk is a very healthy business.

$3.6b in revenues growing at 37% places the business in the top quartile of public software companies. The 78% gross margin is 6 percentage points greater than the public median.

Last week, Databricks announced their Series I financing at $43b. At the same time, they released quarterly figures similar to a public company’s reporting.

Those numbers provide the first public view into one of the most valuable private companies in the world. It’s also an opportunity to compare Databricks to Snowflake.

Two mammoths competing for customers’ workloads offer different architectures. Snowflake’s cloud analytics database thrives on structured data. Databricks’ technology rips through unstructured data in a cloud data lake house.

Last week at Saastr 2023, I had the privilege of hosting a panel with Maggie Hott, GTM leader at OpenAI, Sharon Zhou, cofounder & CEO of Lamini, & Jordan Tigani, founder & CEO of MotherDuck talking about the implications of AI for the software industry broadly.

Four themes resonated throughout the session.

First, large language models like GPT-3 are making AI accessible to the masses. These advanced models allow people to interact conversationally with technology. For example, instead of writing complex SQL queries, users can get database results by describing what they want in plain English. The upshot is that AI can now be utilized by domain experts, not just computer scientists, which opens up the markets for previously technical products to upwards of 10x the user population.

I recently stumbled across a book that profoundly impacted me over 20 years ago. Called the Art of Possibility by Rosemary Stone Zander and Benjamin Zander, conductor of the Boston Philharmonic Orchestra, there it was in the sharing library of a local cafe, nestled underneath a few spy novels.

There it was the bright yellow cover, a banner of optimism contained within the pages.

I’ve been fascinated by dictation software for many years. Speaking has struck me as the most natural interface. It also happens to be at least 3 times faster than typing.

When OpenAI released their dictation model Whisper, I was keen to try it.

So I installed it on my MacBook Pro.

Over the last few weeks, I’ve been using it. As expected, it’s sensational. In fact, this entire blog post is dictated using Whisper.

Historically, software-as-a-service (SaaS) has been built on databases with structured data, as you might find in an Excel spreadsheet.

But the ability of large language models to extract insights from unstructured information changes this architecture : data repositories like data lakes are becoming essential parts of modern SaaS stacks.

For example, next generation marketing software might ingest structured data from Google Analytics and lead capture software as before. But it also might start to analyze customers’ behavior in online webinars, conversations with customer support and sales teams, and more - all of which is unstructured data stored in video & raw text & audio files.

Open source software has evolved from a movement into a big business. Gitlab, Confluent, Databricks, MongoDB, Elastic, Snyk, Hashicorp, Mulesoft, have become huge companies using open source software.

Across, just these they have generated about $100b in market cap.

Early on, open source pioneers viewed software freedom as an end unto itself. They encouraged permissive licensing so anyone could use, modify, and distribute their code.

Large-language models have transformed how millions interact with products : from customer support to code generation to legal document analysis.

These new engagement models invite users through a meaningfully different product journey. Every application now speaks English. Understanding user behavior is essential to building great products.

How should a product manager gauge the customer experience? Lots of new challenges arise :

• First, the models are non-deterministic which means sometimes the same question phrased a bit differently produces unexpected results.
• Second, users can input anything they like which can lead to model drift & unexpected outputs potentially including might include PII (personally identifiable information) or other sensitive content.
• Third, how should a PM think about the user journey through a conversation? What is a successful interaction? It’s no longer straightforward clicking through five screens.

Context.ai is a product analytics platform for LLM-powered applications. Context.ai gives product builders an understanding of how users engage with their products and visibility into product performance, so they can build better experiences.

On Friday, Klaviyo filed their S-1 - one of the first software companies to do so since the beginning of the economic downturn in late 2022. It’s a marvel of business with strong growth, great efficiency & the potential to reignite interest in SaaS IPOs.

Klaviyo is a customer data platform company which ingests marketing data & empowers their customers to automate their marketing with data & AI. The company raised $455m in venture capital, but has used only $15m, growing nearly entirely on profits.