Posts

Eliciting product feedback elegantly is a competitive advantage for LLM-software.

Over the weekend, I queried Google’s Bard, & noticed the elegant feedback loop the product team has incorporated into their product.

I asked Bard to compare the 3rd-row leg room of the leading 7-passenger SUVs.

At the bottom of the post is a little G button, which double-checks the response using Google searches.

Last week, Harvard Business School released its analysis of LLMs on 758 consultants’ performance when using AI. Three key insights emerged.

In the first experiment (inside AI’s capability frontier), consultants randomly assigned access to GPT-4 AI completed 12.2% more tasks on average & 25.1% faster. Quality improved by 40%.

Lower performing consultants benefit the most from AI augmentation, increasing performance by 43% compared to 17% for higher performers.

This rising-tide effect seems common across AI applications. The benefit to the lower quartiles is dramatic across sales, customer support, & consulting. This has broader implications, something I aim to write about later this week.

Artisanal chocolate, artisanal candles, artisanal silk scarves. What about an artisanal email?

LLM’s powerful content generation abilities will soon automate many emails we send. Sales teams have started using software that identifies leads, researches them, identifies commonalities (a shared university affiliation or a passion for a sports team), drafts the email, & sends it. Then it tracks the email for follow up.

New product launches will bring similar tools to everyone.

On October 5th at 9:30am Pacific time, Office Hours will host Oliver Jay, former CRO at Asana & executive at Dropbox. Having worked at two of the most successful bottoms-up SaaS companies, Oliver understands the product-led growth motion better than most.

Recently, he’s written about the PLG trap. “Although the PLG approach yields rapid initial adoption and growth, scaling a PLG company is a different story.”

Yesterday at TechCrunch Disrupt, Harrison Chase, founder of LangChain, Ashe Magalhaes founder of Hearth, & Henry Scott-Green, founder of Context.ai, & I discussed the future of building LLM-enabled applications.

We assembled the panel as a three layer cake : Hearth, the application ; Langchain, the infrastructure ; Context.ai, the product analytics. Here are my takeaways from the conversation.

First, it’s very early in LLM application development in every sense of the word. Few applications are managing significant user volumes. Many remain in testing & are working to develop quality scores for LLM performance before launch. The state of the art is using “vibes” ; how much better did the model feel?

Earlier today, Cisco announced its intention to acquire Splunk for $28b, a 30% premium to the closing price.

Reviewing the financials, we see Splunk is a very healthy business.

$3.6b in revenues growing at 37% places the business in the top quartile of public software companies. The 78% gross margin is 6 percentage points greater than the public median.

Last week, Databricks announced their Series I financing at $43b. At the same time, they released quarterly figures similar to a public company’s reporting.

Those numbers provide the first public view into one of the most valuable private companies in the world. It’s also an opportunity to compare Databricks to Snowflake.

Two mammoths competing for customers’ workloads offer different architectures. Snowflake’s cloud analytics database thrives on structured data. Databricks’ technology rips through unstructured data in a cloud data lake house.

Last week at Saastr 2023, I had the privilege of hosting a panel with Maggie Hott, GTM leader at OpenAI, Sharon Zhou, cofounder & CEO of Lamini, & Jordan Tigani, founder & CEO of MotherDuck talking about the implications of AI for the software industry broadly.

Four themes resonated throughout the session.

First, large language models like GPT-3 are making AI accessible to the masses. These advanced models allow people to interact conversationally with technology. For example, instead of writing complex SQL queries, users can get database results by describing what they want in plain English. The upshot is that AI can now be utilized by domain experts, not just computer scientists, which opens up the markets for previously technical products to upwards of 10x the user population.

I recently stumbled across a book that profoundly impacted me over 20 years ago. Called the Art of Possibility by Rosemary Stone Zander and Benjamin Zander, conductor of the Boston Philharmonic Orchestra, there it was in the sharing library of a local cafe, nestled underneath a few spy novels.

There it was the bright yellow cover, a banner of optimism contained within the pages.

I’ve been fascinated by dictation software for many years. Speaking has struck me as the most natural interface. It also happens to be at least 3 times faster than typing.

When OpenAI released their dictation model Whisper, I was keen to try it.

So I installed it on my MacBook Pro.

Over the last few weeks, I’ve been using it. As expected, it’s sensational. In fact, this entire blog post is dictated using Whisper.