Tinkering with LLMs
I’ve been tinkering with LLMs.
First, I created a chatbot using all my blog posts. Then I created a model to produce blog entries based on my writing.
As I went through this process, I asked myself some questions along the way :
- Does the ML model ingest my posts & keep them? What if I’d like to remove them from the logs or training set or output for others?
- The blogbot often returned no answers for questions I thought would be straightforward. How can I ensure the response rate is 99%+ before launching?
- Which post is better : hand-written, ChatGPT, or custom-trained models? How would I judge? More views? Written in my voice with alliteration & rhetoric? Use of the & instead of “and”? Injects links to other related posts?
- When is a blogbot a better user experience than search? With search, I can know that I’ve plumbed the depths of the blog, looking at every relevant post for an answer. How do I do that with a bot?
- The latency for both is significant : 5 to 25 seconds depending on the query. Google observed 400ms increase in latency decreases traffic by 20%. Will users be more patient with bots than with search?
- What if I reimagined the home page of tomtunguz.com as a chatbot interface rather than a list of all posts? That UI would personalize the experience for each visitor & each session, but it would hamper browsing? Which is the more important use case?
I imagine many product teams are asking analogous questions about how to leverage these new models.
Ethereum revenues have collapsed 82%, which makes the earnings change that much starker. What other business suffered a loss of 80% of revenue & went from burning $2b per quarter to producing more than a hundred million in profit?
This is called a non-deterministic or stochastic answer. Similar inputs do not consistently produce identical outputs. The answers have inconsistent logic.
