I sleep better knowing my agents work through the night. Less work for me in the morning.
My podcast processor transcribes & analyzes conversations. I started on my laptop, needed a little database to collect podcast data & metadata, & booted up a DuckDB instance.
But then the data started to grow, & I wanted the podcast processor to run by itself. I changed two little letters, & the database moved to the cloud :
# Before : local only
conn = duckdb.connect('podcasts.db')
# After : cloud-native
conn = duckdb.connect('md:podcasts.db')
Now, in the small hours, 10 robots listen & summarize podcasts for me while I sleep.
As I collect more & more podcast information, my data has grown. I’m using a larger instance of MotherDuck.
Source : ClickBench
Aside from ease of use, there are real price-performance advantages. MotherDuck systems are two to four times faster than a Snowflake 3XL & from a tenth to a hundredth of the price.
Source : ClickBench
As the amount of data expands & I process more technology podcasts every day, I’m sure I’ll need a data lake. At that point, I can migrate to DuckLake.
Small data becomes big data faster than you know it.
Two letters changed everything. In this era, when those letters aren’t AI it’s worth paying attention.