How AI Redefines User Experience

What if every software spoke English? We asked this question about two years ago but now they do - with AI we can retrofit existing apps to speak English.

I don’t want to have to figure out any particular menu to find a setting or understand how a product manager or designer intended me to use the product.

I just want to talk to my computer and tell it what to do.

So I’ve started to invent tools that help me do this. Here are some movies of them & admittedly they are quite simple.

Sometimes I just want to send a quick email with my voice. I wrote a Python script where I can dictate an email. The computer will figure out what I mean, who I intended to send it to, and if it needs clarification, raise its hand and ask me to disambiguate the recipient.

Here’s an example of me asking for help from PitchBook customer support to reset my password :

Sometimes I’m on the go and I need to dictate a task. Rather than navigating the UI of my to-do app (who has time for 5 clicks?), I created an Android app within about an hour that uses the on-device machine learning model from Google Gemini Nano to summarize and transcribe the task & then send it to my task manager.

It’s not perfect. You can see it doesn’t capture the due date quickly, accurately, but something to focus on improving next weekend.

Over the weekend, I was migrating to a new computer and rather than install all the software I need manually, I used AI coding agents, Claude Code and Cursor, telling them which libraries and repositories I needed, which software I wanted, what environment I preferred. I let them start and 20 minutes later, my computer was 80% ready.

This is the beginning of an era where we are no longer instructing computers, we are delegating tasks to them to figure out, iterate, decide, raise questions when input is needed but otherwise continue.

It’s fundamentally changing both the way we interact with computers and also how designers and product managers will need to design.

If it’s trivial for hundreds of millions of people to develop their own UIs for the core tasks within an app, it highlights a couple of things. The first is very difficult for a PM and a designer to understand all the user segments and use cases.

And the second is the way functionality is exposed will likely need to evolve to make these kinds of custom UIs simpler.