It’s a longer one this week, as I get slightly long-form on my thoughts on the future of UI design in the face of LLMs and then beyond that go fairly non-tech as I get into some food and music bits.
🗣️ 🎮 A new dawn, a new day: Future of User Interfaces
🍲 🥕 Food for thought
🌭 🎵 Kendrick Lamar
⚖️ 📈 LegalTechTrends
LLMs: A new model for interaction
The optimal, and long term, interaction model (or user interface) with LLMs is something I think about regularly, and have been thinking about for some time.
Are we going to end up with the interface to all digital tools being natural language? If LLMs continue to improve and between enhancements to foundational models and tooling being created around them, we start to see the error rate move towards 0, then would a natural language (I’m going to refer to it as voice for the rest of this) interface be the obvious and optimal interface for all digital tools?
If we are able to combine an incredibly deep general knowledge with an expertise in each given tool along with strong reasoning and language skills, then it certainly starts to look like a good idea. It certainly reduces learning curve for everything - if the LLM has the expertise to understand and extract the exact user intent through a conversation then that is undoubtedly going to speed up the process of getting to better end results. It wouldn’t just be useful for beginners, either, I know I can’t be the only person who has found themselves using a tool or software for years before realising some previously undiscovered feature or shortcut.
I’d not be at all surprised if I discovered I was using less than 10% of Excel capabilities. Part of that will be habit, I’ve learnt some basics that are enough to achieve what I want, albeit maybe not the quickest or most efficient way of going about it, but I can usually get what I need doing done. Now imagine if the interface was an expert in Excel who knew every single feature and functional available and when to apply them - I’d be able to just describe my desired outcome and it’d get it done in the best way possible. Of course this would lead to a potential overall de-skilling.
This is an often pitched future. The idea that away from just simple generative AI as we know it, we have a future with fully autonomous tools and software. Autonomous software that can extract your intent from a broad conversation and then work through multi-step processes with deep expertise to build entire end products/outputs. That certainly sounds appealing, and looking at the trajectory of generative AI I don’t think you could bet against this (there are already companies building autonomous agents, so maybe the future is now). But I still mull over if this is a one size fits all solution? Is it going to be the end of the UI as we know it?
I still believe that in some systems, where accuracy and reliability are essential, voice UIs won’t be optimal. Furthermore, I think in some categories of UI there will still be many that just aren’t optimal as voice and benefit far more from a simpler, more binary approach to input (of course that’s not to say that voice doesn’t also win out there, after all, AI and voice commands are neat, right?)
Prompts are going away
One way or another, I’m predicting that prompts are going to become a thing of the past. Those arguing for all UI to be replaced with voice interfaces argue that the models are improving and will continue to improve such that prompt engineering (really wrangling with the LLM trying to optimise the output by providing extensive and detailed instructions) will become irrelevant. I think this is probably quite likely, recent models have already reduced the need for and effectiveness of many prompt engineering tricks.
LLM improvements aside, I have also pitched that prompts need to go away, because user interfaces that are basically modern equivalents to “..say the magic word” are not exactly ideal for anyone.
Considering the following use cases:
Any software or tool where accuracy and reliability are critical
Any software or tool that is expected to work with structured data in a deterministic way
I could argue non voice UIs still feel like they should be better? Any time non-deterministic software is added into any step of the process (and here we are talking about adding it as a UI feature, so possibly just an ease of use solution) it adds risk of error and it adds uncertainty. If we are dealing with 1) critical system which has minimal tolerance for error 2) systems with known data inputs that can just as easily be done with other UI methods, then why not avoid the risk, no matter how small?
But I think as the risk of error reduces it will get to the point that it is less than the error rate of a human (me) checking and double checking. It might be that we get to the point that whilst a deterministic, traditional UI is safer than an LLM UI, the risk of error shifts to the operator, at which point, an LLM might be the more reliable overall system.
For example, consider the two scenarios:
Traditional e-commerce UI such as Amazon - 100% reliable, deterministic
Voice UI for a smart assistant/agent - let’s say we hit 99.99% reliable
As the latter reliability trends upwards, the point of failure is probably me accidentally selecting/purchasing the wrong item. I’m pretty thorough reviewing items before purchase but I’ve definitely ended up with items with slightly wrong spec/capacity/etc a couple times and I can say my kids accuracy and online shopping is probably closer to 50% success - so an LLM voice UI that can clarify relevant details, specs and fully understand the intent and what you are after may well mean it’s reliability for the overall transaction is higher than the deterministic traditional UI.
The scaling of systems
I often think about this non-deterministic element of LLMs and the issue they present, and much like my early musings on tech like the Rabbit gen-AI phone, felt it was it’s undoing. The example I used back then, and still think about now, is would you feel comfortable using a voice interface to book a holiday? We are talking thousands of pounds and potentially your one holiday of the year being ruined (I know this is an extreme scenario, this is imagining you don’t immediately check you booking confirmation details etc but lets ignore that), so to trust a voice interface feels like a pretty bold move. Even when I book online, I double-double check details, dates, prices before confirming.
But of course, pre-internet, booking holidays was done via voice, either on the phone or face-to-face with travel agents. Humans, just like LLMs, are non-deterministic. If you chat to a travel agent and they book your holiday, its just the same. You might have more transparency as to the booking as they tilt their screen towards you to review pictures, but beyond that, its down to you to double check the confirmation, and if there has been any misunderstanding during the conversation then you have to catch it and try to get it corrected.
Likewise telephone or in-person banking. Non-deterministic, voice interfaces in the form of cashiers, interpreting user intent and translating to potentially high value transactions. This is all not mentioning the additional layers of risk that humans bring, such as if they are feeling unwell, tired, distracted etc.
Of course, along came the internet and spoilt us with deterministic systems. Sure, there wasn’t the flexibility of options. You were limited to the specific options provided by the given tool, software or website, but you had relatively high confidence that if you selected a specific date, or destination, you’d reliably get that info. Of course software is made by humans, so it might fail at times, but we looked past that and believed that the systems knew what they were doing.
So maybe the next iteration is back to non-deterministic, but at scale.
We scaled industries and systems up by moving from non-deterministic voice interfaces in the form of person-to-person interaction to highly-available, deterministic software and web applications, so maybe the next step is a web of LLMs - highly available non-deterministic voice based systems at scale.
2024 Food unwrapped
Waitrose has published its 2024 Food trend report, and as always it makes for fun reading. I enjoy these things, and of course, these are shopping trends at Waitrose, so you know, not necessarily representative of the mood across the country. To give you an indication of the demographic this is surveying, I provide the following quote without further comment:
While 38% are making bread from scratch, many simply don’t have the time with today’s hectic schedules
Ok, forget that, I’m gonna comment. 38% making their own bread hahaha, imagine.
A lot of the trends were driven by social media, if you look down the list of trending products, especially in the context of talk about move away from UPFs and with more than 1 in 3 of us making our own bread (..), you might be surprised to see hash browns and instant noodles on there, but digging into it a little, they both make the top list because of various TikTok trends earlier in the year (no shade, hash browns are great and definitely deserve a top spot if you ask me).
Apparently, despite the ongoing cost-of-living challenges in this country, Waitrose-rs are returning to artisanal loaves, spices and olive oil. Apparently taking a bottle of hipster olive oil to a dinner party is the new bottle of wine, who knew.
Probably also unsurprising, non or low alcohol sales are up, although I have worked through a lot of zero-alcohol spirits and none are really up to the price tag. Guinness Zero is the only one I’ve tried thats come close to matching its alcoholic counterpart (although I was charged £6.90 for a can in London the other week, which I probably wouldn’t recommend).
This one I’m leaving here without comment:
(ok, ok, sure - I have a dog, and yes, I’d probably buy this if I saw it)
Kendrick Lamar
Kendrick released a new album the other week. I’ve not massively got into it yet, it’s enjoyable enough but my stand out track on first few listens is TV off. It’s produced by DJ Mustard, who also produced Kendrick’s record breaking Not Like Us (in his recent back-and-forth with Drake)
I’d say Mustard’s style is immediately identifiable (and certainly feels like a sequel to Not Like Us), and its fun that both tracks sampled songs from the same album and you know I’m a nerd for this stuff:
Legal Tech Trends
I’m not sure if I’ve mentioned this on here, but I was once again reading Peter Duffy’s LegalTechTrends this week, and it’s a newsletter definitely worth signing up to. It has loads of news from across the industry as well as his own insights, which are always great.
It usually includes a bunch of data as well as broader industry news with commentary - He speaks on lots of firms/products, which is something I’d love to do but have decided against (entirely to save getting myself in trouble)
