Key AI Trends
Think Transformational and not Incremental
Met with 3 peer CEOs this week, each building something different with GPT-3 and a few common threads emerged.
Think Transformational and not Incremental- Start with the killer use case and work backwards. Seems obvious, but with so much headline chasing these days, it seemed worth restating.
The Unit Economics and Cost Structure of products are evolving. Today's COGs are well understood (how many concurrent users can I squeeze out of a server), but OpenAI is priced by tokens based on the model used. Tokens are effectively "Words in" (prompt) + "Words Out" (response). That means a single request can range between fractions of a penny up to about 2 1/2 cents. That's potentially a lot of variability at scale based on the use case. It's neither better or worse than today, but gaining a deep understanding of the model is critical.
As a result, Infrastructure Optimization is shifting to Prompt Optimization as we move from paying for cloud servers to consuming API calls. Xaqt has already started shutting down GPUs and moving towards a much lighter weight architecture. This is a win 🙌
User Experience is being rethought in order to optimize for prompts and not paying for text generation until you need it. Think cost optimization here, and also getting context right for the end user.
Team composition is radically changing. Development cycles are moving much faster as we no longer need teams of data scientists building and training NLP/NLU models, applications are becoming skinnier, and we'll no longer need much in terms of Devops as our cloud footprint shrinks. Rather, we need people really good at Prompt Design & Engineering (ie, chaining), downstream analytics, vectorization, and a new way of thinking about UX.
This means SaaS pricing may change over-time, from per user to things like per-datapoints (Viable seems to have worked through this well)
Lastly, this thing is truly a gift from the start-up Gods, and it is time to bust a move 🕺