While I have you, do you think it’s a bubble that’s going to pop and go the way of the nft or does it live on and manage to find it’s way into everything digital?
While I have you, do you think it’s a bubble that’s going to pop and go the way of the nft or does it live on and manage to find it’s way into everything digital?
I work in tech and there are definitely a lot of engineers focused on efficient AI usage. Almost all of the best engineers I know are obsessed with efficient usage actually. Context lengths are limited and smaller contexts reduce hallucinations, so more efficient model usage also produces better outputs. The main challenge is getting high quality ‘lean’ context to describe your problem fully enough to the model, and there are many promising attempts at this for code problems which will eventually be reflected in other industries with huge knowledge bases that LLMs struggle with.
There is also tons of work on more efficient and smaller models. There are tradeoffs with smaller models in that they hallucinate more, can’t generalise as effectively, and perform much worse. There is a technical limit on what they can model based on their size, though active research is still ongoing.
Reaching a certain context size for sure sets off red flags that cost and hallucinations are both about to multiply as the models start looping nonsensical tool calls which degrade with each iteration.
I understand why people get obsessed with trying to “one-shot” problems, right now I think it’s the developer equivalent of AI Psychosis, playing the lottery to see if you get lucky or receive a half-finished rewrite.