• 0 Posts
  • 26 Comments
Joined 2 years ago
cake
Cake day: August 4th, 2024

help-circle
  • So they would need to swallow up even more of our chip fab production and push ram and SSD prices even further through the roof for checks notes ah yes… the same functionality as they have on earth.

    AI is already unprofitable because of the insane hardware requirements and the fact that no company has a “moat” so there is a race to the bottom pricing-wise… I can’t imagine anyone also then accounting for building space-hardened kit and getting it into space and dealing with shortened lifespan of the kit is ever gonna see a return.

    All this just so that a chatbot can confidently tell people the wrong stuff






  • I’d love to know more about that 30% reported increase and how real it is (I know this is never going to happen). Is it a) Nvidia<=>OpenAI b2b stuff where they increased revenue by grifting some other CEO b) massaging the numbers to make it look like AI is popular - Microsoft Office+Copilot style or c) there is genuinely something valuable that people are buying

    I feel like there is a whole lot of b) going on with companies baking AI into popular products and then going “ooh line gonup, must be AI” but I could be wrong.








  • Unlike the dotcom bubble, Another big aspect of it is the unit cost to run the models.

    Traditional web applications scale really well. The incremental cost of adding a new user to your app is basically nothing. Fractions of a cent. With LLMs, scaling is linear. Each machine can only handle a few hundred users and they’re expensive to run:

    Big beefy GPUs are required for inference as well as training and they require a large amount of VRAM. Your typical home gaming GPU might have 16gb vram, 32 if you go high end and spend $2500 on it (just the GPU, not the whole pc). Frontier models need like 128gb VRAM to run and GPUs manufactured for data centre use cost a lot more. A state of the art Nvidia h200 costs $32k. The servers that can host one of these big frontier models cost, at best, $20 an hour to run and can only handle a handful of user requests so you need to scale linearly as your subscriber count increases. If you’re charging $20 a month for access to your model, you are burning a user’s monthly subscription every hour for each of these monster servers you have turned on. That’s generous and assumes they’re not paying the “on-demand” price of $60/hr.

    Sam Altman famously said OpenAI are losing money on their $200/mo subscriptions.

    If/when there is a market correction, a huge factor of the amount of continued interest (like with the internet after dotcom) is whether the quality of output from these models reflects the true, unsubsidized price of running them. I do think local models powered by things like llamacpp and ollama and which can run on high end gaming rigs and macbooks might be a possible direction for these models. Currently though you can’t get the same quality as state-of-the-art models from these small, local LLMs.