There’s a lot to cover here but I’ll try to touch on each point:

The key requirement is fast memory that can be addressed by your GPU, and ideally a lot of it - hence the insane cost of this hardware right now.

Remember that you need space for the model’s weights (think of this as its ‘knowledge base’) and the context window, which is basically the data needed for the LLM to keep track of your current conversation with it (effectively its short term memory).

With smaller pools of VRAM (8-16gb) you will have to compromise and either have a more capable model that will lose context quickly and start hallucinating, or a less capable model that can maintain a session for a bit longer but overall less ‘smart’.

For software - there are a couple of options for running the LLM itself, Llama.cpp is one of the more popular tools and is the one that I use. It has a web UI with the usual chat interface, and also exposes an API that you can plug other tools (e.g. opencode) into, depending on your use case.

In terms of hardware recommendations, at 20GB+ of VRAM you do have a bit more headroom compared to more consumer grade GPUs, but to be honest the most cost effective way to get a shitload of VRAM is likely not with a dedicated GPU but actually using a system based around a recent APU.

I got a Minisforum MS-S1 last year for exactly this purpose. It is based on AMD’s Strix Halo platform which it has in common with the Framework Desktop and a couple of other similar devices.

It has 128gb of unified RAM which can be divided between the GPU and CPU however you like, so plenty of capacity for even fairly chunky models. It also uses a tiny amount of power compared to a more traditional system with a dedicated GPU, while also giving really reasonable performance for most AI workloads, more than enough for use in a homelab.

For cloud rental - doable, but pricing is a factor, and of course this will not actually be running locally.

Usability - manage your expectations, but overall for a lot of use cases and of course depending on the model that you are running and the resources you throw at it, it can be comparable with especially older iterations of ChatGPT, Gemini etc.

But remember, you are not a Google or an Anthropic and do not have an infinite pool of compute to throw at your model, nor do you have access to the specific models they are using.

I’m not sure if this is triggering either thalassophobia or submechanophobia but either way thanks I hate it

He is the manifestation of the ‘I made this’ meme. Among other things.

Similar to others, maybe 2 or 3 years old. I was “helping” (probably hindering) my mum paint my bedroom. I distinctly remember waving the paint roller around.

I have witnessed companies make this exact mistake before - they have a legacy system written in $LanguageA that they either cannot find developers to maintain, believe is badly written, or does not support some new feature they want to implement (or some combination of the three) - and decide to solve this by taking the existing codebase and porting/transpiling it to $LanguageB (which is more modern, performant, is easy to hire developers for, etc) - without actually rewriting or rearchitecting anything.

What they are actually doing is substituting one kind of tech debt for another. The existing code that was poorly written and/or not well understood is now just bad code written in a different language. Fixing bugs or implementing new features now takes just as long, if not longer to account for the idiosyncrasies of how the code was ported.

And now this is being done by AI with even less oversight than usual? Recipe for a maintenance disaster.

Personally any game that requires a rootkit in order to play is not a game I’m interested in, from either a gameplay or a security point of view.

I’ve switched both my laptop and desktop over to Linux (Bazzite and Fedora respectively) in the last 6 months.

The last time I tried to daily Linux (over a decade ago) I ended up switching back eventually, but this time I really don’t think I’ll need to. All of the games I play most often work perfectly, the dev tooling is even better than it is on Windows, and the hardware compatibility side has been completely flawless.

Gone are the days of having to hunt down obscure Linux drivers for your touchpad or webcam. Everything just works out of the box.

Hot take: Nintendo peaked with the Wii.

Well, I’m currently writing a service and frontend, both in C# (Blazor for the UI), and using docker-compose to build and deploy them to a Raspberry Pi running Linux. So not only cross-platform, but cross-architecture as well.

This is not a new thing either. Since .NET Core was released almost 10 years ago, it has supported cross platform development.

Assuming your Fiesta produces < 100g/km CO2, looks like you’ll now be paying £20/year (as it was manufactured before 2017, different rules apply) - could be worse!

Not quite. Even existing EVs (with some exceptions for older vehicles) will be charged the new basic VED rate of £195 when they are next due to renew (which could be in up to a year’s time).

What EV purchasers were trying to avoid was the expensive vehicle supplementary tax of £425/year for cars with a list price of over £40k, which EVs purchased/registered since the start of April are no longer exempt from.

As the vast majority of new EVs on the market fall into that price bracket (including all Teslas except the base tier Model 3), suddenly the yearly tax for most new EV purchases jumps from £195 to £620/year.

Right now none of the native clients support SSO. It is a frequently requested feature but, unfortunately, it doesn’t look like it will be implemented any time soon. As with many OSS projects it is probably a case of “you want it, you build it” - but nobody has actually stepped up.

For web access, stick it behind a reverse proxy and use something like Authentik/Authelia/SSO provider of your choice to secure it.

For full access including native clients, set up a VPN.

Oh fuck me, HOW in like 25 years did I not get that pun?!

I think you may have Europe confused with Japan.

There are loads of viable alternatives these days with both legacy auto manufacturers closing the gap and a raft of new EV manufacturers joining the market, mostly from China.

I sold my Model 3 last month and took delivery of a new Polestar 2. The only features that I’ve lost in the switch are the built in dashcam (had to install my own instead) and using my phone as a key (which is supposed to be fixed in the next software update). Other than that, the feature set and specification of the two cars is pretty similar. Except my new one has twice the battery capacity.

We refer to it as kew-bee-cuttle

Play stupid geopolitical games, win stupid prizes…

Sold my Model 3 yesterday. In a sane world, where Tesla is not run by a fascist, I would have considered getting another one as my ownership experience was generally pretty positive.

Instead, my new Polestar arrives next week.

I have a IIIXe (very similar to the one in OP) somewhere. Really limited in what it could do, but very cool for the time. I also have a later model Zire somewhere that had enough horsepower (with a mild overclock) to play Quake.