Im joining in on the reddit ditching thing, and was kinda worried at first that i wouldnt be able to like use it the way i did reddit as it feels like a whole new place, but after engaging with posts and people and actually being a part of lemmy rather than being lurk mode all the time i was pleasantly surprised with how easy it is to become a member of the community, theres a reasonable amount of subs (or whatever the other word for em is) that fit my interests, enough linux content and shitposting for my liking, and the overall random posts made by people equally fed up with Leddit. (also i admit i used reddit a little cus there was this post on the fedora sub showing how to fix a sound issue i been having after a recent update)
Why isn’t there a script that admins can run to do this automatically? It seems like a no-brainer that you would want all communities to show up in search by default.
It’s definitely not a no-brainer to replicate every post/comment in the fediverse to a single-user server where the user is only interested in reading 3 of the communities. That would induce a massive federation/replication overhead both to send and receive posts and comments that no one cares about. Lazy replication that only kicks in when someone asks does admittedly feel weird on a brand new server, where everything is new/empty… but it makes a fair amount of sense if you think about what’s going on behind the scenes. I’m sure a person COULD write a script to subscribe the admin or a dummy account to a seed-list of popular communities to get things started, take a crack at it if you want. It’s not clear to me that it should be the default though, as this doesn’t make sense for single-user or very small instances, they should start empty and decide what they want to subscribe to.
I do think that a useful middle-ground would be to populate the LIST of communities. The all-posts/all-comments feeds really do need to be constrained by what communities people are actually subscribing to. But the number of communities is much smaller, and their rate of creation/change much lower. I think it could be reasonable on federation init to ship the full community list across the wire even on a big server that hosts thousands or tens of thousands of communities, and then eagerly ship community creation/deletion events across the whole fediverse. That seems like it would be a manageable volume of information, it would enable every instance to have a useful /communities/ page from the moment they federate, and it would make subscribing to remote communities much simpler. Right this minute the devs are trying to fast-track performance improvements to keep
lemmy.mlfrom falling over due to the new influx of users though, and that’s good/important work to prioritize now.That’s what I meant, I think they should make it so that all communities show up by default on the community list page.
Also just so I understand what you’re saying, are you saying that I could intentionally sabotage a server just by subscribing to a lot of communities, which would cause the All page to use a ton of resources whenever people look at it?
On a tiny instance with insufficient resources, maybe. But it’s not so much a problem for one person to decide to subscribe to a lot of things. It’s more that you have to be careful about eagerly shipping useless messages around big federated/distributed networks. Imagine a world where Lemmy is very successful, and a network of 10,000 instances federate with each other but maybe 9000 of those are tiny personal instances. If one of those servers has an unpopular sub, there’s a BIG difference between shipping 50 copies of a post made on that sub to the servers with users that care… vs shipping 10k copies to servers where mostly no one cares. Then multiply that by the potentially hundreds or thousands of posts on unpopular subs across the whole network. It’s very easy to ship around millions of messages that no one reads. Good federated designs MUST minimize this.
So if you want to intentionally sabotage the network in the way I’m describing… then you’d set up thousands of lemmy servers and configure users on them to subscribe to everything. People would probably refuse to federate with you and the bad thing would never happen… but yeah… thousands of servers could generate a pathological amount of replication load on the servers hosting active communities.
Hmm interesting, thanks for the explanation.