Our gracious host @JonahAragorn asks that we sign up on a server on join-lemmy.org or sign up on kbin.social instead of directing everybody to use the lemmy.one server specifically, in order to distribute the load.

Thank you!

  • @empireOfLove
    link
    English
    111 year ago

    It looks like about the 1k active user mark is where the Lemmy instance starts to use too many resources. lemmy.ml has blown past that with 1.7k active and almost 40k total registered users, and they’re having a lot of performance issues over there.

    Hope they can build a good scaling framework into the Lemmy backend pretty damn quick, because nobody is going to survive the Reddit influx otherwise.

    • @BurningnnTree
      link
      English
      3
      edit-2
      1 year ago

      To clarify, you’re saying this is an inherent issue in the Lemmy code, not an issue with hosting or whatever? So it’s not currently possible for a Lemmy instance to handle a couple thousand users?

      • @empireOfLove
        link
        English
        51 year ago

        I have a very limited understanding, but, yes. A few thpusand is ok so far, but The database managing post contents becomes the bottleneck as it simply cannot be updated fast enough. No single instance can handle that much traffic, it has to be horizontally scaled somehow.

        • @BurningnnTree
          link
          English
          21 year ago

          That’s interesting, thanks for the info. I hope this issue can be improved before the upcoming Reddit protest. (Or at least the messaging should be improved on the join-lemmy site so that new users understand that the experience is better if you join a smaller server instead of a bigger one)

        • @Randall_Potato
          link
          English
          21 year ago

          I wish I had the time to dedicate to learning enough to know what you’re talking about. This kinda stuff sounds so interesting.

          • @empireOfLove
            link
            English
            9
            edit-2
            9 months ago

            Ok, so the best analogy is: a database is a warehouse. Every user has a shelf in this warehouse, and they add and take stuff to it at random. The workers in the warehouse move stuff in and out of the doors as orders (user requests) come in and out.

            You can build the warehouse taller and wider to hold more shelves (vertical scaling of hardware), but only to a point before the forklift trucks can’t reach and driving across the warehouse for one order would take too long.
            You can similarly add more workers (CPU power) and more doors (faster storage/memory) to your warehouse to handle more orders at one time, but these also have limits because the warehouse is only so big. And there’s eventually a point where no amount of workers or doors or shelf space will improve throughput, because the roads around the warehouse are clogged with trucks.

            At this point the obvious solution is horizontal scaling- build a new warehouse somewhere else nearby, and a small office building (load balancer) that directs new orders to each of the warehouses evenly. Then you have the office occasionally check in with both warehouses to make sure they both have the same stuff to fulfill any given order (synchronizing instances). And as orders continue to increase, you can just keep adding more and more horizontal warehouses- all of which are individually somewhat small and simple, but effectively infinitely scalable compared to the vertical mega-warehouse we tried to build originally.

            Lemmy tries to build in the horizontalness by having separate instances federate and share data automatically without the need for that “front office”. However the issue still stands that individual instances are going to need to handle hundreds of thousands of users for the Lemmy ecosystem to really thrive, and that is going to take a ton of back end database engineering to allow individual instances to scale as well as the fediverse network as a whole.

            • @Randall_Potato
              link
              English
              41 year ago

              Well shit. I’m gonna go add Database Engineering to my resume now. Lol

              Seriously, though. Thanks for the great explanation. Wasn’t expecting that.

  • @arbiter@lemmy.ml
    link
    fedilink
    English
    91 year ago

    I purchased (havent deployed) lemmy.club yesterday and I’m going to look in to deploying via kubernetes this weekend. We’ll need some instances that can horizontally scale so that we can absorb as many users as we can from the APIcalypse.

    I’m slightly worried about the costs, but I’m loving the platform so far and am willing to do as much as I can!

    • @FirstWizardZorander
      link
      English
      41 year ago

      I’m interested in how this would work, especially in regard to the database, which I understand is the most intense of the workloads.

      If this could be solved in a way that scales well, I think we’d be in a good spot. I’ll be looking into the Lemmy backend code in the near future, to see if there is any way to offload the db.

    • @empireOfLove
      link
      English
      2
      edit-2
      1 year ago

      I am also interested in anything you learn about the horizontal scaling. Once I get some free time I’m planning to open my own instance for car culture related stuff and want to be learning ahead of the curve just in case there are legit traffic spikes.

      I was under the impression that the lemmy DB did not scale safely without some core code rewrites.

  • @Gortus
    link
    English
    61 year ago

    Oops, saw this too late. Should I figure out how to delete the account, just not use it, or what?

    • @jonahA
      link
      English
      81 year ago

      You’re welcome to use this account :)

      I just want to avoid everybody joining lemmy.one.

      • @SoManyMinutes
        link
        English
        21 year ago

        I’ve been encouraging everybody at reddit to join lemmy.one in preparation for the exodus.

        Should I stop doing that?

        lemmy.one is the perfect catchall destination name.

        • @jonahA
          link
          English
          51 year ago

          It’s fine to register here, I mostly just don’t want to create a misunderstanding that you have to register on lemmy.one to join local communities here, which can be joined from any instance. And I’m telling Subreddit moderators to link people to join-lemmy.org instead of lemmy.one in announcement posts in the spirit of decentralization.

    • @jab4037
      link
      1
      edit-2
      1 year ago

      Fully without javascript? I’m quite curious how this works for such a website.

      Is it a matter of disabling javascript and everything still works? It definitely loads JS locally, so I’m not sure what you mean

  • @SoManyMinutes
    link
    English
    21 year ago

    What happens when /r/atheism wants to join this community through lemmy.one?

    • @jonahA
      link
      English
      51 year ago

      If the current mods of r/atheism on Reddit want to create a new community here on lemmy.one, they can message me as described here: https://lemmy.one/post/41

      If people are just looking for an existing atheism community here on the Lemmy network, there are atheism communities already on other servers, such as !atheism@lemmy.ml.

      • @turtleatoll
        link
        English
        41 year ago

        I think we should have new users go to join-lemmy and specifically go to like near the bottom and register there to decentralize lemmy thorooughly

      • @SoManyMinutes
        link
        English
        21 year ago

        Thanks. It’s clear I don’t understand how this works. I was wondering if/when reddit implodes if this place would be able to handle the traffic. Again, don’t think I understand how this works. I’m learning. Thanks again.

        • @GuyDudeman
          link
          English
          5
          edit-2
          1 year ago

          So yeah… whatever instance your account is associated with can subscribe to any community from any other instance. You just have to view it “through” your instance that you’re logged into.

          So, say someone has their account with lemmy.ml - they would go to: https://lemmy.ml/c/exchristian@lemmy.one and they could subscribe to it using their lemmy.ml account, and would be able to interact with it through their lemmy.ml account (rather than having to sign up for lemmy.one to subscribe to it).

          It also works visa-versa. Say someone has a Lemmy.one account and they want to subscribe to a community on Lemmy.ca… for instance, WowThisLemmyExists:

          They login to lemmy.one, and then go to https://lemmy.one/c/wowthislemmyexists@lemmy.ca and they could subscribe to it using their lemmy.one account.

          The only caveat is that the instance that your account is on has to have been “federated” (connected) with the instance that the community is on. (Which you can check if you go to the bottom of the page and click “instances”)

    • spaceghotiOPM
      link
      English
      21 year ago

      I don’t think I understand the question. There’s a host of servers to sign up through and they can all access each other. !atheism@beehaw.org can reach us here, no problem.

  • @Bearyhungry
    link
    English
    21 year ago

    Interesting. Good to know thanks