I get the impression that we’re headed for the same issues that pop up when we put all our eggs in one basket with Reddit/FB/whatever. People flock to the largest instance, and someday that instance could go down due to cost or the host losing interest.
I’m wondering whether it would be technically achievable to have servers/instances and federation where the communities are essentially mirrored or have broadly distributed existence - maybe even with user storage a la torrents.
If there’s a large blargh@lemmy.here community and a small blargh@lemmy.there community, all of the discussion, images, contributions to lemmy.here die if the server goes down for good. Yes, the users can relocate to lemmmy.there - even under the same community name - but it’s not the same as having full continuity of a completely mirrored community.
I realize this concept has technical hurdles and would involve a reimagining of how the fediverse works, but I worry we’re just setting up for another blowup at some TBD date when individual sysadmins decide they’ve had enough. If it’s not truly distributed and just functions as a series of interconnected fiefdoms, communities and their information won’t survive outages, deaths, and power struggles.
Defederation and independence is IMO the strength of the Fediverse. How will this work in this scenario? You are basically proposing a fault tolerant Reddit, if I understand it correctly. I apologize in advanced if I read it wrong.
Yeah, trust and federation is a concept that would have to be worked out in this model. I think it would require some critical-mass of federation amongst instances to establish trust. I wonder if this is an actual application for [dodges beer can] blockchain technology…
And how would a blockchain help?
To be honest I don’t have a clue but my lay understanding is that it’s a tool for establishing trust or authenticity in a distributed system. It was a half-serious suggestion because I’ve always heard those who are much more knowledgeable than me say that it has essentially no worthwhile applications.
One minor correction: if lemmy.there has users subscribed to blargh@lemmy.here, the content that is federated to lemmy.there won’t vanish if lemmy.here does: that copy is more or less independent once it’s federated out.
It’s not a 100% complete clone, but it’s also not at risk of totally vanishing off the face of the earth, either.
There are issues with further interaction with that group (since the host instance is gone and it won’t federate back up and then out to other subscribers), but the content does still exist anywhere it was subscribed to.
You could in theory nominate one instance to take over a community. They would then hack their database a little to make it local.
But, I could be wrong. Doesn’t lemmy use the original URL for media? So you would lose that unless the transfer was co-ordinated before they shut down. Kbin does take a copy of the media and self hosts, which of course comes with its own problems.
The media was something I didn’t consider, but yeah the original image is served from the original posting server.
That’s probably trivial to fix, given other fediverse software locally caches and serves images, though yeah, that requires an extra level of trust and more work for admins in the far too likely case of CSAM or other illegal content.
I agree with OP that instances being closed any time is an issue that would need to be resolved fairly soon. A solution in my opinion would be the option to transfer user accounts across instances. This would help with an instance closing and eventually make the fediverse more stable.
A new user currently has a choice for joining from a number of instances but there is no assurance to ongoing existence for them. Along with that, afaik there is no way to transfer user accounts and data across instances. If a user can transfer their accounts and data, there will be less hesitancy to join a new instance, and user accounts and data can be distributed across more instances. This can also work in such a way that if a subset of user data does not meet the criteria for another instance, then that subset of data is not migrated (most likely a community based data filter).
Another issue is with the presence of same community/magazine in multiple instances (let’s say tech@lemmy.this and tech@kbin.that) which is frustrating for users since they need to track multiple communities for similar content and the same content is being copied to multiple communities. This should also be resolved by implementing account migration. We are already seeing that communities on certain instances are becoming the prevalent ones. This creates an incentive for the admin of those instances to not shut down. And if they did decide to shut down the instance, then the users can just migrate to another instance and the prevalent community will also get to keep all its data, just in the new instance.
As a user, if the instance I originally joined is shutdown - I would want a feature that lets me easily transfer my account and data to another instance. But not sure how this would work considering right now I could go make an account with my username in a different instance…
God I’d love to implement a lemmy instance as a byzantine fault tolerant distributed system… Anyone wanna lend me a working brain?
I don’t think it is that easy. But a good feature would be if you could make backups from communities on other instances in case they go down. It would not save the pics, but you should use a third party image service to keep the hosting costs of you instance lower anyway. In the same way, you, but only you, could download a backup of your account.
Also a good idea would be a new protocol which allows syncing feeds/communities between instance, which effectively is mirroring.
This is what federation already does, mostly. Federation means that threads and comments are copied between instances and kept up-to-fate. You are reading this from a copy kept on discuss.tchncs.de. I am reading this from a copy on kbin.social. The original is kept at lemmy.world. If lemmy.world goes down then our copies remain. They are still readable. But you won’t see my new comment and I won’t see your new comment because lemmy.world was responsible for syncing our copies.
This applies to text posts, links, comments and votes. Images and videos would be gone because they are not copied, just linked to.
That’s correct and that’s the problem. If a given community server goes down, that community basically just becomes an archive. It really needs to be able to continue without the host instance, similar to how a mesh works. Each remaining server routes around the dead node.
There is also the problem of search engine indexing… If a given server goes down, that information is lost to the search engine, even though it’s still on other nodes.
Which also leads to duplicate content problem for search engines, as ECU m each node of a given community contains the same information for a given post, making it crappy to index and search.
I think the current model under more mature circumstances is actually the ideal. You want sysadmins to experience the full cost-benefit analysis of running servers. They are functionally countries and should have the risks associated with running a country. A market-driven benevolent dictator/benevolent committee, where the server operators are highly motivated to be good actors, as is currently the case, is the ultimate freedom-security situation.
The risk of a server being able to “vanish” as it were is an essential risk for both community users and community managers to bear.
I agree with OP that instances being closed any time is an issue that would need to be resolved fairly soon.
A new user currently has a choice for joining from a number of instances but there is no assurance to ongoing existence for them. Along with that, there is no way to transfer user accounts and data across instances, afaik. Having the option to transfer user accounts would help with the instance closing and eventually make the fediverse more stable.
If a user can transfer their accounts and data, there will be less hesitancy to join a new instance, and user accounts and data can be distributed across more instances. This can also work in such a way that if a subset of user data does not meet the criteria for another instance, then that subset of data is not migrated.
Another issue is with the presence of same community/magazine in multiple instances (let’s say tech@lemmy.this and tech@kbin.that) which is frustrating for users since they need to track multiple communities for similar content and the same content is being copied to multiple communities. This should also be resolved by implementing account migration. We are already seeing that communities on certain instances are becoming the prevalent ones. This creates an incentive for the admin of those instances to not shut down. And if they did decide to shut down the instance, then the users can just migrate to another instance and the prevalent community will also get to keep all its data, just in the new instance.
That is how Aether works.
Aether is a reddit alternative that I prefer to Lemmy honestly, however it is not being maintained.
I agree with OP that instances being closed any time is an issue that would need to be resolved fairly soon.
A new user currently has a choice for joining from a number of instances but there is no assurance to ongoing existence for them. Along with that, there is no way to transfer user accounts and data across instances, afaik. Having the option to transfer user accounts would help with the instance closing and eventually make the fediverse more stable.
If a user can transfer their accounts and data, there will be less hesitancy to join a new instance, and user accounts and data can be distributed across more instances. This can also work in such a way that if a subset of user data does not meet the criteria for another instance, then that subset of data is not migrated.
Another issue is with the presence of same community/magazine in multiple instances (let’s say tech@lemmy.this and tech@kbin.that) which is frustrating for users since they need to track multiple communities for similar content and the same content is being copied to multiple communities. This should also be resolved by implementing account migration. We are already seeing that communities on certain instances are becoming the prevalent ones. This creates an incentive for the admin of those instances to not shut down. And if they did decide to shut down the instance, then the users can just migrate to another instance and the prevalent community will also get to keep all its data, just in the new instance.
