https://blog.erlend.sh/transitioning-r-rust-to-the-threadiverse
Three months ago I submitted a post to the Rust sub-reddit called 'Building a better /r/rust together'. It quickly rose to the top and ga...
The problem of which instance to host a community on is a big problem for Lemmy. Grouping is an interesting idea but it causes problems as now there are different mods and admins that control subsets of the community.
Picking a single "winner" and letting the others wither seems like the right approach and will probably happen naturally but if the original instance ever shuts down or struggles under the load you will have a mess to migrate to a new instance.
If Lemmy communities were decentralized it would make a huge difference. You could just have a single community but it could survive instances coming and going (as well as many other performance and resiliency benefits). But that would be a huge change to the underlying implementation of communities.
The problem of which instance to host a community on is a big problem for Lemmy.
Seeing the same content posted six times in six communities is a problem. It pollutes the feeds, it fragments the conversation, and prevents the natural death of low-traffic communities.
Avoiding showing identical posts to a user separately seems like a very easy problem to solve.
I think it probably makes sense to host similarly moderated content together.
programming content being grouped together makes sense because it’s moderated to a similar extent between languages and communities.
For example discussions/posts on rust programming and porn are moderated very differently, so they should be on different instances.
I disagree. Grouping communities makes sense if they need the same admin policy. Admin policy sets a floor on what mods are required to police when it comes to things like hate speech and threats, but mods in a programming community aren't going to allow that shit anyway, so admin policy is irrelevant.
I think we agree, and I didn’t know “moderated similarly” was referring to admin policy. Basically you want the people at the top to be generally aligned with the views of the community
Yup, I'm actually interested in working on a truly decentralized lemmy. Basically, I want to use IPFS or Iroh to have it completely decentralized across all users' machines.
However, the big problem remains the same: how does moderation work? With lemmy, you can always go to the instance admin if your mods suck, but if it's completely decentralized, you need some other mechanism. Also with lemmy, if a community starts sucking, there's usually enough redundancy that you can just go to another major instance and find a similar community, but if it's decentralized, I think you'll have the Reddit problem where you'll essentially have to get a large chunk of the community to move if there's an issue.
So I'm not convinced that decentralization is the right way to go, at least until the problem of moderation is resolved. Maybe I'll try building a decentralized instance, which is largely intended to solve the issue of scaling, but I don't think a decentralized platform would be a good replacement for lemmy.
I would also take a look at Matrix. Its rooms are decentralized (although other parts like users are not). It would be an interesting backing protocol for something like Lemmy. You could probably use a model where a community is a room, then posts are references to rooms posted into that room. Comments on posts then occur in the post room.
how does moderation work?
I think it works the same. As part of the community configuration you have a list of mods. Those mods can then post messages such as "ban this user" or "remove this post". If these messages are sent by a mod then they are applied by those in the community.
With lemmy, you can always go to the instance admin if your mods suck, but if it’s completely decentralized, you need some other mechanism.
I think removing the instance admin is actually a feature. IMHO the community is the mods to some degree. If the mods suck then you can either ask them nicely to transition to a new set of mods or you just start a new community. Basically I don't think it is valuable to add a power above mods, because then what if the instance admins suck? I think part of moving communities away from instances would be removing the power of instance admins. They can still ban communities from their instance, but they have no power over the community itself.
if a community starts sucking, [...] you can just go to another major instance and find a similar community, but if it’s decentralized, [...] you’ll essentially have to get a large chunk of the community to move...
I think this is a feature. It is basically democracy for communities. If the community is great then it will gather lots of people and be popular. If it starts sucking then people will leave and find alternatives. It is basically democracy. If there is one "Rust" community with fantastic mods that everyone loves then I don't see it as an issue that there aren't alternatives. But if they start upsetting some subset of the communities you will see alternative communities form and if the majority of people prefer the alternative then it will supplant the "main" community as the new biggest one.
rooms are decentralized
Are they? Or are they federated? I was under the impression that it worked like the rest of Matrix, which operates on ActivityPub, which is a federated network.
I know encryption is run locally, but do the posts also only exist among users' devices? Or is storage centralized/federated?
When I say decentralized, I mean something like BitTorrent, not Lemmy, Matrix, or Mastodon where there are admins managing instances where data is copied.
As part of the community configuration you have a list of mods
Sure, and what happens if one of those mods' accounts gets hacked? Can these attackers add a bunch of other mods to "take over" the community? Can they remove existing mods?
what if the instance admins suck?
Then use another instance. The admin changing significantly isn't all that likely since they have a financial stake in keeping the instance running to their satisfaction. It costs money to run an instance, it doesn't cost money to mod a community.
then people will leave and find alternatives
Sure, and that exists with lemmy as well. But one nice feature lemmy has is namespacing, where you can have multiple similar communities with the same name on different instances, and they can serve as fallbacks to others. Names matter, and they matter a lot more than I think they should, but they do matter.
If !rust@programming.dev sucks, I can move to !rust@lemmy.ml or !rust@lemmy.world or whatever. On Reddit, if /r/rust sucks, I'd move to /r/rust_programming, /r/rust_official, or /r/rust_dev or something (not sure if any of those exist), and it's not exactly clear which I should pick, and new users will likely still flock to /r/rust and blame the Rust community for the online community with an official-sounding name sucking.
There's no easy solution here, but I think namespacing does help, as does having the option of appealing to an admin if a high level mod account is compromised or something.
I think the solution has something to do with users voting and being able to override mod actions, but I'm not sure how to structure that in a way that's unlikely to get manipulated. Perhaps requiring new mods to be approved by a majority of existing mods is an acceptable solution, which should reduce the risk of a hostile takeover. But I'm not a security expert, so I'm not sure what I may be missing.
Regardless, I'm going to work on this decentralized lemmy idea as a hobby at least as long as it holds my interest. I sincerely hope someone beats me to it.
the rest of Matrix, which operates on ActivityPub
Matrix does not use ActivityPub, it has its own protocol.
Sure, and what happens if one of those mods’ accounts gets hacked? Can these attackers add a bunch of other mods to “take over” the community?
Yes. Or it depends on permissions. There have been some ideas for voting-based controls over things like adding and removing mods.
The thing is that you can extend this to what if the instance admin account gets hacked? It seem better to have one point of failure (the mods) rather than two (mods + admins).
what if the instance admins suck?
Then use another instance.
You can't do this if the community is tied to one instance. This is my point, right now you need to trust the instance admins, and sort of the mods. I think it would be better to just have mods. This means that there is a clearly defined source of control over the room rather than two levels.
Names matter, and they matter a lot more than I think they should, but they do matter.
You can still have names like this. The way Matrix does it is actually pretty clever. There is an internal ID for a room which should be mostly invisible to the user. But there are also local aliases. So for example you can have #rust:matrix.org. But if the matrix.org admins decide that that room is not a good target they can update the alias to a different room. These names are controlled by the instances but the room itself is not. I think overall this is a good strategy, trying to have one centralized directory is a recipe for fighting, disagreement and bad results. Pushing the directory responsibilities away from the core protocol makes a lot of sense to me.
I don't think it is a perfect system but it works fairly well. Plus at the end of the day most users are going to come from whatever rust-lang.org points to. Or what they find in their search engine.
Matrix does not use ActivityPub, it has its own protocol.
Ah, I was misinformed. I'll have to read up more on it, because some less-than-authoritative sites claim that it is decentralized (as in, no main/client relationship like lemmy, but instead equal peers), but I'll want to verify that. Matrix claims to be decentralized, but I've seen a lot of misunderstanding on what "decentralized" actually means.
If you have any good, approachable resources, I'm interested.
what if the instance admin account gets hacked?
The admin would probably just reset the instance from a backup. Or access the DB directly to fix things.
A mod doesn't have that control, they only get the permissions granted to them.
You can’t do this if the community is tied to one instance
Right, which is why the Reddit model sucks. But with Lemmy, you can have multiple communities with the same name covering the same content. An enterprising user could even automatically cross-post everything between the communities with a bot, so it could very much operate as a "hot spare." I personally don't even know where most of the communities I engage with are hosted, nor do I particularly care because admins tend to only get involved when asked.
So for example you can have #rust:matrix.org. But if the matrix.org admins decide that that room is not a good target they can update the alias to a different room
I definitely need to read up more on Matrix then. It seems I made a lot of assumptions that just don't hold.
most users are going to come from whatever rust-lang.org points to. Or what they find in their search engine.
Unfortunately, most projects don't include all popular communities. Unless I missed it, rust-lang.org only mentions their own forums, Discord, and Zulip, yet /r/rust has a very significant number users engaging, and that's where I primarily engage as well, so I wouldn't be surprised if many users just assume /r/rust is an official channel (it's not).
So are most users using the Rust forums and those two chat systems? The Rust forums claim ~10k users, whereas /r/rust has ~250k subscribed users when I checked today.
If I search "Rust lemmy," I get !rust@lemmy.ml, and this instance doesn't show up at all in the first 2 pages of DuckDuckGo or Google results. It might get there eventually, but it's a problem any non-centralized platform is going to have, especially if there isn't a common word used in domain names. So if something non-centralized is going to have a shot at all, it needs to be blessed by the project, and even that may not be enough.
I'd like to see truly decentralized platforms work, but I'm just not confident that we've really ironed out enough of the issues to really be successful. But I'm going to try anyway.
claim that it is decentralized
Yeah, the truth is a bit more complicated.
But I think the key thing here is that rooms are decentralized and can survive sever losses. If I create a room on matrix.org and matrix.org disappears it can still be fully used by other servers. (As long as at least one other server was participating when matrix.org went offline, much like you need 1 seeder to keep a torrent alive.) Of course if all admins were accounts on matrix.org then the room will no longer have any admins, so it may be crippled, but if there were admins on multiple servers then the room can effectively live forever and survive any number of homeserver losses.
This is very different from federated protocols like ActivityPub as you can probably tell where the death of an instance will kill any communities on it.
And that's certainly something I'm worried about. If the admin of programming-dev gets tired of running the instance, it could just disappear.
And I'm happy with using servers as an optimization, but IMO it should always allow users to host at least a portion of it themselves without a server. As in, everybody becomes a seeder. That way, if an instance gets hammered with a DDOS or something, the network can still survive by routing around it.
Mod actions could be cryptographically signed using a private keys, and the public keys of the mods would be part of each community's metadata, updated in a way that establishes a chain of custody so only existing mods can add new mods. Each instance would independently verify that mod actions come from a legitimate mod. (I think I basically just described an implementation of NTFs representing mod privileges, BTW.)
only existing mods can add new mods
I prefer it to have multiple mods, ideally a majority. That way, you can't have one mod "go rogue" and add a bunch of alts or whatever, which also means a mod account getting compromised can't "go rogue."
I'm less concerned about mod actions like deleting posts, banning users, etc, since mod actions should always be able to be rolled back since most decentralized systems use immutable data (so a mod action is merely data that instructs clients to ignore or prefer certain other data). However, I don't want a situation where mods become powerless because one of their accounts got compromised.
I'm not concerned here about the rules for how mods are added or removed, just the technical implementation. It's easy enough to require a majority for decisions like that.
There has to be a way to establish with certainty that a user taking mod actions is actually a mod. The fact that you can revert changes in a git repo doesn't make it ok for people to commit without permission, and mod actions are the same. Just allowing unauthorized users to perform mod actions would allow them to fuck up communities faster than the real mods could undo the damage.
Yeah, I agree. All mod actions should be signed by their cryptographic key, and moderator cryptographic keys should probably be separate and stronger than regular user cryptographic keys. Mod actions should be rare enough that this isn't a burden to verify.
One thing I'm not as sure about is if the data is decentralized as well, users would potentially be liable for illegal content. I suppose there could be a system that moderator-removed content gets removed from all regular users' devices, so maybe that's good enough. But then that makes auditing mod actions difficult, since the original data could be much harder to get.
A lot of these problems aren't really technical, but rather UX when designing for a fully decentralized system.
That sounds like a blockchain with signature verification against a previously established and acknowledged set of keys as consensus mechanism. Pretty reasonable, as far as use cases go.
However, it doesn't solve the issue of disagreements and community splitting. If one part of the mod team decides to add another mod, but the rest doesn't, what's to prevent that part from splitting off and continuing their own version of the moderation chain? How is abuse of power handled? And in case of a split, how are community members informed?
Don't get me wrong, I'm not saying it's a poor idea, I'm just saying that it won't solve the issues of community splits, and I'm not sure anything ever can.
I wasn't trying to solve that particular problem, on the assumption that it has already been solved and the same solution can be adapted to the implement I proposed. Someone else who replied to me suggested something like requiring majority approval to add or remove a mod.
Another possibility is for the creator of a community to be a super mod, who can add or remove regular mods, or transfer their super mod status to someone else. That scheme could easily be generalized to allow multiple super mods, or to include a whole hierarchy of mods for large communities.
I don't really think it's discussed in this blog post, but maybe some effort should be placed on trying to see if rust-lang.org/community would be willing to link to a chosen instance. reddit has been partially hostile towards communities that have closed or tried to move their community. I also just think it makes sense for rust's governance to manage a community, but, they might just want to link to one instead for now. (until if/when Mastodon and the fediverse is more successful)
On the website for rust, they do already link to a Forum, a Discord, and a Zulip chat, so maybe they would be willing to list a fediverse community too.
edit: I just realized the poster is Discourse Staff on rust's forum.
edit 2: It's not included in the blog post, but I would really like to be able to use rust's domain in the fediverse. ex. user@fediverse.rust-lang.org
Yep, there’s a clear avenue here. Discourse (where I no longer work, so I shouldn’t really have that staff title any longer) is also implementing ActivityPub, so the Rust forum will actually be able to subscribe to this hypothetical /rust community as well. We just have to give them a good enough reason to do so, by achieving a modicum of unification through this group-follow proposal.
Then it’ll be up to the Rust team to decide which particular /rust community on the fediverse they will hook up the official forum to as their trusted gateway into the larger network.
Discourse (where I no longer work, so I shouldn’t really have that staff title any longer)
I understood the title as staff for rust on a discourse instance. It seems like I misinterpreted the rank, and it actually might mean that you were a developer for the Discourse forum software.
We just have to give them a good enough reason to do so, by achieving a modicum of unification through this group-follow proposal.
I didn't really read the blog post very closely before commenting. It looks like the concern is more focused on how to create a global name system for communities that users understand. I don't agree that it's an issue for rust, but I do think it's an issue for the fediverse in general. I'll cover both quickly.
I think programmers likely understand the fediverse really well once they've used it. I am doubtful there are actually problems with programmers finding their communties, and successfully subscribing to them. I think Discord servers are more comparable to communities than subreddits to communities. On Discord, if a server doesn't have a vanity address, it's partially not a problem because users are actively searching out official services. For programmers, if the organization backing rust officially designates an instance, I think programmers will understand it.
In my other comment on LemmyRS, I actually saw the disorganization of the fediverse as a feature, not an issue.
I don't think users understand it at all. I think it's mostly hopeless to try to explain that this other website that seems similar to their home instance can be followed by returning to their home instance or installing this plugin that will link it for them. I think solving it as a client problem is likely a better approach. It seems like users surprisingly do comprehend how to send and receive emails, but I think there are still user interface problems that need to be solved for the fediverse.
I think trying to solve it at the federation level is likely a bad idea. In my opinion, it just seems like other federated services, like IRC and email, and even IP systems (IPV4 vs IPV6) have been very resilient to change. Though, I do think it's possible to have changes merged into Lemmy.
There are multiple reasons, here are some
(Mastodon created the above clip)
The reddit thread is found at: https://www.reddit.com/r/rust/comments/162keij/transitioning_rrust_to_the_threadiverse/
I assume there will be some discussion there, so go and let them know what you think.
I read that thread, and I remembered I was not missing the average level of negativity Reddit has
As outline in the blog post, https://lemmyrs.org/ would lend itself best to avoid the case of one big instance de-federating from another.
Discoverability can be an issue with smaller instances but I'd argue that can be bypassed by simply linking to it from official resources that previously linked to reddit. Same with linking to that instance from Reddit.
One thing of importance IMO is that should lemmyrs.org be selected as the reddit replacement, there needs to be communication and more importantly help for that instance admin, so that they don't have to carry the weight of supporting one of rust-lang's communication channels.
The problem with lemmyrs.org seems to be that it is poorly maintained, still on Lemmy 17.4 and few moderators aso. Programming.dev otoh is very well maintained, well moderated and still not a huge centralized place. It also host many other interesting resources for programmers. Also, the structure of lemmyrs.org with many smaller communities doesn't work really well at the moment, it would require a huge number of users to avoid these communities being ghost towns.
In short, running an instance requires quite a bit of work, so having a really small instance might be quiet a challenge. Programming.dev is still niche, but large enough to not be a one man show.
I'm biased (I created programming.dev), but I think lemmyrs.org should definitely not be the main site, but that goes for any site with "lemmy" in the name. The entire purpose of federation is that the software doesn't matter. If things go south with lemmy in the future, or people fork it, or just want to migrate to different software you've now completely tied the identity of the site to the software it's running on, which just really seems dumb to me. The reason I chose programming.dev as the name is several reasons.
Of course, these are my own opinions, but I did think for a long time about it before settling on the website name, while it seems most server owners did not really think at all about their site names.
Very good points tbh. My main thought with why I suggested lemmyrs.org is cause it would be an entire instance around Rust, not just a single community on an instance. That being said, there is already an official discourse for Rust so maybe just having a single community on lemmy as opposed to an entire instance is enough in that case ^^
Just pointing out that the pawb.social people are/were also planning on forking Lemmy for similar reasons: https://pawb.social/post/147036 . Not entirely sure how much work has gone into it, but might be worth syncing up with them. Although I'm not sure if it's the "right" thing to do to fork just for ideological reasons, especially since the main lemmy.ml instance seems to be fairly neutral.
I've been thinking about how a single "community" could exist across multiple instances, especially given that the landscape right now is that communities are basically:
Communities following others seems an elegant solution, honestly. Although, I would say that moderators should be able to remove posts of communities they follow, just in case.
However, something stuck out to me when reading the design discussion:
Users who post to a community that follows other communities are offered the choice of whether to post directly to the community they're following, or to one of the communities that are followed by that community. They need not post multiple times, because posting to a more 'upstream' community would cause it to be seen by users of that community as well.
Why not? The lemmy web client at least does a good job at de-duplicating crossposts, and the client used for posting could give you a bullet list of communities you want to send it to. Imagine instances a, b and c where a defederates c, but a also has the largest community for bespoke hatwear or whatever. If you (who is on none of those instances) send your post to just a (because it's the largest), then your content will be unavailable to c. But if you post to both a and c, you reach both communities.
Another thing that confused me while trying to wrap my head around things is this diagram, which I don't think covers a common case:
If a user on b makes a post 1 to the community on c... What happens?
Option 1:
funny@c boosts post 1 as message 2.funny@b is sent 2 and boosts post 1 as message 3.user2@a can see 1 through message 3 because it is posted on b, which they federate with.Option 2:
funny@c boosts post 1 as message 2.funny@b is sent 2 and boosts post 2 as message 3.user2@a cannot see 2 through message 3 because 2 is on c which they do not federate with.How is communities undiscoverable? There are services for this https://lemmyverse.net/communities , of course it would be nice to have that more integrated in to Lemmy, but it is still there.
Okay, maybe "undiscoverable" is kind of the wrong word. I'm thinking about users who don't really understand how lemmy work and don't know that these tools are available.
During the initial "gold rush", I noticed a lot of duplicate communities being created on lemmy.world that already existed elsewhere (e.g. !opossums@lemmy.world for !possums@possumpat.io ). These communities I guess just didn't appear during a search on lemmy.world (since it didn't know about it).
Honestly, it felt like if you made a community anywhere other than lemmy.world, a lemmy.world version would appear and outcompete it unless you linked it elsewhere or it had a dedicated instance.
Not saying that anyone did anything wrong, it's just people being unfamiliar with the platform.
Well, duplicate communities might also be on purpose, to lessen centralization. I see many that try to migrate away from lemmy.ml, and this community is one of them. I agree that it might be a bit confusing, but it is easily worked around by subscribing to both communities.
@erlend_sh I thought the term "Threadiverse" was meant to be "Threads" from Meta. Looking into it, it's an ActivityPub server. I don't like the term Threadiverse, because it is confusing to Threads. Can we call it ActivityPub or Fediverse instead?