Before the scaled sort was introduced, the hope was that it would provide a solution to surface posts from smaller communities, without being overrun by memes and political posts from larger communities. However, the scaled sort has been ineffective so far, as most posts appear with a single vote, making it practically the same as the “New” sort.
The developers have closed all issues related to the scaled sort, even though it fails to address the issues raised in several discussions:
- Rework “Hot” sorting to show posts from more varied communities
- The rank of a post in the aggregated feed should be inversely proportional to the size of the community
- Is there any way to reverse degrowth of the niche communities on Lemmy?
- I hate to say it but I haven’t been very active on lemmy, but I want to be
Personally, I believe the best way to address this issue is through the implementation of tags and custom feeds. With post tags and custom feeds, users could create separate feeds tailored to their preferences by subscribing to a few communities and blocking specific tags or keywords. However, this would require an incentive system similar to imageboards like Safebooru, with a leaderboard to encourage accurate post tagging by users, as also mentioned in The Great Monkey Tagging Army: How Fake Internet Points Can Save Us All!
Although I’ve blocked the largest communities, I still want to see some of that content occasionally.
Do you have any ideas or suggestions on how Lemmy could better surface content from smaller communities?
Edit:
Potential Solutions
Several potential solutions were discussed:
-
Tagging System and Custom Feeds Implementing a tagging system could allow users to create custom feeds by subscribing and blocking specific tags across communities. This could surface niche content by filtering for relevant tags. An incentive system like leaderboards could encourage accurate user tagging.
-
Community Grouping Similar to Reddit’s “Multireddits”, allowing users to group multiple smaller communities together into a single custom feed could boost visibility for those niche communities when browsing that grouped feed.
-
API for Client-Side Sorting Providing an API endpoint that shares metadata for recent posts like post ID, post votes, and comments would allow third-party clients and plugins to experiment with custom sorting algorithms on the front-end tailored to user preferences.
-
“Unanswered” View Having a view that surfaces posts with little or no engagement yet, specifically from smaller communities, could help discover underrepresented niche topics that may need more attention.
-
Server Plugin Architecture If sorting algorithms must be implemented server-side for performance, having a plugin architecture where different instance owners can test out new sorting implementations and formulas could allow faster iteration.
Ideally, a combination of tagging, custom feeds, and improving sort algorithms to factor in community size could provide a multifaceted approach to better surface content from niche communities on Lemmy. Encouraging open discussion around desirable features is valuable to guide development efforts when resources do become available.
We’ve already mentioned that it isn’t possible. You can’t assign percentages to categories or personalize the sorting in any way since it is done in the database.
It would be beneficial to talk more about these desirable features to ensure that when developers do invest time in them, we’ve already come up with a good and robust solution. Otherwise, we may encounter a situation similar to the Scaled sort implementation, where issues remain unresolved despite the feature being added.
This is interesting. So sorting is done server-side, I assume this is for performance. In fact now that I think of it, for a client to implement a custom sort it would need to fetch basically every single post for the entire period, across the entire fediverse, which really doesn’t sound practical. At that point the client has effectively become its own local server.
So that means sorting algorithms need to be implemented server-wide.
I wonder if this could be done with plugins that servers could experiment with and modify more easily, rather than being baked into full version updates?
Edit: I have read further down, I understand this is not possible.
I feel the need to be pedantic here: that quote continues on to the very next (and final) sentence being:
However, anything that is logically possible, is doable, with enough effort & investment - e.g. that infamous quote:
All that quote means is that it would be most simple to do as a back-end task, not a simple front-end one (though even a front-end could, in theory, e.g. slurp up 1000 posts and then use some metric to figure out how to display the most proper subset of 20 from that superset).
But for instance, someone could spin up their own instance, and then add whatever sorting method they wanted to it. However, recall again what happened to DMV.social - anyone who opens up a Lemmy server will be spammed with CSAM, it would seem - so there are other more urgent matters to be attended to first, unless that someone used it purely as a testbed, and made all connections to it to be read-only, or else had a team of moderators willing to put in large amounts of time to fend off those attacks with both manual efforts and also developing automated tools at the same time - e.g. they would need to have technical skill even just to moderate, much less administer the machine (unless, like existing Lemmys, there is a whole team of admins doing the technical parts already). Anyway, I don’t suggest this lightly like it is trivially easy, just to say that it is possible.
Sure, I am not trying to tell you what to do. Just stating that until and unless someone is willing to tinker with actual implementations - and again, right now their attentions seem to be directed elsewhere, plus while Scaled-sorting Hot may not be perfect it is something (I don’t personally have experience to say if it is better than before b/c I was on Kbin.Social at the time which was totally different - but I thought I heard many people say that it is better now?) - then it is going to be a purely theoretical discussion. Which is probably how Scaled sort got implemented too? Though now that it is built, it could be tinkered with, if there is the will to do so.
But unless you are offering to do any of the actual implementation work yourself, I think you would need to discuss this with the actual admins who you would expect to do that work for you - hence you might try Matrix where they hang out, rather than solely discussing it here.
And then, as you said in your OP, when they say “no” and close all GitHub issues, that, as they say, is that. You can’t “force” someone to do work for you for free - and even if you were offering money, or perhaps offering to do all the “design” work yourself for free, they still would need to agree to actually do the implementation.
Moreover, even if you DID offer to do ALL of the implementation work entirely on your own, unless you do want to spin up your own instance to actually run it on, you still would need the buy-in of the instance admins, for which having the buy-in of the developers would go a LONG way.
So you asked:
And my suggestion is that you cannot walk into someone else’s house and tell them how they should do things. Especially when they have ALREADY said no. They know better what their prioritization is, and what they hope to accomplish over the next month, year, and so on. The absolute beauty of the Fediverse is that you can take all of the existing Lemmy code, which is entirely functional, make a fork of it, and spin up your own instance - and not just run it, but even modify the code to do… whatever you want! And then you can share that code, and benefit all the instances that are running Lemmy too! Discuss.Online, Lemmy.World - all of them, well, those that choose to keep your suggestions, though it is up to each one individually to either accept or reject them, and it is ultimately their call. Reddit does not work this way, nor FaceBook/Meta/Threads, Instagram, Xhitter, etc., but we do, b/c it is “open-source”. The caveat to that being… that someone, somewhere, must put in the actual effort to get it done.
And the people who would normally do that, seem to have said no. I gather that you feel frustrated but… it is what it is. Therefore, of what use is it to talk about any of this, when there is no path forward for it? THAT is how to move forward your ideas: either find or become someone yourself who can implement them, and THEN in talking to them you will actually be in an even better position to understand how it all works, and how it might be changed to work even better than it does now.
I dunno, perhaps I should not have replied at all? Sometimes I do overshare my thoughts and if you disliked that here then I apologize:-). It was my hope to help spur your thoughts along these lines that I was thinking, since it seems to me to be the only way forward. But I guess please ignore me if you think I am wrong, and I wish you luck either way!:-)
Fwiw, I do agree that eventually, when the developers are ready to move forward with this again, they indeed might appreciate a ready-made solution if one happened to be already available by then, but again that assumes that one could be made purely on theoretical grounds alone?