!lemmy_project_pri
@bulletintree.comIt isn't just the constant database crashes in Lemmy, GitHub issue 2910...
https://github.com/LemmyNet/lemmy/pull/3708
On July 24, sanitation of HTML was added to the code. But the testing was not called for and it broken titles of postings, link parameter ampersands, discussion of programming code in code blocks
It's data... and now it's very difficult to undo all the damaged data that has been put into the database for weeks now.
Lemmy is a Link aggregator, and it damages Links now... the ampersand parameter deliminator in URL links is now broken because of this code not being tested. Why wasn't there a call for testing to something that was going to alter every new post and comment from both federation and Lemmy itself? how did such obvious things such as a ? parameter list in URL get overlooked... and then new bugfix release comes out after this was known as an issue - and still not fixed.
Database crashing that results in lost data from unsaved post and comments, failure to deliver Federation data without any way client or server operators are notified, and damaged data as fundamental as URL website links...
I'm all for code changes gong in fast, but the lack of actually testing things and spot-checking on Lemmy instead of just changing Rust code without really realizing that a link aggregator uses ampersand in URL links... and not asking people to help think of side-effects...
Development process could even ask just a couple sites with more attentive operators to try out the code for a few days and ask people to report any problems before advising all sites to upgrade and break their URL links.
June 4, 2023 is when I felt I had to get involved. And June 8, 2023 is when I crated my own testing-focused experimental Lemmy install that compiled Main from GitHub source and ready to test the changes I thought were surely going to come to the SQL because of GitHub issue 2910. I wanted to test the code that that developers who had been working and running lemmy.ml for over 4 full years - would surely address. It was June 4 with Issue 2910, the June 30 Reddit API cutoff deadline countdown was well under way. Lemmy.ml put in major hardware upgrades on June 13, 2023 - and I was puzzled why such an easy 2 or 3 hour fix for Issue 2910 wasn't put in... but I still had hope that everyone would see the pending countdown to June 30 API deadline and a fix could go in within 10 days - by June 23... for some slack time before June 30..
Watching Beehaw, up and running for 17 months on Lemmy - crashing constantly... I thought, surely the developers were seeing Issue 2910 happening over there... but June 28 came, June 29 came, June 30... nothing. With 4 years of experience on the Rust code base and such, they were the ones to fix Issue 2910.... but from June 4 to June 30, it just didn't happen.
ORM and SQL .. I mean it's lived up to the bad side of the reputation.
It hides it. As someone said here:
"Osiris on June 18, 2021 | unvote | parent | context | favorite | on: DenoDB
I hate ORMs with the fury of a thousand suns. The problem is that I know SQL but now I have to spend a bunch of time trying to figure out how to convert SQL into ORM X just so it can convert it back to inefficient SQL. SQL mostly translates between various databases but ORMs are unique and you have to learn a new API for each one.
I'm on a project using TypeORM and it has been fantastic at helping developers on my team make really bad schemas due to not understanding how to use TypeORM to make the right relationships.
Currently I'm looking at pg-types because you just write SQL and it just helps by making some TypeScript types for you.
(I have used ORMs in C#, PHP, and JavaScript and I hate all of them)."
In April and May, Lemmy had a great brand... but server crashes and then lemmy.ml closed sign-up.... sort of confusing people and then Lemmy.world became the brand, then it too has had to close home page many times and constant crashes....
The SQL performance issues that were in lemmy for years really - became trouble once data was introduced.
I don't even know how many times people have come to sing-up and heard about Lemmy to find it slow, crashing, errroring.
Social hazing, rock star glamour attitude among the audience who keeps using Lemmy... but these odd trends in 2023 with social media aren't unique to lemmy, are they?
Threads, Twitter to X, Reddit API change, Lemmy crashing and not really caring about the crashes - and Redis or Memcache to mitigate it.... it's all such an odd year. It's as if everyone stirred the pot with social media but no place to really land upon that isn't crashing or in some form of Elon Musk kind of chaos.
I remember the rise of Reddit, the rise of Gmail - and it just felt like the growth was being dealt with. In 2023, Twitter seems to be leading the entire audience into accepting a kind of crash bad experience and 'stick around, no matter how bad it is' that Reddit users seem to have accepted.
I don't want to be a Lemmy developer... I want the people who know Rust Diesel and such to actually make it work - or choose something else that does work. I want Lemmy to actually not crash all the time and at least be where Reddit was in 2008 when there were just a few programmers doing it.
The SQL statements and server crashes in Lemmy speak for them self, just like the chaos of Reddit and Elon Musk Twitter speaks for itself... but a lot of people accept it. The audience is not asking for stability.
Strange times in 2023!
Unbelievable today the push-back on filtering by what is in-essence, the entire key to lemmy - published date
Ok, taking a total fresh approach based on all the months I've been reading people's suggestions and !lemmywishlist@lemmy.ml kind of things....
I think server operators should at minimum want the ability to view Remote-only, and even per-instance. Further, I think proxy of API to a community should be something to head towards... where a Lemmy API call can be forwarded as an API call on another Lemmy server for a specific community.
Some stock multi-community lists, some of which are dynamically generated like Small / Medium / Large based on number of active posts, users, or other tunable parameters. Encourage people to engage in topics that they normally would not see...
I think foundation is that it should use words and not numbers. Right now, the entire Lemmy system is built upon using localized index numbers for community. Even if it just becomes a JSON blob to throw into PostgreSQL and recall by name...
Maybe have them like communities. And people can subscribe/unsubscribe to a specific list. And the list can have moderators who regulate it (editors). And an option to clone a list to new name.
/mc/ multi-community, name. And no ID numbers. A Trigger or something would have to build the ID numbers in the background.
And a browser of these, much like communities are browsed... and maybe even voting on them. Lemmy doesn't have voting on communities - subscribe alone - but sometimes you don't want to subscribe because they have too much content - but you would still vote for it or recommend it.
Weeks ago I had my moment of facing the attitude of keeping all this secret.
Just casually mention join_collapse_limit was tried behind-the scenes a month ago, then why are there zero post or comments in the entire Lemmy search for join_collapse_limit? I searched the entire GitHub project - no mention of join_collapse_limit. But Ready on the Spot to reveal the secret private communications tried join_collapse_limit log ago.
You know what join_collapse_limit is telling yo8u? Too many JOIN is a performance problem! The entire ORM Rust code and reliance on new JOIN is going to lead to other unpredictable performance problems that varies when there are 10,000 posts vs 2 million posts! And that's exactly the history of 2023... watching the code performance wildly swing based on the size of communities being queried, etc.
What I see is that pull request for ideas get created only after noise is made on a subject. There is a lack of openness to make mistakes in public.
For me,** the server crashes are what annoys me**, not human beings working on solutions. But for most of the people on the project, what seems to anthem is needing to have proper tabs vs. spaces on source code and even adding layers of SQL formatting tools in the middle of what clearly can be described as an SQL performance crisis.
Things keep getting broken: the HTML sanitation took a few hours to add to the project but now weeks of broken titles, HTML code blocks, even URL parameters are now broken on everyday links. The changes to delete behavior have orphaned comments and that has gone on for weeks now.