Changelog

2024.06.26 - upgraded lemmy to 0.19.5
2024.06.09 - upgraded lemmy to 0.19.4, postgres to 16, pict-rs to 0.5.15
2024.01.23 - upgraded lemmy to 0.19.3
2024.01.11 - upgraded lemmy to 0.19.2
2023.12.21 - upgraded lemmy to 0.19.1
2023.12.17 - upgraded lemmy to 0.19.0
2023.10.12 - upgrade VPS to 160GB, other specs unchanged
2023.08.9 - upgraded lemmy and lemmy-ui to 0.18.4
2023.07.29 - upgraded lemmy and lemmy-ui to 0.18.3
2023.07.11 - upgraded lemmy and lemmy-ui to 0.18.2
2023.07.10 - upgraded lemmy-ui to 0.18.2-rc.1 to mitigate XSS vulnerability
2023.07.10 - VPS upgraded to 8GB RAM (required to upgrade storage, needed anyway.. only $2pm)
2023.07.08 - upgraded lemmy and lemmy-ui 0.18.1 🎉
2023.07.06 - upgraded lemmy to 0.18.1-rc.10 and lemmy-ui 0.18.1-rc.11
2023.07.04 - upgraded lemmy-ui to 0.18.1-rc.10
2023.07.04 - upgraded lemmy and lemmy-ui to 0.18.1-rc.9
2023.07.03 - upgraded Lemmy to 0.18.1-rc.4 and lemmy-ui 0.18-rc.7
2023.06.28 - VPS upgraded from 2 to 4 vCPU, other specs the same.
2023.06.24 - upgraded Lemmy to 0.18.0
2023.06.23 - upgraded Lemmy to 0.18.0-rc.6
2023.06.13 - upgraded Lemmy to 0.17.4. Increased federation workers and reduced logging storage at the same time.
2023.06.10 - VPS storage upgraded from 40GB to 80GB, other specs the same.
2023.06.09 - VPS upgraded to 4GB RAM, 2 vCPU, other specs the same.
2023.06.08 - aussie.zone created, running Lemmy 0.17.3. OVH VPS was 2GB RAM, 1 vCPU, 40GB NVME

Nerd Stuff

The aussie.zone server is currently an OVH VPS in Sydney:
8GB RAM, 4 vCPU, 160GB NVME storage

Images are stored in an object store bucket on Wasabi, also in Sydney.

I post updates every week or so randomly with current server resource graphs:
Nerd update 20/4/24
Nerd update 2/9/23
Nerd update 13/8/23
Nerd update 5/8/23
Nerd update 29/7/23
Nerd update 22/7/23
Nerd update 15/7/23
Nerd update 7/7/23
Nerd update 30/6/23

If you have any questions, please post.

I've noticed a lot of broken images around the place and can't quite work out what's happening. It doesn't seem to be a caching issue, at least not on my end, as trying to visit the image URL directly returns a 404.

As noted in a post in the Melbourne daily thread a few weeks ago, there are also a lot of broken community icons about the place. And as I think was raised a while ago, thumbnails seem to be broken most of the time as well.

As an example, here's what the trending communities tab looks like:

The communities page is a little better, but it's about 50/50 as to whether the icon does or doesn't load:

Here's a list of my own posts with missing images I've noticed recently

That last image seems to be super broken at the time of writing. Instead of just returning a generic 404, it returns:
{"error":"unknown","message":"Request error: error sending request for url (http://pictrs:8080/image/original/b4df332b-82a9-4ec8-b115-ac1b1b1824b0.jpeg): connection error: Connection reset by peer (os error 104)"}

Additionally, this is how thumbnails on links have been previewing:

Right-clicking and opening in new tab, it seems to be pulling a random pixel from somewhere in the image and then enlarging it and proxying it (500% zoom):

Example URL

But then other URL thumbnails don't work at all, for instance this post on trending:

Which links to Wikipedia, but fails for whatever reason, then tries to load an image called "broken-image-fallback.png", and, ironically, can't even load that (leads to a 404 page).

And some thumbnails on working images don't work, for example this post in c/GardeningAustralia doesn't load - even if I click on the thumbnail (which should normally load the full size/original image), but does work if I click the image URL itself.

One more thing: this new image proxy thing is getting super annoying. I realise it's a Lemmy-side implementation issue, but a couple of sites (Imgur, for example), don't like all the image requests from a single server/IP and end up rate limiting us. This effectively means that Imgur is unusable and no images load.

I actually thought there might have been another mass image purge like last years' incident which required the purging of 48 hours of images, but there hasn't been any announcements about it, and the other broken images dotted around the joint makes me think Pictrs is on its deathbed.

Any ideas what's going on?

Sidenote: image uploads whether in posts or as embedded images within posts/comments seem to fail about half the time for me on both mobile and desktop. I thought this was an issue with Boost again, but it's happening to me on PC as well. On mobile, it usually either returns no error at all, or a 500. On PC, no UI errors are displayed, it just doesn't embed the image, but network tools tell me that something somewhere is returning a 500, likely the image uploader thingy.

I thought the recent patches were supposed to fix the slow federation, but we're still not getting comments or vote tallies through for over a week.

About to reboot the server, hold onto your hats.

I've been seeing various 5xx error codes (usually 504s, but occasionally 502) sporadically most of the day. It doesn't seem to be a constant thing, just every 5-10 or so loads/clicks, it errors out. I've also noticed it's sometimes taking a while to load. Seems 50/50 whether it takes 1-2 seconds, or 6-10 seconds to load.

Is this something on my end, or is today a lazy Sunday for the server, too?

(I took a screenshot, but I'm getting errors when I try to upload that, too. A classic "ffprobe failed with exit code 1: success")

I'm the developer of Fediverser Project, which is a set of services to make it easy for people on Reddit to migrate to the Fediverse. It lets people use their Reddit credentials (OAuth) to sign up and create an account on a Lemmy server.

It also offers a cool onboarding feature: during signup, we can fetch the user's subscribed subreddits, and we use this information to automatically subscribe them to the corresponding Lemmy (or Kbin/Mbin) community. This "subreddit -> fediverse group" map is crowdsourced and people can sign up if they want to contribute. The "main" site also provides a "Find an instance" feature: it can track all the servers that use the Fediverse software and redirect users to their closest instance.

To enable this service, the Lemmy admin needs to add a couple of docker services to their setup and needs to get their own Reddit API key (which is used only for authentication, so well within the rate limits and certainly not incurring any prices).

I'd really like to see aussie.zone becoming part of the network. I believe this would make it faster and simpler to get more people in the fediverse, and I'm willing to provide all the support and help needed to get the "country-based" services getting started with it.

Any questions, don't hesitate to ask.

This is a simple software bug. You see, the way Lemmy federates is that updates to a community only federate to instances that have a user subscribed to that community. If you take a community like !libertyhub@lemmy.blahaj.zone, I'm the only person on all of Aussie Zone who subscribed to it. I know it because when I subscribed, we started getting updates from Blahaj.Zone about that community.

A few days ago, I was mistakenly banned from that community. And this caused Blahaj.Zone to stop sending our instance updates on the community. I spoke with the admin of the community and got unbanned, but the unban didn't federate. The instance will only let us know I'm unbanned if someone here subscribes over there. And I can't subscribe, because this server thinks I'm banned. It's a deadlock. If it wasn't for the ban, I could subscribe and the server would realise there isn't a ban anymore. But I can't, because this server thinks there is still a ban.

When I try and upload an image for my avatar, I receive the message:

{"data":{"files":null,"msg":"ffprobe Failed with exit status: 1"},"state":"success"}

This happened with both a WebP and a PNG file, so I suspect it's a bug.

I can upload images fine here, and when I export my account details it shows a valid image URL for my avatar.

So, as per images below, when you search for an Australian community associated with lemmy, lemmy.world is more likely to come up than Aussie Zone in all i've tried, bar Melbourne our most active community.

My question: Is this a problem we should consider intentional action to correct? And if so what could we do?