It doesn't seem to be doing anything for me, even on large websites like YouTube or Amazon, it basically just copies the link as-is.
I took some photos at an event and I need to go through them and get rid of the bad ones (eyes closed, things in the shot, out of focus, blurred etc.) I'm not a pro photographer so no idea where to begin with photo apps. I've used RawTherapee and Gimp a bit.
What app will let me quickly browse the photos and handle (delete/tag) photo formats together (both the RAW and the JPG)?
I wanted to run my VPN/Tailscale setup past you, see if anybody has suggestions on how I could do things better.
10.0.0.0/24
), router+DNS on 10.0.0.1
, server running docker containers on 10.0.0.2
.*.local.dom.tld
to the server, public DNS points *.dom.tld
to my dynamic public IP.10.0.0.2:443
, forwards to various other services.Goals for Tailscale:
Goals in general:
How I progresed with Tailscale:
100.64.0.2
) available on the host's default network stack. Containers can use "ports:" to map to 100.64.0.2
(tailscale) and/or 10.0.0.2
(LAN). Bad: tailscale would mess with /etc/resolv.conf
on host. Also bad: tailscale0 on host picked up stuff that binds to 0.0.0.0
.network_mode: host
). Made it leave /etc/resolv.conf
alone. tailscale0 on host stack still picks up everything on 0.0.0.0
.This is kinda where I'm stuck. I can make the tailscale container bridged which would put the tailscale0 interface inside the container. It wouldn't pick up 0.0.0.0
from host but how would I publish ports to it?
network_mode: container:tailscale
). This would prevent said containers from using "ports:" to map to host anymore. Also, everything they publish locally would end up on tailscale0 whether I like it or not.10.1.1.1
, mirror that from the tailscale container, and target it from other containers explicitly with "ports:" when I want to publish a port to tailscale. Downside: 10.1.1.1
would be in the host's network stack so still picks up 0.0.0.0
.192.168.1.0/24
and use tailscale serve
to forward specific ports to other containers over that subnet. Unfortunately serve
is fairly limited; it can't do UDP and technically it refuses to forward TCP either to non-localhost (but you can dump the serve config to JSON, and hack that config, and use it with TS_SERVE_CONFIG=
🤮).I'm posting this in selfhosted because Gandi increasing prices actually helped me a lot with being more serious about selfhosting, made me look into things like DNS and reverse proxies and VPN and docker and also ended up saving me money by re-evaluating my service needs.
For background, Gandi.net is a large and old (25 years) domain registrar and hosting provider in the EU, who after two successive rounds of being acquired by investment funds have hiked up prices across the board for all their services.
In July 2023 when they announced the changes for November I was using their services for pretty much everything because I manage domains for friends and family. That means a wide selection of domains registered with them (both TLDs and European ccTLDs), LAMP hosting, and was taking advantage of their free email hosting for multiple domains.
For the record I don't hold the price hike against them, it was just unsustainable for us. Their email prices (~5€/mailbox/mo) are in line with market prices and so are hosting prices. Their domain prices are however exaggerated (€25-30/yr is their lower price now). I also think they could've been smarter about email, they could've offered lower prices if you keep domains registered with them. [These prices include the VAT for my country btw. They will appear lower in USD.]
What I did:
Domains: looked into alternative registrars with decent prices, support for all the ccTLDs I needed, DNSSEC, enforced whois privacy, and representative services (some ccTLDs require a local contact). Went with INWX.com (Germany) and Netim.com (France). Saved about €70/yr. Could have saved more for .org/.net/.com domains with an American registrar but didn't want to spread too thin.
DNS: learned to use a dedicated DNS service, especially now that I was using multiple registrars since I didn't want to manage DNS in multiple places. Wanted something with support for DNSSEC and API. Went with deSEC.io (Germany) as main service and Bunny.net (Slovenia) as backup. deSEC is free, more on Bunny pricing below. Learned a lot about DNS in the process.
Email: having multiple low-volume mailboxes forced me to look into volume-based providers who charge for storage and emails sent/received not mailboxes. I've found Migadu (Swiss with servers in France at OVH), MXRoute (self-hosted in Texas) and PurelyMail (don't know). Fair warning, they're all 1-2 man operations. But their prices are amazing because you pay a flat fee per year and can have any number of domains and mailboxes instead of monthly fees for one mailbox at one domain. Saved €130/yr. Learned a lot about MX records and SPF/DKIM/DMARC.
Hosting: had a revelation that none of the webpages I was hosting actually needed live dynamic services (like PHP and MySQL). Those that were using a CMS like WordPress or PHP photo galleries could be self-hosted in docker containers because only one person was using each, and the static output hosted on a CDN. Enter Bunny.net, who also offer CDN and static storage services. For Europe and North America it costs 1 cent per GB with a $1 minimum/mo, so basically $12/yr since all websites are low traffic personal websites. Saved another €130/yr. Learned a lot about Docker, reverse proxies and self-hosting in general.
Keep in mind that I already had a decent PC for self-hosting, but at €330 saved per year I could've afforded buying a decent machine and some storage either way.
I think separating registrars, DNS, email and hosting was a good decision because it allows a lot of flexibility should any of them have any issues, price hikes etc.
It does complicate things if I should kick the bucket – compared to having everything in one place – which is something I'll have to consider. I've put together written details for now.
Any comments or questions are welcome. If there are others that have gone through similar migrations I'd be curious what you chose.
I'm thinking of putting all my email archive (55k messages, about 6 GB) on a private IMAP server but I'm wondering how to access it remotely when needed.
Obviously I'd need a webmail client but is there any that can deal with that amount of data and also be able to search through To, From, Subject and body efficiently?
I can also set up a standalone search engine of some sort (the messages are stored one per file in regular folders) but then how do I view the message once I locate it?
I can also expose the IMAP server itself and see if I can find a mobile app that fits the bill but I'd rather not do that. A webmail client would be much easier to reverse proxy and protect.
I've repurposed a 32 GB M.2 SATA SSD as a bootable "USB stick" and I'm putting useful tools on it. So far I've got memtest, seatools, gparted live, system rescue, clonezilla, and a live install iso of the distro installed on my PC. What other great bootable tools am I sleeping on?
I use multiple workspaces and I open text files all the time.
Once upon a time Mousepad used to behave sanely and would open them in a new tab if there was already an instance on the current workspace, or open a new window (on the current workspace) if there wasn't.
They broke that at some point. Now it's anybody's guess where the file will open. Maybe it opens in a tab in an existing window on this workspace. Maybe in a tab in a window on a random workspace. Maybe a new window on this workspace even though there's one open. I've given up trying to figure it out.
As a last resort I can use wmctrl
to figure out how to open the files and can script a sane launcher myself – provided that the editor has --tab
and --window
options AND lets you specify the window instance. Mousepad has the former but not the latter.
So, do you know any editor that can do it by itself or has those options so I can do it myself? TIA
Docker is a lot less complicated than it was made out to be.
Docker is a way of taking a service (something like Plex) and making it work in a sort of "slice" cut out of the real machine's resources (CPU, RAM and disk space). These slices are called containers.
There are several benefits:
docker-compose up -d
: run this into the same dir as a magical yaml file to create a container for the first time.docker stop cups
, docker start cups
, docker restart cups
will stop/start/restart the cups container.docker container list
shows all containers you've created.docker rm cups
removes the cups container (if it's not running).docker image list
shows the software images that the containers are using.docker rmi olbat/cupsd
will remove the olbat/cupsd image, but only if the cups container that is based on it has been removed (and stopped).docker exec cups ls /etc/cups
will execute a command inside the container. You can execute /bin/sh
or /bin/bash
to explore inside the container machine.ctop
is a nice tool that will show you all containers and let you start/stop/restart them.There are a couple of things you need to add to a fresh Debian install in order to use Docker:
docker.io
.ctop
is a nice CLI tool that shows your containers and lets you do stuff with them (stop, start, restart, enter shell, view logs etc.) It's basically a simple CLI version of Portainer (which I never bothered installing after all).The following tools are indirectly related to services inside docker containers:
vainfo
will verify that GPU-accelerated video encoding/decoding is working for AMD and Intel GPUs. This will be useful for many media streaming containers. See the Arch wiki for more.avahi
(which is avahi-daemon
on Debian) and avahi-dnsconfd
will help autodiscover some services on LAN between Linux machines. Only applicable if you have more than one Linux machine on your LAN, of course, and it's only relevant to some services (eg. CUPS).Should you use Docker or Podman? If you're a beginner just use Docker. It's a lot simpler. Yes, there are good reasons to use Podman but you don't need the extra headache when you're starting out. You will be able to transition into Podman easier later.
Use restart: "always"
in your compose yaml's and save yourself unnecessary trouble. Some people try to micromanage their containers and end up writing sysctl scripts for each of them and so on and so forth. With this restart policy your containers will stay stopped if stopped manually, but will start each time the docker daemon [re]starts, which most likely means at boot, which is probably all you want right now.
The one docker issue that will give you the most trouble is mapping users from the real machine to the container machine and back. You want the service in the container to run as a certain user, and maybe you want to give it access to some files or devices on the real machine too. But some docker images were made by people who apparently don't understand how Linux permissions work. A good docker image will let you specify what users and groups it needs to work with (emby/embyserver
is a very good example). A bad image will make up some UID and GID that's completely unrelated to anything on your machine and give you no way to change them; for such images you can try to force them to use root (UID and GID 0) but that negates some of the benefits a container was supposed to give you. So when looking for images see if they have a description, and if it mentions any UID and GID mapping. Otherwise you will probably have a bad time.
How do you find (good) docker images? On hub.docker.com, just search for what you need (eg. "cups"). I suggest sorting images by most recently updated, and have a look at how many downloads and stars it has, too. It's also very helpful if they have a description and instructions.
One last thing before we get to the good stuff. I made a dir on my RAID array, /mnt/array/docker
where I make one subdir for each service (eg. /mnt/array/docker/cups
), where I keep the magical yaml (compose.yaml) for each service, and sometimes I map config files from the container, so they persist even if the container is deleted. I also use git
in those dirs to keep track of the changes to the yaml files.
Emby is a media server that you use to index movies and series and watch them remotely on TV or your phone and tablet. Some other popular alternatives are Plex, Jellyfin and Serviio. See this comparison chart.
Here's the docker-compose.yaml
, explanations below:
version: "2.3"
services:
emby:
image: emby/embyserver
container_name: emby
#runtime: nvidia # for NVIDIA GPUs
#network_mode: host # if you need DLNA or Wake-on-Lan
environment:
- UID=1000 # The UID to run emby as
- GID=100 # The GID to run emby as
- GIDLIST=100,44,105 # extra groups for /dev/dri/* devices
volumes:
- "./data:/config" # emby data dir (note the dot at the start)
- "/mnt/nas/array/multimedia:/mnt/nas/array/multimedia"
ports:
- "8096:8096/tcp" # HTTP port
- "8920:8920/tcp" # HTTPS port
devices:
- "/dev/dri:/dev/dri" # VAAPI/NVDEC/NVENC render nodes
restart: always
/dev
devices that are owned by 3rd party users like video
and render
. Very nice.Let's look at another nicely made Docker image. Deluge is a BitTorrent client, what we put in the Docker container is actually just the server part. The server can deal with the uploads/downloads but needs an UI app to manage it. The UI apps can be installed on your phone for example (I like Transdroid) but the Deluge server also includes a web interface on port 8112.
version: "2.1"
services:
deluge:
image: lscr.io/linuxserver/deluge:latest
container_name: deluge
environment:
- PUID=1000
- PGID=1000
- DELUGE_LOGLEVEL=error
volumes:
- "./config:/config" # mind the dot at the start
- "/mnt/nas/array/deluge:/downloads"
ports:
- "8112:8112/tcp" # web UI
- "60000:60000/tcp" # BT transfers
- "60000:60000/udp" # BT transfers
- "58846:58846/tcp" # daemon remote control (for Transdroid)
restart: always
Most of this is covered above with Emby so I won't repeat everything, just the important distinctions:
Navidrome is a music indexer and streaming server (sort of like your own Spotify). It's follows the Subsonic spec so any client app that works with Subsonic will work with Navidrome (I like Substreamer). It also includes a web UI.
version: "3"
services:
navidrome:
image: deluan/navidrome:latest
container_name: navidrome
environment:
ND_SCANSCHEDULE: 1h
ND_LOGLEVEL: info
ND_BASEURL: ""
ND_PORT: 4533
ND_DATAFOLDER: /data
ND_MUSICFOLDER: /music
volumes:
- "./data:/data"
- "/mnt/nas/array/music:/music:ro"
ports:
- "4533:4533/tcp"
restart: "always"
Again, mostly self-explanatory:
This server can do some interesting things. Its bread and butter is DLNA. It has a companion Android app called, you guessed it, BubbleUPnP, which acts as a DLNA controller. The server part here can do local transcoding so the Android phone doesn't have to (subject to some caveats, for example the phone, the DLNA source and the Bubble server need to be on the same LAN; and it can only transcode one stream at a time).
It can also identify media providers (like Emby or Plex) on the LAN and media renderers (like Chromecast or Home Mini speaker) and DLNA-enables them so they appear in the Bubble app as well as on other DLNA-aware devices.
version: "3.3"
services:
deluge:
image: bubblesoftapps/bubbleupnpserver-openj9-leap
container_name: bubbleupnpserver
network_mode: "host"
user: "0:0"
devices:
- "/dev/dri:/dev/dri:rw"
volumes:
- "./data/configuration.xml:/opt/bubbleupnpserver/configuration.xml:rw"
restart: "always"
/dev/dri
which are restricted to video
and render
groups to access GPU-accelerated transcoding. So using root is the only solution here (short of looking for a nicer image).Normally I'd install samba on the host machine but Debian wanted me to install like 30 packages for it so I think that's a valid reason to use a container.
version: "2.3"
services:
samba:
image: twistify/anonymous-samba
container_name: samba
volumes:
- "./etc/samba:/etc/samba:ro"
- "/mnt/nas/array:/mnt/nas/array"
ports:
- "445:445/tcp" # SMB
- "139:139/tcp" # NetBIOS
restart: "always"
Normally this should be a simple enough setup, and it is simple as far as docker is concerned. Map some ports, map the config files, map the array so you can give out shares, done. But the image doesn't offer UID customization and just runs as root.
For reference I give the /etc/samba/smb.conf
here too because I know it's tricky. This one only offers anonymous read-only shares, which mainly worked out of the box (hence why I stuck to this image in spite of the root thing).
[global]
workgroup = WORKGROUP
log file = /dev/stdout
security = user
map to guest = Bad User
log level = 2
browseable = yes
read only = yes
create mask = 666
directory mask = 555
guest ok = yes
guest only = yes
force user = root
[iso]
path=/mnt/nas/multimedia/iso
You can add your own shares aside from [iso], as long as the paths are mapped in the yaml.
Notice the ugly use of root in the Samba config too.
CUPS is a printer server, which I need because my printer is connected via USB to the server and I want to be able to print from my desktop machine.
version: "2.3"
services:
cups:
image: aguslr/cups:latest
container_name: cups
privileged: "yes"
environment:
- CUPS_USER=admin
- CUPS_PASS=admin
volumes:
- "/dev/bus/usb:/dev/bus/usb" # keep this under volumes, not devices
- "/run/dbus:/run/dbus"
ports:
- "631:631/tcp" # CUPS
restart: "always"
The docker setup is not terribly complicated if you overlook things like /dev/bus/usb
needing to be a volume mapping not a device mapping, or the privileged mode.
CUPS is complicated because it's a complex beast, so I'll try to add some pointers below (mostly unrelated to docker):
lpstat -p
inside the host to check if CUPS know about your printer, and /usr/lib/cups/backend/usb
to check if it knows about the USB printer in particular.See you next time with more docker recipes! As usual any and all comments and suggestions are welcome, including "omg you're so dumb, that thing could be done easier like this".
<< Episode i: outline
>> Episode iii: docker recipes
In this episode: installing a secondary disk in the machine, making a system backup, installing Debian, and configuring essential service.
Adding a secondary disk to the machine
Why? Because I'd like to keep the old system around while tinkering with the new one, just in case something goes south. Also the old system is full of config files that are still useful.
To this end I grabbed a spare SSD I had lying around and popped it into the machine.
...except it was actually a bit more involved than I expected. (Skip ahead if you're not interested in this.) The machine has two M.2 slots with the old system disk occupying one of them, and 6 SATA ports on the motherboard, being used by the 6 HDDs. The spare SSD would need a 7th SATA port.
I could go get another M.2 but filling the second M.2 takes away one SATA channel, so I would be back to being one port short. 🤦
The solution was a PCI SATA expansion card which I happened to have around. Alternatively I could've disconnected one of the less immediately useful arrays and free up some SATA ports that way.
Taking a system backup
This was simplified by the fact I used a single partition for the old system, so there are no multiple partitions for things like /home, /var, /tmp, swap etc.
So:
fdisk
to wipe the partition table on the backup SSD and create one primary Linux partition across the whole disk. In fdisk that basically means hitting Enter-Enter-Enter and accepting all defaults.cp -avx
to copy the root filesystem to it.Regarding this last point: I've seen many people recommend dd
or clonezilla but that also duplicates the block ID, which you then need to change or there would be trouble. Others go for rsync
or piping through tar
, with complicated options, but cp -ax
is perfectly fine for this.
A backup is not complete without making the SSD bootable as well.
blkid
to find the UUID of the SSD partition and change it in the SSD's /etc/fstab
./swapfile
in the root which I created with dd
and formatted with mkswap
, which was copied by cp
. I mount it with /swapfile swap swap defaults 0 0
so that stays the same.mount --bind
/dev, /sys and /proc.Here's an example for that last point, taken from this nice reddit post:
mkdir /mnt/chroot
mount /dev/SDD-partition-1 /mnt/chroot
mount --bind /dev /mnt/chroot/dev
mount --bind /proc /mnt/chroot/proc
mount --bind /sys /mnt/chroot/sys
mount --bind /tmp /mnt/chroot/tmp
chroot /mnt/chroot
Verify /etc/grub.d/30-os_prober or similar is installed and executable
Update GRUB: update-grub
Install GRUB: grub-install /dev/SDD-whole-disk
Update the initrd: update-initramfs -u
I verified that I could boot the backup disk before proceeding.
Installing Debian stable
Grabbed an amd64 ISO off their website and put it on a flash stick. I used the graphical gnome-disk-utility application for that but you can also do dd bs=4M if=image.iso of=/dev/stick-device
. Usual warnings apply, make sure it's the right /dev, umount any pre-existing flash partition first etc.
Booting the flash stick should have been uneventful but the machine would not see it in BIOS or in the quick-boot BIOS menu so I had to poke around and disable some kind of secure feature (which will be of no use to you on your BIOS so good luck with that).
During the install I selected the usual things like keymap, timezone, the disk to install to (the M.2, not the SSD backup with the old system), chose to use the whole disk in one partition as before, created a user, disallowed login as root, and requested explicit installation of a SSH server. Networking works via DHCP so I didn't need to configure anything.
After install
SSH'd into the machine using user + password, copied the public SSH key from my desktop machine into the Debian ~/.ssh/authorized_keys
, and then disabled password logins in /etc/ssh/sshd_config
and service ssh restart
. Made sure I could login with public key.
Installed ntpdate
and added 0 * * * * /usr/sbin/ntpdate router.lan &>/dev/null
to the root crontab.
Mounting RAID arrays
The RAID arrays are MD (the Linux software driver) so they should have been already detected by the Linux kernel. A quick look at /proc/mdstat
and /etc/mdadm/mdadm.conf
confirms that the arrays have indeed been detected and configured and are running fine.
All that's left is to mount the arrays. After creating /mnt
directories I added them to /etc/fstab
with entries such as this:
UUID=array-uuid-here /mnt/nas/array ext4 rw,nosuid,nodev 0 0
...and then systemctl daemon-reload
to pick up the new fstab right away, followed by a mount -a
as root to mount the arrays.
Publishing the arrays over NFS
Last step in restoring basic functionality to my LAN is to publish the mounted arrays over NFS.
I installed and used aptitude
to poke around Debian packages for a suitable NFS server, then installed nfs-kernel-server
.
Next I added the arrays to /etc/exports
with entries such as this:
/mnt/nas/array desktop.lan(rw,async,secure,no_subtree_check,mp,no_root_squash)
And after a service nfs-kernel-server restart
the desktop machine was able to mount the NFS shares without any issues.
For completion's sake, the desktop machine is Manjaro, uses nfs-utils
and mounts NFS shares in /etc/fstab
like this:
nas:/mnt/nas/array /mnt/nas/array nfs vers=4,rw,hard,intr,noatime,timeo=10,retrans=2,retry=0,rsize=1048576,wsize=1048576 0 0
The first one (with nas: prefix) is the remote dir, the second is the local dir. I usually mirror the locations on the server and clients but you can of course use different ones.
All done
That's essential functionality restored, with SSH and NFS + RAID working.
In the next episode I will attempt to install something non-essential, like Emby or Deluge, in a docker container. I intend to keep the system installed on the metal very basic, basically just ssh, nfs and docker over the barebones Debian install. Everything else should go into docker containers.
My goals when working with docker containers:
Basically the idea is that if I ever lose the system disk I should be able to simply reinstall a barebones Debian and redo the containers using the stored configs, and all the important mutable files would be on RAID.
See you next time, and meanwhile I appreciate any comments about what I could've done better as well as suggestions for next steps!.
I need a very simple method for non-advanced users to share each other's screen explicitly when they need help. They're running XFCE on Manjaro and the machines involved are using Tailscale. Edit: SSH access is also available, with key authentication.
I need something super simple because they are remote from me and from each other and any graphical setup will have to be assisted sight-unseen over phone. So ideally just (1) install something (which I can do for them over SSH), (2) pick something from the Applications menu and maybe (3) press a big "START" button.
It's also ok-ish if the remote capability is present all the time and I can connect without their explicit permission, but you can see why it would be best if they did something to enable it...
I've been looking for a solution but all I find is stuff that's way too complicated OR starts a new desktop session instead of showing the current one.
Edited: to clarify I'm not the one who will be remoting-in and to mention SSH is available.
TIA
@lemmyvore
@feddit.nl