They added the Maria 10.11 module stream to dnf for AlmaLinux 9.4, enabling an officially supported upgrade route.
Just run sudo dnf module install mariadb:10.11/server then sudo dnf update and it will upgrade you to 10.11.6. I ran the upgrade day before yesterday and it was quick and seamless. No need to remove first.
Don’t forget to then run mariadb-upgrade.