The error is due to an incorrect or missing / malformed DNS record.
Are you using Virtualmin’s DNS for the domain or are you using the domain registrar’s dns / some third party DNS? If the latter, than you must insure that the DKIM related DNS records that you have manually created match the ones exactly that Virtualmin’s DNS has created for the domain at Virtualmin → Server Configuration → DNS Records.
You must be careful when copying over the DKIM DNS record. Some third-party DNS services (for example AWS’ Route 53) do not accept the long content that txt records such as DKIM have and expect these long content records to be broken up in chunks of 64 bytes or less. If you don’t know how to do that, see Need help about DKIM concept & setup - #2 by calport