Finally, I was able to get this resolved.
Our virtualmin server is on a subnet of a full class "C" network. Our ISP has allotted to us 10 addresses and registered these addresses with ARIN to officially allocate them to our organization.
A big clue to decipher how to do this:
-In particular, steps 6 and 7 below: (1-5 are for the ISP)
6 On the DNS server for the sub-network, create a new master zone for the reverse network 192.168.1.100-110. Webmin will automatically convert this to the correct in-addr.arpa format for you.
7 Add Reverse Address records to the new zone as normal for IP addresses like 192.168.1.100-110.101. Adding a record for the IP 192.168.1.101 will not work.
Another big clue was using DNSstuff’s tools to do an “advanced” reverse IP trace - which walked through each of the DNS queries for our the allotted IP address we checked.
Parsing through the queries and responses explained the issue plainly.
[code:1]Asking e.root-servers.net for 188.8.131.52.in-addr.arpa PTR record:
e.root-servers.net says to go to henna.arin.net. (zone: 207.in-addr.arpa.)
Asking henna.arin.net. for 184.108.40.206.in-addr.arpa PTR record:
henna.arin.net [220.127.116.11] says to go to ns1.lightpointcommunications.net. (zone: 214.162.207.in-addr.arpa.)
Asking ns1.lightpointcommunications.net. for 18.104.22.168.in-addr.arpa PTR record: Got CNAME referral (with no NS) to d.root-servers.net (zone 232.224/22.214.171.124.in-addr.arpa.) [from 126.96.36.199]
Asking d.root-servers.net for 232.224/188.8.131.52.in-addr.arpa. PTR record:
d.root-servers.net [184.108.40.206] says to go to figwort.ARIN.NET. (zone: 207.in-addr.arpa.)
Asking figwort.ARIN.NET. for 232.224/220.127.116.11.in-addr.arpa. PTR record:
figwort.arin.net [18.104.22.168] says to go to ns.lightpointcommunications.com. (zone: 214.162.207.in-addr.arpa.)
Asking ns.lightpointcommunications.com. for 232.224/22.214.171.124.in-addr.arpa. PTR record:
ns.lightpointcommunications.com [126.96.36.199] says to go to ns1.lightonthenet.org. (zone: 224/188.8.131.52.in-addr.arpa.)
Asking ns1.lightonthenet.org. for 232.224/184.108.40.206.in-addr.arpa. PTR record: Reports that no PTR records exist [from 220.127.116.11].[/code:1]
So that means our ISP is forwarding the 224/18.104.22.168 zone request to our server and then the specific IP is queried (looking for a PTR for the IP in our allocated block that ends in .232).
In order to fix this - I had to know what zone query our ISP was forwarding - in this case : “224/22.214.171.124.in-addr.arpa.” AND I had to have a PTR in the correct format to delineate specific IP’s on our allocated block - again, the hint was in good ol’ DNSstuff tools: “232.224/126.96.36.199.in-addr.arpa.”
In the webmin interface for BIND this is easy to do whence you know the way to accomplish it.
create a master zone, select it to be “reverse” and then enter the zone that your ISP is querying - you don’t have to enter it “backwards” just the 188.8.131.52/28 or similar will work - webmin will convert it for you.
Add reverse records to this master zone - again you don’t need to enter them in reverse - just 184.108.40.206/28.232 will do the trick.
For posterities sake, I also added another reverse master zone for the whole class “C” network - 207.214.162 And added all the reverse records in this address as well. This was significant for host xxx.xxx.xxx.xxx queries to resolve correctly. Seeing as everything is working as expected at this point, I don’t plan on seeing the impact of removing this zone will have at this point.
It’s great to finally put this to rest and I hope this thread will help others find answers to similar problems…