Moving nameservers

I am going to have to move 2 nameserver soon and just want to run a plan by to see if anyone sees pitfalls I don't.

The setup:
ns1: running DNS, IIS. mailserver, and SQL. This is the production server with all live sites and databases.
ns2: running DNS. This is only a development server and all DNS zones are secondary.

Plan I am considering right now is to first change the ip of the 2nd nameserver and around day 2 after that move it to the new location and promote all the DNS zones to primary (but leave the IP the same as it is now).
Then, once this is done and all dns queries show ns2 at the new ip, submit the change of ip for ns1. This will be done around a weekend like on a Thursday. On a Saturday move the production server to the new location and turn off DNS services for this box. Also would then modify the DNS in ns2 to have all host records point to the new ip. What I am hoping here is that when ns1 is not found by a web request that it will kick over to the secondary name server which will then throw out the correct location of the production box running IIS.
Then once the new IP for ns1 has propergated through the net turn DNS back on and also change all the DNS zones on ns2 back to secondary.

I can see that for a day, maybe two, I might get some 404 errors on clients from ISPs not flushing their cache as often as they should, but once past that it should be all good. Also considered running a 3rd server that is copy of ns1 and just have 2 servers running at different locations for a time that are the same, but the replication scheme needed for SQL to keep the two databases the same would be more trouble then it is worth I think.