Configuring 2 routers in the same LAN

[Sneakers]

I have been trying to build an internet redundancy with two routers.

The coniguration is as follows:

I got 1 router (A) connected to ISP-X.
I got 1 router (B) connected to ISP-Y.

The ISPs are off course different, and both are connected just fine.

I have configured my DNS servers like this (by priority):

1) Main DC
2) Router (A)
3) Router (B)

Now, everything works just fine (all clients have their network configured automatically, and all connect to the internet), however, when Router (A) disconnects from ISP-X, the clients can't resolve the DNS since they won't switch to router (B).
I have both routers configured in the DHCP scope options (prioritized as shown above), I also have them configured in the DHCP scope - DNS options.

Why won't my clients try to use Router (B) when Router (A) DNS routing is down?