Tuesday, November 12, 2013

Using CARP with VMWare ESXi

If you want to use CARP on your VMWare guest VMs, you will probably find that it doesn't work out of the box. This is due to ESXi rejecting promiscuous mode on the virtual switch by default. To enable promiscuous mode, go to the Network configuration section for the host (in vSphere Client), and click properties for the vSwitch. Edit the properties for the vSwitch, and change the setting of "Promiscuous Mode" to Accept under the "Security" tab.

For bonus points, if you are using NIC Teaming on ESXi (even with just a standby adapter), you will find that your CARP interfaces always remain in BACKUP state, and your logs fill with the following messages.

Nov 12 11:25:51  kernel: carp0: MASTER -> BACKUP (more frequent advertisement received)
Nov 12 11:25:51  kernel: carp0: link state changed to DOWN
Nov 12 11:25:54  kernel: carp0: link state changed to UP

This is because ESXi is rebroadcasting CARP advertisements that come back down the other members of the team. To correct this, you need to dig into the Advanced Settings, under Software. Change Net.ReversePathFwdCheckPromisc to 1. Annoyingly, you will need to reboot the host to affect these changes, but it works.