Changing VMware ESX Service Console IP’s without downtime

Very recently I came across a situation where I needed to change the IP range that a group of ESX hosts were using for Service Console connections, without causing a disruption in service to the VM’s.

This was something I thought would be possible, but I hit a few issues, so thought I’d share the process that worked without failure, in case anyone else out there has the same issues.

Let’s start with some background…

The customer had a single IP range for their entire network, and now wanted to change the addressing so that management services are segregated from “normal” network traffic. So, we had to migrate the Service Console, vCenter IP and the VM Kernel port group IP’s to a whole new range. The first port of call was to ensure that the new range could still communicate with the current IP range, so there wouldn’t be a problem getting traffic across the boundary. For now, this was open to all traffic to ensure that there were no port blocking rules interfering with the transition, and this would be locked down once all migrations were complete.

The site uses both HA and DRS and so these had to be worked around when it came to moving the IP range. On top of all this, they had turned off admission control as they were running low on resources and the new host hadn’t arrived yet, so they didn’t have a spare host that they could temporarily put into maintenance mode…

So, we made a plan, and we tried to stick to it…

The plan was:

  1. Disable HA as a precautionary step. We actually removed HA from the cluster altogether.
  2. Set DRS to partially automated.
  3. Clear the host files in each ESX host (HA fills these in but doesn’t empty it when removing HA). Also clear the host files in vCenter if needed.
  4. Change the IP of the vCenter host, and alter the DNS records for it. Ensure that vCenter can ping each ESX host.
  5. Check that each ESX host can resolve the new vCenter IP.
  6. Add an additional Service Console port group onto each ESX host with its new IP. Each time, add the new IP to the vCenter server’s host file after an error is generated.
  7. Right click the host and choose “Connect”. It should now reconnect using its new IP.
  8. Add the new VM Kernel port groups, with their new IP’s and enable them for vMotion.
  9. Alter the DNS entries for each host in your DNS system, and flush the DNS cache.
  10. Change the Gateway in each Service Console to the new router IP. Change the “Gateway Device” setting to the new vSwif (most likely vSwif1).
  11. Remove the old Service Console port groups.
  12. Re-enable DRS.
  13. Re-enable HA.

The reason for modifying the host file in vCenter if you’re wondering was because I found that whilst you add a service console and the vCenter server is on its new IP address, even if DNS resolves the ESX hostname to the old IP for the host, the new service console would respond, causing errors with routing and the firewalling that was in place on this site. Changing the host file was far easier than editing DNS and flushing all the relevant DNS caches each time, as it provided an immediate change. Once DNS was then altered, caches cleared, the host file could be emptied, and the system would continue to function.

Share

Recent Posts

Microsoft

McCann & MullenLowe – Microsoft Teams Case Study

When bringing companies with different technology systems together, it can be difficult to efficiently collaborate. #Microsoft Teams can help. Learn how its single platform system provided McCann and MullenLowe with the solution they needed to enable their employees to work together. Check out this video:

Read More »