Upgrading Field Programmable Hardware Devices for Cisco Catalyst 9800 Series Wireless Controllers

This method is not the official way of updating the ROMMON software, as presented by Cisco. You can find more details at this link:

Upgrading Field Programmable Hardware Devices for Cisco Catalyst 9800 Series Wireless Controllers

If someone is thinking about updating the ROMMON software on Cisco Catalyst 9800 Series Wireless Controllers, I have good news. It’s not as complicated as it might seem at first glance. Moreover, there’s no need to anticipate downtime in an HA (High Availability) solution. Personally, this was a significant issue for me, as scheduling downtime and potential issues afterward were, to put it mildly, problematic. I realized that we can also carry out such an update differently, just as I wanted and transparently for users.

Let’s start with what ROMMON (ROM Monitor) actually is, and then quickly move on to how to easily and quickly perform such an update.

A ROMMON image is a software package used by ROM Monitor (ROMMON) software on a controller. The software package is separate from the consolidated package normally used to boot the controller.

Scenario: Pair of 9800-80 Cisco Catalyst Wireless Controllers located in two seperate Data Centers

IOS XE Software installed: Cisco IOS XE Software, Version: 17.09.03

ROMMON version: System Bootstrap, Version: 16.12(5r)

Upgrade path: 16.12(5R) > 17.3(3R)

Step-by-step instruction that I performed for the above setup:

Copy the file C9800-80-rommon.173-3r.pkg from the server, e.g. sftp, to the controller:

WLC#copy sftp: flash:
Source username [admin]? 
Source filename []? C9800-80-rommon.173-3r.pkg
Destination filename [C9800-80-rommon.173-3r.pkg]?
Password:
!!!!!
15270932 bytes copied in 15.864 secs (962615 bytes/sec)

2. Check the signature after copying.

WLC#verify /sha512 bootflash:C9800-80-rommon.173-3r.pkg
..........................................................................................................................................................................................................................................Done!
verify /sha512 (bootflash:C9800-80-rommon.173-3r.pkg) = 7c632068218441ad75bc49ca165a52bfa48cdd7b34f859549dd81ecf6a5941fdcd671ff35819fbf587120841d4a66fc5687cfc2d7859092071fc7b242ed71bff

3. Check the current version of ROMMON software.

WLC#show rom-monitor chassis active r0
==========================================================

System Bootstrap, Version 16.12(5r), RELEASE SOFTWARE
Copyright (c) 1994-2019  by cisco Systems, Inc.

WLC#show rom-monitor chassis standby r0
==========================================================

System Bootstrap, Version 16.12(5r), RELEASE SOFTWARE
Copyright (c) 1994-2019  by cisco Systems, Inc.

4. Initiate upgrade on the active chassis.

WLC#upgrade rom-monitor filename bootflash:C9800-80-rommon.173-3r.pkg chassis active r0
Verifying the code signature of the ROMMON package...
Chassis model C9800-80-K9 has a single rom-monitor.

Upgrade rom-monitor

Target copying rom-monitor image file

Secure update of the ROMMON image will occur after a reload.

131072+0 records in
131072+0 records out
131072 bytes (131 kB, 128 KiB) copied, 2.4692 s, 53.1 kB/s
Copying ROMMON environment
131072+0 records in
131072+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.880704 s, 149 kB/s
131072+0 records in
131072+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.881267 s, 149 kB/s
ROMMON upgrade complete.
To make the new ROMMON permanent, you must restart the RP.

5. Force switchover between boxes

Thanks to the command below, the chassis will restart and replace the ROMMON version, in line with the above message, at point 4.

WLC#redundancy force-switchover

Thanks to this simple action, all services are still available to users provided by the Wi-Fi infrastructure. Furthermore, the previously active chassis updates the ROMMON software version to version 17.3(3R). The entire process, along with the chassis reboot and HA setup, took me:

18 minutes, 15 seconds, and 665 milliseconds.

6. Check the HA status.

WLC#show chassis 
Chassis/Stack Mac Address : <output hidden> - Local Mac Address
Mac persistency wait time: Indefinite
Local Redundancy Port Type: FIBRE
                                             H/W   Current
Chassis#   Role    Mac Address     Priority Version  State                 IP
-------------------------------------------------------------------------------------
 1       Standby   <output hidden>     2      V02     Ready                <output hidden> 
*2       Active   <output hidden>     1      V02     Ready                <output hidden>

7. Check the current version of ROMMON software.

WLC#show rom-monitor chassis active r0
==========================================================

System Bootstrap, Version 16.12(5r), RELEASE SOFTWARE
Copyright (c) 1994-2019  by cisco Systems, Inc.

WLC#show rom-monitor chassis standby r0
==========================================================

System Bootstrap, Version 17.3(3r), RELEASE SOFTWARE
Copyright (c) 1994-2020  by cisco Systems, Inc.

8. In the end, repeat the entire procedure again for the currently active unit.

This post was created thanks to the post published by Rasika Nayanajith on his blog and my own experience with updating the ROMMON software at a client’s site.