Blog

Upgrade Tips

Here are some suggestions to help you upgrade or patch your ServiceNow instance.  Whether you are new to upgrades and patching, or want to know more about it, this article will help you get updated to the newest ServiceNow version.

ServiceNow's upgrade process is awesome.

I have prior experience upgrading other applications and it was a difficult and stressful experience.  Some of my worst experiences in programming were during system upgrades of systems other than ServiceNow!  I will always remember those weekends of hell,  hours on the support line, and the general anxiousness and trepidation of having to upgrade.  It was not fun, no way.

Not to worry however, ServiceNow is much, much easier in comparison.  If you are on ServiceNow, you can upgrade without fear!  Especially if you are prepared and ready for the upgrade.

1. Prepare for Upgrade

Learn about upgrades and upgrade preparation

Build an Upgrade Team

When I used to update HP Service Manager at clients, most often I would upgrade the system alone.  Some of the most stressful situations in my career were failed upgrades and backouts.  It was often because I was by myself and had little support to resolve the issue.

If you have the opportunity to bring a team together to assist on the upgrade this can greatly help.  Try to build an upgrade team that helps each other and each person provides value. People who only criticize issues, panic, and provide no assistance should not attend the upgrade.

Establish Production Verification Test

If you have testers, make sure their verification testing is documented beforehand and done in the test instance.  If not, they will be tempted to test things they have never tested before in production.  If they are doing that it is too late, they can only test in "Test".  Make sure they know this beforehand.

2. Specific Upgrade Prep

For the ServiceNow Geneva version, there are a couple suggestions I have to improve your upgrade.

Fix Existing Data Issues

Do a review of your current system and see if you have any issues today you can fix before the upgrade.  Here are some articles you can use to check.

3. Read about the latest Patches and Upgrades

You can read release notes for all the latest patches and upgrades here.  ServiceNow Wiki - Versions

It is also a good idea to browse the ServiceNow Community to see user's opinions on the upgrade or patch you are interested in.

Note that for patches, ServiceNow is moving towards quarterly mandatory patch levels.  Being aware of upcoming patches will be more important in the near future.

4. Clone Back

You should always clone Production over Dev, Test, QA, Sandbox instances before you patch or upgrade.  That way you have accurate and most current data and code to test against during Testing.

You can Request a Clone in the Left Navigator of ServiceNow under System Clone > Request Clone

More information about System Clone: ServiceNow Wiki - System Clone

Remember in-flight update sets will be cloned over.  Back those up to xml before cloning.

5. SCHEDULE DEVELOPMENT Upgrade

After all ServiceNow instances are cloned over (except Production of course), you are ready to organize an upgrade to a Development/Test Instance for testing.

How to request a upgrade/patch

  1. Login into ServiceNow HI support
  2. Click the "Manage Upgrades" box
  3. Select the instance to upgrade and click schedule.  Pick a Dev/Test instance so you can test the upgrade/patch before going to production.

If you don't see the version, you might need to click the "Need a version not listed?" link.  When you click this, you can "Request a version entitlement" and ask for a new version.  ServiceNow will approve/reject the entitlement and if approved, you can go back to the Manage Upgrades box and request the version you want.  Sometimes versions are not available yet as ServiceNow only wants to deploy to a few customers at once, or other reasons they might have.

 I suggest upgrading in this order:

  1. Upgrade Sandbox Instance(if available). Test Upgrade in Sandbox Instance
  2. Upgrade Test  Instance (if available).  Test upgrade in Test Instance.
  3. Upgrade QA  Instance (if available).  Test upgrade in QA Instance.
  4. Upgrade Dev  Instance (if available).  Test upgrade in Dev Instance.
  5. Upgrade Production

Another option is to wait to upgrade one of the instances for two weeks after going to Production.  That way you have an older version of ServiceNow in case you want to test a defect that you think the upgrade may have caused.

6. CONFIRM DEVELOPMENT Upgrade

After you schedule your upgrade.  You can check the upgrade status using upgrade status and see when it completed.

Review Upgrade Monitor

Once the upgrade is completed,  review the upgrade monitor and revert certain customization to OOB as needed.  Sometimes you only made a slight change to some code, and you no longer need it.  Other times the ServiceNow code is more powerful or newer, and you may decide to use that code instead.

After an upgrade, you want to go to the Upgrade Monitor (Left Nav Bar > Upgrade Monitor).  Click the "Review Skipped Updates" button.

Review all the skipped updates and selectively choose which ones to revert to base.  There may be no updates to revert.  Less is more in this respect, as reverting to base code can have adverse effects on the instance.

Testing

After the instance is upgraded and you went through the upgrade history, you can begin testing.  

  1. Create Test Plans for each ServiceNow application you use
  2. Have selected users and testers test the upgraded/patched feature using the test plans you created.  
  3. Test new or updated features as part of the upgrade
  4. Test Inbound and Outbound Email
  5. Test User Authentication (Logging in/out)
  6. Test Full ServiceNow and CMS Sites for any potential UI issues
  7. Important. Notify users that a upgrade is coming.  You can do this via email, corporate social network sites, or with Overview help or a header message.  It is important that you do this.  They may ignore your message, but you made an attempt and make sure to document that attempt to notify them.

Fix Issues

If an issue found is the result of a customization you created, you should attempt to fix it, recording your fix in an Upgrade Set.  If there is an issue that you think ServiceNow might have a fix for or didn't notice, create an incident at ServiceNow HI support.  They have been really helpful with upgrade issues I have had, often providing quick workarounds or are aware of the issue and will fix the issue on the next release.

7. Schedule and Upgrade Production

Once you have completed your testing and fixes, you are ready to go to Production.

Never Rollback

I suggest adopting a policy of never backing out or rolling back an upgrade or reverting to a backup.  If though ServiceNow can do this (maybe), it should never be done.  If you have to rollback, it is huge mistake and huge decision. Don't rollback unless it is only absolutely necessary.  Don't rollback for a minor reason.

It is important to have worst case scenarios and solutions available. However once an upgrade starts, it is difficult if not impossible to stop.  I am not even sure if rolling back is even possible anymore.  I have heard of people rolling back earlier upgrades like Aspen, but that did not go well for people who made that decision.

Going into an upgrade with a never rollback philosophy will insure you prepared enough before the upgrade and are willing to fix issues that occur as well.

Schedule during normal awake hours

Schedule the upgrade during hours you are normally awake.  Scheduling things at 2am on Saturday is not a great idea.  You will be tired, if you need assistance the best support will be offline, and so will other employees who could assist.  I have seen some great mistakes made at 2am, and sometimes even at work!

When they say you must upgrade at 2am, "discuss" that 6pm is a much better idea.  Say it is "ServiceNow's Best Practices".  You will be happy you did.  

Follow the Previous Upgrade Process

When you upgraded dev, test, qa, follow that process and repeat with Production.  Do not deviate from what you did previously on those upgrades.

Usually the process is upgrade and then apply the update set.  Sometimes there are manual steps as well.

Issues During the Upgrade

Even with a lot of preparation and planning, an upgrade can go wrong.  The important thing is not to panic.  Do not start making changes in production as quick fixes.  Carefully think of solutions to the issue.

Node upgrade error - ServiceNow Geneva

Recently during a ServiceNow Geneva upgrade I attended, one of the production nodes failed to upgrade to Geneva.  The upgrade monitor stopped working properly (See Picture) and ServiceNow was not working correctly. This was because 2 of the nodes were Geneva and 1 of the nodes was Fuji yet due to an error in the upgrade process.

We can't fix node issues, so ServiceNow HI support was contacted.  They were very helpful and fixed the node issue, initiated upgrade on the database to Geneva version, and after completion, fully restored service.

It was stressful and took a while to fix, but it was important we did our preparation and upgrade testing beforehand.  That insured it wasn't our fault because we knew it was on ServiceNow side.

8. Wait, Wait, Celebrate

People like to celebrate immediately after the upgrade completes.  I like to wait a week later, get though any issues found, and celebrate!  Have a party, then get back into it, continuing to do amazing things with ServiceNow.