macadmin.fraserhess.com

Cisco Umbrella, NXDOMAIN, and noNXDOMAIN

,

Cisco Umbrella provides DNS security for organizations. It is built on Cisco’s purchase of OpenDNS in 2015. Cloud DNS resolvers provide filtering tailored to your organization. Roaming clients are protected by a module in Cisco Secure Client. The Umbrella client captures DNS queries and forwards them to the cloud resolvers.

This week, I discovered a surprising behavior in the Umbrella client:

If the Umbrella client receives an NXDOMAIN (non-existent domain) from the cloud resolvers in response to a DNS query, the query is retried against local DNS resolvers.

This behavior broke my mental model of cloud-based DNS security1.

Before I describe how to disable this behavior, I want to explore why this might be the default.

Umbrella’s DNS protections work by intercepting DNS queries in the client network stack and forwarding them to cloud resolvers. But this isn’t always what we might want to happen. Many organizations still have local DNS resolvers serving internal domains. Umbrella has several methods to keep internal domain queries local:

  • Configure internal domains in the Umbrella dashboard
  • Search domains provided by DHCP are considered local2
  • Using Umbrella DNS appliances on internal networks. The Umbrella client defers all traffic to these.

If none of these bypasses have been set up, then local re-querying of NXDOMAIN responses makes sense. It serves as a backstop to not break local DNS domains.

noNXDOMAIN

To disable the re-query behavior, there is a setting one can make in the OrgInfo.json file3:

"noNXDOMAIN" : "1"

This is not well documented. If you Google for Cisco Umbrella "noNXDOMAIN" you’ll get three results (or maybe four by the time you read this) and none are about installing Cisco Secure Client on macOS.

The “1” value must be a string. If it’s an integer then the acumbrellagent process will crash.

According to Umbrella support, the setting can only be changed in a couple of scenarios:

  • Initial installation
  • Full uninstall/re-install of the Umbrella module
  • Disabling the Umbrella client, removing all files in /opt/cisco/secureclient/umbrella, placing a new OrgInfo.json file in the same folder, and then enabling the client again

All three of these methods create a new computer object in the Umbrella dashboard.

What led to this discovery?

I have been helping to test Netskope Private Access. This product also intercepts DNS queries. For the first time, we have two agents inspecting DNS queries. With both enabled, Cisco Umbrella would get into a re-query loop when an NXDOMAIN response was returned. The query counts in both the Umbrella client and dashboard would skyrocket. Searching for any clue as to how Umbrella handled NXDOMAIN, we found this setting. Ultimately, we resolved this interoperability issue by enabling noNXDOMAIN.

  1. I consider myself quite familiar with Cisco Umbrella. I even have some notoriety in this area. ↩︎
  2. The current set of local domains is found in /opt/cisco/secureclient/umbrella/data/mergedlocaldomains ↩︎
  3. The configuration file that bootstraps an Umbrella client ↩︎

Leave a comment