NetScaler (yeah, yeah, we’re supposed to call it ADC now) has a very cool IP Reputation feature that allows admins to easily block traffic from known-bad IP addresses. Unfortunately, because it’s so straightforward to use there’s not a ton of information available on how to troubleshoot it when it doesn’t work. I’ve been fighting an issue with it, and a recent NetScaler update helped me finally solve it.

Other folks like Citrix and Jacob have great information on how to configure the feature and get it blocking the bad stuff. It (normally) takes about 10 minutes to set up and after that Just Works™.

Unfortunately for me, that hasn’t been the case. My lab appliance has been failing to update it’s database and I’ve been looking for a while to figure out why.

Before diving in, let’s start with a couple basics on the IP Reputation service. Once enabled (see the links above), it checks every 5 minutes for database updates. If it finds any, it downloads them and applies them to the local copy of the database. It logs all activity to /var/log/iprep.log. To work properly, the NetScaler must be able to resolve and access the following three addresses on port TCP/443:

• localdb-ip-daily.brightcloud.com
• localdb-ip-rtu.brightcloud.com
• api.bcti.brightcloud.com

If you search for issues with iprep, almost every article, blog post or documentation link will point you towards issues communicating with these addresses. Usually it’s either a DNS configuration problem on the NetScaler or a firewall blocking access to these addresses from the NSIP. In my case, both of those were working fine.

The iprep update process was failing with the following in the iprep.log. They key error is “URL using bad/illegal format or missing URL”. This is documented widely as a standard error produced by curl, but isn’t discussed anywhere in relation to NetScaler.

Jan 15 09:10:00 <local2.info> Net-NS1 iprep: iprep process started...
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: iprep_get_schema_version:134 current schema version:1.1
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: IPREP update versions: major version:0 minor version:0 update version:0 total ips:0 last update time:0
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: PE update versions: major version:0 minor version:0 update version:0 total ips:0 last update time:0
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: outfile:/var/nslog/iprep/webroot_http_resp_1579097400.xml
Jan 15 09:10:00 <local2.err> Net-NS1 iprep: iprep_curl_download:90 curl_easy_perform failed. Error code:3 Err msg:URL using bad/illegal format or missing URL
Jan 15 09:10:00 <local2.err> Net-NS1 iprep: Not able to connect/resolve WebRoot.
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: Not able to connect to WebRoot.
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: IPREP update versions: major version:0 minor version:0 update version:0 total ips:0 last update time:0
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: PE update versions: major version:0 minor version:0 update version:0 total ips:0 last update time:0
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: iprep and PE are at same version.
Jan 15 09:10:00 <local2.info> Net-NS1 iprep: iprep process exiting with error code:0.

I’ve been trying to find the fix for a long time, and have stumped several NetScaler experts with it. However, I looked again after updating to 13.0 build 47.24 (if you haven’t done this upgrade yet, stop reading and go do it right now). After the upgrade, the output of the error had changed:

Feb  6 10:10:00 <local2.info> Net-NS1 iprep: iprep process started...
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: iprep_get_schema_version:134 current schema version:1.1
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: IPREP update versions: major version:1 minor version:2450 update version:127 total ips:940191 last update time:1579109702
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: PE update versions: major version:1 minor version:2450 update version:127 total ips:940191 last update time:1579109702
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: outfile:/var/nslog/iprep/webroot_http_resp_1581001800.xml
Feb  6 10:10:00 <local2.err> Net-NS1 iprep: iprep_curl_download:90 curl_easy_perform failed. Error code:5 Err msg:Couldn't resolve proxy name
Feb  6 10:10:00 <local2.err> Net-NS1 iprep: Not able to connect/resolve WebRoot.
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: Not able to connect to WebRoot.
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: IPREP update versions: major version:1 minor version:2450 update version:127 total ips:940191 last update time:1579109702
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: PE update versions: major version:1 minor version:2450 update version:127 total ips:940191 last update time:1579109702
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: iprep and PE are at same version.
Feb  6 10:10:00 <local2.info> Net-NS1 iprep: iprep process exiting with error code:0.

Now it’s complaining that the proxy server configuration is wrong. The proxy configuration is on the Change Reputation Settings screen: Security > Reputation > Click “Change Reputation Settings”. It looked like this:

Since there’s no proxy in use, the port should not be set at all. On this screen, delete the contents of the Proxy Port field and click OK. If you open the configuration again, you’ll see this:

Once that change was made, the iprep.log showed the database doing a full update the next time it ran.

So, I’m putting this up, in the hopes that the keywords from the error make it into Google and some poor admin find this page instead of cursing at IP Reputation for months.