Sometimes, people want to know, how to extract data from APPFW logs. That’s easy, it is in /var/log/ns.log (and it’s predecessors, these ns.log.XX.gz).

grep APPFW ns.log

will extract all application firewall logs.

zcat ns.log.*.gz |grep APPFW

will do the same to the old logs. Unfortunately this will give you a terrible mess of output. It’s hardly possible to find false positives in there.

From now on, I will deal with old logs only. If you want to examine the current one, you’ll have to replace zcat ns.log.*.gz with cat ns.log.

---

The solution

So, what can we do? I would not be interested in details, the only thing I’d like to know: which URLs trigger rules? Next, of course, would be: which rules get triggered? A Citrix ADC (NetScaler) shell runs on BSD, and BSD is UNIX. Like always, in UNIX, it’s a one liner in bash:

zcat ns.log.*.gz |grep APPFW | awk -F '//' '{print $2}'|cut -d " " -f1 |sort | uniq -c |sort -n |less

---

The solution explained (for these non-UNIX guys)

procede here if you are able to understand my script and don’t need my explaination.

This will:

zcat ns.log.*.gz
expand all ns.log.*.gz files and output it’s content;

grep APPFW
filter out logs containing APPFW. The string APPFW is used by Citrix ADC (NetScaler) to mark web application firewall (WAF) logs;

awk -F '//' '{print $2}'
we are not interested in details now, so I will search for // (from https://) as a start point and will print everything behind this;

cut -d " " -f1
next, I will remove everything behind the first blank (that’s behind the actual URL);

sort
I’ll have to sort the output in alphabetical order;

uniq -c
make the output unique, and count it at the same time;

sort -n
sort this by numbers;

less
and give a page wise output. You may terminate it using q. Skip this command, if you don’t like to gat output page wise;

This is the output of my script. The number in front is the number of occurence of a certain string. you see the URLs behind this numbers. You can easily see, someone tried to get backups from my blog, but it got blocked. There is also still access to a blog I deleted some years ago (training.blog.norz.at) and some more.

---

This will give you a quick overview. It will mix all kind of scans, so start URLs, deny URLs, SQL injections, everything into one output. Maybe you don’t agree to this? You want to see just startURLs? So replace APPFW with APPFW_STARTURL. Easy like that.

---

What to do, if you want to run this script on a regular base? Go to /var/log

vi examine_logs.bash

(I know, everyone hates vi, but hey, follow my instructions, it works like a charm. Different to nano, vi is everywhere, even on a Citrix ADC)

Type [i] (to change into insert mode)

paste this script into vi

press [ESC] and than [X] and [ENTER]

chmod +755 examine_logs.bash

from now on, you max execute this script at any time using ./examine_logs.bash

---

Require more data?

There might not be logs enough. Log rotation in Citrix ADC is like that:

There is a cron job running every hour. This cron job checks the size of the current ns.log file. If this file is bigger than 100 kbt it gets compressed to ns.log.0.gz. There is a maximum of 25 ns.log.xx.gz files. Following CTX121898, we could increase both, the number of log files and the size of logs. This is governed by the /etc/newsyslog.conf file. It would be easy, to overwrite this file. Unfortunately it gets reverted after reboot. If you want to keep this updated file for ever, you have to copy it to the /flash/nsconfig directory.

so open this file, scroll down to /var/log/ns.log. Change values for size and count as needed.
Don’t forget to copy this file to /flash/nsconfig!

---

Possible problems:

I have tested my script using CEF and Geolocation logging set appfw settings -GeoLocationLogging ON -CEFLogging ON; that’s not the default structure of log files. I hope, it would also work for standard logging, but I didn’t test. Tell me if you face problems.

I tested this script from: Citrix ADC bash, Kali Linux and Windows 10 bash. It worked with all of them.