User Agent 007

Published: 2011-08-26
Last Updated: 2011-08-26 00:11:28 UTC
by Daniel Wesemann (Version: 1)
7 comment(s)

Recently, while conducting an audit at a financial services company, I decided to verify their claim that their "desktop build is standardized" and "no other devices are on the network". The network team provided access to a SPAN port on their Internet uplink, where I attached my pen-test workstation to take a look.

$sudo ngrep -qt -W single -s1514 -d eth0 -P~ 'User-Agent:' 'port 80'

"ngrep" works like grep, but on network traffic. Thus, the above command digs through everything on port 80 (http) that the span port provides, and searches for the string "User-Agent:", which commonly contains the "signature" of the web client making the access. A little bit of cleanup was needed to make the output usable:

| sed 's/.*User-Agent/User-Agent/' | sed 's/~.*//' | sed '/^$/d'

This takes care of empty lines, and throws out everything that isn't part of the User-Agent: string.  Collect the output into a file for a while, and then tally:

$cat output.txt | sort | uniq -c | sort -rn

And lookie, we ended up with about 80 distinct user agents. In only five minutes of traffic. Well, so far for "standardized desktop build" and "nothing else on the network". Among the user agent strings seen were

User-Agent: Mozilla/4.0 (compatible; Lotus-Notes/6.0; Windows-NT)
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1;.NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: Gecko/20110303 Firefox/3.6.15 (.NET CLR 3.5.30729)

Hmm, peculiar, some users are surfing with IE7 on Windows XP, while others are using an oooold version of Lotus Notes, and again others are using a vulnerable version of Firefox ??

User-Agent: Apple-iPhone3C1/812.1
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
User-Agent: BlackBerry9000/
User-Agent: BlackBerry9700/

A couple of mobile devices ... with what looks like a Windows7/IE9 system thrown in for good measure. The mobile devices turned out to be most interesting, because unless there is a WiFi gateway hooked into the corporate LAN, these devices usually surf via the mobile phone network, and shouldn't show up in the company's outbound Internet traffic. Guess what we found a couple minutes later ...: a little unauthorized wireless network extension, using WEP and the company name as SSID. Duh...!

And,  last but not least, we found some odd ducks that certainly warranted a closer look ..:

User-Agent: core
User-Agent: n1ghtCrawler
User-Agent: curl/7.8.1 (sparc-sun-solaris2.6) libcurl 7.9.6 (OpenSSL 0.9.6c)
User-Agent: Mozilla/4.0 (banzai)

Moral of the story: While your IDS probably alerts on "unusual" User Agent strings, it might nonetheless be a good idea to check out the full set of client applications that you have communicating with the Internet. The "User-Agent" string isn't failsafe, but it's a good start. You never know, you might just uncover a Secret (User) Agent who is busy squirreling away your data.

If you have other clever ways of auditing the user agent strings on your perimeter, please share in the comments below!


7 comment(s)


I'm glad you brought up odd user-agents, because I've been stuck searching for a user-agent I saw on my network:

USER-AGENT: TSA Browser Agent

My google-fu is lacking because the only thing I found that dealt with this user agent (and not the government agency, TSA) was a snort signature.

Do you believe this is in fact the user agent for the Target Saver spyware?
You can put multiple actions into a single sed command:

| sed 's/.*User-Agent/User-Agent/

Good diary entry about a simple way to do some good intelligence.
You can use tshark and avoid all the sed stuff:

$sudo tshark -f "port 80" -R "http contains \"User-Agent:\"" -Tfields -e http.user_agent > output.txt

$cat output.txt | sort | uniq -c | sort -rn

Sample output:

132 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0
13 Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20101130 Ubuntu/9.10 (karmic) Firefox/3.5.7
1 Microsoft-CryptoAPI/6.1
Blue Coat Proxy (Or Bluecoat Reporter) administrators can get this pretty easily as well.

In Blue-Coat Reporter - There's a built-in report for User-Agents.

Or, if you want to watch real-time action flying by, you can go into your Blue Coat and start a tail in Access Logging. You'll be doing a lot of scrolling this way, but it shows the required information.
This has given me an idea..for finding rogue access points.

Search for iPhone or iPad user agent strings with source address which you know should only be wired and you probably have yourself a rogue AP there.

Just ran this report on my Bluecoat and found a few possibles.
I suppose I'd freak you out then Matt. I have been known to browse with FF but user agent reporting iPhone.
What about HTTP requests that do not set the user agent header? I imagine they may also be interesting and worth examining.

Diary Archives