|
This diary is a bit unusual in that the problem here is very close to home, the ISC/DShield website. But I figure among all of our readers, there may be one who can help. I have seen others describing the same issue, but so far I haven't found a solution. The problem: Users who download our log submission client using Internet Explorer frequently receive truncated files. Firefox appears to download them fine. In either case, the server logs a "200" status and the file size in our Apache access log is correct (about 2.2 MBytes). However, the users only receives 200-300kBytes. A packet capture confirms that only 200-300kBytes got transfered. As MSIE starts the download, it does display the correct file size (and the content-length header is correct) Some of the issues we excluded: mod_security, firewall, IPS Also note that the downloads work fine with Firefox, so the server is perfectly capable of sending the file. Any help is appreciated. Link to the file: http://isc.sans.org/clients/cvtwin-setup.exe Here is a packet dump of the end of the connection: IP client.54436 > server.80: Flags [.], ack 193105, win 32850, length 0 IP server.80 > client.54436: Flags [.], ack 646, win 1783, length 1460 IP client.54436 > server.80: Flags [.], ack 196025, win 32850, length 0 IP server.80 > client.54436: Flags [.], ack 646, win 1783, length 1460 IP server.80 > client.54436: Flags [FP.], seq 215005:216465, ack 646, win 1783, length 1460 IP client.54436 > server.80: Flags [.], ack 198945, win 32120, length 0 IP client.54436 > server.80: Flags [.], ack 200405, win 32850, length 0 IP client.54436 > server.80: Flags [.], ack 203325, win 32850, length 0 IP client.54436 > server.80: Flags [.], ack 207705, win 32850, length 0 IP client.54436 > server.80: Flags [.], ack 210625, win 32850, length 0 IP client.54436 > server.80: Flags [.], ack 212085, win 32120, length 0 IP client.54436 > server.80: Flags [.], ack 213545, win 32850, length 0 IP client.54436 > server.80: Flags [.], ack 216466, win 32120, length 0 IP client.54436 > server.80: Flags [F.], seq 646, ack 216466, win 32850, length 0 IP server.80 > client.54436: Flags [.], ack 647, win 1783, length 0 ------ |
Johannes 4164 Posts ISC Handler Oct 14th 2009 |
| Thread locked Subscribe |
Oct 14th 2009 1 decade ago |
|
how is ending the trace? you should be able to determine which end is closing the connection.
|
Anonymous |
| Quote |
Oct 14th 2009 1 decade ago |
|
I added the last few packets from my packet capture above. The server is sending the first "FIN".
|
Johannes 4164 Posts ISC Handler |
| Quote |
Oct 14th 2009 1 decade ago |
|
Have you tried right clicking on the file from a directory view and then save-as?? If that works then the chances are the browser is reading the Character Set of the last page it was on and using it for the transfer type. If no default character set is listed it may default to a 7-bit Latin character set (1251 I believe) as many windows things do.
Try setting up a doctype in the html.. <meta http-equiv="content-type" content="text/html; charset=utf-8"> I know you can set the default character set in Apache as well, which may solve your problem. The other option is to set up an FTP link instead. Just a thought.. Good luck, Al |
Al of Your Data Center 80 Posts |
| Quote |
Oct 14th 2009 1 decade ago |
|
I'm fairly confident this is NOT your problem, but I thought I'd mention it just in case. I had a somewhat similar issue that was resolved using the Apache EnableSendFile directive:
http://securityonion.blogspot.com/2008/10/apache-enablesendfile-directive.html For more information, please see: http://httpd.apache.org/docs/2.0/mod/core.html#enablesendfile Thanks, Doug Burks |
Al of Your Data Center 2 Posts |
| Quote |
Oct 14th 2009 1 decade ago |
|
Charset: already setting it in HTTP header and as a META tag
EnableSendFile: Tried it... so far no difference (but a good catch!) . |
Johannes 4164 Posts ISC Handler |
| Quote |
Oct 14th 2009 1 decade ago |
|
I set up two test files:
isc.sans.org/… (all "0") isc.sans.org/… (random content) both files are 1,000,000 bytes long. I am still having the same issue with these files as I have with the cvtwin-setup.exe file. |
Johannes 4164 Posts ISC Handler |
| Quote |
Oct 14th 2009 1 decade ago |
|
Because the .exe is a zip file, could it be related to the mod_deflate module in Apache?
http://www.waterworld.com.hk/en/node/58 |
Johannes 1 Posts |
| Quote |
Oct 14th 2009 1 decade ago |
|
Just a quick observation. Doubt it means anything...
C:\Users\ignorance>nc isc.sans.org 80 GET /clients/cvtwin-setup.exe HTTP/1.0 HTTP/1.1 302 Found Date: Wed, 14 Oct 2009 18:21:54 GMT Server: nc -l -p 80 Location: https://blogs.sans.org/appsecstreetfighter//clients/cvtwin-setup.exe Content-Length: 252 Connection: close Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="https://blogs.sans.org/appsecstreetfighter//clients/cvtwin-setup. exe">here</a>.</p> </body></html> Same results (302 redirect and double-slash in redirect) with telnet. |
Johannes 2 Posts |
| Quote |
Oct 14th 2009 1 decade ago |
|
Just a quick observation. Doubt it means anything...
C:\Users\ignorance>nc isc.sans.org 80 GET /clients/cvtwin-setup.exe HTTP/1.0 HTTP/1.1 302 Found Date: Wed, 14 Oct 2009 18:21:54 GMT Server: nc -l -p 80 Location: https://blogs.sans.org/appsecstreetfighter//clients/cvtwin-setup.exe Content-Length: 252 Connection: close Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="https://blogs.sans.org/appsecstreetfighter//clients/cvtwin-setup. exe">here</a>.</p> </body></html> Same results (302 redirect and double-slash in redirect) with telnet. |
Johannes 2 Posts |
| Quote |
Oct 14th 2009 1 decade ago |
|
I am not having this problem in ie8 using your test files.
This reminds me of a gzip issue I ran into before. This is an IE problem and they have fixed it in ie8. There is a work around you can apply in apache though. http://support.microsoft.com/kb/871205 |
Anonymous |
| Quote |
Oct 14th 2009 1 decade ago |
|
I think the problem you are running into is with some missing HTTP headers in the server response. There are several known issues with Internet Explorer that deal with receiving application/octet-stream MIME-typed files while missing certain response headers...
|
Anonymous |
| Quote |
Oct 14th 2009 1 decade ago |
|
I could only replicate this issue when using a caching proxy (ISA server). The proxy would download the whole file from your server but would send my client only 675KB. Subsequent requests returned the full file from the proxy's cache to my browser. Could be coincidental though..
Was that packet capture done on the client or the server? |
Anonymous |
| Quote |
Oct 15th 2009 1 decade ago |
|
Purely an point regarding my ignorance - but has anyone tested doing this in IE8 and IE8's "compatability mode" (which resolves most issues I have seen with IE8)?
|
Anonymous |
| Quote |
Oct 15th 2009 1 decade ago |
|
Not sure if it'll help, but here are my observations from IE 8. I downloaded the null.bin file three times. The first time, the saved file was 166440 bytes. The second time it was the entire file, 1000000 bytes. The third time, the saved file was 152568 bytes. So the incomplete chunks are also inconsistent in size.
What's puzzling is that unlike Johannes, in my testing the full 1000000 bytes *did* come across the wire every time. The server was sending me the entire file, the browser wasn't saving all of it. So I'd concur that this is definitely an IE problem. |
parseword 9 Posts |
| Quote |
Oct 16th 2009 1 decade ago |
|
I am a bit late in the game, and I don't know if it has anything to do with this particular failure mode, but I recently had some problems downloading longish DShield reports. The way to reproduce the problem (somewhat) reliably was to do a rate-limited curl download. The same method also seems to cause problems when downloading your test binaries:
$ curl --limit-rate 10K "http://isc.sans.org/clients/cvtwin-setup.exe" > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 49 2202k 49 1098k 0 0 10168 0 0:03:41 0:01:50 0:01:51 8494 curl: (18) transfer closed with 1130052 bytes remaining to read For the record, this happens on OSX 10.5. On Debian lenny the effect is inconsistent. Go figure. |
parseword 1 Posts |
| Quote |
Oct 22nd 2009 1 decade ago |
Sign Up for Free or Log In to start participating in the conversation!
