Nginx log analysis method

First, the nginx log intercepted from nigx - 122.59.14.12 122.59.14.12 - - [24/Apr/2011:10:37:06 +0800] "GET /product/sellerscore. Do?returnpage=0&supplierid=ff8080811fdc4c29011feaa9ed165c11 HTTP/1.1" 499 0 "http://www.dhgate.com/wholesale/store.do?act=sellerStore&sellerid=ff8080811fdc4c29011feaa9ed165c11&datatype=&winid=all& Sortinfo=operatedate&sorttype=down&sort=operateate,down&keyword=i phone&freeshipping=0&wholesale=0&price=0&minprice=&maxprice=&pagesize=40&catalogid=" "Mozilla /4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; FunWebProducts; GTB6.6; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; CPNTDF; OfficeLiveConnector.1.5; OfficeLivePatch.1.3; .NET4.0C)" rBABeU2RGgBXQj3OA46vAg== Second, the configuration of the nginx log in the nginx configuration file $remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" $secure_dhgate_auth ' Third, the following is an explanation of each record of nginx log records 1,122.59.14.12 This is a request to the client ip of the nginx server ($remote_addr). 2, 122.59.14.12 This is a client user ($remote_user) requesting to the nginx server. 3. [24/Apr/2011:10:37:06 +0800]([$time_local]) Record the request time (in the format [day/month/year:hour:minute:second zone], the last +0800 means the server The time zone is Dongba District. 4, "GET /product/sellerscore.do?returnpage=0&supplierid=ff8080811fdc4c29011feaa9ed165c11 HTTP/1.1" ($request) The most useful information in this entire record, first of all, It tells our server that it receives a GET request, followed by the resource path requested by the client. Third, the protocol used by the client is HTTP/1.1. The entire format is "%m %U%q %H" , ie "Request Method/Access Path/Protocol" 5, 499 (I will write an article for explanation of the 499 status code)--($status) This is a status code sent by the server to the client. End, it tells us whether the client's request is successful, or is redirected, or what kind of error is encountered. This value is 200, indicating that the server has successfully responded to the client's request. In general, this value is The beginning of 2 indicates that the request was successful, and the beginning of 3 indicates redirection to 4 There are some errors in the header of the client, and there are some errors on the server side beginning with 5. For details, see the HTTP specification (RFC2616 section 10). [http://www.w3.org/Protocols/rfc2616/Rfc2616.txt] 6, 0 ($body_bytes_sent) This indicates how many bytes the server sends to the client. When the log analyzes the statistics, add these bytes to know that the server is always at a certain time. What is the amount of data sent? 7. "http://www.dhgate.com/wholesale/store.do?act=sellerStore&sellerid=ff8080811fdc4c29011feaa9ed165c11&datatype=&winid=all&sortinfo=operatedate&sorttype=down& Sort=operatedate,down&keyword=i phone&freeshipping=0&wholesale=0&price=0&minprice=&maxprice=&pagesize=40&catalogid=" ($http_referer) This item is ($http_referer ) The http request entered by the client. 8. Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; FunWebProducts; GTB6.6; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; CPNTDF; OfficeLiveConnector.1.5; OfficeLivePatch.1.3; .NET4.0C)" ($http_user_agent) This main record client information 9, rBABeU2RGgBXQj3OA46vAg== $secure_dhgate_auth Fourth, practical log analysis script 1. View nginx Number of processes # ps -aux| Grep nginx| Wc -l 2, analyze the log to see the number of ip connections on the day # grep "23/Apr/2011" logs/www.dhgate.access.log| Wc -l 3. See what url the specified ip visited on the day # grep "23/Apr/2011" logs/www.dhgate.access.log| Grep "69.248.213.128"| Awk '{print $9}' 4. View the top 10 urls for the day of the visit # grep "23/Apr/2011" logs/www.dhgate.access.log| Awk '{print $9}'| Sort | Uniq -c | Sort -nr | Head -n 10 5. View the maximum number of visits in hours (hotspots) # grep "24/Apr/2011" logs/www.dhgate.access.log| Awk '{print $6}'| Cut -c14-15| Sort | Uniq -c | Sort -nr| Head
zh-CNzh-CN