Shell script analysis nginx log access times and most time-consuming pages (slow query)

When the server pressure is relatively large, it is very laborious to run. We often do site page optimization, we will find those pages with more visits and more time-consuming. Find those addresses that are highly visited and time-consuming, and you'll get immediate results with immediate optimization. Below is a shell script that I often use when doing optimization. This can also be counted as a slow page of the web page's slowpage, like mysql slowquery.

The following is mine: nginx configuration

log_format main '$remote_addr - $remote_user [$time_local] $request ''"$status" $body_bytes_sent "$http_referer" ''" ;$http_user_agent" "$http_x_forwarded_for" $request_time';

access_log /var/log/nginx/access.log main buffer=32k;

Configure from above, you can see: ip In the first column, the page time is in the last column, separated by spaces. So in awk, you can use :$1$NF to read the current value. Where NF is a constant, representing the entire number of columns.

The following is the shell file for analyzing the code, which can be saved as slow.sh

#!/bin/sh

export PATH=/usr/Bin:/bin:/usr/local/bin:/usr/X11R6/bin;export LANG=zh_CN.GB2312;

function usage(){echo "$0 filelog options";exit 1;}< Br>

function slowlog(){#set -x;field=$2;files=$1;end=2;msg="";

[[ $2 == '1' ]] && field=1&&end=2&&msg="Total Visits Statistics";[[ $2 == '2' ]] && field=3&&end=4& ;&msg="Average Access Time Statistics";

echo -e "\ \ \ \ ";echo -n "$msg";seq -s '#' 30 |  Sed -e 's/[0-9]*//g';

awk '{split($7,bbb,"?");arr[bbb[1]]=arr[bbb [1]]+$NF; arr2[bbb[1]]=arr2[bbb[1]]+1; } END{for ( i in arr ) { print i":"arr2[i]":" ;arr[i]":"arr[i]/arr2[i]}}' $1 |  Sort -t: +$field -$end -rn | Grep "pages" | Head -30 |  Sed 's/:/\\t/g'}

[[ $# < 2 ]] && usage;

slowlog $1 $2;

only Need to execute: slow.sh log file 1 or 2 1: 30 access to the most mundane page 2: 30 access to the most time-consuming page

Execution results are as follows:

chmod + x ./slow.sh

chmod +x slow.sh ./slow.sh /var/log/nginx/./slow.sh /var/log/nginx/access.log 2

Average access time statistics############################################################### 120.456 30.114/pages/########2.php 1 16.161 16.161/pages/########3.php 212 1122.49 5.29475/pages/########4.php 6 28.645 4.77417

..................

I hope the above script will help you.