Practical tips: What to do when the Linux system "crash"

                  If the problem can be reproduced, then the problem has been solved 80%. For the core of the operating system, if there is a problem with the reproduction method, then it can be said that 99% has been solved. A common problem is that the system can run for a while and then crash. If it is not good to reproduce the problem, then only the analysis is based on what remains on the scene of the crash. If the system is not dead, such as disk interrupts and file system is good, then there may be log information kept in the file, but I have never encountered such good luck. If the keyboard interrupt can still respond (press Num Lock, you can see the keyboard light off), then luck is good enough, then you can sacrifice the sysrq method, and press Alt-Sysrq-T to get the process system stack information. Press Alt-Sysrq-M to get the memory allocation information, and press Alt-Sysrq-W to get the current register information. Linux/Documentation/sysrq.txt. In addition, it is best to turn off the automatic blank function of the terminal, so that at least some information can be seen from the screen when the system dies. The setting method is: # echo 1 > /proc/sys/kernel/sysrq # setterm -blank These two settings are best added to the system startup script (such as /etc/rc.d/rc.local), guaranteed each time Startup can be run. If unfortunately, the keyboard is dead, and (more unfortunately, this is very common), then it is not only a way to wait for death, then you can use the serial console to send system information to another On the system, this can be used to locate the problem by analyzing this information. The setting method is as follows: Preparation 1. A monitored server, a PC for monitoring work. 2. A serial port is directly connected. Configuration 1. On the server, add a new grub project and add the core parameter "console=ttyS0 console=tty1", such as: kernel /boot/vmlinuz-2.4.21-9.30AXsmp ro root=LABEL=/1 console=ttyS0 Console=tty1 2. On the server, modify /etc/sysconfig/syslog and add the klogd option "-c 7" to ensure that more kernel information is output. For example: KLOGD_OPTIONS="-x -c 7" 3. Restart the server 4. Connect the two machines with a serial cable. Test: 1) Run "cat /dev/ttyS0" on the PC and run "echo" on the server. Hi > /dev/ttyS0", see if there is a "hi" output on the PC. 2) Run "cat /dev/ttyS0" on the PC and run "echo w > /proc/sysrq-trigger" on the server to see if there is any corresponding kernel information output on the PC. 3) Run "cat /dev/ttyS0" on the PC and run "modprobe loop" on the server to see if there is any corresponding kernel information output on the PC. 5. If the test passes, run on the PC: cat /dev/ttyS0