Nginx reload未生效问题排查处理
修改Nginx配置文件后，nginx -t检查通过，通过nginx -s reload重新加载配置，发现配置未生效。
无异常报错日志，以下是对该问题的排查分析过程。
1.获取nginx主进程PID
swift@Server:/etc/nginx/conf.d$ ps -ef |grep nginx
root 10643 1 0 15:29 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 10645 10643 0 15:29 ? 00:00:04 nginx: worker process
www-data 10647 10643 0 15:29 ? 00:00:11 nginx: worker process
www-data 10648 10643 0 15:29 ? 00:00:14 nginx: worker process
www-data 10649 10643 0 15:29 ? 00:00:18 nginx: worker process
www-data 10650 10643 0 15:29 ? 00:00:13 nginx: worker process
www-data 10651 10643 0 15:29 ? 00:00:13 nginx: worker process
www-data 10652 10643 0 15:29 ? 00:00:07 nginx: worker process
www-data 10653 10643 0 15:29 ? 00:00:05 nginx: worker process
www-data 10654 10643 0 15:29 ? 00:00:00 nginx: cache manager process
swift 12160 3536 0 16:22 pts/0 00:00:00 grep --color=auto nginx2.使用strace跟踪nginx主进程
strace -p 106433.运行nginx -s reload命令，通过strace查看nginx进程调用相关信号
...
open("/var/log/nginx/patient-h5-xxxxxyy-test_error.log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 1022
fcntl(1022, F_SETFD, FD_CLOEXEC) = 0
open("/var/log/nginx/patient-standard-xxxxy-test_access.log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 1023
fcntl(1023, F_SETFD, FD_CLOEXEC) = 0
open("/var/log/nginx/patient-standard-xxxx-test_error.log", O_WRONLY|O_CREAT|O_APPEND, 0644) = -1 EMFILE (Too many open files)
gettid() = 10643
write(6, "2022/08/04 16:21:30 [emerg] 1064"..., 136) = 136
close(648) = 0
...打开日志文件失败，提示Too many open files
4.发现Too many open files，查看系统进程限制文件
swift@Server:/etc/nginx/conf.d$ cat /proc/10643/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 95594 95594 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 95594 95594 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us发现Max open files配置是1024,过小，nginx -t reload过程中，重新打开日志文件，达到limit限制，导致reload中断，配置未生效。
5.对nginx服务进行调整配置参数
编辑/lib/systemd/system/nginx.service文件，在 [Service]中添加 LimitNOFILE=65535限制即可
[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid
TimeoutStopSec=5
KillMode=mixed
LimitNOFILE=655356.重新加载service配置，重启nginx服务
sudo systemctl daemon-reload
sudo systemctl restart nginx7.验证
检查limit，已生效
swift@Server:/etc/nginx/conf.d$ ps -ef |grep nginx
root 12761 1 0 16:35 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 12764 12761 0 16:35 ? 00:00:00 nginx: worker process
www-data 12765 12761 0 16:35 ? 00:00:00 nginx: worker process
www-data 12766 12761 1 16:35 ? 00:00:00 nginx: worker process
www-data 12767 12761 1 16:35 ? 00:00:00 nginx: worker process
www-data 12768 12761 0 16:35 ? 00:00:00 nginx: worker process
www-data 12769 12761 1 16:35 ? 00:00:00 nginx: worker process
www-data 12770 12761 0 16:35 ? 00:00:00 nginx: worker process
www-data 12771 12761 0 16:35 ? 00:00:00 nginx: worker process
www-data 12772 12761 0 16:35 ? 00:00:00 nginx: cache manager process
www-data 12773 12761 0 16:35 ? 00:00:00 nginx: cache loader process
swift 12776 3536 0 16:35 pts/0 00:00:00 grep --color=auto nginx
swift@Server:/etc/nginx/conf.d$ cat /proc/12761/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 95594 95594 processes
Max open files 65535 65535 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 95594 95594 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us修改nginx配置文件后，nginx -t reload恢复正常
扩展
为什么本地ulimit配置了最大文件打开数，service服务中没有生效？
本地limit限制中，open files配置的是65535
swift@Server:~$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 15576
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 15576
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited查看nginx服务limit限制是1024，而不是65535
swift@Server:~$ sudo systemctl start nginx.service
swift@Server:~$ ps -ef |grep nginx
root 4001 1 0 18:25 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 4003 4001 0 18:25 ? 00:00:00 nginx: worker process
www-data 4004 4001 0 18:25 ? 00:00:00 nginx: worker process
swift 4008 1614 0 18:25 pts/0 00:00:00 grep --color=auto nginx
swift@Server:~$ cat /proc/4001/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 15576 15576 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 15576 15576 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
swift@Server:~$
原因：
本地执行的ulimit对于Nginx的Service服务不生效，通常是因为nginx服务不是通过当前shell启动的，而是通过systemd服务启动的，systemd服务有自己独立的资源限制配置。
核心原因可以理解为：
ulimit 只影响当前 shell 及其子进程
systemd service 不属于这个进程树解决方法：
修改Nginx这个Service服务的最大文件打开数limit限制，可以通过修改systemd的全局配置文件，或者nginx独立的nginx.service文件来进行处理。
1.通过修改systemd的全局配置文件/etc/systemd/system.conf
# 修改进程的Max open files限制
DefaultLimitNOFILE=65535systemd默认配置，会对所有systemd管理的service服务生效。
2.通过nginx自身的service文件/lib/systemd/system/nginx.service修改限制
[Service]
LimitNOFILE=65535仅对该nginx服务生效，且会覆盖/etc/systemd/system.conf中的DefaultLimitNOFILE配置。