请教下 supervisor 配置的问题

[MrMike]

为了监控 elasticsearch 死掉后，能自动重启，网上查了下,supervisor 可以实现这个要求，以下是配置： [program=sticsearch] command=/usr/share/elasticsearch/bin/elasticsearch autostart=true autorestart=true stdout_logfile=/home/demoer/log/elasticsearch/elasticsearch-stdout.log stderr_logfile=/home/demoer/log/elasticsearch/elasticsearch-stderr.log startsecs=5

我把这个直接写在 supervisor.conf ，在启动 supervisorctl reload, reread, update,都说没有找到配置（ No config updates to processes ）。 我用 include_file 单独来引入，也是同样的结果。 请问下配置代码是否正确呢？

[RIcter]

得重启 supervisor 服务…

[MrMike]

supervisord -c /etc/supervisord.conf

[root tmp]# ps aux | grep supervisord
root 10926 0.0 0.3 220120 11836 ? Ss Apr14 0:11 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 12554 0.0 0.2 217092 10664 ? Ss Apr14 0:10 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 12564 0.0 0.2 217128 10716 ? Ss Apr14 0:10 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 12575 0.0 0.2 217280 10824 ? Ss Apr14 0:11 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 12764 0.0 0.2 219728 11464 ? Ss Apr14 0:11 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 13844 0.0 0.2 217588 11164 ? Ss 00:11 0:05 /usr/bin/python /usr/bin/supervisord
root 14070 0.0 0.2 216964 9828 ? Ss 01:00 0:04 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 14117 0.0 0.2 217144 10644 ? Ss 01:02 0:04 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 14128 0.0 0.2 217520 11144 ? Ss 01:04 0:04 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 14936 0.0 0.2 216960 9832 ? Ss 08:07 0:00 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root 14938 0.0 0.0 112648 960 pts/0 S+ 08:07 0:00 grep --color=auto supervisord

[root tmp]# supervisorctl
supervisor> reload
Really restart the remote supervisord process y/N? y
Restarted supervisord
supervisor> reread
No config updates to processes
supervisor> update
supervisor> status
supervisor>

已经重启过很多次，还是一样的结果。

下面是 supervisor.conf 和 elasticsearch.conf 的配置内容。

supervisor.conf:

[include]
files = /etc/supervisord/conf.d/elasticsearch.conf

elasticsearch.conf:

[program=sticsearch]
command=systemctl restart elasticsearch.service
;command=/usr/share/elasticsearch/bin/elasticsearch
;directory=/usr/share/elasticsearch
autostart=true
autorestart=true
startsecs=10
startretries=3
redirect_stderr=true
user=root
stdout_logfile=/home/public/log/elasticsearch/elasticsearch-stdout.log
stderr_logfile=/home/public/log/elasticsearch/elasticsearch-stderr.log

[zzlyzq]

主配置里面默认会包含所有 conf.d 下面的 conf 后缀文件 回头我看看我的配置

[MrMike]

@zzlyzq 我安装完 supervisor 后，在 etc 下面没有 /etc/supervisord 文件夹的，我是自己创建的 /etc/supervisord/con.d/文件夹，然后把 elasticsearch.conf 放在里面。

[zzlyzq]

恩 看起来没啥错，我的 centos6.8 安装后，主文件是 包括如下两行

[include]
files = /etc/supervisor.d/*.conf

你的配置看起来也没啥问题的，要不试试
ls /etc/supervisord/conf.d/elasticsearch.conf
看看这个文件是不是真正的存在呢？ 不存在的概率可能比较小，不过工作中经常会遇到这种情况，呵呵。

[MrMike]

@zzlyzq 文件是存在的。郁闷。。一直找不到原因。

[zzlyzq]

@MrMike 既然文件已经加载了，怀疑还是配置的问题，
1. 是不是配置的 program 名字有冲突？
2. 打开界面，可能更一目了然

在 supervisord.conf 开头加上或者修改如下，打开 http ，一目了然

[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
[inet_http_server] ; inet (TCP) server disabled by default
port=0.0.0.0:1009 ; (ip_address:port specifier, *:port for all iface)
username=ops ; (default is no username (open server))
password=123123 ; (default is no password (open server))

[zzlyzq]

@MrMike 我大概知道了， supervisor 在执行命令的时候要求比较苛刻，比如全路径呀，环境变量报错呀啥的， 刚刚的 http 页面可以让你看到
1. elasticsearch 进程是否配置到 supervisord
2. 启动情况如何，是 ok 还是 fail
3. 瞅瞅 log 里面有啥吧

[MrMike]

@zzlyzq supervisord.conf 用的是默认设置，我没修过过。

[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
chmod=0777 ; socket file mode (default 0700)
;chown=nobody:nogroup ; socket file uid:gid owner
;username=user ; (default is no username (open server))
;password=123 ; (default is no password (open server))

;[inet_http_server] ; inet (TCP) server disabled by default
;port=127.0.0.1:9001 ; (ip_address:port specifier, *:port for all iface)
;username=user ; (default is no username (open server))
;password=123 ; (default is no password (open server))

supervisord.log

2017-04-15 08:08:21,706 INFO RPC interface 'supervisor' initialized
2017-04-15 08:08:21,706 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2017-04-15 08:08:21,707 INFO supervisord started with pid 14936
2017-04-15 08:43:04,969 CRIT Supervisor running as root (no user in config file)
2017-04-15 08:43:04,969 INFO Included extra file "/etc/supervisord/conf.d/elasticsearch.conf" during parsing
2017-04-15 08:43:04,980 INFO RPC interface 'supervisor' initialized
2017-04-15 08:43:04,980 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2017-04-15 08:43:04,981 INFO daemonizing the supervisord process
2017-04-15 08:43:04,981 INFO supervisord started with pid 15044

貌似没看出有啥问题呢。也没有报错。

[zzlyzq]

Included extra file "/etc/supervisord/conf.d/elasticsearch.conf" during parsing

上面的一行说明你的 elasticsearch.conf 已经被加载了，现在我越来越猜测是 elasticsearch.conf 配置问题导致没有启动

既然没有打开 http ，那就使用如下命令看看 status 吧

supervisorctl status

[zsz]

1.supervisor 配置中的 command 一定要处于挂起运行，把标准输出，是他的子进程，运行期间不能返回
2.你需要用命令行启动 es 的方式，配置到 command
3.既然用 systemd 和 supervisor 用一个就可以，他们都是守护


有问题可以加群问，效率高些，面向初学者的 python Linux 学习群， 由一群有经验的工程师组成， python qq 群号： 278529278 ， PHP qq 群号： 476648701 ，非商业性质，拒绝广告，只接收真正想学这方面技术的朋友，交流学习，申请请说明来自 v2ex

[yzongyue]

这样写试试 [program:sticsearch]

[ksupertu]

新版本先要启动 supervisord 然后在启动 supervisorctl ，不然会一直报错……

[ksupertu]

而且你 command 那里最好写成 shell 脚本去执行，错误概率小一点

[MrMike]

@ksupertu 已经启动过了。 command 我用 /usr/share/elasticsearch/bin/elasticsearch 也是不能用的。在日志里面也看不到错误提示呢。

[zsz]

@MrMike ，你看的什么错误日志，是 supervisor 的？还是 supervisor 配置文件中 es 的？这个启动不了应该是环境问题，一般是 java 环境， supervisor 自己的错误日志中应该有信息才对

[MrMike]

@zsz 是 supervisord.log

[a128445]

[program=sticsearch]

改为
[program:sticsearch]

[zsz]

@MrMike 这两个 log 呢？ 有错误信息

stdout_logfile=/home/public/log/elasticsearch/elasticsearch-stdout.log
stderr_logfile=/home/public/log/elasticsearch/elasticsearch-stderr.log

[zsz]

先按照上面说的把配置写对 [program:sticsearch]

[MrMike]

@zsz
@a128445

elasticsearch.con ：

[program:elasticsearch]
command=/usr/share/elasticsearch/bin/elasticsearch
directory=/usr/share/elasticsearch
autostart=true
autorestart=true
startsecs=10
startretries=3
redirect_stderr=true
user=root
stdout_logfile=/home/demoer/log/elasticsearch/elasticsearch-stdout.log
stderr_logfile=/home/demoer/log/elasticsearch/elasticsearch-stderr.log


elasticsearch-stdout.log ：

Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
at org.elasticsearch.cli.Command.main(Command.java:53)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
Refer to the log for complete error details.
Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
at org.elasticsearch.cli.Command.main(Command.java:53)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
Refer to the log for complete error details.
Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
at org.elasticsearch.cli.Command.main(Command.java:53)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
Refer to the log for complete error details.
Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
at org.elasticsearch.cli.Command.main(Command.java:53)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
Refer to the log for complete error details.


elasticsearch-stderr.log 内容为空。

我在重启 supervisor 后，进入 supervisorctl ，提示“ elasticsearch FATAL Exited too quickly (process log may have details)”

[MrMike]

elasticsearch 安装后，在 /usr/share/elasticsearch/就没有 config 这个文件的。

[zsz]

1. 你命令行成功启动过 es 么？
2.如果你只是在 systemd 中成功启动过，你的配置文件应该在系统级的目录下，找到他，先用命令行能启动，再把命令配置进去， es 在命令行可以指定配置文件的

[MrMike]

@zsz ES 是可以正常启动和关闭的。我再看看这个错误提示到底是咋回事。。

[MrMike]

@zsz 我试试用命令行启动下。

[MrMike]

@zsz
@a128445

配置文件改成这样的：

[program:elasticsearch]
command=/etc/init.d/elasticsearch start
directory=/usr/share/elasticsearch
autostart=true
autorestart=true
startsecs=10
startretries=3
redirect_stderr=true
user=root
stdout_logfile=/home/demoer/log/elasticsearch/elasticsearch-stdout.log
stderr_logfile=/home/demoer/log/elasticsearch/elasticsearch-stderr.log

我手动关闭 es 后，等了几秒钟 es 没有启动，就使用 supervisorctl restart elasticsearch 启动，
elasticsearch: ERROR (not running)
elasticsearch: ERROR (spawn error)

得到这样的错误提示，但是 es 却重新启动了。不能自动启动 es 呢。。

[MrMike]

@zsz stop es 后，我目前的配置，只能手动执行 supervisorctl reload ，才能再让 es 启动。

[zsz]

1. 上面我指出过 supervisor 中 command 不能使用 systemctl 或者 init.d 的方式，因为这两种是 daemon 运行，命令执行完就返回了
2. 你加我上面的发群我用 qq 跟你说，这样解释太慢了，非商业化的群，不用担心

[MrMike]

@zsz 加了，回你信息了