[Linux][运维]SaltStack安装Apache/Mysql/PHP部署Wordpress

salt介绍

SaltStack是一个服务器基础架构集中化管理平台，具备配置管理、远程执行、监控等功能，基于Python语言实现，结合轻量级消息队列（ZeroMQ）与Python第三方模块（Pyzmq、PyCrypto、Pyjinjia2、python-msgpack和PyYAML等）构建。

SaltStack 采用 C/S模式，server端就是salt的master，client端就是minion，minion与master之间通过ZeroMQ消息队列通信。

master监听4505和4506端口，4505对应的是ZMQ的PUB system，用来发送消息，4506对应的是REP system是来接受消息的。

命令执行步骤

具体步骤如下

• Salt stack的Master与Minion之间通过ZeroMq进行消息传递，使用了ZeroMq的发布-订阅模式，连接方式包括tcp，ipc
• salt命令，将cmd.run ls命令从salt.client.LocalClient.cmd_cli发布到master，获取一个Jodid，根据jobid获取命令执行结果。
• master接收到命令后，将要执行的命令发送给客户端minion。
• minion从消息总线上接收到要处理的命令，交给minion._handle_aes处理
• minion._handle_aes发起一个本地线程调用cmdmod执行ls命令。线程执行完ls后，调用minion._return_pub方法，将执行结果通过消息总线返回给master
• master接收到客户端返回的结果，调用master._handle_aes方法，将结果写的文件中
• salt.client.LocalClient.cmd_cli通过轮询获取Job执行结果，将结果输出到终端。

saltstack安装

测试环境

• CentOS6.5
• master：172.20.22.46 (主机名：hadoop1) # 之前配置Spark改的名字
• slave：172.20.22.159 (主机名：hadoop2)

分为以下几步

关闭selinux

/etc/selinux/config，把SELINUX=enforcing改为SELINUX=disabled

master安装salt-minion, salt-master

Google

slave安装salt-minion

Google

配置salt-master

参考

#指定master，冒号后有一个空格
master: 192.168.2.22
user: root

#-------以下为可选--------------
# salt运行的用户，影响到salt的执行权限
user: root
#s alt的运行线程，开的线程越多一般处理的速度越快，但一般不要超过CPU的个数
worker_threads: 10
# master的管理端口
publish_port : 4505
# master跟minion的通讯端口，用于文件服务，认证，接受返回结果等
ret_port : 4506
# 如果这个master运行的salt-syndic连接到了一个更高层级的master,那么这个参数需要配置成连接到的这个高层级master的监听端口
syndic_master_port : 4506
# 指定pid文件位置
pidfile: /var/run/salt-master.pid
# saltstack 可以控制的文件系统的开始位置
root_dir: /
# 日志文件地址
log_file: /var/log/salt_master.log
# 分组设置
nodegroups:
  group_all: '*'
# salt state执行时候的根目录
file_roots:
  base:
    - /srv/salt/
# 设置pillar 的根目录
pillar_roots:
  base:
    - /srv/pillar

本文主要改动

file_roots:
  base:
    - /srv/salt/base   dev:
    - /srv/salt/dev   test:
    - /srv/salt/test   prod:
    - /srv/salt/prod 

配置salt-minion

参考

#指定master，冒号后有一个空格
master: 192.168.2.22
id: minion-01
user: root

#-------以下为可选--------------
# minion的识别ID，可以是IP，域名，或是可以通过DNS解析的字符串
id: 192.168.0.100
# salt运行的用户权限
user: root
# master的识别ID，可以是IP，域名，或是可以通过DNS解析的字符串
master : 192.168.0.100
# master通讯端口
master_port: 4506
# 备份模式，minion是本地备份，当进行文件管理时的文件备份模式
backup_mode: minion
# 执行salt-call时候的输出方式
output: nested 
# minion等待master接受认证的时间
acceptance_wait_time: 10
# 失败重连次数，0表示无限次，非零会不断尝试到设置值后停止尝试
acceptance_wait_time_max: 0
# 重新认证延迟时间，可以避免因为master的key改变导致minion需要重新认证的syn风暴
random_reauth_delay: 60
# 日志文件位置
log_file: /var/logs/salt_minion.log
# 文件路径基本位置
file_roots:
  base:
    - /etc/salt/minion/file
# pillar基本位置
pillar_roots:
  base:
    - /data/salt/minion/pillar

本文主要改动

master : 172.20.22.46
grains:
  roles:
    - nginx   env:
    - test   myname:
    - yzd 

启动/重启salt

sudo service salt-master start/restart

sudo service salt-minion start/restart

master认证minion的key

Google

检查安装是否完成

[root@hadoop1 Desktop]# salt-run manage.status
down:
up:
    - hadoop1     - hadoop2 
[root@hadoop1 Desktop]# salt '*' grains.item os myname
hadoop1:
    ----------     myname:
    os:
        CentOS
hadoop2:
    ----------     myname: # 自定义的grains
        - yzd     os:
        CentOS

其它

关于grains/pillar/state等可以查阅文档

部署WordPress

环境要求

摘自官网
We recommend servers running version 7.2 or greater of PHP and MySQL version 5.6 OR MariaDB version 10.0 or greater.
We also recommend either Apache or Nginx as the most robust options for running WordPress, but neither is required.

最后的完整文件树

[root@hadoop1 base]# tree
.
├── mysql
│   ├── conf.sls
│   ├── files
│   │   ├── conf.sh
│   │   ├── my.cnf
│   │   ├── mysql-5.5.60.tar.gz
│   │   ├── mysql-5.6.40.tar.gz
│   │   ├── mysqld
│   │   └── mysqllns.sh
│   ├── init.sls
│   └── install.sls
├── top.sls
├── web
│   ├── apache.sls
│   ├── ap.sls
│   └── files
│       ├── apache-conf.d
│       │   ├── mod_dnssd.conf
│       │   ├── php.conf
│       │   ├── README
│       │   └── welcome.conf
│       ├── httpd.conf
│       └── php.ini
└── wordpress
    ├── files
    │   ├── wordpress-latest.tar.gz
    │   └── wordpress-yang.tar.gz
    └── wp-install.sls

7 directories, 21 files

安装Apache和Php

流程

• 安装apache和php依赖
• 将主机的apache和php配置文件覆盖从机文件
• 运行apache服务

web/ap.sls：

lamp-install:
  pkg.installed:
    - pkgs:       - httpd       - php       - php-pdo       - php-mysql       - php-gd 
apache-config:
  file.managed:
    - name: /etc/httpd/conf/httpd.conf #服务实际使用的文件路径     - source: salt://web/files/httpd.conf #salt的源文件用于分发到minion上面 路径是base目录下面的web 这里也支持http和ftp方式     - user: root     - group: root     - mode: 644     - require:       - pkg: lamp-install 
php-config:
  file.managed:
    - name: /etc/php.ini     - source: salt://web/files/php.ini     - user: root     - group: root     - mode: 644 
#使用watch在apache配置文件发送变化时，重新加载apache配置
lamp-service:
  service.running:
    - name: httpd     - enable: True     - reload: True #如果不加reload 默认会重启服务     - watch: #增加       - file: apache-config #监控上面的apache-config ID 所以说 一个ID在一个状态只能出现一次 
apache-conf:
  file.recurse:
    - name: /etc/httpd/conf.d     - source: salt://web/files/apache-conf.d

运行结果：

[root@hadoop1 base]# salt 'hadoop2' state.sls web.ap
hadoop2: ----------
 ID: lamp-install
 Function: pkg.installed
 Result: True
 Comment: 4 targeted packages were installed/updated. The following packages were already installed: httpd
 Started: 09:14:41.225378
 Duration: 64220.577 ms
 Changes: 
 ----------
 libXpm:
 ----------
 new:
 3.5.10-2.el6
 old:
 php:
 ----------
 new:
 5.3.3-49.el6
 old:
 php-cli:
 ----------
 new:
 5.3.3-49.el6
 old:
 php-common:
 ----------
 new:
 5.3.3-49.el6
 old:
 php-gd:
 ----------
 new:
 5.3.3-49.el6
 old:
 php-mysql:
 ----------
 new:
 5.3.3-49.el6
 old:
 php-pdo:
 ----------
 new:
 5.3.3-49.el6
 old: ----------
 ID: apache-config
 Function: file.managed
 Name: /etc/httpd/conf/httpd.conf
 Result: True
 Comment: File /etc/httpd/conf/httpd.conf is in the correct state
 Started: 09:15:45.451376
 Duration: 14.631 ms
 Changes: ----------
 ID: php-config
 Function: file.managed
 Name: /etc/php.ini
 Result: True
 Comment: File /etc/php.ini is in the correct state
 Started: 09:15:45.466131
 Duration: 4.601 ms
 Changes: ----------
 ID: lamp-service
 Function: service.running
 Name: httpd
 Result: True
 Comment: Service httpd has been enabled, and is running
 Started: 09:15:45.477085
 Duration: 477.583 ms
 Changes: 
 ----------
 httpd:
 True ----------
 ID: apache-conf
 Function: file.recurse
 Name: /etc/httpd/conf.d
 Result: True
 Comment: Recursively updated /etc/httpd/conf.d
 Started: 09:15:45.954926
 Duration: 149.052 ms
 Changes: 
 ----------
 /etc/httpd/conf.d/welcome.conf:
 ----------
 diff:
 --- 
 +++ 
 @@ -9,3 +9,4 @@
 ErrorDocument 403 /error/noindex.html
 </LocationMatch>


Summary ------------
Succeeded: 5 (changed=3)
Failed: 0 ------------
Total states run:     5

安装Mysql5.6

注意：CentOS6 yum自带的Mysql版本是5.1，达不到wordpress的最低版本要求。这里我们使用源码安装mysql5.6。步骤比较多，坑也很多。

流程

• 传输mysql源码包至从机
• 解压源码包
• 安装编译所需依赖
• 源码安装mysql
• 将主机my.cnf覆盖从机文件
• 运行conf.sh，使用scripts/mysql_install_db建立数据库
• 运行mysqllns.sh创建所需要的软连接
• 将主机mysqld覆盖从机文件
• 运行mysqld

init.sls

include:
  - mysql.install   - mysql.conf

install.sls

#install source mysql
mysql_source:
  file.managed:
    - name: /home/mysql-5.6.40.tar.gz     - unless: test -e /home/mysql-5.6.40.tar.gz     - source: salt://mysql/files/mysql-5.6.40.tar.gz 
#tar source mysql
extract_mysql:
  cmd.run:
    - cwd: /home     - names:         - tar xf mysql-5.6.40.tar.gz     - unless: test -d /home/mysql-5.6.40     - require:         - file: mysql_source 
#useradd for mysql
mysql_user:
  user.present:
    - name: mysql     - uid: 1024     - createhome: False     - gid_from_name: True     - shell: /sbin/nologin 
#mysql pkg.install
mysql_pkg:
  pkg.installed:
    - pkgs:       - gcc       - gcc-c++       - autoconf       - automake       - openssl       - openssl-devel       - zlib       - zlib-devel       - ncurses-devel       - libtool-ltdl-devel       - cmake 
#mysql source install
mysql_commpile:
  cmd.run:
    - cwd: /home/mysql-5.6.40     - names:         - chown root:root /home/mysql-5.6.40 -R         - cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/usr/local/mysql/data -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATTON=utf8_cuicode_ci -DWITH_READLINE=1 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=1 -DENABLED_LOCAL_INFILE=1 -DDEFAULT_COLLATION=utf8_general_ci -DWITH_MYISAM_STORAGE_ENGINE=1 -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_DEBUG=0         - make         - make install     - require:         - cmd: extract_mysql         - pkg: mysql_pkg     - unless: test -d /usr/local/mysql

conf.sls

include:
  - mysql.install 
# mysql for config
mysql_cnf:
  file.managed:
    - name: /etc/my.cnf     - user: root     - mode: 755     - source: salt://mysql/files/my.cnf 
# mysql init
salt://mysql/files/conf.sh:
  cmd.script:
      - env:           - BATCH: 'yes'       - require:           - cmd: mysql_commpile           - pkg: mysql_pkg 
# mysql lnk
salt://mysql/files/mysqllns.sh:
  cmd.script:
    - env:       - BATCH: 'yes'     - require:       - cmd: mysql_commpile       - pkg: mysql_pkg 
# mysql server
mysql_service:
   file.managed:
     - name: /etc/init.d/mysqld      - user: root      - mode: 755      - source: salt://mysql/files/mysqld    cmd.run:
     - names:          - /sbin/chkconfig --add mysqld          - /sbin/chkconfig --level 35 mysqld on      - unless: /sbin/chkconfig --list mysqld    service.running:
     - name: mysqld      - enable: True      - reload: True

conf.sh：

#!/bin/bash
/usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql/ --datadir=/usr/local/mysql/data/

mysqllns.sh

#!/bin/bash
ln -sv /usr/local/mysql/bin/mysql /usr/bin 
ln -sv /usr/local/mysql/bin/mysqladmin /usr/bin/
ln -sv /usr/local/mysql/bin/mysqldump /usr/bin/
#mysql competence
/bin/chown -R mysql.mysql /usr/local/mysql/ && /bin/chown -R mysql.mysql /usr/local/mysql/data/
#for wordpress
mkdir /var/lib/mysql
ln -sv /usr/local/mysql/data/mysql.sock /var/lib/mysql/

mysqld

#!/bin/sh
# chkconfig: 2345 10 50
# description: mysqld
basedir=/usr/local/mysql/
datadir=/usr/local/mysql/data/

注意：
需要添加下面两行来保证chkconfig设置成功

#!/bin/sh
# chkconfig: 2345 10 50

运行结果

由于首次运行编译返回日志过长，为了美观的结果，运行了两次，所以软连接脚本显示失败（文件已存在）

[root@hadoop1 base]# salt 'hadoop2' state.sls mysql.init
hadoop2: ----------
 ID: mysql_source
 Function: file.managed
 Name: /home/mysql-5.6.40.tar.gz
 Result: True
 Comment: unless execution succeeded
 Started: 10:09:23.030270
 Duration: 888.041 ms
 Changes: ----------
 ID: extract_mysql
 Function: cmd.run
 Name: tar xf mysql-5.6.40.tar.gz
 Result: True
 Comment: unless execution succeeded
 Started: 10:09:23.919593
 Duration: 7.14 ms
 Changes: ----------
 ID: mysql_user
 Function: user.present
 Name: mysql
 Result: True
 Comment: User mysql is present and up to date
 Started: 10:09:23.927631
 Duration: 2.501 ms
 Changes: ----------
 ID: mysql_pkg
 Function: pkg.installed
 Result: True
 Comment: All specified packages are already installed.
 Started: 10:09:23.931484
 Duration: 2006.185 ms
 Changes: ----------
 ID: mysql_commpile
 Function: cmd.run
 Name: cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/usr/local/mysql/data -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATTON=utf8_cuicode_ci -DWITH_READLINE=1 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=1 -DENABLED_LOCAL_INFILE=1 -DDEFAULT_COLLATION=utf8_general_ci -DWITH_MYISAM_STORAGE_ENGINE=1 -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_DEBUG=0
 Result: True
 Comment: unless execution succeeded
 Started: 10:09:25.939461
 Duration: 8.829 ms
 Changes: ----------
 ID: mysql_commpile
 Function: cmd.run
 Name: make
 Result: True
 Comment: unless execution succeeded
 Started: 10:09:25.948913
 Duration: 5.983 ms
 Changes: ----------
 ID: mysql_commpile
 Function: cmd.run
 Name: make install
 Result: True
 Comment: unless execution succeeded
 Started: 10:09:25.955397
 Duration: 6.567 ms
 Changes: ----------
 ID: mysql_cnf
 Function: file.managed
 Name: /etc/my.cnf
 Result: True
 Comment: File /etc/my.cnf is in the correct state
 Started: 10:09:25.962408
 Duration: 4.894 ms
 Changes: ----------
 ID: salt://mysql/files/conf.sh
 Function: cmd.script
 Result: True
 Comment: Command 'salt://mysql/files/conf.sh' run
 Started: 10:09:25.967900
 Duration: 200740.417 ms
 Changes: 
 ----------
 pid:
 28643
 retcode:
 0
 stderr:
 2018-07-24 10:09:26 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
 2018-07-24 10:09:26 0 [Note] Ignoring --secure-file-priv value as server is running with --bootstrap.
 2018-07-24 10:09:26 0 [Note] /usr/local/mysql//bin/mysqld (mysqld 5.6.40) starting as process 28647 ...
 2018-07-24 10:11:06 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
 2018-07-24 10:11:06 0 [Note] Ignoring --secure-file-priv value as server is running with --bootstrap.
 2018-07-24 10:11:06 0 [Note] /usr/local/mysql//bin/mysqld (mysqld 5.6.40) starting as process 28744 ...
 stdout:
 Installing MySQL system tables...OK

 Filling help tables...OK

 To start mysqld at boot time you have to copy
 support-files/mysql.server to the right place for your system

 PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
 To do so, start the server, then issue the following commands:

 /usr/local/mysql//bin/mysqladmin -u root password 'new-password'
 /usr/local/mysql//bin/mysqladmin -u root -h 192.168.253.62 password 'new-password'

 Alternatively you can run:

 /usr/local/mysql//bin/mysql_secure_installation

 which will also give you the option of removing the test
 databases and anonymous user created by default. This is
 strongly recommended for production servers.

 See the manual for more instructions.

 You can start the MySQL daemon with:

 cd . ; /usr/local/mysql//bin/mysqld_safe &

 You can test the MySQL daemon with mysql-test-run.pl

 cd mysql-test ; perl mysql-test-run.pl

 Please report any problems at http://bugs.mysql.com/

 The latest information about MySQL is available on the web at

 http://www.mysql.com

 Support MySQL by buying support/licenses at http://shop.mysql.com

 WARNING: Found existing config file /usr/local/mysql//my.cnf on the system.
 Because this file might be in use, it was not replaced,
 but was used in bootstrap (unless you used --defaults-file)
 and when you later start the server.
 The new default config file was created as /usr/local/mysql//my-new.cnf,
 please compare it with your file and take the changes you need.

 WARNING: Default config file /etc/my.cnf exists on the system
 This file will be read by default by the MySQL server
 If you do not want to use this, either remove it, or use the
 --defaults-file argument to mysqld_safe when starting the server ----------
 ID: salt://mysql/files/mysqllns.sh
 Function: cmd.script
 Result: False
 Comment: Command 'salt://mysql/files/mysqllns.sh' run
 Started: 10:12:46.710451
 Duration: 262.12 ms
 Changes: 
 ----------
 pid:
 28810
 retcode:
 1
 stderr:
 ln: creating symbolic link `/usr/bin/mysql': File exists
 ln: creating symbolic link `/usr/bin/mysqladmin': File exists
 ln: creating symbolic link `/usr/bin/mysqldump': File exists
 mkdir: cannot create directory `/var/lib/mysql': File exists
 ln: creating symbolic link `/var/lib/mysql/mysql.sock': File exists
 stdout: ----------
 ID: mysql_service
 Function: file.managed
 Name: /etc/init.d/mysqld
 Result: True
 Comment: File /etc/init.d/mysqld is in the correct state
 Started: 10:12:46.972872
 Duration: 4.936 ms
 Changes: ----------
 ID: mysql_service
 Function: cmd.run
 Name: /sbin/chkconfig --add mysqld
 Result: True
 Comment: unless execution succeeded
 Started: 10:12:46.977924
 Duration: 10.248 ms
 Changes: ----------
 ID: mysql_service
 Function: cmd.run
 Name: /sbin/chkconfig --level 35 mysqld on
 Result: True
 Comment: unless execution succeeded
 Started: 10:12:46.988419
 Duration: 6.787 ms
 Changes: ----------
 ID: mysql_service
 Function: service.running
 Name: mysql
 Result: True
 Comment: Service mysql has been enabled, and is in the desired state
 Started: 10:12:46.995438
 Duration: 236.708 ms
 Changes: 
 ----------
 mysql:
 True

Summary -------------
Succeeded: 13 (changed=3)
Failed: 1 -------------
Total states run:     14

安装wordpress

流程

• 将wordpress解压至/var/www/html
• 修改wp-config.php并发送至从机
• 在数据库新建名为wordpress数据库

wp-install.sls

# copy tar.gz
wordpress_source:
  file.managed:
    - name: /home/wordpress-yang.tar.gz     - unless: test -e /home/wordpress-yang.tar.gz     - source: salt://wordpress/files/wordpress-yang.tar.gz 
#tar source
extract_wordpress:
  cmd.run:
    - cwd: /home     - names:         - tar xf wordpress-yang.tar.gz         - chown root:root /home/wordpress -R     - unless: test -d /home/wordpress     - require:         - file: wordpress_source 
# move to www
move_wordpress:
  cmd.run:
    - cwd: /home     - names:       - cp -rf wordpress/* /var/www/html/       - mysql -uroot -e "create database IF NOT EXISTS wordpress"     - require:       - cmd: extract_wordpress

运行结果

[root@hadoop1 base]# salt 'hadoop2' state.sls wordpress.wp-install
hadoop2: ----------
 ID: wordpress_source
 Function: file.managed
 Name: /home/wordpress-yang.tar.gz
 Result: True
 Comment: unless execution succeeded
 Started: 10:22:32.580145
 Duration: 1199.729 ms
 Changes: ----------
 ID: extract_wordpress
 Function: cmd.run
 Name: tar xf wordpress-yang.tar.gz
 Result: True
 Comment: unless execution succeeded
 Started: 10:22:33.780510
 Duration: 9.208 ms
 Changes: ----------
 ID: extract_wordpress
 Function: cmd.run
 Name: chown root:root /home/wordpress -R
 Result: True
 Comment: unless execution succeeded
 Started: 10:22:33.790282
 Duration: 6.722 ms
 Changes: ----------
 ID: move_wordpress
 Function: cmd.run
 Name: cp -rf wordpress/* /var/www/html/
 Result: True
 Comment: Command "cp -rf wordpress/* /var/www/html/" run
 Started: 10:22:33.797953
 Duration: 2065.049 ms
 Changes: 
 ----------
 pid:
 29030
 retcode:
 0
 stderr:
 stdout: ----------
 ID: move_wordpress
 Function: cmd.run
 Name: mysql -uroot -e "create database IF NOT EXISTS wordpress"
 Result: True
 Comment: Command "mysql -uroot -e "create database IF NOT EXISTS wordpress"" run
 Started: 10:22:35.863419
 Duration: 53.151 ms
 Changes: 
 ----------
 pid:
 29033
 retcode:
 0
 stderr:
 stdout:

Summary ------------
Succeeded: 5 (changed=2)
Failed: 0 ------------
Total states run:     5

总结

成功通过master的saltstack为slave安装lamp环境，并将openstack部署成功。

遇到的问题整理

无法运行state.sls

报错the function “state.highstate” is running as PID xxxx
kill掉slave的进程

php admin不显示网页

重启apache
sudo service httpd restart

MySQL启动报：[ERROR] The server quit without updating PID file

https://blog.csdn.net/ydyang1126/article/details/72473828

cmd.run的顺序控制

https://www.jianshu.com/p/624b9cf51c64

查看顺序
salt ‘hadoop2’ state.show_sls wordpress.wp-install
order越小越早执行，-1是最后

Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock’ (2)

http://aiezu.com/article/mysql_cant_connect_through_socket.html

service mysqld does not support chkconfig

https://www.cnblogs.com/niocai/archive/2012/07/12/2587780.html

unless命令

http://blog.51cto.com/732233048/1677843

unless：用于检查的命令，仅当unless选项指向的命令返回false时才执行name指向的命令

若minion端不存在/tmp/nginx-1.4.5.tar.gz这个文件，才会执行这个file模块

参考

https://www.jianshu.com/p/624b9cf51c64
https://blog.csdn.net/chengyuqiang/article/details/78119322
https://www.linuxidc.com/Linux/2017-12/149615.html
http://www.cnblogs.com/xiewenming/p/7674806.html