MariaDB – Adamhuan's Data Center – 【逻辑驱动数据】

MariaDB Galera – error：Job for mariadb.service failed because the control process exited with error code.

报错如题所示，具体如下：

[root@mariadb-cluster-3 ~]# service mysql start
Starting mysql (via systemctl):  Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.
                                                           [FAILED]
[root@mariadb-cluster-3 ~]#

[root@mariadb-cluster-3 ~]# service mysql start

Starting mysql (via systemctl): Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.

[FAILED]

[root@mariadb-cluster-3 ~]#

日志：

Dec 22 10:02:41 mariadb-cluster-3 systemd: Starting MariaDB database server...
Dec 22 10:02:43 mariadb-cluster-3 sh: WSREP: Failed to start mysqld for wsrep recovery: '2017-12-22 10:02:41 140276076894336 [Note] /usr/sbin/mysqld (mysqld 10.2.11-MariaDB) starting as process 20861 ...
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Uses event mutexes
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Compressed tables use zlib 1.2.7
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Using Linux native AIO
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Number of pools: 1
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Using SSE2 crc32 instructions
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Completed initialization of buffer pool
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140275306231552 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [ERROR] InnoDB: Header page consists of zero bytes in datafile: ./ibdata1, Space ID:0, Flags: 0. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [ERROR] InnoDB: Corrupted page [page id: space=0, page number=0] of datafile './ibdata1' could not be found in the doublewrite buffer.
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [Note] InnoDB: Starting shutdown...
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Plugin 'InnoDB' init function returned error.
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [Note] Plugin 'FEEDBACK' is disabled.
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Could not open mysql.plugin table. Some plugins may be not loaded
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Unknown/unsupported storage engine: InnoDB
Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Aborting'
Dec 22 10:02:43 mariadb-cluster-3 systemd: mariadb.service: control process exited, code=exited status=1
Dec 22 10:02:43 mariadb-cluster-3 systemd: Failed to start MariaDB database server.
Dec 22 10:02:43 mariadb-cluster-3 systemd: Unit mariadb.service entered failed state.
Dec 22 10:02:43 mariadb-cluster-3 systemd: mariadb.service failed.

Dec 22 10:02:41 mariadb-cluster-3 systemd: Starting MariaDB database server...

Dec 22 10:02:43 mariadb-cluster-3 sh: WSREP: Failed to start mysqld for wsrep recovery: '2017-12-22 10:02:41 140276076894336 [Note] /usr/sbin/mysqld (mysqld 10.2.11-MariaDB) starting as process 20861 ...

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Uses event mutexes

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Compressed tables use zlib 1.2.7

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Using Linux native AIO

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Number of pools: 1

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Using SSE2 crc32 instructions

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [Note] InnoDB: Completed initialization of buffer pool

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140275306231552 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [ERROR] InnoDB: Header page consists of zero bytes in datafile: ./ibdata1, Space ID:0, Flags: 0. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting-datadict.html for how to resolve the issue.

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [ERROR] InnoDB: Corrupted page [page id: space=0, page number=0] of datafile './ibdata1' could not be found in the doublewrite buffer.

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:41 140276076894336 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [Note] InnoDB: Starting shutdown...

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Plugin 'InnoDB' init function returned error.

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [Note] Plugin 'FEEDBACK' is disabled.

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Could not open mysql.plugin table. Some plugins may be not loaded

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Unknown/unsupported storage engine: InnoDB

Dec 22 10:02:43 mariadb-cluster-3 sh: 2017-12-22 10:02:42 140276076894336 [ERROR] Aborting'

Dec 22 10:02:43 mariadb-cluster-3 systemd: mariadb.service: control process exited, code=exited status=1

Dec 22 10:02:43 mariadb-cluster-3 systemd: Failed to start MariaDB database server.

Dec 22 10:02:43 mariadb-cluster-3 systemd: Unit mariadb.service entered failed state.

Dec 22 10:02:43 mariadb-cluster-3 systemd: mariadb.service failed.

原因：配置文件写的有问题。

先移动文件：

[root@mariadb-cluster-3 mysql]# mkdir /backup/mysql_20171222
[root@mariadb-cluster-3 mysql]# 
[root@mariadb-cluster-3 mysql]# pwd
/var/lib/mysql
[root@mariadb-cluster-3 mysql]# ll
total 159764
-rw-rw---- 1 mysql mysql    16384 Dec 22 10:02 aria_log.00000001
-rw-rw---- 1 mysql mysql       52 Dec 22 10:02 aria_log_control
-rw-rw---- 1 mysql mysql 12582912 Dec 22 01:29 ibdata1
-rw-r----- 1 mysql mysql 50331648 Dec 22 01:49 ib_logfile0
-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile1
-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile101
-rw-rw---- 1 mysql mysql        0 Dec 22 01:21 sst_in_progress
[root@mariadb-cluster-3 mysql]# 
[root@mariadb-cluster-3 mysql]# ls -ltr ib*
-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile1
-rw-rw---- 1 mysql mysql 12582912 Dec 22 01:29 ibdata1
-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile101
-rw-r----- 1 mysql mysql 50331648 Dec 22 01:49 ib_logfile0
[root@mariadb-cluster-3 mysql]# 
[root@mariadb-cluster-3 mysql]# mv ib* /backup/mysql_20171222/
[root@mariadb-cluster-3 mysql]#

[root@mariadb-cluster-3 mysql]# mkdir /backup/mysql_20171222

[root@mariadb-cluster-3 mysql]#

[root@mariadb-cluster-3 mysql]# pwd

/var/lib/mysql

[root@mariadb-cluster-3 mysql]# ll

total 159764

-rw-rw---- 1 mysql mysql 16384 Dec 22 10:02 aria_log.00000001

-rw-rw---- 1 mysql mysql 52 Dec 22 10:02 aria_log_control

-rw-rw---- 1 mysql mysql 12582912 Dec 22 01:29 ibdata1

-rw-r----- 1 mysql mysql 50331648 Dec 22 01:49 ib_logfile0

-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile1

-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile101

-rw-rw---- 1 mysql mysql 0 Dec 22 01:21 sst_in_progress

[root@mariadb-cluster-3 mysql]#

[root@mariadb-cluster-3 mysql]# ls -ltr ib*

-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile1

-rw-rw---- 1 mysql mysql 12582912 Dec 22 01:29 ibdata1

-rw-rw---- 1 mysql mysql 50331648 Dec 22 01:29 ib_logfile101

-rw-r----- 1 mysql mysql 50331648 Dec 22 01:49 ib_logfile0

[root@mariadb-cluster-3 mysql]#

[root@mariadb-cluster-3 mysql]# mv ib* /backup/mysql_20171222/

[root@mariadb-cluster-3 mysql]#

[……]

Galera error – WSREP: Failed to start mysqld for wsrep recovery: ‘2017-12-22 1:19:58 140304668907648 [Note] /usr/sbin/mysqld (mysqld 10.2.11-MariaDB) starting as process 3777 …

具体日志如下：

Dec 22 01:19:49 mariadb-cluster-3 systemd: Starting MariaDB database server...
Dec 22 01:20:01 mariadb-cluster-3 sh: WSREP: Failed to start mysqld for wsrep recovery: '2017-12-22  1:19:58 140304668907648 [Note] /usr/sbin/mysqld (mysqld 10.2.11-MariaDB) starting as process 3777 ...
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:19:59 140304668907648 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:19:59 140304668907648 [Note] InnoDB: Uses event mutexes
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:19:59 140304668907648 [Note] InnoDB: Compressed tables use zlib 1.2.7
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:19:59 140304668907648 [Note] InnoDB: Using Linux native AIO
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:19:59 140304668907648 [Note] InnoDB: Number of pools: 1
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:19:59 140304668907648 [Note] InnoDB: Using SSE2 crc32 instructions
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [Note] InnoDB: Completed initialization of buffer pool
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140303903000320 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [ERROR] InnoDB: The innodb_system data file 'ibdata1' must be writable
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [ERROR] InnoDB: The innodb_system data file 'ibdata1' must be writable
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [Note] InnoDB: Starting shutdown...
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [ERROR] Plugin 'InnoDB' init function returned error.
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [Note] Plugin 'FEEDBACK' is disabled.
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:00 140304668907648 [ERROR] Could not open mysql.plugin table. Some plugins may be not loaded
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:01 140304668907648 [ERROR] Unknown/unsupported storage engine: InnoDB
Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22  1:20:01 140304668907648 [ERROR] Aborting'
Dec 22 01:20:01 mariadb-cluster-3 systemd: mariadb.service: control process exited, code=exited status=1
Dec 22 01:20:01 mariadb-cluster-3 systemd: Failed to start MariaDB database server.
Dec 22 01:20:01 mariadb-cluster-3 systemd: Unit mariadb.service entered failed state.
Dec 22 01:20:01 mariadb-cluster-3 systemd: mariadb.service failed.
Dec 22 01:20:01 mariadb-cluster-3 systemd: Started Session 71 of user root.
Dec 22 01:20:01 mariadb-cluster-3 systemd: Starting Session 71 of user root.

Dec 22 01:19:49 mariadb-cluster-3 systemd: Starting MariaDB database server...

Dec 22 01:20:01 mariadb-cluster-3 sh: WSREP: Failed to start mysqld for wsrep recovery: '2017-12-22 1:19:58 140304668907648 [Note] /usr/sbin/mysqld (mysqld 10.2.11-MariaDB) starting as process 3777 ...

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:19:59 140304668907648 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:19:59 140304668907648 [Note] InnoDB: Uses event mutexes

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:19:59 140304668907648 [Note] InnoDB: Compressed tables use zlib 1.2.7

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:19:59 140304668907648 [Note] InnoDB: Using Linux native AIO

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:19:59 140304668907648 [Note] InnoDB: Number of pools: 1

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:19:59 140304668907648 [Note] InnoDB: Using SSE2 crc32 instructions

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [Note] InnoDB: Completed initialization of buffer pool

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140303903000320 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [ERROR] InnoDB: The innodb_system data file 'ibdata1' must be writable

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [ERROR] InnoDB: Plugin initialization aborted with error Generic error

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [Note] InnoDB: Starting shutdown...

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [ERROR] Plugin 'InnoDB' init function returned error.

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [Note] Plugin 'FEEDBACK' is disabled.

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:00 140304668907648 [ERROR] Could not open mysql.plugin table. Some plugins may be not loaded

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:01 140304668907648 [ERROR] Unknown/unsupported storage engine: InnoDB

Dec 22 01:20:01 mariadb-cluster-3 sh: 2017-12-22 1:20:01 140304668907648 [ERROR] Aborting'

Dec 22 01:20:01 mariadb-cluster-3 systemd: mariadb.service: control process exited, code=exited status=1

Dec 22 01:20:01 mariadb-cluster-3 systemd: Failed to start MariaDB database server.

Dec 22 01:20:01 mariadb-cluster-3 systemd: Unit mariadb.service entered failed state.

Dec 22 01:20:01 mariadb-cluster-3 systemd: mariadb.service failed.

Dec 22 01:20:01 mariadb-cluster-3 systemd: Started Session 71 of user root.

Dec 22 01:20:01 mariadb-cluster-3 systemd: Starting Session 71 of user root.

文件系统权限的问题：【must be writable】

修正权限：

[root@mariadb-cluster-3 ~]# chown -R mysql.mysql /var/lib/mysql/
[root@mariadb-cluster-3 ~]#

1 2	[root@mariadb-cluster-3 ~]# chown -R mysql.mysql /var/lib/mysql/ [root@mariadb-cluster-3 ~]#

然后再启动就没问题了。
———————————————[……]

MariaDB Galera – error：Job for mariadb.service failed because a timeout was exceeded.

在集群节点服务器上以集群的方式启动数据库可能出现如题所示的错误，具体如下：

[root@mariadb-cluster-2 ~]# service mysql start
Starting mysql (via systemctl):  
Job for mariadb.service failed because a timeout was exceeded. See "systemctl status mariadb.service" and "journalctl -xe" for details.
                                                           [FAILED]
[root@mariadb-cluster-2 ~]#

[root@mariadb-cluster-2 ~]# service mysql start

Starting mysql (via systemctl):

Job for mariadb.service failed because a timeout was exceeded. See "systemctl status mariadb.service" and "journalctl -xe" for details.

[FAILED]

[root@mariadb-cluster-2 ~]#

日志：

Dec 21 20:36:55 mariadb-cluster-2 systemd: Starting MariaDB database server...
Dec 21 20:38:25 mariadb-cluster-2 systemd: mariadb.service start-pre operation timed out. Terminating.
Dec 21 20:38:25 mariadb-cluster-2 systemd: Failed to start MariaDB database server.
Dec 21 20:38:25 mariadb-cluster-2 systemd: Unit mariadb.service entered failed state.
Dec 21 20:38:25 mariadb-cluster-2 systemd: mariadb.service failed.

Dec 21 20:36:55 mariadb-cluster-2 systemd: Starting MariaDB database server...

Dec 21 20:38:25 mariadb-cluster-2 systemd: mariadb.service start-pre operation timed out. Terminating.

Dec 21 20:38:25 mariadb-cluster-2 systemd: Failed to start MariaDB database server.

Dec 21 20:38:25 mariadb-cluster-2 systemd: Unit mariadb.service entered failed state.

Dec 21 20:38:25 mariadb-cluster-2 systemd: mariadb.service failed.

有可能是系统防火墙的干扰：
[crayon-5bee8fd60[……]

MariaDB Galera – error：[ERROR] WSREP: Failed to prepare for ‘xtrabackup-v2’ SST. Unrecoverable.

如题所示的错误，发生在，在集群节点服务器上以集群的方式启动数据库，具体如下：

[root@mariadb-cluster-2 ~]# service mysql start
Starting mysql (via systemctl):  Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.
                                                           [FAILED]
[root@mariadb-cluster-2 ~]#

[root@mariadb-cluster-2 ~]# service mysql start

Starting mysql (via systemctl): Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.

[FAILED]

[root@mariadb-cluster-2 ~]#

日志：

Dec 21 19:57:47 mariadb-cluster-2 mysqld: WSREP_SST: [INFO] Streaming with xbstream (20171221 19:57:47.901)
Dec 21 19:57:47 mariadb-cluster-2 mysqld: WSREP_SST: [INFO] Using socat as streamer (20171221 19:57:47.904)
Dec 21 19:57:47 mariadb-cluster-2 mysqld: which: no socat in (/usr/sbin:/sbin:/usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)
Dec 21 19:57:47 mariadb-cluster-2 mysqld: WSREP_SST: [ERROR] socat not found in path: /usr/sbin:/sbin:/usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin (20171221 19:57:47.908)
Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258381383424 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.158.1.174' --datadir '/var/lib/mysql/'   --parent '5001'  ''
Dec 21 19:57:47 mariadb-cluster-2 mysqld: Read: '(null)'
Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258381383424 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.158.1.174' --datadir '/var/lib/mysql/'   --parent '5001'  '' : 2 (No such file or directory)
Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258602903296 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258602903296 [ERROR] Aborting
Dec 21 19:58:01 mariadb-cluster-2 mysqld: Error in my_thread_global_end(): 1 threads didn't exit
Dec 21 19:58:01 mariadb-cluster-2 systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE
Dec 21 19:58:01 mariadb-cluster-2 systemd: Failed to start MariaDB database server.
Dec 21 19:58:01 mariadb-cluster-2 systemd: Unit mariadb.service entered failed state.
Dec 21 19:58:01 mariadb-cluster-2 systemd: mariadb.service failed.

Dec 21 19:57:47 mariadb-cluster-2 mysqld: WSREP_SST: [INFO] Streaming with xbstream (20171221 19:57:47.901)

Dec 21 19:57:47 mariadb-cluster-2 mysqld: WSREP_SST: [INFO] Using socat as streamer (20171221 19:57:47.904)

Dec 21 19:57:47 mariadb-cluster-2 mysqld: which: no socat in (/usr/sbin:/sbin:/usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)

Dec 21 19:57:47 mariadb-cluster-2 mysqld: WSREP_SST: [ERROR] socat not found in path: /usr/sbin:/sbin:/usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin (20171221 19:57:47.908)

Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258381383424 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.158.1.174' --datadir '/var/lib/mysql/' --parent '5001' ''

Dec 21 19:57:47 mariadb-cluster-2 mysqld: Read: '(null)'

Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258381383424 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.158.1.174' --datadir '/var/lib/mysql/' --parent '5001' '' : 2 (No such file or directory)

Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258602903296 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.

Dec 21 19:57:47 mariadb-cluster-2 mysqld: 2017-12-21 19:57:47 140258602903296 [ERROR] Aborting

Dec 21 19:58:01 mariadb-cluster-2 mysqld: Error in my_thread_global_end(): 1 threads didn't exit

Dec 21 19:58:01 mariadb-cluster-2 systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE

Dec 21 19:58:01 mariadb-cluster-2 systemd: Failed to start MariaDB database server.

Dec 21 19:58:01 mariadb-cluster-2 systemd: Unit mariadb.service entered failed state.

Dec 21 19:58:01 mariadb-cluster-2 systemd: mariadb.service failed.

原因，需要安装socat软件包。

该软件包默认Linux不支[……]

MariaDB Galera – error：[ERROR] WSREP: gcs/src/gcs_gcomm.cpp:gcomm_open():791: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)

具体报错如下：

[root@mariadb-cluster-1 ~]# service mysql start
Starting mysql (via systemctl):  Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.
                                                           [FAILED]
[root@mariadb-cluster-1 ~]#

[root@mariadb-cluster-1 ~]# service mysql start

Starting mysql (via systemctl): Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.

[FAILED]

[root@mariadb-cluster-1 ~]#

日志：/var/log/messages

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/evs_proto.hpp:close():290: evs::proto(bf49540b, OPERATIONAL, view_id(REG,bf49540b,1)) closing in state 5
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_view():617: bf49540b view(view_id(TRANS,bf49540b,1) memb {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: })
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_trans():519: bf49540b
Dec 21 16:49:15 mariadb-cluster-1 mysqld: current view view(view_id(REG,bf49540b,1) memb {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: })
Dec 21 16:49:15 mariadb-cluster-1 mysqld: next view view(view_id(TRANS,bf49540b,1) memb {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: })
Dec 21 16:49:15 mariadb-cluster-1 mysqld: pc view view(view_id(NON_PRIM,bf49540b,1) memb {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {
Dec 21 16:49:15 mariadb-cluster-1 mysqld: })
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_trans():551: bf49540b quorum ok
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:shift_to():388: bf49540b shift_to: NON_PRIM -> TRANS prim 0 last prim view_id(NON_PRIM,00000000,0) to_seq -1
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_view():617: bf49540b view((empty))
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:shift_to():388: bf49540b shift_to: TRANS -> NON_PRIM prim 0 last prim view_id(NON_PRIM,00000000,0) to_seq -1
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] WSREP: gcomm/src/pc_proto.cpp:deliver_view():229: view((empty))
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcs/src/gcs_gcomm.cpp:handle_up():440: handle_up: self leave
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/gmcast.cpp:handle_stable_view():1508: GMCast::handle_stable_view: view((empty))
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/gmcast.cpp:handle_stable_view():1613: proto: v=0,lu=bf49540b,ru=00000000,ls=0,rs=0,la=tcp://0.0.0.0:4567,ra=tcp://10.158.1.174:4567,mc=,gn=wsrep_cluster,ch=0,st=HANDSHAKE_WAIT,pr=0,tp=0x55d41b5c0de0,ts=110542285158
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:shift_to():388: bf49540b shift_to: NON_PRIM -> CLOSED prim 0 last prim view_id(NON_PRIM,00000000,0) to_seq -1
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/gmcast.cpp:close():336: gmcast bf49540b close
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:~AsioTcpSocket():32: dtor for 0x55d41b5b0278
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:close():223: closing 0x55d41b5c0e88 state 1 send_q size 0
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/protonet.cpp:erase():26: erase pstack 0x55d41b5a2a78
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: gcs/src/gcs_gcomm.cpp:gcomm_open():791: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)
Dec 21 16:49:15 mariadb-cluster-1 mysqld: at gcomm/src/pc.cpp:connect():158
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():208: Failed to open backend connection: -110 (Connection timed out)
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:~AsioTcpSocket():32: dtor for 0x55d41b5d2508
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:~AsioTcpSocket():32: dtor for 0x55d41b5c0e88
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: gcs/src/gcs.cpp:gcs_open():1458: Failed to open channel 'wsrep_cluster' at 'gcomm://10.158.1.113,10.158.1.174,10.158.1.178': -110 (Connection timed out)
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: galera/src/replicator_smm.cpp:connect():317: gcs connect failed: Connection timed out
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: wsrep::connect(gcomm://10.158.1.113,10.158.1.174,10.158.1.178) failed: 7
Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] Aborting
Dec 21 16:49:16 mariadb-cluster-1 systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE
Dec 21 16:49:16 mariadb-cluster-1 systemd: Failed to start MariaDB database server.
Dec 21 16:49:16 mariadb-cluster-1 systemd: Unit mariadb.service entered failed state.
Dec 21 16:49:16 mariadb-cluster-1 systemd: mariadb.service failed.

Dec 21 16:50:01 mariadb-cluster-1 systemd: Started Session 4 of user root.
Dec 21 16:50:01 mariadb-cluster-1 systemd: Starting Session 4 of user root.

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/evs_proto.hpp:close():290: evs::proto(bf49540b, OPERATIONAL, view_id(REG,bf49540b,1)) closing in state 5

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_view():617: bf49540b view(view_id(TRANS,bf49540b,1) memb {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: })

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_trans():519: bf49540b

Dec 21 16:49:15 mariadb-cluster-1 mysqld: current view view(view_id(REG,bf49540b,1) memb {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: })

Dec 21 16:49:15 mariadb-cluster-1 mysqld: next view view(view_id(TRANS,bf49540b,1) memb {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: })

Dec 21 16:49:15 mariadb-cluster-1 mysqld: pc view view(view_id(NON_PRIM,bf49540b,1) memb {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: bf49540b,0

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } joined {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } left {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: } partitioned {

Dec 21 16:49:15 mariadb-cluster-1 mysqld: })

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_trans():551: bf49540b quorum ok

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:shift_to():388: bf49540b shift_to: NON_PRIM -> TRANS prim 0 last prim view_id(NON_PRIM,00000000,0) to_seq -1

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:handle_view():617: bf49540b view((empty))

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:shift_to():388: bf49540b shift_to: TRANS -> NON_PRIM prim 0 last prim view_id(NON_PRIM,00000000,0) to_seq -1

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] WSREP: gcomm/src/pc_proto.cpp:deliver_view():229: view((empty))

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcs/src/gcs_gcomm.cpp:handle_up():440: handle_up: self leave

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/gmcast.cpp:handle_stable_view():1508: GMCast::handle_stable_view: view((empty))

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/gmcast.cpp:handle_stable_view():1613: proto: v=0,lu=bf49540b,ru=00000000,ls=0,rs=0,la=tcp://0.0.0.0:4567,ra=tcp://10.158.1.174:4567,mc=,gn=wsrep_cluster,ch=0,st=HANDSHAKE_WAIT,pr=0,tp=0x55d41b5c0de0,ts=110542285158

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/pc_proto.cpp:shift_to():388: bf49540b shift_to: NON_PRIM -> CLOSED prim 0 last prim view_id(NON_PRIM,00000000,0) to_seq -1

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/gmcast.cpp:close():336: gmcast bf49540b close

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:~AsioTcpSocket():32: dtor for 0x55d41b5b0278

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:close():223: closing 0x55d41b5c0e88 state 1 send_q size 0

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/protonet.cpp:erase():26: erase pstack 0x55d41b5a2a78

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: gcs/src/gcs_gcomm.cpp:gcomm_open():791: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)

Dec 21 16:49:15 mariadb-cluster-1 mysqld: at gcomm/src/pc.cpp:connect():158

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():208: Failed to open backend connection: -110 (Connection timed out)

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:~AsioTcpSocket():32: dtor for 0x55d41b5d2508

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [Note] [Debug] WSREP: gcomm/src/asio_tcp.cpp:~AsioTcpSocket():32: dtor for 0x55d41b5c0e88

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: gcs/src/gcs.cpp:gcs_open():1458: Failed to open channel 'wsrep_cluster' at 'gcomm://10.158.1.113,10.158.1.174,10.158.1.178': -110 (Connection timed out)

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: galera/src/replicator_smm.cpp:connect():317: gcs connect failed: Connection timed out

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] WSREP: wsrep::connect(gcomm://10.158.1.113,10.158.1.174,10.158.1.178) failed: 7

Dec 21 16:49:15 mariadb-cluster-1 mysqld: 2017-12-21 16:49:15 140612430137472 [ERROR] Aborting

Dec 21 16:49:16 mariadb-cluster-1 systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE

Dec 21 16:49:16 mariadb-cluster-1 systemd: Failed to start MariaDB database server.

Dec 21 16:49:16 mariadb-cluster-1 systemd: Unit mariadb.service entered failed state.

Dec 21 16:49:16 mariadb-cluster-1 systemd: mariadb.service failed.

Dec 21 16:50:01 mariadb-cluster-1 systemd: Started Session 4 of user root.

Dec 21 16:50:01 mariadb-cluster-1 systemd: Starting Session 4 of user root.

原因：第一次启动集群的时候，应该用以下两个中的一个：
1. galera_new_clust[……]