Cloudera Manager + CDH 6.1.1：Install on OEL 7.6
参考文档：
安装前
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/installation_reqts.html
安装时
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/install_cm_cdh.html
安装后与排错
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/cm_ig_troubleshooting.html
————————————————
一、安装前准备：
数据库部分：
这里，我选择MySQL的5.7的版本，数据库厂商选择：Percona。
上传安装介质，并准备好YUM环境：
上传，并解压：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
[root@c65 ~]# cd /software [root@c65 software]# ls [root@c65 software]# [root@c65 software]# ls -ltr total 286052 -rw-r--r-- 1 root root 292915200 Oct 17 10:45 Percona-Server-5.7.23-23-r500fcf5-el7-x86_64-bundle.tar [root@c65 software]# [root@c65 software]# tar -xf Percona-Server-5.7.23-23-r500fcf5-el7-x86_64-bundle.tar [root@c65 software]# [root@c65 software]# ls -ltr total 572108 -rw-rw-r-- 1 root root 201542592 Sep 4 03:44 Percona-Server-57-debuginfo-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 7163764 Sep 4 03:44 Percona-Server-client-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 10907916 Sep 4 03:44 Percona-Server-rocksdb-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 1116096 Sep 4 03:44 Percona-Server-devel-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 766260 Sep 4 03:44 Percona-Server-shared-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 40724340 Sep 4 03:44 Percona-Server-server-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 1301156 Sep 4 03:44 Percona-Server-shared-compat-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 27621540 Sep 4 03:45 Percona-Server-test-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 1759572 Sep 4 03:45 Percona-Server-tokudb-57-5.7.23-23.1.el7.x86_64.rpm -rw-r--r-- 1 root root 292915200 Oct 17 10:45 Percona-Server-5.7.23-23-r500fcf5-el7-x86_64-bundle.tar [root@c65 software]# [root@c65 software]# createrepo /software/ Spawning worker 0 with 2 pkgs Spawning worker 1 with 2 pkgs Spawning worker 2 with 2 pkgs Spawning worker 3 with 1 pkgs Spawning worker 4 with 1 pkgs Spawning worker 5 with 1 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@c65 software]# [root@c65 software]# ls -ltr total 572112 -rw-rw-r-- 1 root root 201542592 Sep 4 03:44 Percona-Server-57-debuginfo-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 7163764 Sep 4 03:44 Percona-Server-client-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 10907916 Sep 4 03:44 Percona-Server-rocksdb-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 1116096 Sep 4 03:44 Percona-Server-devel-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 766260 Sep 4 03:44 Percona-Server-shared-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 40724340 Sep 4 03:44 Percona-Server-server-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 1301156 Sep 4 03:44 Percona-Server-shared-compat-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 27621540 Sep 4 03:45 Percona-Server-test-57-5.7.23-23.1.el7.x86_64.rpm -rw-rw-r-- 1 root root 1759572 Sep 4 03:45 Percona-Server-tokudb-57-5.7.23-23.1.el7.x86_64.rpm -rw-r--r-- 1 root root 292915200 Oct 17 10:45 Percona-Server-5.7.23-23-r500fcf5-el7-x86_64-bundle.tar drwxr-xr-x 2 root root 4096 Feb 25 16:41 repodata [root@c65 software]# [root@c65 software]# |
配置YUM：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
[root@c65 software]# vi /etc/yum.repos.d/local.repo [root@c65 software]# [root@c65 software]# cat /etc/yum.repos.d/local.repo [local] NAME=local BASEURL=file:///software GPGCHECK=0 [root@c65 software]# [root@c65 software]# yum repolist Loaded plugins: langpacks, ulninfo local | 2.9 kB 00:00:00 local/primary_db | 13 kB 00:00:00 repo id repo name status base/x86_64 CentOS-7 - Base 10,019 epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 12,902 extras/x86_64 CentOS-7 - Extras 371 local local 9 ol7_UEKR5/x86_64 Latest Unbreakable Enterprise Kernel Release 5 for Oracle Linux 7Server (x86_64) 125 ol7_latest/x86_64 Oracle Linux 7Server Latest (x86_64) 12,218 updates/x86_64 CentOS-7 - Updates 1,098 repolist: 36,742 [root@c65 software]# [root@c65 software]# yum list | grep local Percona-Server-57-debuginfo.x86_64 5.7.23-23.1.el7 local Percona-Server-client-57.x86_64 5.7.23-23.1.el7 local Percona-Server-devel-57.x86_64 5.7.23-23.1.el7 local Percona-Server-rocksdb-57.x86_64 5.7.23-23.1.el7 local Percona-Server-server-57.x86_64 5.7.23-23.1.el7 local Percona-Server-shared-57.x86_64 5.7.23-23.1.el7 local Percona-Server-shared-compat-57.x86_64 5.7.23-23.1.el7 local Percona-Server-test-57.x86_64 5.7.23-23.1.el7 local Percona-Server-tokudb-57.x86_64 5.7.23-23.1.el7 local boost-locale.i686 1.53.0-27.el7 base boost-locale.x86_64 1.53.0-27.el7 base boost148-locale.x86_64 1.48.0-7.el7 epel drupal7-domain_locale.noarch 1.0-0.11.beta6.el7 epel ghc-old-locale.x86_64 1.0.0.5-26.4.el7 epel ghc-old-locale-devel.x86_64 1.0.0.5-26.4.el7 epel libstoragemgmt-local-plugin.noarch 1.6.2-4.el7 base maven-local.noarch 3.4.1-11.el7 base mintlocale.noarch 1.4.4-2.el7 epel mintlocale-set-default-locale.noarch 1.4.4-2.el7 epel mono-locale-extras.x86_64 4.6.2-4.el7 epel openlmi-scripts-locale.noarch 0.4.0-4.el7 epel perl-local-lib.noarch 1.008010-4.el7 base php-symfony-locale.noarch 2.8.12-2.el7 epel popub-local.x86_64 0-0.3.20171007git6ffa11c.el7 python-tzlocal.noarch 1.2-2.el7 epel python2-dnf-plugin-local.noarch 2.1.5-5.el7 extras yum-plugin-local.noarch 1.1.31-50.el7 base [root@c65 software]# |
注意，对于数据库的安装方式，官方有以下限制或要求的说明：
如果你选择的后端数据库是【MySQL】：
Use UTF8 encoding for all custom databases. MySQL and MariaDB must use the MySQL utf8 encoding, not utf8mb4.
字符集需要【utf8】，而非【utf8mb4】
For MySQL 5.7, you must install the MySQL-shared-compat or MySQL-shared package. This is required for the Cloudera Manager Agent installation.
如果是MySQL 5.7，那么你除了server包，还需要安装以下两个包中的一个：
1. MySQL-shared-compat
2. MySQL-shared
MySQL GTID-based replication is not supported.
MySQL的基于GTID模式的主从架构，在Cloudera的数据库中是不支持的。
GTID模式，是通过修改参数【gtid_mode】开启的：
在5.6中，改参数是静态的，修改参数需要停数据库，在5.7中，改参数已经变成了动态参数。
Hue requires the default MySQL/MariaDB version (if used) of the operating system on which it is installed. For more information, see Hue Databases.
Both the Community and Enterprise versions of MySQL are supported, as well as MySQL configured by the AWS RDS service.
MySQL的社区版和企业版都是被Cloudera支持的。
如果你选择的后端数据库是【Oracle】：
Cloudera supports Oracle Exadata and RAC instances when they serve as backend databases for CDH components without high availability. Cloudera software is architected with the assumption of a single database instance, and supports normal operations between Cloudera Enterprise and Oracle Exadata (or RAC) in such an environment. Cloudera cannot reasonably test and certify on each strategy for each RDBMS. Cloudera expects HA solutions for RDBMS to be transparent to Cloudera software, and therefore are not supported and debugged by Cloudera. It is the responsibility of the customer to provision, configure, and manage the RDBMS HA deployment, so that Cloudera software behaves exactly as it would when interfacing with a single, non-HA service. Cloudera will support and help customers troubleshoot issues when a cluster has HA enabled. While diagnosing database-related problems in Cloudera components, customers may be required to temporarily disable or bypass HA mechanisms for troubleshooting purposes. If an HA-related issue is found, it is the responsibility of the customer to engage with the database vendor so that a solution to that issue can be found.
Before upgrading from CDH 5 to CDH 6, check the value of the COMPATIBLE initialization parameter in the Oracle Database using the following SQL query:
|
1 |
SELECT name, value FROM v$parameter WHERE name = 'compatible' |
如果后端数据库是Oracle，并且你要对CDH做升级，那么升级前，你需要执行上面的SQL，查询【compatible】的相关信息。
————————————————
安装MySQL：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
[root@c65 software]# yum install Percona-Server-server-57 Loaded plugins: langpacks, ulninfo Resolving Dependencies --> Running transaction check ---> Package Percona-Server-server-57.x86_64 0:5.7.23-23.1.el7 will be installed --> Processing Dependency: Percona-Server-shared-57 for package: Percona-Server-server-57-5.7.23-23.1.el7.x86_64 --> Processing Dependency: Percona-Server-client-57 for package: Percona-Server-server-57-5.7.23-23.1.el7.x86_64 --> Running transaction check ---> Package Percona-Server-client-57.x86_64 0:5.7.23-23.1.el7 will be installed ---> Package Percona-Server-shared-57.x86_64 0:5.7.23-23.1.el7 will be installed --> Processing Dependency: Percona-Server-shared-compat-57 for package: Percona-Server-shared-57-5.7.23-23.1.el7.x86_64 --> Running transaction check ---> Package Percona-Server-shared-compat-57.x86_64 0:5.7.23-23.1.el7 will be obsoleting ---> Package mariadb-libs.x86_64 1:5.5.60-1.el7_5 will be obsoleted --> Finished Dependency Resolution Dependencies Resolved =============================================================================================================================================== Package Arch Version Repository Size =============================================================================================================================================== Installing: Percona-Server-server-57 x86_64 5.7.23-23.1.el7 local 39 M Percona-Server-shared-compat-57 x86_64 5.7.23-23.1.el7 local 1.2 M replacing mariadb-libs.x86_64 1:5.5.60-1.el7_5 Installing for dependencies: Percona-Server-client-57 x86_64 5.7.23-23.1.el7 local 6.8 M Percona-Server-shared-57 x86_64 5.7.23-23.1.el7 local 748 k Transaction Summary =============================================================================================================================================== Install 2 Packages (+2 Dependent packages) Total download size: 48 M Is this ok [y/d/N]: y Downloading packages: ----------------------------------------------------------------------------------------------------------------------------------------------- Total 166 MB/s | 48 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : Percona-Server-shared-compat-57-5.7.23-23.1.el7.x86_64 1/5 Installing : Percona-Server-shared-57-5.7.23-23.1.el7.x86_64 2/5 Installing : Percona-Server-client-57-5.7.23-23.1.el7.x86_64 3/5 Installing : Percona-Server-server-57-5.7.23-23.1.el7.x86_64 4/5 ------------- * The suggested mysql options and settings are in /etc/percona-server.conf.d/mysqld.cnf * If you want to use mysqld.cnf as default configuration file please make backup of /etc/my.cnf * Once it is done please execute the following commands: rm -rf /etc/my.cnf update-alternatives --install /etc/my.cnf my.cnf "/etc/percona-server.cnf" 200 ------------- Percona Server is distributed with several useful UDF (User Defined Function) from Percona Toolkit. Run the following commands to create these functions: mysql -e "CREATE FUNCTION fnv1a_64 RETURNS INTEGER SONAME 'libfnv1a_udf.so'" mysql -e "CREATE FUNCTION fnv_64 RETURNS INTEGER SONAME 'libfnv_udf.so'" mysql -e "CREATE FUNCTION murmur_hash RETURNS INTEGER SONAME 'libmurmur_udf.so'" See http://www.percona.com/doc/percona-server/5.7/management/udf_percona_toolkit.html for more details Erasing : 1:mariadb-libs-5.5.60-1.el7_5.x86_64 5/5 Verifying : Percona-Server-shared-compat-57-5.7.23-23.1.el7.x86_64 1/5 Verifying : Percona-Server-client-57-5.7.23-23.1.el7.x86_64 2/5 Verifying : Percona-Server-shared-57-5.7.23-23.1.el7.x86_64 3/5 Verifying : Percona-Server-server-57-5.7.23-23.1.el7.x86_64 4/5 Verifying : 1:mariadb-libs-5.5.60-1.el7_5.x86_64 5/5 Installed: Percona-Server-server-57.x86_64 0:5.7.23-23.1.el7 Percona-Server-shared-compat-57.x86_64 0:5.7.23-23.1.el7 Dependency Installed: Percona-Server-client-57.x86_64 0:5.7.23-23.1.el7 Percona-Server-shared-57.x86_64 0:5.7.23-23.1.el7 Replaced: mariadb-libs.x86_64 1:5.5.60-1.el7_5 Complete! [root@c65 software]# [root@c65 software]# |
启动服务，并设置开机启动
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
[root@c65 software]# service mysql status Redirecting to /bin/systemctl status mysql.service ● mysqld.service - MySQL Server Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled) Active: inactive (dead) Docs: man:mysqld(8) http://dev.mysql.com/doc/refman/en/using-systemd.html [root@c65 software]# [root@c65 software]# service mysql start Redirecting to /bin/systemctl start mysql.service [root@c65 software]# [root@c65 software]# service mysql status Redirecting to /bin/systemctl status mysql.service ● mysqld.service - MySQL Server Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2019-02-25 17:26:35 CST; 12s ago Docs: man:mysqld(8) http://dev.mysql.com/doc/refman/en/using-systemd.html Process: 9174 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS) Process: 9107 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS) Main PID: 9180 (mysqld) Tasks: 28 CGroup: /system.slice/mysqld.service └─9180 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid Feb 25 17:26:01 c65 systemd[1]: Starting MySQL Server... Feb 25 17:26:35 c65 systemd[1]: Started MySQL Server. [root@c65 software]# [root@c65 software]# systemctl enable mysql Failed to execute operation: Too many levels of symbolic links [root@c65 software]# [root@c65 software]# systemctl enable mysqld [root@c65 software]# [root@c65 software]# |
修改临时口令，并登陆MySQL：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
[root@c65 software]# cat /var/log/mysqld.log | grep "password" 2019-02-25T09:26:29.961215Z 1 [Note] A temporary password is generated for root@localhost: cRzK(9s3C:uJ [root@c65 software]# [root@c65 software]# mysqladmin -u root -p password '***随便看别人口令可不是好孩子***'; Enter password: mysqladmin: [Warning] Using a password on the command line interface can be insecure. Warning: Since password will be sent to server in plain text, use ssl connection to ensure password safety. [root@c65 software]# [root@c65 software]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.23-23 Percona Server (GPL), Release 23, Revision 500fcf5 Copyright (c) 2009-2018 Percona LLC and/or its affiliates Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.00 sec) mysql> |
磁盘与挂载点的推荐配置可以查看官方文档：
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/cm_ig_reqs_space.html
我这里因为是实验环境，所以为了简便：
1. 所有的cloudera程序与数据文件存放位置，根目录下的相对应的目录中
2. 根分区剩余空间在50G以上
网络相关的配置。
文件：/etc/hosts
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
[root@c61 ~]# cat /etc/hosts #127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 #::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # Local 127.0.0.1 localhost # Pub 10.158.1.99 c61 10.158.1.111 c62 10.158.1.112 c63 10.158.1.113 c64 10.158.1.116 c65 [root@c61 ~]# |
网卡配置，在配置文件中写成静态分配：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
[root@c61 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens192 TYPE="Ethernet" PROXY_METHOD="none" BROWSER_ONLY="no" BOOTPROTO="none" DEFROUTE="yes" IPV4_FAILURE_FATAL="no" IPV6INIT="yes" IPV6_AUTOCONF="yes" IPV6_DEFROUTE="yes" IPV6_FAILURE_FATAL="no" IPV6_ADDR_GEN_MODE="stable-privacy" NAME="ens192" UUID="54ab8060-6dca-4ed9-a056-c48554f31dc4" DEVICE="ens192" ONBOOT="yes" IPADDR="10.158.1.99" PREFIX="23" GATEWAY="10.158.0.1" DNS1="114.114.114.114" IPV6_PRIVACY="no" [root@c61 ~]# |
主机名的配置：
|
1 2 3 |
[root@c61 ~]# cat /etc/hostname c61 [root@c61 ~]# |
禁用防火墙与SELINUX
|
1 2 3 4 5 6 7 8 9 |
[root@c61 ~]# systemctl status firewalld ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: man:firewalld(1) [root@c61 ~]# [root@c61 ~]# sestatus SELinux status: disabled [root@c61 ~]# |
时间同步服务
1. NTP
2. 我的环境因为是在VMware上的，所以通过VMware TOOLBOX就可以了
|
1 2 3 4 5 |
[root@c61 ~]# vmware-toolbox-cmd timesync status Disabled [root@c61 ~]# vmware-toolbox-cmd timesync enable Enabled [root@c61 ~]# |
CDH 和 CM开放的端口：
配置本地的Parcel库。
安装HTTPD：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
[root@c61 ~]# yum list | grep httpd darkhttpd.x86_64 1.11-1.el7 epel httpd.x86_64 2.4.6-88.0.1.el7 ol7_latest httpd-devel.x86_64 2.4.6-88.0.1.el7 ol7_latest httpd-itk.x86_64 2.4.7.04-2.el7 epel httpd-manual.noarch 2.4.6-88.0.1.el7 ol7_latest httpd-tools.x86_64 2.4.6-88.0.1.el7 ol7_latest iipsrv-httpd-fcgi.noarch 1.0.0-6.0.el7 epel keycloak-httpd-client-install.noarch 0.6-3.el7 base libmicrohttpd.i686 0.9.33-2.el7 base libmicrohttpd.x86_64 0.9.33-2.el7 base libmicrohttpd-devel.i686 0.9.33-2.el7 base libmicrohttpd-devel.x86_64 0.9.33-2.el7 base libmicrohttpd-doc.noarch 0.9.33-2.el7 base lighttpd.x86_64 1.4.53-1.el7 epel lighttpd-fastcgi.x86_64 1.4.53-1.el7 epel lighttpd-mod_authn_gssapi.x86_64 1.4.53-1.el7 epel lighttpd-mod_authn_mysql.x86_64 1.4.53-1.el7 epel lighttpd-mod_authn_pam.x86_64 1.4.53-1.el7 epel lighttpd-mod_geoip.x86_64 1.4.53-1.el7 epel lighttpd-mod_mysql_vhost.x86_64 1.4.53-1.el7 epel mirmon-httpd.noarch 2.11-1.el7 epel nextcloud-httpd.noarch 10.0.4-2.el7 epel opensips-httpd.x86_64 1.10.5-4.el7 epel owncloud-httpd.noarch 9.1.5-1.el7 epel python2-keycloak-httpd-client-install.noarch python2-sphinxcontrib-httpdomain.noarch 1.7.0-1.el7 epel radicale-httpd.noarch 1.1.2-1.el7 epel sysusage-httpd.noarch 5.7-2.el7 epel thttpd.x86_64 2.29-2.el7 epel viewvc-httpd-fcgi.noarch 1.1.26-1.el7 epel viewvc-httpd-wsgi.noarch 1.1.26-1.el7 epel web-assets-httpd.noarch 5-1.el7 epel [root@c61 ~]# [root@c61 ~]# yum install -y httpd Loaded plugins: langpacks, ulninfo Resolving Dependencies --> Running transaction check ---> Package httpd.x86_64 0:2.4.6-88.0.1.el7 will be installed --> Processing Dependency: httpd-tools = 2.4.6-88.0.1.el7 for package: httpd-2.4.6-88.0.1.el7.x86_64 --> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-88.0.1.el7.x86_64 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-88.0.1.el7.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-88.0.1.el7.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.8-3.el7_4.1 will be installed ---> Package apr-util.x86_64 0:1.5.2-6.0.1.el7 will be installed ---> Package httpd-tools.x86_64 0:2.4.6-88.0.1.el7 will be installed ---> Package mailcap.noarch 0:2.1.41-2.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =============================================================================================================================================== Package Arch Version Repository Size =============================================================================================================================================== Installing: httpd x86_64 2.4.6-88.0.1.el7 ol7_latest 1.2 M Installing for dependencies: apr x86_64 1.4.8-3.el7_4.1 base 103 k apr-util x86_64 1.5.2-6.0.1.el7 ol7_latest 91 k httpd-tools x86_64 2.4.6-88.0.1.el7 ol7_latest 90 k mailcap noarch 2.1.41-2.el7 base 31 k Transaction Summary =============================================================================================================================================== Install 1 Package (+4 Dependent packages) Total download size: 1.5 M Installed size: 4.3 M Downloading packages: (1/5): apr-1.4.8-3.el7_4.1.x86_64.rpm | 103 kB 00:00:01 (2/5): apr-util-1.5.2-6.0.1.el7.x86_64.rpm | 91 kB 00:00:03 (3/5): httpd-2.4.6-88.0.1.el7.x86_64.rpm | 1.2 MB 00:00:04 (4/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:00:00 (5/5): httpd-tools-2.4.6-88.0.1.el7.x86_64.rpm | 90 kB 00:00:02 ----------------------------------------------------------------------------------------------------------------------------------------------- Total 227 kB/s | 1.5 MB 00:00:06 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : apr-1.4.8-3.el7_4.1.x86_64 1/5 Installing : apr-util-1.5.2-6.0.1.el7.x86_64 2/5 Installing : httpd-tools-2.4.6-88.0.1.el7.x86_64 3/5 Installing : mailcap-2.1.41-2.el7.noarch 4/5 Installing : httpd-2.4.6-88.0.1.el7.x86_64 5/5 Verifying : httpd-2.4.6-88.0.1.el7.x86_64 1/5 Verifying : apr-util-1.5.2-6.0.1.el7.x86_64 2/5 Verifying : mailcap-2.1.41-2.el7.noarch 3/5 Verifying : httpd-tools-2.4.6-88.0.1.el7.x86_64 4/5 Verifying : apr-1.4.8-3.el7_4.1.x86_64 5/5 Installed: httpd.x86_64 0:2.4.6-88.0.1.el7 Dependency Installed: apr.x86_64 0:1.4.8-3.el7_4.1 apr-util.x86_64 0:1.5.2-6.0.1.el7 httpd-tools.x86_64 0:2.4.6-88.0.1.el7 mailcap.noarch 0:2.1.41-2.el7 Complete! [root@c61 ~]# |
配置文件：/etc/httpd/conf/httpd.conf
|
1 2 3 4 5 6 7 8 9 10 |
[root@c61 ~]# vi /etc/httpd/conf/httpd.conf [root@c61 ~]# [root@c61 ~]# cat /etc/httpd/conf/httpd.conf | grep -v "#" | grep -B 5 "\.parcel" TypesConfig /etc/mime.types AddType application/x-compress .Z AddType application/x-gzip .gz .tgz .parcel [root@c61 ~]# |
如上，在【IfModule mime_module】中的【x-gzip】中，增加了【.parcel】
然后重启HTTP服务：
|
1 2 3 |
[root@c61 ~]# service httpd restart Redirecting to /bin/systemctl restart httpd.service [root@c61 ~]# |
下载官网的Parcel文件：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
[root@c61 parcel]# cd /var/www/html/ [root@c61 html]# ls -ltr total 0 [root@c61 html]# [root@c61 html]# mkdir cloudera-repos [root@c61 html]# [root@c61 html]# vi download_cloudera_parcels.sh [root@c61 html]# [root@c61 html]# ls -ltr total 4 drwxr-xr-x 2 root root 6 Feb 25 19:43 cloudera-repos -rw-r--r-- 1 root root 275 Feb 25 19:46 download_cloudera_parcels.sh [root@c61 html]# [root@c61 html]# cat download_cloudera_parcels.sh wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.1.0/parcels/ -P /var/www/html/cloudera-repos sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.1.0/parcels/ -P /var/www/html/cloudera-repos [root@c61 html]# [root@c61 html]# ls -ltr /var/www/html/cloudera-repos total 0 [root@c61 html]# [root@c61 html]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 7.9G 0 7.9G 0% /dev tmpfs 7.9G 0 7.9G 0% /dev/shm tmpfs 7.9G 9.4M 7.9G 1% /run tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup /dev/mapper/ol-root 57G 9.4G 48G 17% / /dev/sda1 1014M 212M 803M 21% /boot tmpfs 1.6G 12K 1.6G 1% /run/user/42 tmpfs 1.6G 0 1.6G 0% /run/user/0 [root@c61 html]# [root@c61 html]# nohup sh download_cloudera_parcels.sh & [1] 30347 [root@c61 html]# nohup: ignoring input and appending output to ‘nohup.out’ [root@c61 html]# [root@c61 html]# ps -ef | grep download root 30347 29197 0 19:48 pts/0 00:00:00 sh download_cloudera_parcels.sh root 30548 29197 0 19:51 pts/0 00:00:00 grep --color=auto download [root@c61 html]# [root@c61 html]# du -sh cloudera-repos/ 8.0M cloudera-repos/ [root@c61 html]# [root@c61 html]# ls -ltr cloudera-repos/ total 4 -rw-r--r-- 1 root root 836 Mar 6 2018 robots.txt drwxr-xr-x 3 root root 19 Feb 25 19:48 cdh6 [root@c61 html]# [root@c61 html]# tail -f nohup.out 5350K .......... .......... .......... .......... .......... 0% 56.3K 10h28m 5400K .......... .......... .......... .......... .......... 0% 78.5K 10h26m 5450K .......... .......... .......... .......... .......... 0% 49.2K 10h26m 5500K .......... .......... .......... .......... .......... 0% 66.0K 10h25m 5550K .......... .......... .......... .......... .......... 0% 79.0K 10h23m 5600K .......... .......... .......... .......... .......... 0% 133K 10h20m 5650K .......... .......... .......... .......... .......... 0% 98.6K 10h17m 5700K .......... .......... .......... .......... .......... 0% 98.1K 10h15m 5750K .......... .......... .......... .......... .......... 0% 79.1K 10h13m 5800K .......... .......... .......... .......... .......... 0% 66.0K 10h12m 5850K .......... .......... .......... .......... .......... 0% 65.8K 10h11m 5900K .......... .......... .......... .......... .......... 0% 56.4K 10h11m 5950K .......... .......... .......... .......... .......... 0% 29.9K 10h15m 6000K .......... .......... .......... ......... （过多的输出。） |
可以看到，脚本正在运行，执行下载。
等待下载完成。
等待一段时间后，脚本执行完成：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
[root@c61 html]# pwd /var/www/html [root@c61 html]# [root@c61 html]# ps -ef | grep download root 13428 13264 0 09:05 pts/0 00:00:00 grep --color=auto download [root@c61 html]# [root@c61 html]# ls -ltr total 12268 -rw-r--r-- 1 root root 275 Feb 25 19:46 download_cloudera_parcels.sh drwxr-xr-x 4 root root 54 Feb 26 08:01 cloudera-repos -rw------- 1 root root 12555570 Feb 26 08:05 nohup.out [root@c61 html]# [root@c61 html]# cat nohup.out | tail -n 12 Reusing existing connection to archive.cloudera.com:443. HTTP request sent, awaiting response... 200 OK Length: 1053 (1.0K) [application/json] Saving to: ‘/var/www/html/cloudera-repos/gplextras6/6.1.0/parcels/manifest.json’ 0K . 100% 70.6M=0s 2019-02-26 08:05:46 (70.6 MB/s) - ‘/var/www/html/cloudera-repos/gplextras6/6.1.0/parcels/manifest.json’ saved [1053/1053] FINISHED --2019-02-26 08:05:46-- Total wall clock time: 4m 11s Downloaded: 11 files, 12M in 4m 7s (50.3 KB/s) [root@c61 html]# [root@c61 html]# cat nohup.out | grep -B 4 "Downloaded:" 2019-02-26 08:01:35 (3.57 MB/s) - ‘/var/www/html/cloudera-repos/cdh6/6.1.0/parcels/manifest.json’ saved [28173/28173] FINISHED --2019-02-26 08:01:35-- Total wall clock time: 12h 13m 0s Downloaded: 11 files, 7.4G in 12h 12m 49s (176 KB/s) -- 2019-02-26 08:05:46 (70.6 MB/s) - ‘/var/www/html/cloudera-repos/gplextras6/6.1.0/parcels/manifest.json’ saved [1053/1053] FINISHED --2019-02-26 08:05:46-- Total wall clock time: 4m 11s Downloaded: 11 files, 12M in 4m 7s (50.3 KB/s) [root@c61 html]# [root@c61 html]# du -sh cloudera-repos/ 7.6G cloudera-repos/ [root@c61 html]# [root@c61 html]# tree /var/www/html/cloudera-repos/ /var/www/html/cloudera-repos/ ├── cdh6 │ └── 6.1.0 │ └── parcels │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-el6.parcel │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-el6.parcel.sha256 │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha256 │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-sles12.parcel │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-sles12.parcel.sha256 │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-xenial.parcel │ ├── CDH-6.1.0-1.cdh6.1.0.p0.770702-xenial.parcel.sha256 │ ├── index.html │ └── manifest.json ├── gplextras6 │ └── 6.1.0 │ └── parcels │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-el6.parcel │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-el6.parcel.sha256 │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-el7.parcel │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-el7.parcel.sha256 │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-sles12.parcel │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-sles12.parcel.sha256 │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-xenial.parcel │ ├── GPLEXTRAS-6.1.0-1.gplextras6.1.0.p0.770702-xenial.parcel.sha256 │ ├── index.html │ └── manifest.json └── robots.txt 6 directories, 21 files [root@c61 html]# |
因为配置了HTTP，所以可以在网页上访问：
http://c61/cloudera-repos/
以上是配置了CDH6的组件
在官方网站的文档中，还可以看到其他的几个组件的下载方式：
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/cm_ig_create_local_parcel_repo.html
Apache Accumulo for CDH
|
1 |
wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/parcels/1.7.2/ -P /var/www/html/cloudera-repos |
CDS Powered By Apache Spark 2 for CDH
|
1 |
wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/spark2/parcels/2.3.0.cloudera3/ -P /var/www/html/cloudera-repos |
Cloudera Navigator Key Trustee Server
Cloudera Navigator Key Trustee KMS and HSM KMS
Sqoop Connectors
|
1 |
wget --recursive --no-parent --no-host-directories http://archive.cloudera.com/sqoop-connectors/parcels/latest/ -P /var/www/html/cloudera-repos |
下载官方的RPM包：
CM6 – Cloudera Manager 6
https://archive.cloudera.com/cm6/6.1.0/redhat7/
Cloudera CDH6
https://archive.cloudera.com/cdh6/6.1.0/redhat7/
https://archive.cloudera.com/gplextras6/6.1.0/redhat7/
下载RPM后，上传服务器：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
[root@c61 software]# pwd /software [root@c61 software]# [root@c61 software]# ls -ltr total 13860 -rw-r--r--. 1 root root 1718 Feb 20 21:57 RPM-GPG-KEY-cloudera -rw-r--r--. 1 root root 14185448 Feb 20 22:05 enterprise-debuginfo-6.1.1-853290.el7.x86_64.rpm drwxr-xr-x 4 root root 29 Feb 26 09:54 cloudera-rpm [root@c61 software]# [root@c61 software]# tree cloudera-rpm/ cloudera-rpm/ ├── cdh6 │ ├── avro-doc-1.8.2+cdh6.1.1-875250.el7.noarch.rpm │ ├── avro-libs-1.8.2+cdh6.1.1-875250.el7.noarch.rpm │ ├── avro-tools-1.8.2+cdh6.1.1-875250.el7.noarch.rpm │ ├── bigtop-jsvc-1.0.10+cdh6.1.1-875250.el7.x86_64.rpm │ ├── bigtop-jsvc-debuginfo-1.0.10+cdh6.1.1-875250.el7.x86_64.rpm │ ├── bigtop-utils-0.7.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── flume-ng-1.8.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── flume-ng-agent-1.8.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── flume-ng-doc-1.8.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── hadoop-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-client-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-conf-pseudo-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-debuginfo-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-doc-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-datanode-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-fuse-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-journalnode-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-namenode-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-nfs3-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-secondarynamenode-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-hdfs-zkfc-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-httpfs-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-kms-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-kms-server-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-libhdfs-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-libhdfs-devel-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-lzo-0.4.15+cdh6.1.0-770702.el7.x86_64.rpm │ ├── hadoop-lzo-debuginfo-0.4.15+cdh6.1.0-770702.el7.x86_64.rpm │ ├── hadoop-mapreduce-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-mapreduce-historyserver-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-yarn-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-yarn-nodemanager-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-yarn-proxyserver-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-yarn-resourcemanager-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hadoop-yarn-timelinereader-3.0.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hbase-2.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hbase-doc-2.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hbase-master-2.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hbase-regionserver-2.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hbase-rest-2.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hbase-solr-1.5+cdh6.1.1-875250.el7.noarch.rpm │ ├── hbase-solr-doc-1.5+cdh6.1.1-875250.el7.noarch.rpm │ ├── hbase-solr-indexer-1.5+cdh6.1.1-875250.el7.noarch.rpm │ ├── hbase-thrift-2.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── hive-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-hbase-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-hcatalog-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-jdbc-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-metastore-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-server-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-server2-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-webhcat-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hive-webhcat-server-2.1.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── hue-3.9.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── impala-3.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── impala-catalog-3.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── impala-debuginfo-3.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── impala-lzo-3.1.0+cdh6.1.0-770702.el7.x86_64.rpm │ ├── impala-lzo-debuginfo-3.1.0+cdh6.1.0-770702.el7.x86_64.rpm │ ├── impala-server-3.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── impala-shell-3.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── impala-state-store-3.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── impala-udf-devel-3.1.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── kafka-2.0.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── kafka-mirror-maker-2.0.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── kafka-server-2.0.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── kite-1.0.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── kudu-1.8.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── kudu-client0-1.8.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── kudu-client-devel-1.8.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── kudu-debuginfo-1.8.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── kudu-master-1.8.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── kudu-tserver-1.8.0+cdh6.1.1-875250.el7.x86_64.rpm │ ├── oozie-5.0.0_beta1+cdh6.1.1-875250.el7.noarch.rpm │ ├── oozie-client-5.0.0_beta1+cdh6.1.1-875250.el7.noarch.rpm │ ├── parquet-1.9.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── parquet-format-2.3.1+cdh6.1.1-875250.el7.noarch.rpm │ ├── pig-0.17.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── search-1.0.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── sentry-2.1.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── sentry-hdfs-plugin-2.1.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── sentry-store-2.1.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── solr-7.4.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── solr-crunch-1.0.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── solr-doc-7.4.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── solr-mapreduce-1.0.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── solr-server-7.4.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── spark-core-2.4.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── spark-history-server-2.4.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── spark-netlib-1.1+cdh6.1.0-770702.el7.x86_64.rpm │ ├── spark-python-2.4.0+cdh6.1.1-875250.el7.noarch.rpm │ ├── sqoop-1.4.7+cdh6.1.1-875250.el7.noarch.rpm │ ├── sqoop-metastore-1.4.7+cdh6.1.1-875250.el7.noarch.rpm │ ├── zookeeper-3.4.5+cdh6.1.1-875250.el7.x86_64.rpm │ ├── zookeeper-debuginfo-3.4.5+cdh6.1.1-875250.el7.x86_64.rpm │ ├── zookeeper-native-3.4.5+cdh6.1.1-875250.el7.x86_64.rpm │ └── zookeeper-server-3.4.5+cdh6.1.1-875250.el7.x86_64.rpm └── cm6 ├── cloudera-manager-agent-6.1.1-853290.el7.x86_64.rpm ├── cloudera-manager-daemons-6.1.1-853290.el7.x86_64.rpm ├── cloudera-manager-installer.bin ├── cloudera-manager.repo ├── cloudera-manager-server-6.1.1-853290.el7.x86_64.rpm ├── cloudera-manager-server-db-2-6.1.1-853290.el7.x86_64.rpm └── oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm 2 directories, 105 files [root@c61 software]# |
然后，在HTTP中配置软链接：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[root@c61 html]# pwd /var/www/html [root@c61 html]# [root@c61 html]# ls -ltr total 12268 -rw-r--r-- 1 root root 275 Feb 25 19:46 download_cloudera_parcels.sh drwxr-xr-x 4 root root 54 Feb 26 08:01 cloudera-repos -rw------- 1 root root 12555570 Feb 26 08:05 nohup.out [root@c61 html]# [root@c61 html]# ln -s /software/cloudera-rpm/ cloudera-rpms [root@c61 html]# [root@c61 html]# ls -ltr total 12268 -rw-r--r-- 1 root root 275 Feb 25 19:46 download_cloudera_parcels.sh drwxr-xr-x 4 root root 54 Feb 26 08:01 cloudera-repos -rw------- 1 root root 12555570 Feb 26 08:05 nohup.out lrwxrwxrwx 1 root root 23 Feb 26 09:57 cloudera-rpms -> /software/cloudera-rpm/ [root@c61 html]# [root@c61 html]# |
然后网页查看：
http://c61/cloudera-rpms/
然后配置本地的YUM库：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
[root@c61 software]# createrepo cloudera-rpm/cdh6/ Spawning worker 0 with 17 pkgs Spawning worker 1 with 17 pkgs Spawning worker 2 with 16 pkgs Spawning worker 3 with 16 pkgs Spawning worker 4 with 16 pkgs Spawning worker 5 with 16 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@c61 software]# [root@c61 software]# createrepo cloudera-rpm/cm6/ Spawning worker 0 with 1 pkgs Spawning worker 1 with 1 pkgs Spawning worker 2 with 1 pkgs Spawning worker 3 with 1 pkgs Spawning worker 4 with 1 pkgs Spawning worker 5 with 0 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@c61 software]# [root@c61 software]# ls -ltr cloudera-rpm/cdh6/ | grep repo drwxr-xr-x 2 root root 4096 Feb 26 10:06 repodata [root@c61 software]# ls -ltr cloudera-rpm/cm6/ | grep repo -rw-r--r--. 1 root root 232 Feb 20 21:57 cloudera-manager.repo drwxr-xr-x 2 root root 4096 Feb 26 10:09 repodata [root@c61 software]# [root@c61 software]# cat /etc/yum.repos.d/local_cloudera.repo [cdh6] name=cdh6 baseurl=http://c61/cloudera-rpms/cdh6 gpgcheck=0 [cm6] name=cm6 baseurl=http://c61/cloudera-rpms/cm6 gpgcheck=0 [root@c61 software]# [root@c61 software]# yum repolist Loaded plugins: langpacks, ulninfo cdh6 | 2.9 kB 00:00:00 cm6 | 2.9 kB 00:00:00 (1/2): cdh6/primary_db | 58 kB 00:00:01 (2/2): cm6/primary_db | 8.2 kB 00:00:02 repo id repo name status base/x86_64 CentOS-7 - Base 10,019 cdh6 cdh6 98 cm6 cm6 5 epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 12,907 extras/x86_64 CentOS-7 - Extras 371 ol7_UEKR5/x86_64 Latest Unbreakable Enterprise Kernel Release 5 for Oracle Linux 7Server (x86_64) 125 ol7_latest/x86_64 Oracle Linux 7Server Latest (x86_64) 12,218 updates/x86_64 CentOS-7 - Updates 1,103 repolist: 36,846 [root@c61 software]# [root@c61 software]# scp /etc/yum.repos.d/local_cloudera.repo c62:/etc/yum.repos.d/ root@c62's password: local_cloudera.repo 100% 130 74.5KB/s 00:00 [root@c61 software]# scp /etc/yum.repos.d/local_cloudera.repo c63:/etc/yum.repos.d/ root@c63's password: local_cloudera.repo 100% 130 49.0KB/s 00:00 [root@c61 software]# scp /etc/yum.repos.d/local_cloudera.repo c64:/etc/yum.repos.d/ root@c64's password: local_cloudera.repo 100% 130 50.8KB/s 00:00 [root@c61 software]# scp /etc/yum.repos.d/local_cloudera.repo c65:/etc/yum.repos.d/ root@c65's password: local_cloudera.repo 100% 130 15.3KB/s 00:00 [root@c61 software]# |
Manually Install Cloudera Software Packages | 手动安装Cloudera软件包
Cloudera Manager Server
在我的环境里，主节点是【c61】：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 |
[root@c61 ~]# hostname c61 [root@c61 ~]# [root@c61 ~]# yum list | grep cloudera-manager cloudera-manager-agent.x86_64 6.1.1-853290.el7 cm6 cloudera-manager-daemons.x86_64 6.1.1-853290.el7 cm6 cloudera-manager-server.x86_64 6.1.1-853290.el7 cm6 cloudera-manager-server-db-2.x86_64 6.1.1-853290.el7 cm6 [root@c61 ~]# [root@c61 ~]# yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server Loaded plugins: langpacks, ulninfo Resolving Dependencies --> Running transaction check ---> Package cloudera-manager-agent.x86_64 0:6.1.1-853290.el7 will be installed cm6/filelists_db | 109 kB 00:00:00 --> Processing Dependency: python-psycopg2 for package: cloudera-manager-agent-6.1.1-853290.el7.x86_64 base/x86_64/filelists_db | 7.1 MB 00:00:02 --> Processing Dependency: openssl-devel for package: cloudera-manager-agent-6.1.1-853290.el7.x86_64 ol7_latest/x86_64/filelists | 10 MB 00:00:10 --> Processing Dependency: mod_ssl for package: cloudera-manager-agent-6.1.1-853290.el7.x86_64 --> Processing Dependency: MySQL-python for package: cloudera-manager-agent-6.1.1-853290.el7.x86_64 --> Processing Dependency: /lib/lsb/init-functions for package: cloudera-manager-agent-6.1.1-853290.el7.x86_64 cdh6/filelists_db | 427 kB 00:00:00 epel/x86_64/filelists_db | 11 MB 00:00:03 extras/x86_64/filelists_db | 231 kB 00:00:00 ol7_UEKR5/x86_64/filelists | 1.2 MB 00:00:02 updates/x86_64/filelists_db | 1.9 MB 00:00:00 --> Processing Dependency: libpq.so.5()(64bit) for package: cloudera-manager-agent-6.1.1-853290.el7.x86_64 ---> Package cloudera-manager-daemons.x86_64 0:6.1.1-853290.el7 will be installed ---> Package cloudera-manager-server.x86_64 0:6.1.1-853290.el7 will be installed --> Running transaction check ---> Package MySQL-python.x86_64 0:1.2.5-1.el7 will be installed ---> Package mod_ssl.x86_64 1:2.4.6-88.0.1.el7 will be installed ---> Package openssl-devel.x86_64 1:1.0.2k-16.0.1.el7 will be installed --> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-16.0.1.el7.x86_64 --> Processing Dependency: zlib-devel(x86-64) for package: 1:openssl-devel-1.0.2k-16.0.1.el7.x86_64 ---> Package postgresql-libs.x86_64 0:9.2.24-1.el7_5 will be installed ---> Package python-psycopg2.x86_64 0:2.5.1-3.el7 will be installed ---> Package redhat-lsb-core.x86_64 0:4.1-27.0.1.el7 will be installed --> Processing Dependency: redhat-lsb-submod-security(x86-64) = 4.1-27.0.1.el7 for package: redhat-lsb-core-4.1-27.0.1.el7.x86_64 --> Processing Dependency: /usr/bin/patch for package: redhat-lsb-core-4.1-27.0.1.el7.x86_64 --> Processing Dependency: /usr/bin/m4 for package: redhat-lsb-core-4.1-27.0.1.el7.x86_64 --> Processing Dependency: spax for package: redhat-lsb-core-4.1-27.0.1.el7.x86_64 --> Running transaction check ---> Package krb5-devel.x86_64 0:1.15.1-37.el7_6 will be installed --> Processing Dependency: krb5-libs(x86-64) = 1.15.1-37.el7_6 for package: krb5-devel-1.15.1-37.el7_6.x86_64 --> Processing Dependency: libkadm5(x86-64) = 1.15.1-37.el7_6 for package: krb5-devel-1.15.1-37.el7_6.x86_64 --> Processing Dependency: libverto-devel for package: krb5-devel-1.15.1-37.el7_6.x86_64 --> Processing Dependency: libselinux-devel for package: krb5-devel-1.15.1-37.el7_6.x86_64 --> Processing Dependency: libcom_err-devel for package: krb5-devel-1.15.1-37.el7_6.x86_64 --> Processing Dependency: keyutils-libs-devel for package: krb5-devel-1.15.1-37.el7_6.x86_64 ---> Package m4.x86_64 0:1.4.16-10.el7 will be installed ---> Package patch.x86_64 0:2.7.1-10.el7_5 will be installed ---> Package redhat-lsb-submod-security.x86_64 0:4.1-27.0.1.el7 will be installed ---> Package spax.x86_64 0:1.5.2-13.el7 will be installed ---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed --> Running transaction check ---> Package keyutils-libs-devel.x86_64 0:1.5.8-3.el7 will be installed ---> Package krb5-libs.x86_64 0:1.15.1-34.el7 will be updated ---> Package krb5-libs.x86_64 0:1.15.1-37.el7_6 will be an update ---> Package libcom_err-devel.x86_64 0:1.42.9-13.el7 will be installed ---> Package libkadm5.x86_64 0:1.15.1-37.el7_6 will be installed ---> Package libselinux-devel.x86_64 0:2.5-14.1.el7 will be installed --> Processing Dependency: libsepol-devel(x86-64) >= 2.5-10 for package: libselinux-devel-2.5-14.1.el7.x86_64 --> Processing Dependency: pkgconfig(libsepol) for package: libselinux-devel-2.5-14.1.el7.x86_64 --> Processing Dependency: pkgconfig(libpcre) for package: libselinux-devel-2.5-14.1.el7.x86_64 ---> Package libverto-devel.x86_64 0:0.2.5-4.el7 will be installed --> Running transaction check ---> Package libsepol-devel.x86_64 0:2.5-10.el7 will be installed ---> Package pcre-devel.x86_64 0:8.32-17.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =============================================================================================================================================== Package Arch Version Repository Size =============================================================================================================================================== Installing: cloudera-manager-agent x86_64 6.1.1-853290.el7 cm6 9.7 M cloudera-manager-daemons x86_64 6.1.1-853290.el7 cm6 1.1 G cloudera-manager-server x86_64 6.1.1-853290.el7 cm6 8.7 k Installing for dependencies: MySQL-python x86_64 1.2.5-1.el7 base 90 k keyutils-libs-devel x86_64 1.5.8-3.el7 base 37 k krb5-devel x86_64 1.15.1-37.el7_6 ol7_latest 271 k libcom_err-devel x86_64 1.42.9-13.el7 base 31 k libkadm5 x86_64 1.15.1-37.el7_6 ol7_latest 177 k libselinux-devel x86_64 2.5-14.1.el7 base 187 k libsepol-devel x86_64 2.5-10.el7 base 77 k libverto-devel x86_64 0.2.5-4.el7 base 12 k m4 x86_64 1.4.16-10.el7 base 256 k mod_ssl x86_64 1:2.4.6-88.0.1.el7 ol7_latest 111 k openssl-devel x86_64 1:1.0.2k-16.0.1.el7 ol7_latest 1.5 M patch x86_64 2.7.1-10.el7_5 base 110 k pcre-devel x86_64 8.32-17.el7 base 480 k postgresql-libs x86_64 9.2.24-1.el7_5 base 234 k python-psycopg2 x86_64 2.5.1-3.el7 base 132 k redhat-lsb-core x86_64 4.1-27.0.1.el7 ol7_latest 37 k redhat-lsb-submod-security x86_64 4.1-27.0.1.el7 ol7_latest 15 k spax x86_64 1.5.2-13.el7 base 260 k zlib-devel x86_64 1.2.7-18.el7 base 50 k Updating for dependencies: krb5-libs x86_64 1.15.1-37.el7_6 ol7_latest 803 k Transaction Summary =============================================================================================================================================== Install 3 Packages (+19 Dependent packages) Upgrade ( 1 Dependent package) Total download size: 1.1 G Is this ok [y/d/N]: y Downloading packages: No Presto metadata available for ol7_latest (1/23): cloudera-manager-agent-6.1.1-853290.el7.x86_64.rpm | 9.7 MB 00:00:00 (2/23): cloudera-manager-server-6.1.1-853290.el7.x86_64.rpm | 8.7 kB 00:00:00 (3/23): MySQL-python-1.2.5-1.el7.x86_64.rpm | 90 kB 00:00:01 (4/23): keyutils-libs-devel-1.5.8-3.el7.x86_64.rpm | 37 kB 00:00:02 (5/23): libcom_err-devel-1.42.9-13.el7.x86_64.rpm | 31 kB 00:00:03 (6/23): krb5-devel-1.15.1-37.el7_6.x86_64.rpm | 271 kB 00:00:04 (7/23): libselinux-devel-2.5-14.1.el7.x86_64.rpm | 187 kB 00:00:00 (8/23): libverto-devel-0.2.5-4.el7.x86_64.rpm | 12 kB 00:00:00 (9/23): m4-1.4.16-10.el7.x86_64.rpm | 256 kB 00:00:00 (10/23): libkadm5-1.15.1-37.el7_6.x86_64.rpm | 177 kB 00:00:00 (11/23): libsepol-devel-2.5-10.el7.x86_64.rpm | 77 kB 00:00:01 (12/23): mod_ssl-2.4.6-88.0.1.el7.x86_64.rpm | 111 kB 00:00:04 (13/23): patch-2.7.1-10.el7_5.x86_64.rpm | 110 kB 00:00:00 (14/23): krb5-libs-1.15.1-37.el7_6.x86_64.rpm | 803 kB 00:00:10 (15/23): pcre-devel-8.32-17.el7.x86_64.rpm | 480 kB 00:00:02 (16/23): openssl-devel-1.0.2k-16.0.1.el7.x86_64.rpm | 1.5 MB 00:00:04 (17/23): redhat-lsb-core-4.1-27.0.1.el7.x86_64.rpm | 37 kB 00:00:01 (18/23): redhat-lsb-submod-security-4.1-27.0.1.el7.x86_64.rpm | 15 kB 00:00:01 (19/23): python-psycopg2-2.5.1-3.el7.x86_64.rpm | 132 kB 00:00:15 (20/23): spax-1.5.2-13.el7.x86_64.rpm | 260 kB 00:00:00 (21/23): zlib-devel-1.2.7-18.el7.x86_64.rpm | 50 kB 00:00:00 (22/23): postgresql-libs-9.2.24-1.el7_5.x86_64.rpm | 234 kB 00:00:00 (23/23): cloudera-manager-daemons-6.1.1-853290.el7.x86_64.rpm | 1.1 GB 00:01:24 ----------------------------------------------------------------------------------------------------------------------------------------------- Total 13 MB/s | 1.1 GB 00:01:26 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : krb5-libs-1.15.1-37.el7_6.x86_64 1/24 Installing : postgresql-libs-9.2.24-1.el7_5.x86_64 2/24 （过多的输出，...） Verifying : patch-2.7.1-10.el7_5.x86_64 14/24 Verifying : libsepol-devel-2.5-10.el7.x86_64 15/24 Verifying : libkadm5-1.15.1-37.el7_6.x86_64 16/24 Verifying : cloudera-manager-daemons-6.1.1-853290.el7.x86_64 17/24 Verifying : spax-1.5.2-13.el7.x86_64 18/24 Verifying : libselinux-devel-2.5-14.1.el7.x86_64 19/24 Verifying : 1:openssl-devel-1.0.2k-16.0.1.el7.x86_64 20/24 Verifying : cloudera-manager-server-6.1.1-853290.el7.x86_64 21/24 Verifying : postgresql-libs-9.2.24-1.el7_5.x86_64 22/24 Verifying : 1:mod_ssl-2.4.6-88.0.1.el7.x86_64 23/24 Verifying : krb5-libs-1.15.1-34.el7.x86_64 24/24 Installed: cloudera-manager-agent.x86_64 0:6.1.1-853290.el7 cloudera-manager-daemons.x86_64 0:6.1.1-853290.el7 cloudera-manager-server.x86_64 0:6.1.1-853290.el7 Dependency Installed: MySQL-python.x86_64 0:1.2.5-1.el7 keyutils-libs-devel.x86_64 0:1.5.8-3.el7 krb5-devel.x86_64 0:1.15.1-37.el7_6 libcom_err-devel.x86_64 0:1.42.9-13.el7 libkadm5.x86_64 0:1.15.1-37.el7_6 libselinux-devel.x86_64 0:2.5-14.1.el7 libsepol-devel.x86_64 0:2.5-10.el7 libverto-devel.x86_64 0:0.2.5-4.el7 m4.x86_64 0:1.4.16-10.el7 mod_ssl.x86_64 1:2.4.6-88.0.1.el7 openssl-devel.x86_64 1:1.0.2k-16.0.1.el7 patch.x86_64 0:2.7.1-10.el7_5 pcre-devel.x86_64 0:8.32-17.el7 postgresql-libs.x86_64 0:9.2.24-1.el7_5 python-psycopg2.x86_64 0:2.5.1-3.el7 redhat-lsb-core.x86_64 0:4.1-27.0.1.el7 redhat-lsb-submod-security.x86_64 0:4.1-27.0.1.el7 spax.x86_64 0:1.5.2-13.el7 zlib-devel.x86_64 0:1.2.7-18.el7 Dependency Updated: krb5-libs.x86_64 0:1.15.1-37.el7_6 Complete! [root@c61 ~]# |
Cloudera Manager Agent
在我的环境里，从节点是其他四个：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
[root@c62 ~]# ls -ltr /etc/yum.repos.d/ total 24 -rw-r--r--. 1 root root 13093 Nov 2 15:02 public-yum-ol7.repo -rw-r--r--. 1 root root 2492 Feb 25 14:10 tsinghua.repo -rw-r--r-- 1 root root 130 Feb 26 10:12 local_cloudera.repo [root@c62 ~]# [root@c62 ~]# yum install cloudera-manager-agent cloudera-manager-daemons Loaded plugins: langpacks, ulninfo base | 3.6 kB 00:00:00 cdh6 | 2.9 kB 00:00:00 cm6 | 2.9 kB 00:00:00 epel | 4.7 kB 00:00:00 extras | 3.4 kB 00:00:00 ol7_UEKR5 | 1.2 kB 00:00:00 ol7_latest | 1.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/5): cdh6/primary_db | 58 kB 00:00:00 (2/5): cm6/primary_db | 8.2 kB 00:00:00 (3/5): epel/x86_64/updateinfo | 956 kB 00:00:00 (4/5): updates/x86_64/primary_db | 2.4 MB 00:00:00 (5/5): epel/x86_64/primary_db | 6.6 MB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package cloudera-manager-agent.x86_64 0:6.1.1-853290.el7 will be installed cm6/filelists_db | 109 kB 00:00:00 （过多的输出，...） |
启动服务：
CM Agent
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
[root@c61 ~]# systemctl start cloudera-scm-agent [root@c61 ~]# systemctl enable cloudera-scm-agent [root@c61 ~]# [root@c61 ~]# systemctl status cloudera-scm-agent ● cloudera-scm-agent.service - Cloudera Manager Agent Service Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-agent.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2019-02-26 10:58:19 CST; 54s ago Main PID: 20503 (cmagent) CGroup: /system.slice/cloudera-scm-agent.service └─20503 /usr/bin/python2 /opt/cloudera/cm-agent/bin/cm agent Feb 26 10:58:27 c61 cm[20503]: [26/Feb/2019 10:58:27 +0000] 20503 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent/flood Feb 26 10:58:27 c61 cm[20503]: [26/Feb/2019 10:58:27 +0000] 20503 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent/supervisor/include Feb 26 10:58:27 c61 cm[20503]: [26/Feb/2019 10:58:27 +0000] 20503 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent/cgroups Feb 26 10:58:27 c61 cm[20503]: [26/Feb/2019 10:58:27 +0000] 20503 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent/process Feb 26 10:58:27 c61 cm[20503]: [26/Feb/2019 10:58:27 +0000] 20503 MainThread tmpfs INFO Successfully mounted tmpfs at /var/run/cloudera-scm-agent/process Feb 26 10:58:27 c61 cm[20503]: [26/Feb/2019 10:58:27 +0000] 20503 MainThread logging INFO Logging to /var/log/cloudera-scm-agent/cloudera-scm-agent.log Feb 26 10:58:34 c61 cm[20503]: status_server: added process group Feb 26 10:58:34 c61 cm[20503]: flood: added process group Feb 26 10:58:34 c61 cm[20503]: /opt/cloudera/cm-agent/lib/python2.7/site-packages/psutil/_pslinux.py:477: RuntimeWarning: dirty, writeback, mapped, commit_limit memory...were set to 0 Feb 26 10:58:34 c61 cm[20503]: warnings.warn(msg, RuntimeWarning) Hint: Some lines were ellipsized, use -l to show in full. [root@c61 ~]# |
CM Server
启动CM Server的服务之前，需要配置CM的后端数据库。
配置数据库参考官方文档：
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/cm_ig_mysql.html#cmig_topic_5_5
在我的环境里，MySQL的服务端位于【c65】
修改MySQL服务端的配置，并重启数据库服务：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
[root@c65 ~]# service mysqld stop Redirecting to /bin/systemctl stop mysqld.service [root@c65 ~]# [root@c65 ~]# vi /etc/my.cnf [root@c65 ~]# [root@c65 ~]# cat /etc/my.cnf [mysqld] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock transaction-isolation = READ-COMMITTED # Disabling symbolic-links is recommended to prevent assorted security risks; # to do so, uncomment this line: symbolic-links = 0 key_buffer_size = 32M max_allowed_packet = 32M thread_stack = 256K thread_cache_size = 64 query_cache_limit = 8M query_cache_size = 64M query_cache_type = 1 max_connections = 550 #expire_logs_days = 10 #max_binlog_size = 100M #log_bin should be on a disk with enough free space. #Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your #system and chown the specified folder to the mysql user. log_bin=/var/lib/mysql/mysql_binary_log #In later versions of MySQL, if you enable the binary log and do not set #a server_id, MySQL will not start. The server_id must be unique within #the replicating group. server_id=1 binlog_format = mixed read_buffer_size = 2M read_rnd_buffer_size = 16M sort_buffer_size = 8M join_buffer_size = 8M # InnoDB settings innodb_file_per_table = 1 innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 64M innodb_buffer_pool_size = 4G innodb_thread_concurrency = 8 innodb_flush_method = O_DIRECT innodb_log_file_size = 512M [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid sql_mode=STRICT_ALL_TABLES [root@c65 ~]# [root@c65 ~]# ps -ef | grep mysql root 7472 7312 0 11:15 pts/0 00:00:00 tail -f /var/log/mysqld.log root 7475 7353 0 11:15 pts/1 00:00:00 grep --color=auto mysql [root@c65 ~]# [root@c65 ~]# service mysqld start Redirecting to /bin/systemctl start mysqld.service [root@c65 ~]# [root@c65 ~]# ps -ef | grep mysql root 7472 7312 0 11:15 pts/0 00:00:00 tail -f /var/log/mysqld.log mysql 7525 1 20 11:15 ? 00:00:02 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid root 7567 7353 0 11:15 pts/1 00:00:00 grep --color=auto mysql [root@c65 ~]# [root@c65 ~]# service mysqld status Redirecting to /bin/systemctl status mysqld.service ● mysqld.service - MySQL Server Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2019-02-26 11:15:35 CST; 1min 23s ago Docs: man:mysqld(8) http://dev.mysql.com/doc/refman/en/using-systemd.html Process: 7522 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS) Process: 7492 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS) Main PID: 7525 (mysqld) Tasks: 38 CGroup: /system.slice/mysqld.service └─7525 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid Feb 26 11:15:33 c65 systemd[1]: Starting MySQL Server... Feb 26 11:15:35 c65 systemd[1]: Started MySQL Server. [root@c65 ~]# |
该过程中的日志：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
[root@c65 ~]# tail -f /var/log/mysqld.log 2019-02-26T03:08:26.994808Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 190226 11:08:26 2019-02-26T03:08:27.595170Z 0 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool 2019-02-26T03:08:29.108204Z 0 [Note] InnoDB: Shutdown completed; log sequence number 2594400 2019-02-26T03:08:29.111047Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2019-02-26T03:08:29.111094Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA' 2019-02-26T03:08:29.111167Z 0 [Note] Shutting down plugin 'sha256_password' 2019-02-26T03:08:29.111180Z 0 [Note] Shutting down plugin 'mysql_native_password' 2019-02-26T03:08:29.111710Z 0 [Note] Shutting down plugin 'binlog' 2019-02-26T03:08:29.113004Z 0 [Note] /usr/sbin/mysqld: Shutdown complete ================== 2019-02-26T03:15:34.704064Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2019-02-26T03:15:34.707116Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.23-23-log) starting as process 7525 ... 2019-02-26T03:15:34.715958Z 0 [Note] InnoDB: PUNCH HOLE support available 2019-02-26T03:15:34.716046Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2019-02-26T03:15:34.716064Z 0 [Note] InnoDB: Uses event mutexes 2019-02-26T03:15:34.716078Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2019-02-26T03:15:34.716119Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 2019-02-26T03:15:34.716134Z 0 [Note] InnoDB: Using Linux native AIO 2019-02-26T03:15:34.716982Z 0 [Note] InnoDB: Number of pools: 1 2019-02-26T03:15:34.717255Z 0 [Note] InnoDB: Using CPU crc32 instructions 2019-02-26T03:15:34.721537Z 0 [Note] InnoDB: Initializing buffer pool, total size = 4G, instances = 8, chunk size = 128M 2019-02-26T03:15:34.872527Z 0 [Note] InnoDB: Completed initialization of buffer pool 2019-02-26T03:15:34.912843Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2019-02-26T03:15:34.957052Z 0 [Note] InnoDB: Crash recovery did not find the parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite 2019-02-26T03:15:34.960484Z 0 [Note] InnoDB: Highest supported file format is Barracuda. 2019-02-26T03:15:35.356991Z 0 [Note] InnoDB: Created parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite, size 31457280 bytes 2019-02-26T03:15:35.516805Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2019-02-26T03:15:35.517020Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2019-02-26T03:15:35.644609Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2019-02-26T03:15:35.645941Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. 2019-02-26T03:15:35.645982Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active. 2019-02-26T03:15:35.646703Z 0 [Note] InnoDB: Waiting for purge to start 2019-02-26T03:15:35.698351Z 0 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.7.23-23 started; log sequence number 2594884 2019-02-26T03:15:35.698608Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2019-02-26T03:15:35.699366Z 0 [Note] Plugin 'FEDERATED' is disabled. 2019-02-26T03:15:35.734432Z 0 [Note] InnoDB: Buffer pool(s) load completed at 190226 11:15:35 2019-02-26T03:15:35.738292Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them. 2019-02-26T03:15:35.738333Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory. 2019-02-26T03:15:35.739260Z 0 [Warning] CA certificate ca.pem is self signed. 2019-02-26T03:15:35.739344Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory. 2019-02-26T03:15:35.740064Z 0 [Note] Server hostname (bind-address): '*'; port: 3306 2019-02-26T03:15:35.740144Z 0 [Note] IPv6 is available. 2019-02-26T03:15:35.740180Z 0 [Note] - '::' resolves to '::'; 2019-02-26T03:15:35.740219Z 0 [Note] Server socket created on IP: '::'. 2019-02-26T03:15:35.769329Z 0 [Note] Event Scheduler: Loaded 0 events 2019-02-26T03:15:35.769698Z 0 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.7.23-23-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release 23, Revision 500fcf5 |
MySQL服务端，添加远端访问权限
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
[root@c65 ~]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.23-23-log Percona Server (GPL), Release 23, Revision 500fcf5 Copyright (c) 2009-2018 Percona LLC and/or its affiliates Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.04 sec) mysql> mysql> select user,host from mysql.user; +---------------+-----------+ | user | host | +---------------+-----------+ | mysql.session | localhost | | mysql.sys | localhost | | root | localhost | +---------------+-----------+ 3 rows in set (0.00 sec) mysql> mysql> grant all privileges on *.* to root@'%' identified by '***密码是不能被看到的哦***'; Query OK, 0 rows affected, 1 warning (0.01 sec) mysql> flush privileges; Query OK, 0 rows affected (0.01 sec) mysql> select user,host from mysql.user; +---------------+-----------+ | user | host | +---------------+-----------+ | root | % | | mysql.session | localhost | | mysql.sys | localhost | | root | localhost | +---------------+-----------+ 4 rows in set (0.00 sec) mysql> |
然后从【c61】访问MySQL
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
[root@c61 ~]# mysql -h c65 -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.23-23-log Percona Server (GPL), Release 23, Revision 500fcf5 Copyright (c) 2009-2018 Percona LLC and/or its affiliates Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.00 sec) mysql> select database(); +------------+ | database() | +------------+ | NULL | +------------+ 1 row in set (0.00 sec) mysql> exit Bye [root@c61 ~]# |
可以看到，已经可以成功访问了。
YUM: MySQL JDBC Driver
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 |
[root@c61 ~]# yum list | grep mysql | grep java mysql-connector-java.noarch 1:5.1.25-3.el7 base [root@c61 ~]# [root@c61 ~]# yum install -y mysql-connector-java Loaded plugins: langpacks, ulninfo Resolving Dependencies --> Running transaction check ---> Package mysql-connector-java.noarch 1:5.1.25-3.el7 will be installed --> Processing Dependency: jta >= 1.0 for package: 1:mysql-connector-java-5.1.25-3.el7.noarch --> Processing Dependency: slf4j for package: 1:mysql-connector-java-5.1.25-3.el7.noarch --> Running transaction check ---> Package geronimo-jta.noarch 0:1.1.1-17.el7 will be installed ---> Package slf4j.noarch 0:1.7.4-4.el7_4 will be installed --> Processing Dependency: mvn(log4j:log4j) for package: slf4j-1.7.4-4.el7_4.noarch --> Processing Dependency: mvn(javassist:javassist) for package: slf4j-1.7.4-4.el7_4.noarch --> Processing Dependency: mvn(commons-logging:commons-logging) for package: slf4j-1.7.4-4.el7_4.noarch --> Processing Dependency: mvn(commons-lang:commons-lang) for package: slf4j-1.7.4-4.el7_4.noarch --> Processing Dependency: mvn(ch.qos.cal10n:cal10n-api) for package: slf4j-1.7.4-4.el7_4.noarch --> Running transaction check ---> Package apache-commons-lang.noarch 0:2.6-15.el7 will be installed ---> Package apache-commons-logging.noarch 0:1.1.2-7.el7 will be installed --> Processing Dependency: mvn(logkit:logkit) for package: apache-commons-logging-1.1.2-7.el7.noarch --> Processing Dependency: mvn(avalon-framework:avalon-framework-api) for package: apache-commons-logging-1.1.2-7.el7.noarch ---> Package cal10n.noarch 0:0.7.7-4.el7 will be installed ---> Package javassist.noarch 0:3.16.1-10.el7 will be installed ---> Package log4j.noarch 0:1.2.17-16.el7_4 will be installed --> Processing Dependency: mvn(org.apache.geronimo.specs:geronimo-jms_1.1_spec) for package: log4j-1.2.17-16.el7_4.noarch --> Processing Dependency: mvn(javax.mail:mail) for package: log4j-1.2.17-16.el7_4.noarch --> Running transaction check ---> Package avalon-framework.noarch 0:4.3-10.el7 will be installed --> Processing Dependency: xalan-j2 for package: avalon-framework-4.3-10.el7.noarch ---> Package avalon-logkit.noarch 0:2.1-14.el7 will be installed --> Processing Dependency: tomcat-servlet-3.0-api for package: avalon-logkit-2.1-14.el7.noarch ---> Package geronimo-jms.noarch 0:1.1.1-19.el7 will be installed ---> Package javamail.noarch 0:1.4.6-8.el7 will be installed --> Running transaction check ---> Package tomcat-servlet-3.0-api.noarch 0:7.0.76-8.el7_5 will be installed ---> Package xalan-j2.noarch 0:2.7.1-23.el7 will be installed --> Processing Dependency: xerces-j2 for package: xalan-j2-2.7.1-23.el7.noarch --> Processing Dependency: osgi(org.apache.xerces) for package: xalan-j2-2.7.1-23.el7.noarch --> Running transaction check ---> Package xerces-j2.noarch 0:2.11.0-17.el7_0 will be installed --> Processing Dependency: xml-commons-resolver >= 1.2 for package: xerces-j2-2.11.0-17.el7_0.noarch --> Processing Dependency: xml-commons-apis >= 1.4.01 for package: xerces-j2-2.11.0-17.el7_0.noarch --> Processing Dependency: osgi(org.apache.xml.resolver) for package: xerces-j2-2.11.0-17.el7_0.noarch --> Processing Dependency: osgi(javax.xml) for package: xerces-j2-2.11.0-17.el7_0.noarch --> Running transaction check ---> Package xml-commons-apis.noarch 0:1.4.01-16.el7 will be installed ---> Package xml-commons-resolver.noarch 0:1.2-15.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================================= Package Arch Version Repository Size ======================================================================================================================================================================================= Installing: mysql-connector-java noarch 1:5.1.25-3.el7 base 1.3 M Installing for dependencies: apache-commons-lang noarch 2.6-15.el7 base 276 k apache-commons-logging noarch 1.1.2-7.el7 base 78 k avalon-framework noarch 4.3-10.el7 base 88 k avalon-logkit noarch 2.1-14.el7 base 87 k cal10n noarch 0.7.7-4.el7 base 36 k geronimo-jms noarch 1.1.1-19.el7 base 31 k geronimo-jta noarch 1.1.1-17.el7 base 20 k javamail noarch 1.4.6-8.el7 base 758 k javassist noarch 3.16.1-10.el7 base 627 k log4j noarch 1.2.17-16.el7_4 base 444 k slf4j noarch 1.7.4-4.el7_4 base 170 k tomcat-servlet-3.0-api noarch 7.0.76-8.el7_5 ol7_latest 211 k xalan-j2 noarch 2.7.1-23.el7 base 1.9 M xerces-j2 noarch 2.11.0-17.el7_0 base 1.1 M xml-commons-apis noarch 1.4.01-16.el7 base 227 k xml-commons-resolver noarch 1.2-15.el7 base 108 k Transaction Summary ======================================================================================================================================================================================= Install 1 Package (+16 Dependent packages) Total download size: 7.4 M Installed size: 10 M Downloading packages: (1/17): apache-commons-logging-1.1.2-7.el7.noarch.rpm | 78 kB 00:00:00 (2/17): apache-commons-lang-2.6-15.el7.noarch.rpm | 276 kB 00:00:00 (3/17): avalon-framework-4.3-10.el7.noarch.rpm | 88 kB 00:00:00 (4/17): cal10n-0.7.7-4.el7.noarch.rpm | 36 kB 00:00:00 (5/17): geronimo-jms-1.1.1-19.el7.noarch.rpm | 31 kB 00:00:00 (6/17): avalon-logkit-2.1-14.el7.noarch.rpm | 87 kB 00:00:02 (7/17): geronimo-jta-1.1.1-17.el7.noarch.rpm | 20 kB 00:00:04 (8/17): javamail-1.4.6-8.el7.noarch.rpm | 758 kB 00:00:03 (9/17): log4j-1.2.17-16.el7_4.noarch.rpm | 444 kB 00:00:03 (10/17): javassist-3.16.1-10.el7.noarch.rpm | 627 kB 00:00:07 (11/17): tomcat-servlet-3.0-api-7.0.76-8.el7_5.noarch.rpm | 211 kB 00:00:04 (12/17): mysql-connector-java-5.1.25-3.el7.noarch.rpm | 1.3 MB 00:00:07 (13/17): slf4j-1.7.4-4.el7_4.noarch.rpm | 170 kB 00:00:07 (14/17): xalan-j2-2.7.1-23.el7.noarch.rpm | 1.9 MB 00:00:07 (15/17): xml-commons-apis-1.4.01-16.el7.noarch.rpm | 227 kB 00:00:00 (16/17): xerces-j2-2.11.0-17.el7_0.noarch.rpm | 1.1 MB 00:00:07 xml-commons-resolver-1.2-15.el FAILED https://mirrors.tuna.tsinghua.edu.cn/centos/7/os/x86_64/Packages/xml-commons-resolver-1.2-15.el7.noarch.rpm: [Errno 12] Timeout on https://mirrors.tuna.tsinghua.edu.cn/centos/7/os/x86_64/Packages/xml-commons-resolver-1.2-15.el7.noarch.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. (17/17): xml-commons-resolver-1.2-15.el7.noarch.rpm | 108 kB 00:00:00 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 135 kB/s | 7.4 MB 00:00:55 Running transaction check Running transaction test Transaction test succeeded Running transaction Warning: RPMDB altered outside of yum. Installing : geronimo-jms-1.1.1-19.el7.noarch 1/17 Installing : xml-commons-apis-1.4.01-16.el7.noarch 2/17 Installing : xml-commons-resolver-1.2-15.el7.noarch 3/17 Installing : xalan-j2-2.7.1-23.el7.noarch 4/17 Installing : xerces-j2-2.11.0-17.el7_0.noarch 5/17 Installing : geronimo-jta-1.1.1-17.el7.noarch 6/17 Installing : javassist-3.16.1-10.el7.noarch 7/17 Installing : javamail-1.4.6-8.el7.noarch 8/17 Installing : log4j-1.2.17-16.el7_4.noarch 9/17 Installing : cal10n-0.7.7-4.el7.noarch 10/17 Installing : tomcat-servlet-3.0-api-7.0.76-8.el7_5.noarch 11/17 Installing : apache-commons-logging-1.1.2-7.el7.noarch 12/17 Installing : avalon-logkit-2.1-14.el7.noarch 13/17 Installing : avalon-framework-4.3-10.el7.noarch 14/17 Installing : apache-commons-lang-2.6-15.el7.noarch 15/17 Installing : slf4j-1.7.4-4.el7_4.noarch 16/17 Installing : 1:mysql-connector-java-5.1.25-3.el7.noarch 17/17 Verifying : xml-commons-apis-1.4.01-16.el7.noarch 1/17 Verifying : log4j-1.2.17-16.el7_4.noarch 2/17 Verifying : apache-commons-logging-1.1.2-7.el7.noarch 3/17 Verifying : geronimo-jms-1.1.1-19.el7.noarch 4/17 Verifying : apache-commons-lang-2.6-15.el7.noarch 5/17 Verifying : avalon-logkit-2.1-14.el7.noarch 6/17 Verifying : avalon-framework-4.3-10.el7.noarch 7/17 Verifying : tomcat-servlet-3.0-api-7.0.76-8.el7_5.noarch 8/17 Verifying : 1:mysql-connector-java-5.1.25-3.el7.noarch 9/17 Verifying : cal10n-0.7.7-4.el7.noarch 10/17 Verifying : slf4j-1.7.4-4.el7_4.noarch 11/17 Verifying : javamail-1.4.6-8.el7.noarch 12/17 Verifying : xalan-j2-2.7.1-23.el7.noarch 13/17 Verifying : javassist-3.16.1-10.el7.noarch 14/17 Verifying : geronimo-jta-1.1.1-17.el7.noarch 15/17 Verifying : xerces-j2-2.11.0-17.el7_0.noarch 16/17 Verifying : xml-commons-resolver-1.2-15.el7.noarch 17/17 Installed: mysql-connector-java.noarch 1:5.1.25-3.el7 Dependency Installed: apache-commons-lang.noarch 0:2.6-15.el7 apache-commons-logging.noarch 0:1.1.2-7.el7 avalon-framework.noarch 0:4.3-10.el7 avalon-logkit.noarch 0:2.1-14.el7 cal10n.noarch 0:0.7.7-4.el7 geronimo-jms.noarch 0:1.1.1-19.el7 geronimo-jta.noarch 0:1.1.1-17.el7 javamail.noarch 0:1.4.6-8.el7 javassist.noarch 0:3.16.1-10.el7 log4j.noarch 0:1.2.17-16.el7_4 slf4j.noarch 0:1.7.4-4.el7_4 tomcat-servlet-3.0-api.noarch 0:7.0.76-8.el7_5 xalan-j2.noarch 0:2.7.1-23.el7 xerces-j2.noarch 0:2.11.0-17.el7_0 xml-commons-apis.noarch 0:1.4.01-16.el7 xml-commons-resolver.noarch 0:1.2-15.el7 Complete! [root@c61 ~]# |
数据库创建脚本：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
[root@c65 ~]# cat init_cloudera_database_mysql.sql # cloudera manager server CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '***不要看密码***'; # activity monitor CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '***不要看密码***'; # reports manager CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '***不要看密码***'; # hue CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '***不要看密码***'; # hive metastore server CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '***不要看密码***'; # sentry server CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '***不要看密码***'; # cloudera navigator audit server CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '***不要看密码***'; # cloudera navigator metadata server CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '***不要看密码***'; # oozie CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '***不要看密码***'; [root@c65 ~]# |
执行：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
[root@c65 ~]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 Server version: 5.7.23-23-log Percona Server (GPL), Release 23, Revision 500fcf5 Copyright (c) 2009-2018 Percona LLC and/or its affiliates Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.00 sec) mysql> exit Bye [root@c65 ~]# [root@c65 ~]# mysql -u root -p < init_cloudera_database_mysql.sql Enter password: [root@c65 ~]# [root@c65 ~]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 9 Server version: 5.7.23-23-log Percona Server (GPL), Release 23, Revision 500fcf5 Copyright (c) 2009-2018 Percona LLC and/or its affiliates Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | amon | | hue | | metastore | | mysql | | nav | | navms | | oozie | | performance_schema | | rman | | scm | | sentry | | sys | +--------------------+ 13 rows in set (0.00 sec) mysql> mysql> select user,host from mysql.user; +---------------+-----------+ | user | host | +---------------+-----------+ | amon | % | | hive | % | | hue | % | | nav | % | | navms | % | | oozie | % | | rman | % | | root | % | | scm | % | | sentry | % | | mysql.session | localhost | | mysql.sys | localhost | | root | localhost | +---------------+-----------+ 13 rows in set (0.00 sec) mysql> |
可以看到，执行成功。
然后，配置Cloudera与MySQL的关联。
安装JAVA支持：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
[root@c61 ~]# ls -ltr /software/cloudera-rpm/cm6/ total 1316136 -rw-r--r--. 1 root root 833386 Feb 20 21:55 cloudera-manager-installer.bin -rw-r--r--. 1 root root 232 Feb 20 21:57 cloudera-manager.repo -rw-r--r--. 1 root root 10992 Feb 20 21:57 cloudera-manager-server-db-2-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 8956 Feb 20 21:57 cloudera-manager-server-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 10148300 Feb 20 22:02 cloudera-manager-agent-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 184988341 Feb 20 22:04 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm -rw-r--r--. 1 root root 1151713276 Feb 21 21:16 cloudera-manager-daemons-6.1.1-853290.el7.x86_64.rpm drwxr-xr-x 2 root root 4096 Feb 26 10:09 repodata [root@c61 ~]# [root@c61 ~]# yum list | grep oracle | grep j2sdk oracle-j2sdk1.8.x86_64 1.8.0+update181-1 cm6 [root@c61 ~]# [root@c61 ~]# yum install -y oracle-j2sdk1.8 Loaded plugins: langpacks, ulninfo Resolving Dependencies --> Running transaction check ---> Package oracle-j2sdk1.8.x86_64 0:1.8.0+update181-1 will be installed --> Finished Dependency Resolution Dependencies Resolved =============================================================================================================================================== Package Arch Version Repository Size =============================================================================================================================================== Installing: oracle-j2sdk1.8 x86_64 1.8.0+update181-1 cm6 176 M Transaction Summary =============================================================================================================================================== Install 1 Package Total download size: 176 M Installed size: 364 M Downloading packages: oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm | 176 MB 00:00:12 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : oracle-j2sdk1.8-1.8.0+update181-1.x86_64 1/1 Verifying : oracle-j2sdk1.8-1.8.0+update181-1.x86_64 1/1 Installed: oracle-j2sdk1.8.x86_64 0:1.8.0+update181-1 Complete! [root@c61 ~]# |
Cloudera Manager Server DB:
|
1 2 3 4 5 6 7 8 9 |
[root@c61 ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh -h c65 mysql scm scm Enter SCM password: JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera Verifying that we can write to /etc/cloudera-scm-server Creating SCM configuration file in /etc/cloudera-scm-server Executing: /usr/java/jdk1.8.0_181-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly! [root@c61 ~]# |
因为数据库服务器不在本地，所以需要增加【-h】选项指定。
查看配置：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[root@c61 ~]# cat /etc/cloudera-scm-server/db.properties # Auto-generated by scm_prepare_database.sh on Tue Feb 26 13:30:54 CST 2019 # # For information describing how to configure the Cloudera Manager Server # to connect to databases, see the "Cloudera Manager Installation Guide." # com.cloudera.cmf.db.type=mysql com.cloudera.cmf.db.host=c65 com.cloudera.cmf.db.name=scm com.cloudera.cmf.db.user=scm com.cloudera.cmf.db.setupType=EXTERNAL com.cloudera.cmf.db.password=***不要看密码哦*** [root@c61 ~]# |
这是前面的脚本自动配置的。
最后，启动服务就好了：
|
1 2 |
[root@c61 ~]# systemctl start cloudera-scm-server [root@c61 ~]# systemctl enable cloudera-scm-server |
成功启动后：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
[root@c61 ~]# cat /var/log/cloudera-scm-server/cloudera-scm-server.log | tail -n 8 2019-02-26 13:36:06,619 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating documents:2019-02-26T05:36:06.619Z 2019-02-26 13:36:06,666 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num docs:221 2019-02-26 13:36:06,668 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Constructing repo:2019-02-26T05:36:06.668Z 2019-02-26 13:36:07,439 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2019-02-26T05:36:07.439Z 2019-02-26 13:36:07,542 INFO WebServerImpl:org.eclipse.jetty.server.Server: jetty-9.4.6.v20170531 2019-02-26 13:36:07,572 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@243c862d{HTTP/1.1,[http/1.1]}{0.0.0.0:7180} 2019-02-26 13:36:07,573 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started @147216ms 2019-02-26 13:36:07,573 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server. [root@c61 ~]# [root@c61 ~]# netstat -tupln | grep 7180 tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 31047/java [root@c61 ~]# [root@c61 ~]# service cloudera-scm-server status Redirecting to /bin/systemctl status cloudera-scm-server.service ● cloudera-scm-server.service - Cloudera CM Server Service Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2019-02-26 13:33:40 CST; 3min 25s ago Main PID: 31047 (java) Tasks: 132 CGroup: /system.slice/cloudera-scm-server.service └─31047 /usr/java/jdk1.8.0_181-cloudera/bin/java -cp .:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-conn... Feb 26 13:33:40 c61 systemd[1]: Started Cloudera CM Server Service. Feb 26 13:33:40 c61 cm-server[31047]: JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera Feb 26 13:33:40 c61 cm-server[31047]: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 Feb 26 13:33:43 c61 cm-server[31047]: ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system...ion logging. Feb 26 13:33:53 c61 cm-server[31047]: 13:33:53.244 [main] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table 'scm.CM_VERSION' doesn't exist Feb 26 13:34:27 c61 cm-server[31047]: Security framework of XStream not initialized, XStream is probably vulnerable. Feb 26 13:34:41 c61 cm-server[31047]: Security framework of XStream not initialized, XStream is probably vulnerable. Hint: Some lines were ellipsized, use -l to show in full. [root@c61 ~]# |
只有端口监听起来了，并且服务器日志如上所示，才表明服务真的成功启动了。
接下来就是在网页上图形化的安装了：
http://c61:7180/cmf/login
二、安装
访问URL：
http://c61:7180/cmf/login
admin/admin
这里要改成：http://c61/cloudera-rpms/cm6/
并且对改路径做出如下配置：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
[root@c61 ~]# cd /var/www/html/cloudera-rpms/cm6/ [root@c61 cm6]# pwd /var/www/html/cloudera-rpms/cm6 [root@c61 cm6]# [root@c61 cm6]# ls -ltr total 1316136 -rw-r--r--. 1 root root 833386 Feb 20 21:55 cloudera-manager-installer.bin -rw-r--r--. 1 root root 232 Feb 20 21:57 cloudera-manager.repo -rw-r--r--. 1 root root 10992 Feb 20 21:57 cloudera-manager-server-db-2-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 8956 Feb 20 21:57 cloudera-manager-server-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 10148300 Feb 20 22:02 cloudera-manager-agent-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 184988341 Feb 20 22:04 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm -rw-r--r--. 1 root root 1151713276 Feb 21 21:16 cloudera-manager-daemons-6.1.1-853290.el7.x86_64.rpm drwxr-xr-x 2 root root 4096 Feb 26 10:09 repodata [root@c61 cm6]# [root@c61 cm6]# wget https://archive.cloudera.com/cm6/6.1.1/allkeys.asc --2019-02-26 13:54:01-- https://archive.cloudera.com/cm6/6.1.1/allkeys.asc Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.228.167 Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.228.167|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 14041 (14K) [text/plain] Saving to: ‘allkeys.asc’ 100%[=============================================================================================================================================>] 14,041 --.-K/s in 0.1s 2019-02-26 13:54:02 (131 KB/s) - ‘allkeys.asc’ saved [14041/14041] [root@c61 cm6]# [root@c61 cm6]# ls -ltr total 1316152 -rw-r--r-- 1 root root 14041 Feb 19 17:57 allkeys.asc -rw-r--r--. 1 root root 833386 Feb 20 21:55 cloudera-manager-installer.bin -rw-r--r--. 1 root root 232 Feb 20 21:57 cloudera-manager.repo -rw-r--r--. 1 root root 10992 Feb 20 21:57 cloudera-manager-server-db-2-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 8956 Feb 20 21:57 cloudera-manager-server-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 10148300 Feb 20 22:02 cloudera-manager-agent-6.1.1-853290.el7.x86_64.rpm -rw-r--r--. 1 root root 184988341 Feb 20 22:04 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm -rw-r--r--. 1 root root 1151713276 Feb 21 21:16 cloudera-manager-daemons-6.1.1-853290.el7.x86_64.rpm drwxr-xr-x 2 root root 4096 Feb 26 10:09 repodata [root@c61 cm6]# |
如果没有这个【allkeys.asc】，那么在后面Install Agents的步骤，会出现因为无法找到该文件而导致的自动安装进程中断的错误。