ELK – Adamhuan's Data Center – 【逻辑驱动数据】

elasticsearch,Install plugin：HEAD

执行安装：

[elasticme@elk elasticsearch-2.3.4]$ pwd
/opt/elasticsearch-2.3.4
[elasticme@elk elasticsearch-2.3.4]$ 
[elasticme@elk elasticsearch-2.3.4]$ whoami
elasticme
[elasticme@elk elasticsearch-2.3.4]$ 
[elasticme@elk elasticsearch-2.3.4]$ ls plugins/
[elasticme@elk elasticsearch-2.3.4]$ 
[elasticme@elk elasticsearch-2.3.4]$ bin/plugin install mobz/elasticsearch-head
-> Installing mobz/elasticsearch-head...
Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ...
Downloading ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONE
Verifying https://github.com/mobz/elasticsearch-head/archive/master.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
Installed head into /opt/elasticsearch-2.3.4/plugins/head
[elasticme@elk elasticsearch-2.3.4]$ 
[elasticme@elk elasticsearch-2.3.4]$ ls -ltr plugins/
total 4
drwxrwxr-x 5 elasticme elasticme 4096 Jul 18 01:21 head
[elasticme@elk elasticsearch-2.3.4]$ 
[elasticme@elk elasticsearch-2.3.4]$ ls -ltr plugins/head/
total 48
-rw-rw-r-- 1 elasticme elasticme  559 Jul 18 01:21 LICENCE
drwxrwxr-x 5 elasticme elasticme 4096 Jul 18 01:21 _site
-rw-rw-r-- 1 elasticme elasticme  795 Jul 18 01:21 package.json
-rw-rw-r-- 1 elasticme elasticme 1082 Jul 18 01:21 index.html
-rw-rw-r-- 1 elasticme elasticme 3482 Jul 18 01:21 grunt_fileSets.js
-rw-rw-r-- 1 elasticme elasticme  104 Jul 18 01:21 elasticsearch-head.sublime-project
drwxrwxr-x 4 elasticme elasticme 4096 Jul 18 01:21 test
-rw-rw-r-- 1 elasticme elasticme 5146 Jul 18 01:21 README.textile
-rw-rw-r-- 1 elasticme elasticme  100 Jul 18 01:21 plugin-descriptor.properties
drwxrwxr-x 4 elasticme elasticme 4096 Jul 18 01:21 src
-rw-rw-r-- 1 elasticme elasticme 2171 Jul 18 01:21 Gruntfile.js
[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ pwd

/opt/elasticsearch-2.3.4

[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ whoami

elasticme

[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ ls plugins/

[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ bin/plugin install mobz/elasticsearch-head

-> Installing mobz/elasticsearch-head...

Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ...

Downloading ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONE

Verifying https://github.com/mobz/elasticsearch-head/archive/master.zip checksums if available ...

NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)

Installed head into /opt/elasticsearch-2.3.4/plugins/head

[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ ls -ltr plugins/

total 4

drwxrwxr-x 5 elasticme elasticme 4096 Jul 18 01:21 head

[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ ls -ltr plugins/head/

total 48

-rw-rw-r-- 1 elasticme elasticme 559 Jul 18 01:21 LICENCE

drwxrwxr-x 5 elasticme elasticme 4096 Jul 18 01:21 _site

-rw-rw-r-- 1 elasticme elasticme 795 Jul 18 01:21 package.json

-rw-rw-r-- 1 elasticme elasticme 1082 Jul 18 01:21 index.html

-rw-rw-r-- 1 elasticme elasticme 3482 Jul 18 01:21 grunt_fileSets.js

-rw-rw-r-- 1 elasticme elasticme 104 Jul 18 01:21 elasticsearch-head.sublime-project

drwxrwxr-x 4 elasticme elasticme 4096 Jul 18 01:21 test

-rw-rw-r-- 1 elasticme elasticme 5146 Jul 18 01:21 README.textile

-rw-rw-r-- 1 elasticme elasticme 100 Jul 18 01:21 plugin-descriptor.properties

drwxrwxr-x 4 elasticme elasticme 4096 Jul 18 01:21 src

-rw-rw-r-- 1 elasticme elasticme 2171 Jul 18 01:21 Gruntfile.js

[elasticme@elk elasticsearch-2.3.4]$

编辑ES配置文件：

[elasticme@elk elasticsearch-2.3.4]$ pwd
/opt/elasticsearch-2.3.4
[elasticme@elk elasticsearch-2.3.4]$ 
[elasticme@elk elasticsearch-2.3.4]$ cat config/elasticsearch.yml | grep -v ^#
[elasticme@elk elasticsearch-2.3.4]$ 
[elasticme@elk elasticsearch-2.3.4]$ vi config/elasticsearch.yml
[elasticme@elk ~]$ cat /opt/elasticsearch-2.3.4/config/elasticsearch.yml | grep -v ^#
cluster.name: es_cluster_adamhuan

node.name: 192.168.232.151

path.data: /opt/elasticsearch-2.3.4/data

path.logs: /opt/elasticsearch-2.3.4/logs

network.host: 192.168.232.151
http.port: 9200
[elasticme@elk ~]$ 
[elasticme@elk ~]$ ls -ltr /opt/elasticsearch-2.3.4/data
total 8
drwxrwxr-x 3 elasticme elasticme 4096 Jul 17 19:46 elasticsearch
drwxrwxr-x 3 elasticme elasticme 4096 Jul 18 01:30 es_cluster_adamhuan
[elasticme@elk ~]$ 
[elasticme@elk ~]$ ls -ltr /opt/elasticsearch-2.3.4/logs
total 12
-rw-r--r-- 1 elasticme elasticme    0 Jul 17 19:42 elasticsearch_deprecation.log
-rw-r--r-- 1 elasticme elasticme    0 Jul 17 19:42 elasticsearch_index_indexing_slowlog.log
-rw-r--r-- 1 elasticme elasticme    0 Jul 17 19:42 elasticsearch_index_search_slowlog.log
-rw-r--r-- 1 elasticme elasticme 3992 Jul 17 22:57 elasticsearch.log.2016-07-17
-rw-rw-r-- 1 elasticme elasticme  441 Jul 18 01:24 elasticsearch.log
-rw-rw-r-- 1 elasticme elasticme    0 Jul 18 01:30 es_cluster_adamhuan_deprecation.log
-rw-rw-r-- 1 elasticme elasticme    0 Jul 18 01:30 es_cluster_adamhuan_index_indexing_slowlog.log
-rw-rw-r-- 1 elasticme elasticme    0 Jul 18 01:30 es_cluster_adamhuan_index_search_slowlog.log
-rw-rw-r-- 1 elasticme elasticme 2525 Jul 18 01:34 es_cluster_adamhuan.log
[elasticme@elk ~]$

[elasticme@elk elasticsearch-2.3.4]$ pwd

/opt/elasticsearch-2.3.4

[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ cat config/elasticsearch.yml | grep -v ^#

[elasticme@elk elasticsearch-2.3.4]$

[elasticme@elk elasticsearch-2.3.4]$ vi config/elasticsearch.yml

[elasticme@elk ~]$ cat /opt/elasticsearch-2.3.4/config/elasticsearch.yml | grep -v ^#

cluster.name: es_cluster_adamhuan

node.name: 192.168.232.151

path.data: /opt/elasticsearch-2.3.4/data

path.logs: /opt/elasticsearch-2.3.4/logs

network.host: 192.168.232.151

http.port: 9200

[elasticme@elk ~]$

[elasticme@elk ~]$ ls -ltr /opt/elasticsearch-2.3.4/data

total 8

drwxrwxr-x 3 elasticme elasticme 4096 Jul 17 19:46 elasticsearch

drwxrwxr-x 3 elasticme elasticme 4096 Jul 18 01:30 es_cluster_adamhuan

[elasticme@elk ~]$

[elasticme@elk ~]$ ls -ltr /opt/elasticsearch-2.3.4/logs

total 12

-rw-r--r-- 1 elasticme elasticme 0 Jul 17 19:42 elasticsearch_deprecation.log

-rw-r--r-- 1 elasticme elasticme 0 Jul 17 19:42 elasticsearch_index_indexing_slowlog.log

-rw-r--r-- 1 elasticme elasticme 0 Jul 17 19:42 elasticsearch_index_search_slowlog.log

-rw-r--r-- 1 elasticme elasticme 3992 Jul 17 22:57 elasticsearch.log.2016-07-17

-rw-rw-r-- 1 elasticme elasticme 441 Jul 18 01:24 elasticsearch.log

-rw-rw-r-- 1 elasticme elasticme 0 Jul 18 01:30 es_cluster_adamhuan_deprecation.log

-rw-rw-r-- 1 elasticme elasticme 0 Jul 18 01:30 es_cluster_adamhuan_index_indexing_slowlog.log

-rw-rw-r-- 1 elasticme elasticme 0 Jul 18 01:30 es_cluster_adamhuan_index_search_slowlog.log

-rw-rw-r-- 1 elasticme elasticme 2525 Jul 18 01:34 es_cluster_adamhuan.log

[elasticme@elk ~]$

启动ES：

[elasticme@elk elasticsearch-2.3.4]$ bin/elasticsearch
[2016-07-18 01:46:55,332][WARN ][bootstrap                ] unable to install syscall filter: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed
[2016-07-18 01:46:55,608][INFO ][node                     ] [192.168.232.151] version[2.3.4], pid[28187], build[e455fd0/2016-06-30T11:24:31Z]
[2016-07-18 01:46:55,608][INFO ][node                     ] [192.168.232.151] initializing ...
[2016-07-18 01:46:56,328][INFO ][plugins                  ] [192.168.232.151] modules [reindex, lang-expression, lang-groovy], plugins [head], sites [head]
[2016-07-18 01:46:56,345][INFO ][env                      ] [192.168.232.151] using [1] data paths, mounts [[/ (/dev/sda3)]], net usable_space [20.8gb], net total_space [26.1gb], spins? [possibly], types [ext4]
[2016-07-18 01:46:56,345][INFO ][env                      ] [192.168.232.151] heap size [1015.6mb], compressed ordinary object pointers [true]
[2016-07-18 01:46:56,345][WARN ][env                      ] [192.168.232.151] max file descriptors [4096] for elasticsearch process likely too low, consider increasing to at least [65536]
[2016-07-18 01:46:59,528][INFO ][node                     ] [192.168.232.151] initialized
[2016-07-18 01:46:59,528][INFO ][node                     ] [192.168.232.151] starting ...
[2016-07-18 01:46:59,712][INFO ][transport                ] [192.168.232.151] publish_address {192.168.232.151:9300}, bound_addresses {192.168.232.151:9300}
[2016-07-18 01:46:59,723][INFO ][discovery                ] [192.168.232.151] es_cluster_adamhuan/963bUKriRoC0E5vLE-oixQ
[2016-07-18 01:47:02,854][INFO ][cluster.service          ] [192.168.232.151] new_master {192.168.232.151}{963bUKriRoC0E5vLE-oixQ}{192.168.232.151}{192.168.232.151:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2016-07-18 01:47:02,890][INFO ][http                     ] [192.168.232.151] publish_address {192.168.232.151:9200}, bound_addresses {192.168.232.151:9200}
[2016-07-18 01:47:02,890][INFO ][node                     ] [192.168.232.151] started
[2016-07-18 01:47:03,023][INFO ][gateway                  ] [192.168.232.151] recovered [1] indices into cluster_state
[2016-07-18 01:47:03,678][INFO ][cluster.routing.allocation] [192.168.232.151] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[logstash-2016.07.18][1]] ...]).

[elasticme@elk elasticsearch-2.3.4]$ bin/elasticsearch

[2016-07-18 01:46:55,332][WARN ][bootstrap ] unable to install syscall filter: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed

[2016-07-18 01:46:55,608][INFO ][node ] [192.168.232.151] version[2.3.4], pid[28187], build[e455fd0/2016-06-30T11:24:31Z]

[2016-07-18 01:46:55,608][INFO ][node ] [192.168.232.151] initializing ...

[2016-07-18 01:46:56,328][INFO ][plugins ] [192.168.232.151] modules [reindex, lang-expression, lang-groovy], plugins [head], sites [head]

[2016-07-18 01:46:56,345][INFO ][env ] [192.168.232.151] using [1] data paths, mounts [[/ (/dev/sda3)]], net usable_space [20.8gb], net total_space [26.1gb], spins? [possibly], types [ext4]

[2016-07-18 01:46:56,345][INFO ][env ] [192.168.232.151] heap size [1015.6mb], compressed ordinary object pointers [true]

[2016-07-18 01:46:56,345][WARN ][env ] [192.168.232.151] max file descriptors [4096] for elasticsearch process likely too low, consider increasing to at least [65536]

[2016-07-18 01:46:59,528][INFO ][node ] [192.168.232.151] initialized

[2016-07-18 01:46:59,528][INFO ][node ] [192.168.232.151] starting ...

[2016-07-18 01:46:59,712][INFO ][transport ] [192.168.232.151] publish_address {192.168.232.151:9300}, bound_addresses {192.168.232.151:9300}

[2016-07-18 01:46:59,723][INFO ][discovery ] [192.168.232.151] es_cluster_adamhuan/963bUKriRoC0E5vLE-oixQ

[2016-07-18 01:47:02,854][INFO ][cluster.service ] [192.168.232.151] new_master {192.168.232.151}{963bUKriRoC0E5vLE-oixQ}{192.168.232.151}{192.168.232.151:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)

[2016-07-18 01:47:02,890][INFO ][http ] [192.168.232.151] publish_address {192.168.232.151:9200}, bound_addresses {192.168.232.151:9200}

[2016-07-18 01:47:02,890][INFO ][node ] [192.168.232.151] started

[2016-07-18 01:47:03,023][INFO ][gateway ] [192.168.232.151] recovered [1] indices into cluster_state

[2016-07-18 01:47:03,678][INFO ][cluster.routing.allocation] [192.168.232.151] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[logstash-2016.07.18][1]] ...]).

查看运行：
[crayon[……]

Posted on: 2016年7月18日 2016年7月18日
Author: Adamhuan
Categories: ELK
Tags: Elasticsearch, head, Plugin
Leave a comment: elasticsearch,Install plugin：HEAD

Deploy ELK on CentOS 6.8

本文仅会涉及到一台服务器。

如题所示，这台服务器为：CentOS 6.8。

[root@elk ~]# cat /etc/redhat-release
CentOS release 6.8 (Final)
[root@elk ~]#

[root@elk ~]# cat /etc/redhat-release

CentOS release 6.8 (Final)

[root@elk ~]#

关于这台服务器的其他相关信息如下：

[root@elk ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:CC:EC:E9  
          inet addr:192.168.232.151  Bcast:192.168.232.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fecc:ece9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13209 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6425 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:16349985 (15.5 MiB)  TX bytes:396212 (386.9 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:960 (960.0 b)  TX bytes:960 (960.0 b)

[root@elk ~]# 
[root@elk ~]# sestatus 
SELinux status:                 disabled
[root@elk ~]# 
[root@elk ~]# service iptables status
iptables: Firewall is not running.
[root@elk ~]#
[root@elk ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.232.0   0.0.0.0         255.255.255.0   U     1      0        0 eth0
0.0.0.0         192.168.232.2   0.0.0.0         UG    0      0        0 eth0
[root@elk ~]#
[root@elk ~]# mkdir /software
[root@elk ~]# mkdir /iso
[root@elk ~]# mkdir /backup
[root@elk ~]# mkdir /script
[root@elk ~]#
[root@elk ~]# hostname
elk
[root@elk ~]# 
[root@elk ~]# cat /etc/hosts
# Local
127.0.0.1       localhost

# Pub
192.168.232.151 elk
[root@elk ~]# 
[root@elk ~]# cat /etc/resolv.conf 
# Generated by NetworkManager
domain localdomain
search localdomain
nameserver 192.168.232.2
[root@elk ~]# 
[root@elk ~]# ping -c 3 baidu.com
PING baidu.com (111.13.101.208) 56(84) bytes of data.
64 bytes from 111.13.101.208: icmp_seq=1 ttl=128 time=36.8 ms
64 bytes from 111.13.101.208: icmp_seq=2 ttl=128 time=37.7 ms
64 bytes from 111.13.101.208: icmp_seq=3 ttl=128 time=48.4 ms

--- baidu.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 7706ms
rtt min/avg/max/mdev = 36.875/41.016/48.458/5.278 ms
[root@elk ~]#

[root@elk ~]# ifconfig

eth0 Link encap:Ethernet HWaddr 00:0C:29:CC:EC:E9

inet addr:192.168.232.151 Bcast:192.168.232.255 Mask:255.255.255.0

inet6 addr: fe80::20c:29ff:fecc:ece9/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:13209 errors:0 dropped:0 overruns:0 frame:0

TX packets:6425 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:16349985 (15.5 MiB) TX bytes:396212 (386.9 KiB)

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:65536 Metric:1

RX packets:16 errors:0 dropped:0 overruns:0 frame:0

TX packets:16 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:960 (960.0 b) TX bytes:960 (960.0 b)

[root@elk ~]#

[root@elk ~]# sestatus

SELinux status: disabled

[root@elk ~]#

[root@elk ~]# service iptables status

iptables: Firewall is not running.

[root@elk ~]#

[root@elk ~]# route -n

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

192.168.232.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0

0.0.0.0 192.168.232.2 0.0.0.0 UG 0 0 0 eth0

[root@elk ~]#

[root@elk ~]# mkdir /software

[root@elk ~]# mkdir /iso

[root@elk ~]# mkdir /backup

[root@elk ~]# mkdir /script

[root@elk ~]#

[root@elk ~]# hostname

elk

[root@elk ~]#

[root@elk ~]# cat /etc/hosts

# Local

127.0.0.1 localhost

# Pub

192.168.232.151 elk

[root@elk ~]#

[root@elk ~]# cat /etc/resolv.conf

# Generated by NetworkManager

domain localdomain

search localdomain

nameserver 192.168.232.2

[root@elk ~]#

[root@elk ~]# ping -c 3 baidu.com

PING baidu.com (111.13.101.208) 56(84) bytes of data.

64 bytes from 111.13.101.208: icmp_seq=1 ttl=128 time=36.8 ms

64 bytes from 111.13.101.208: icmp_seq=2 ttl=128 time=37.7 ms

64 bytes from 111.13.101.208: icmp_seq=3 ttl=128 time=48.4 ms

--- baidu.com ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 7706ms

rtt min/avg/max/mdev = 36.875/41.016/48.458/5.278 ms

[root@elk ~]#

我将会在这台服务器上搭建ELK架构[……]

Posted on: 2016年7月18日 2016年7月18日
Author: Adamhuan
Categories: ELK
Tags: Elasticsearch, ELK, Install, kibana, Linux, Logstash
Leave a comment: Deploy ELK on CentOS 6.8