Modify by lyc at 2020-3-24
博文参考：
elastci中文官网
Elasticsearch7集群搭建

1.服务器基础配置

IP 角色 配置 系统
192.168.100.161 elasticsearch-node-01 8C/16G/500G CentOS7
192.168.100.162 elasticsearch-node-02 8C/16G/500G CentOS7
192.168.100.163 elasticsearch-node-03 8C/16G/500G CentOS7

配置hosts

1
2
3
4
$ vim /etc/hosts
192.168.100.161 elasticsearch-node-01
192.168.100.162 elasticsearch-node-02
192.168.100.163 elasticsearch-node-03

系统优化

1
2
3
4
5
6
7
8
$ vim /etc/security/limits.conf
* hard nofile 102400
* soft nofile 102400
* soft nproc 102400
* hard nproc 102400
# allow user 'elasticsearch' mlockall
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited

内核参数优化

1
2
3
$ vim /etc/sysctl.conf
vm.swappiness = 1
vm.max_map_count = 655360

2.elasticsearch 安装

下载安装

  • elasticsearch 7以上版本会自带jdk12的安装包，所以不需要另外安装jdk
  • elasticsearch 7以下版本需要另外安装jdk
1
2
3
4
5
6
7
8
9
10
# 下载安装
$ cd /usr/local/src
$ wget -c https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.1-x86_64.rpm
$ yum localinstall -y elasticsearch-7.5.1-x86_64.rpm

# 创建日志与数据目录
$ mkdir -pv /data/backup/elasticsearch_backup
$ mkdir -pv /data/elasticsearch/{elasticsearch_data,elasticsearch_logs}
$ chown elasticsearch:elasticsearch /data/backup/elasticsearch_backup
$ chown -R elasticsearch:elasticsearch /data/elasticsearch

elasticsearch相关文件句柄修改

1
2
3
4
5
6
$ mkdir -p /etc/systemd/system/elasticsearch.service.d
$ vim /etc/systemd/system/elasticsearch.service.d/override.conf
[Service]
DefaultLimitNOFILE=65536
DefaultLimitNPROC=32000
LimitMEMLOCK=infinity

开机自启动

1
2
$ systemctl daemon-reload
$ systemctl enable elasticsearch.service

3.配置文件

/etc/elasticsearch/elasticsearch.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
$ vim elasticsearch.yml
# ---------------------------------- Cluster -----------------------------------
cluster.name: inner-es-cluster   # 集群名称

# ------------------------------------ Node ------------------------------------
node.name: node-100161      # 节点名称

# ----------------------------------- Paths ------------------------------------
path.data: /data/elasticsearch/elasticsearch_data   # 数据目录
path.logs: /data/elasticsearch/elasticsearch_logs   # 日志目录
path.repo: ["/data/backup/elasticsearch_backup"]    #

# ----------------------------------- Memory -----------------------------------
bootstrap.memory_lock: true     # 锁住内存禁止交换

# ---------------------------------- Network -----------------------------------
network.host: 192.168.100.161   # 服务地址
http.port: 9200                 # 服务端口
transport.tcp.port: 9300        # 集群节点间通信端口

# --------------------------------- Discovery ----------------------------------
# 手动指定可以成为 mater 的所有节点的 name 或者 ip，这些配置将会在第一次选举中进行计算（可配多个）
cluster.initial_master_nodes: ["elasticsearch-node-01", "elasticsearch-node-02", "elasticsearch-node-03"]
# 集群发现种子节点列表
discovery.seed_hosts: ["elasticsearch-node-01", "elasticsearch-node-02", "elasticsearch-node-03"]
cluster.no_master_block: write

# ---------------------------------- Gateway -----------------------------------
gateway.recover_after_nodes: 2          # 重新启动整个集群直到N个节点启动后的块初始恢复

# ---------------------------------- Various -----------------------------------
action.destructive_requires_name: true  # 删除索引时需要显式名称
http.cors.enabled: true                 # 允许跨域
http.cors.allow-origin: "*"             # 允许跨域，源所有

/etc/elasticsearch/jvm.options

分配服务器一半的内存给JVM

1
2
3
4
$ vim jvm.options
....
-Xms8g
-Xmx8g

4.验证

pretty 参数标识响应以json格式输出
$domain 修改成负载均衡的域名
如有对es集群配置密码，$password 改成密码

1
2
3
4
5
6
7
8
9
10
11
# 访问数据节点
$ curl 192.168.100.161:9200

# 查看集群节点:
$ curl -XGET -u elastic:$password 'http://$domain/_cat/nodes?pretty'  

# 查看集群健康状态
$ curl -i -XGET -u elastic:$password 'http://$domain/_cluster/health?pretty'

# 查看索引
$ curl -XGET -u elastic:$password 'http://$domain/_cat/indices?v'

5.使用nginx负载均衡配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
$ cat $domain.conf 
upstream es_load_balancer {
    server 192.168.100.161:9200 max_fails=2 fail_timeout=5s;
    server 192.168.100.162:9200 max_fails=2 fail_timeout=5s;
    server 192.168.100.163:9200 max_fails=2 fail_timeout=5s;
    keepalive 60;
}

server {
    listen 80;
    server_name $domain;

    client_max_body_size 100m;
    client_body_buffer_size  256k;

    location / {
        proxy_pass http://es_load_balancer;
        proxy_redirect off;

        proxy_http_version 1.1;
        # proxy_set_header Connection "";
        proxy_set_header Connection "Keep-Alive";
        proxy_set_header Proxy-Connection "Keep-Alive";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;

        proxy_connect_timeout       90;
        proxy_read_timeout          90;
        proxy_send_timeout          90;

        proxy_buffer_size           128k;
        proxy_buffers               32 32k;
        proxy_busy_buffers_size     256k;
        proxy_temp_file_write_size  128k;
    }
    error_log  /data/httplogs/$domain-error.log;
    #access_log /data/httplogs/$domain-access.log weblog;
    access_log off;
}