devbox
Cpp五条
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

Centos7 elasticsearch-7.7.0 集群搭建，启用x-pack验证 Kibana7.4用户管理

作者：Cpp五条 | 2024-04-02 21:42:26

赞

踩

Centos7 elasticsearch-7.7.0 集群搭建，启用x-pack验证 Kibana7.4用户管理

前言

Elasticsearch 是一个分布式、RESTful 风格的搜索和数据分析引擎，能够解决不断涌现出的各种用例。 作为 Elastic Stack 的核心，它集中存储您的数据，帮助您发现意料之中以及意料之外的情况。

环境准备

软件                          OS系统      IP                     节点
Elasticsearch7.7.0    centos 7    10.16.101.27    node-1
Elasticsearch7.7.0    centos 7    10.16.101.28    node-2
Elasticsearch7.7.0    centos 7    10.16.101.29    node-3 

1.JDK环境安装以及配置

Elasticsearch7.7.0需要jdk11的允许环境，所有我们安装Jdk11

  1. # 已经JDK保存在OSS，所有从OSS下载
  2. cd root
  3. wget https://starshop.obs.cn-south-1.myhuaweicloud.com/other/jdk-11.0.14_linux-x64_bin.tar.gz
  4.  
  5. # 解压 jdk-11.0.14_linux-x64_bin.tar.gz 移动到/usr/java/jdk11目录下
  6. tar xf jdk-11.0.14_linux-x64_bin.tar.gz
  7. mv jdk-11.0.14_linux-x64_bin /usr/java/jdk11
  8.  
  9. # 赋予java 执行权限
  10. chmod 777 /usr/java/jdk11/bin/java
  11. chmod 777 /usr/java/jdk11/jre/bin/java

 配置jdk环境变量

  1. 配置环境变量
  2. vi /etc/profile
  3. 在文本末添加以下内容
  4. ---------------------------------------------------------------
  5. #java环境变量
  6. export JAVA_HOME=/usr/java/jdk1.8
  7. export PATH=$JAVA_HOME/bin:$PATH
  8. export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
  9. ----------------------------------------------------------------
  10. 执行一下命令 立即生效
  11. source  /etc/profile
  12.  
  13. 查看java版本
  14. java -version

2.开端口

  1. # Elasticsearch7.7.0 需要开启9200 9300 9100端口进行通信
  2. firewall-cmd --zone=public --add-port=9100/tcp --permanent && firewall-cmd --zone=public --add-port=9200/tcp --permanent && firewall-cmd --zone=public --add-port=9300/tcp --permanent && firewall-cmd --reload

3.创建新用户，用户启动Elasticsearch

  1. # 因为Elasticsearch不能使用root用户启动，所有必须新建用户
  2. # 新建用户
  3. adduser es  
  4. # 给新用户添加密码
  5. passwd es

安装 Elasticsearch7.7.0

https://www.elastic.co/cn/downloads/past-releases#elasticsearch

  1. # 下载es
  2. cd /root
  3. wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.7.0-linux-x86_64.tar.gz
  4. # 解压文件到,移动到/usr/local/elastic/elasticsearch7
  5. tar xf elasticsearch-7.7.0-linux-x86_64.tar.gz
  6. mv elasticsearch-7.7.0-linux-x86_64 /usr/local/elastic/elasticsearch7
  7. # 将elasticsearch7目录及路径下所有文件所有者均设为es用户
  8. chown es:es -R /usr/local/elastic/elasticsearch7

1.修改系统配置文件

  1. # 打开sysctl.conf文件。Linux默认不允许任何用户和应用程序直接开辟这么大的虚拟内存
  2. vi /etc/sysctl.conf
  3. # 文件sysctl.conf末尾加入
  4. vm.max_map_count=655360
  5. # 保存退出后，为了让系统控制权限配置生效
  6. sysctl -p

2.修改limits.conf文件

  1. vi /etc/security/limits.conf
  2. # 在文件内结束符前加上下面给出的几行代码,保存即可
  3. * soft nofile 65536
  4. * hard nofile 131072
  5. * soft nproc 4096
  6. * hard nproc 4096
  7. * soft memlock unlimited
  8. * hard memlock unlimited

3.修改elasticsearch.yml(在27,28,29 编辑elasticsearch.yml)

  1. # 切换到es用户
  2. su es
  3. # 编辑elasticsearch.yml
  4. cd /usr/local/elastic/elasticsearch7/config
  5. vi /usr/local/elastic/elasticsearch7/config/elasticsearch.yml
  6. ########################下面是文件内容########################################
  7. # 集群名称，同一集群下每台服务器的名称要一致否则会报错
  8. cluster.name: my-elasticsearch
  9. # 节点名称，集群下每台机的节点名称唯一（举例的机器节点名称设为node-1，其他分别为node-2，node-3）
  10. node.name: node-1
  11. # ES数据储存路径（保证该目录存在不存在则报错）
  12. path.data: /usr/local/elastic/elasticsearch7/data
  13. # ES运行日志文件路径（保证该目录存在不存在则报错）
  14. path.logs: /usr/local/elastic/elasticsearch7/logs
  15. # 需求锁住物理内存，避免操作系统的内存Swaping
  16. bootstrap.memory_lock: true
  17. # 外界访问ES ip地址(设置0.0.0.0表示可以通过该机器的任何ip访问)
  18. network.host: 0.0.0.0
  19. # 加入此配置可以解决集群的警告
  20. network.publish_host: 10.16.101.27
  21. # ES的访问端口号（不设置则默认为9200）
  22. http.port: 9200
  23. # 集群下所有机器的访问ES的url集合
  24. discovery.seed_hosts: ["10.16.101.27:9300", "10.16.101.28:9300", "10.16.101.29:9300"]
  25. # 集群下所有节点集合
  26. cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
  27.

4 . 修改elasticsearch的VM配置

vi /usr/local/elastic/elasticsearch7/config/jvm.options

修改VM为机器内存的一半

启动elasticsearch

1.设置elasticsearch开机启动

  1. # 切换到root用户
  2. su  root
  3. # 切换到/etc/rc.d/init.d目录
  4. cd /etc/rc.d/init.d
  5. # 编辑elasticsearch启动文件
  6. vi elasticsearch
  7. ####################### 贴入下面的文本保存即可 ###############################
  8. #!/bin/sh
  9. #chkconfig: 2345 80 05
  10. #description: elasticsearch 6.2.2
  11. export JAVA_HOME=/usr/java/jdk11 
  12. export ES_PATH=/usr/local/elastic/elasticsearch7
  13. case "$1" in
  14. start)
  15.     su es<<!
  16.     cd $ES_PATH
  17.     ./bin/elasticsearch -d -p pid
  18. !
  19.     echo "elasticsearch startup"
  20.     ;;  
  21. stop)
  22.     es_pid=`ps aux|grep elasticsearch | grep -v 'grep elasticsearch' | awk '{print $2}'`
  23.     kill -9 $es_pid
  24.     echo "elasticsearch stopped"
  25.     ;;  
  26. restart)
  27.     es_pid=`ps aux|grep elasticsearch | grep -v 'grep elasticsearch' | awk '{print $2}'`
  28.     kill -9 $es_pid
  29.     echo "elasticsearch stopped"
  30.     su es<<!
  31.     cd $ES_PATH
  32.     ./bin/elasticsearch -d -p pid
  33. !
  34.     echo "elasticsearch startup"
  35.     ;;  
  36. *)
  37.     echo "start|stop|restart"
  38.     ;;  
  39. esac
  40. exit $?
  41.  
  42. # 刚编辑的文件需要赋予权限
  43. chmod -X elasticsearch 
  44. # 添加到系统服务
  45. chkconfig --add elasticsearch
  46.  
  47. # 开启服务
  48. service elasticsearch start
  49. # 停止服务
  50. service elasticsearch stop
  51. # 重启服务
  52. service elasticsearch restart
  53. # 设置开机启动
  54. chkconfig elasticsearch on

在浏览器中输入 http://10.16.101.27:9200 代表安装成功
在这里插入图片描述查看集群是否成功 在浏览器输出入 http://10.16.101.27:9200/_cat/health?v 显示3节点代表成功

中文分词和拼音分词器安装

中文ik下载:https://github.com/medcl/elasticsearch-analysis-ik/releases
拼音分词器下载:https://github.com/medcl/elasticsearch-analysis-pinyin/releases

根据elasticsearch版本下载对应的分词器版本,必须版本对应

  1. # 文件下载在root目录下
  2. cd /root
  3. #下载ik
  4. wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.7.0/elasticsearch-analysis-ik-7.7.0.zip
  5. # 下载pinyin
  6. wget https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v7.7.0/elasticsearch-analysis-pinyin-7.7.0.zip
  7.  
  8. # 在elasticsearch 的plugins目录下新疆ik和pinyin文件夹
  9. mkdir /usr/local/elastic/elasticsearch7/plugins/ik
  10. mkdir /usr/local/elastic/elasticsearch7/plugins/pinyin
  11. # unzip 将上面的2个zip直接解压到对应ik和pinyin目录下
  12. unzip elasticsearch-analysis-ik-7.7.0.zip -d /usr/local/elastic/elasticsearch7/plugins/ik
  13. unzip elasticsearch-analysis-pinyin-7.7.0.zip -d /usr/local/elastic/elasticsearch7/plugins/pinyin
  14. # 赋予权限，重启elasticsearch
  15. chown es:es -R /usr/local/elastic/elasticsearch7
  16. service elasticsearch restart

使用crul命令，输入下面的URL地址，验证分词器是否成功

curl -X GET -H "Content-Type: application/json"  "http://localhost:9200/_analyze?pretty=true" -d'{"text":"中华五千年华夏"}';

若elasticsearch无法关闭，可使用kill -9 pid 强制关闭在重启

  1. # 查看pid
  2. ps aux|grep elasticsearch
  3. # 强制关闭
  4. kill -9 xx
  5. # 启动elasticsearch
  6. service elasticsearch start

安装浏览器ealsticsearch-head插件

ealsticsearch只是后端提供各种api，那么怎么直观的使用它呢？elasticsearch-head将是一款专门针对于elasticsearch的客户端工具
elasticsearch-head 需要nodejs的支持

1.下载安装nodejs

  1. # 文件下载在root下
  2. cd /root
  3. # 下载最新稳定版nodejs
  4. wget https://nodejs.org/dist/v16.15.1/node-v16.15.1-linux-x64.tar.xz
  5. # 解压文件
  6. tar xf node-v16.15.1-linux-x64.tar.xz
  7. # 移动解压后目录到软件目录,并将目录重命名nodejs    
  8. mv node-v16.15.1-linux-x64 /usr/local/nodejs
  9. # 查看node的版本
  10. /usr/local/nodejs/bin/node -v
  11. # 配置环境变量
  12. vim /etc/profile
  13.  
  14. #################################将下面文本贴入profile文件中保存###################
  15. # 添加node_home变量
  16. export NODE_HOME=/usr/local/nodejs
  17. # 把node_home变量添加到环境变量中
  18. export PATH=$PATH:$NODE_HOME/bin
  19.  
  20. # 重新加载一下配置文件使新的配置生效
  21. source /etc/profile
  22. # 查看nodejs版本
  23. node -version

至此，centos7上的nodejs运行环境安装成功，下面我们安装elasticsearch-head

2.安装grunt构建工具

可以进行打包压缩、测试、执行等等的工作，head插件就是通过grunt启动的。因此需要安装grunt

  1. # 切换到nodejs安装目录
  2. cd /usr/local/nodejs
  3. # 配置阿里镜像
  4. npm config set registry https://registry.npm.taobao.org
  5. # 安装grunt
  6. npm install -g grunt-cli
  7. # 查看是否安装成功
  8. grunt -version

2.下载安装elasticsearch-head插件 GitHub - mobz/elasticsearch-head: A web front end for an elastic search cluster下载elasticsearch-head-master.zip，将文件拷贝到/root目录下

  1. cd /root
  2. # 解压elasticsearch-head-master
  3. unzip elasticsearch-head-master.zip
  4. # 将文件移动到/usr/local/elastic/elasticsearch-head目录
  5. mv elasticsearch-head-master /usr/local/elastic/elasticsearch-head
  6. # 切换到/usr/local/elastic/elasticsearch-head目录下
  7. cd /usr/local/elastic/elasticsearch-head
  8. # 编译elasticsearch-head
  9. npm install
  10. # 运行 elasticsearch-head 后台运行
  11. nohup npm run start &

浏览器中输入上面的地址http://localhost:9100，如图说明head插件已经安装成功了。

在这里插入图片描述

上图不能显示集群健康值,需要在配置一下elasticsearch.yml

  1. cd /usr/local/elastic/elasticsearch7/config
  2. vi elasticsearch.yml
  3. # 在elasticsearch.yml文件中加入下面配置,保存即可
  4. http.cors.enabled: true 
  5. http.cors.allow-origin: "*"
  6. # 重启 elasticsearch
  7. service elasticsearch restart

3.ealsticsearch-head 开机启动 

上面的配置ealsticsearch-head Xshell客户端关闭，后导致9100端口也关闭，所有必须配置开机自启动

  1. cd /etc/init.d
  2. # 新建elasticsearch-head启动脚本
  3. vi elasticsearch-head
  4. ################## 写入下面的文本,根据自己时间安装情况配置#####################
  5. #!/bin/sh
  6. #chkconfig: 2345 80 05
  7. #description: elasticsearch-head
  8. # nodejs 安装的路径
  9. export NODE_PATH=/usr/local/nodejs
  10. export PATH=$PATH:$NODE_PATH/bin
  11. # elasticsearch-head 的路径
  12. cd /usr/local/elastic/elasticsearch-head
  13. nohup npm run start >/usr/local/elastic/elasticsearch-head/nohup.out 2>&1 &
  14. # 赋予 elasticsearch-head 权限
  15. chmod +x elasticsearch-head
  16. # 设置开机执行sh
  17. chkconfig --add elasticsearch-head
  18. # 也可手动启动
  19. service elasticsearch-head start

启用x-pack验证

第一种

切换到es的用户下，使用下面命令生成证书

bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""

启动elasticsearch

./elasticsearch -d

自动生成默认用户和密码

  1. bin/elasticsearch-setup-passwords auto
  2. # 手动设置
  3. # bin/elasticsearch-setup-passwords interactive

修改elastic超级用户密码

  1. #  假设elastic 默认密码为 xxxxxxx
  2. # curl -XPUT -u elastic:xxxxxxx 'http://localhost:9200/_xpack/security/user/elastic/_password' -d '{ "password" : "your_passwd" }'

浏览器验证

 

第二种

启动 Elasticsearch 程序

  1. [elastic@console bin]$ ./elasticsearch -d
  2.     future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_181/jre] does not meet this requirement

创建密码

  1. [elastic@console bin]$ ./elasticsearch-setup-passwords interactive
  2. future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_181/jre] does not meet this requirement
  3.  
  4. Unexpected response code [500] from calling GET http://192.168.108.126:9200/_security/_authenticate?pretty
  5. It doesn't look like the X-Pack security feature is enabled on this Elasticsearch node.
  6. Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.
  8. ERROR: X-Pack Security is disabled by configuration.

需要设置 X-Pack

  1. [elastic@console bin]$ vim ../config/elasticsearch.yml
  2.     http.cors.enabled: true
  3.     http.cors.allow-origin: "*"
  4.     http.cors.allow-headers: Authorization
  5.     xpack.security.enabled: true
  6.     xpack.security.transport.ssl.enabled: true

添加密码

  1. [elastic@console bin]$ ./elasticsearch-setup-passwords interactive
  2. future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_181/jre] does not meet this requirement
  3. Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
  4. You will be prompted to enter passwords as the process progresses.
  5. Please confirm that you would like to continue [y/N]y
  6.  
  7.  
  8. Enter password for [elastic]:
  9. Reenter password for [elastic]:
  10. Passwords do not match.
  11. Try again.
  12. Enter password for [elastic]:
  13. Reenter password for [elastic]:
  14. Enter password for [apm_system]:
  15. Reenter password for [apm_system]:
  16. Enter password for [kibana]:
  17. Reenter password for [kibana]:
  18. Enter password for [logstash_system]:
  19. Reenter password for [logstash_system]:
  20. Enter password for [beats_system]:
  21. Reenter password for [beats_system]:
  22. Enter password for [remote_monitoring_user]:
  23. Reenter password for [remote_monitoring_user]:
  24. Changed password for user [apm_system]
  25. Changed password for user [kibana]
  26. Changed password for user [logstash_system]
  27. Changed password for user [beats_system]
  28. Changed password for user [remote_monitoring_user]
  29. Changed password for user [elastic]

修改kibana

  1. [root@console bin]# vim ../config/kibana.yml
  2. 	elasticsearch.username: "elastic"
  3. 	elasticsearch.password: "passwd"

 修改密码

  1. POST /_security/user/elastic/_password
  2. {
  3.   "password": "123456"
  4. }

修改密码之后，需要重新设置kibana的配置文件，才可以重新使用kibana

java 连接认证

  1. package elasticSearch.highLevelClient;
  2.  
  3. import org.apache.http.HttpHost;
  4. import org.apache.http.auth.AuthScope;
  5. import org.apache.http.auth.UsernamePasswordCredentials;
  6. import org.apache.http.client.CredentialsProvider;
  7. import org.apache.http.impl.client.BasicCredentialsProvider;
  8. import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
  9. import org.elasticsearch.client.RestClient;
  10. import org.elasticsearch.client.RestClientBuilder;
  11. import org.elasticsearch.client.RestHighLevelClient;
  12. import org.elasticsearch.client.XPackClient;
  13.  
  14. import java.io.IOException;
  15.  
  16. /**
  17.  * @author [tu.tengfei]
  18.  * @description
  19.  * @date 2019/8/10
  20.  */
  21. public class ESHighClient {
  22.     public static RestHighLevelClient client;
  23.     public static void getESClient(){
  24.  
  25. //        client = new RestHighLevelClient(RestClient.builder(
  26. //                new HttpHost("slave01", 9200, "http"),
  27. //                new HttpHost("master01", 9200, "http")
  28. //        ));
  29.  
  30.         //需要用户名和密码的认证
  31.         final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
  32.         credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "passwd"));
  33.         RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost("192.168.108.126", 9200, "http"))
  34.                 .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
  35.                     @Override
  36.                     public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpAsyncClientBuilder) {
  37.                         return httpAsyncClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
  38.                     }
  39.                 });
  40.         client = new RestHighLevelClient(restClientBuilder);
  41.     }
  42.  
  43.     public static void clientClose(){
  44.         if (client!=null){
  45.             try {
  46.                 client.close();
  47.             } catch (IOException e) {
  48.                 e.printStackTrace();
  49.             }
  50.         }
  51.     }
  52. }
  53.

 

 Kibana7.4用户管理

1.下载kibana以及修改相应配置

  1. cd /opt
  2. wget https://artifacts.elastic.co/downloads/kibana/kibana-7.4.2-linux-x86_64.tar.gz
  3. tar -zxvf kibana-7.4.2-linux-x86_64.tar.gz

2.修改配置

cd /opt/kibana-7.4.2-linux-x86_64/config
vim kibana.yml

  1. # 中文
  2. i18n.locale: "zh-CN"
  3. server.port: 15601
  4. server.host: "0.0.0.0"
  5. elasticsearch.hosts: ["http://192.168.18.126:19200"]
  6. elasticsearch.username: "kibana_system"
  7. elasticsearch.password: "xxxxx"
  8. kibana.index: ".kibana"
  9. xpack.reporting.encryptionKey: "a_random_string"
  10. xpack.security.encryptionKey: "something_at_least_32_characters"
  11. # 是否开启安全策略
  12. # xpack.security.enabled: false

3.kibana用户管理

/opt/kibana-7.4.2-linux-x86_64/bin/kibana --allow-root

添加用户并赋予相应的权限
在这里插入图片描述 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Cpp五条/article/detail/353871
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号