ELK是什么？
轻量级日志统计分析组件，包含elasticsearch、filebeat、kibana
ELK环境准备
Elasticsearch 下载地址
https://www.elastic.co/downloads/past-releases/elasticsearch-6-4-2
Elasticsearch 参考文档
https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
Filebeat 下载地址
https://www.elastic.co/downloads/past-releases/filebeat-6-4-2
Filebeat 参考文档
https://www.elastic.co/guide/en/beats/filebeat/6.4/index.html
Kibana 下载地址
https://www.elastic.co/downloads/past-releases/kibana-6-4-2
Kibana 参考文档
https://www.elastic.co/guide/en/kibana/6.4/index.html
ELK安装
下载elasticsearch-6.4.2.tar.gz解压tar –zvxf elasticsearch-6.4.2.tar.gz下载filebeat-6.4.2-linux-x86_64.tar.gz解压 tar –zvxf filebeat-6.4.2-linux-x86_64.tar.gz下载kibana-6.4.2-linux-x86_64.tar.gz解压 tar –zvxf kibana-6.4.2-linux-x86_64.tar.gz
什么是Filebeat
如官网图片所示，Filebeat是一个轻量级的日志收集、过滤的中间件。可以向elasticsearch、logstash等推送数据。
Filebeat配置
Input配置filebeat.inputs:# 文档类型- type: log# 是否开启 true or falseenabled: true# 日志路径paths:- /var/log/*.log#- c:\programdata\elasticsearch\logs\*
设置Kibana
setup.kibana:host: "kibanahost:5601"
设置output
output.elasticsearch:# es服务器配置hosts: ["Elasticsearchhost:9200"]# 指定索引文件名称index: "filebeat-%{[beat.version]}-%{+yyyy.MM.dd}"
Filebeat Nginx Module
Filebeat默认已经支持很多Module，通过官网可知如下
参考文档：https://www.elastic.co/guide/en/beats/filebeat/6.4/filebeat-module-nginx.html
Nginx moduleedit The nginx module parses access and error logs created
by the Nginx HTTP server.
When you run the module, it performs a few tasks under the hood:
Sets the default paths to the log files (but don’t worry, you can
override the defaults) Makes sure each multiline log event gets sent
as a single event Uses ingest node to parse and process the log lines,
shaping the data into a structure suitable for visualizing in Kibana
Deploys dashboards for visualizing the log data Compatibilityedit This
module requires theingest-user-agentandingest-geoipElasticsearch
plugins.
The Nginx module was tested with logs from version 1.10.
On Windows, the module was tested with Nginx installed from the
Chocolatey repository.
官网已经明确说明，我们需要安装ingest-user-agent和ingest-geoip elasticsearch插件
这里说明一下，是要在elasticsearch下安装
安装ingest-user-agent
参考文档
https://www.elastic.co/guide/en/elasticsearch/plugins/6.4/ingest-user-agent.html
下载离线包
https://artifacts.elastic.co/downloads/elasticsearch-plugins/ingest-user-agent/ingest-user-agent-6.4.2.zip.
这里我们选择离线安装
https://www.elastic.co/guide/en/elasticsearch/plugins/6.4/plugin-management-custom-url.html
下载完成后通过以下命令安装插件
sudo bin/elasticsearch-plugin install file:///path/to/plugin.zip
file:///path/to/plugin.zip 为离线插件所在路径
安装ingest-geoip
参考文档
https://www.elastic.co/guide/en/elasticsearch/plugins/6.4/ingest-geoip.html
下载离线包
https://artifacts.elastic.co/downloads/elasticsearch-plugins/ingest-geoip/ingest-geoip-6.4.2.zip.
这里我们选择离线安装
https://www.elastic.co/guide/en/elasticsearch/plugins/6.4/plugin-management-custom-url.html
下载完成后通过以下命令安装插件
sudo bin/elasticsearch-plugin install file:///path/to/plugin.zip
file:///path/to/plugin.zip 为离线插件所在路径
开启nginx module
./filebeat modules enable nginx
通过以下命令查看是否开启成功
./filebeat modules list
配置nginx module
进入/home/elk/filebeat-6.4.2-linux-x86_64/modules.d 路径下
vi nginx.yml
这里配置自己的access.log和error.log日志路径即可，这里是作者环境下的nginx日志路径。
启动Elasticsearch
运行
./bin/elasticsearch -d
【需要创建非root账号运行】
chown -R elasticesearch:elasticesearch elasticsearch-6.4.2
这个时候可能发现除去本机以外的机器都无法访问,即使放开了9200端口也不行。解决方案如下：
/p/211982465c3b
启动Kibana
配置Kibana
设置当前IP
server.host: "Kibana Host"
后台运行
./bin/kibana &
启动filebeat
nohup ./filebeat -e -c filebeat.yml > filebeat.log &
启动 dashboard
./filebeat setup --dashboards
访问Kibana
一般是localhost:5601
Nginx Dashboard
找到Nginx模块
这里说明一下，如果部署和配置过程中有试错的情况 注意 • 启动了之后 filebeat.yml中 input 不需要改变 • 之前有数据模板不对 需要清空，否则无法使用
Ok 到了这里Nginx模块我们已经搭建完成。
