|
logstash安装 下载路径: https://www.elastic.co/downloads/logstash（安装方法参考官网安装步骤） 要读取nginx日志，配置nginx日志格式 vim nginx.conf 修改nginx记录日志格式，从http模块下 log_format main '$remote_addr | $time_local | $request | $uri | ' '$status | $body_bytes_sent | $bytes_sent | $gzip_ratio | $http_referer | ' '"$http_user_agent" | $http_x_forwarded_for | $upstream_addr | $upstream_response_time | $upstream_status | $request_time'; 修改完成后保存，使用./nginx -s reload 重新加载 /etc/logstash/conf.d下创建nginx日志配置文件 touch nginx_access.conf sudo vim nginx_access.conf input { file { path => [ "/usr/local/nginx/logs/adsapi.access.log" ] type => "nginx_access" } } filter { grok { match => [ "message", "%{IPORHOST:clientip} \| %{HTTPDATE:timestamp} \| (?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:http_version})?|-) \| %{URIPATH:uripath} \| %{NUMBER:response} \| (?:%{NUMBER:body_bytes_sent}|-) \| (?:%{NUMBER:bytes_sent}|-) \| (?:%{NOTSPACE:gzip_ratio}|-) \| (?:%{QS:http_referer}|-) \| %{QS:user_agent} \| (?:%{QS:http_x_forwarded_for}|-) \| (%{URIHOST:upstream_addr}|-) \| (%{BASE16FLOAT:upstream_response_time}) \| %{NUMBER:upstream_status} \| (%{BASE16FLOAT:request_time})" ] } geoip { source => "clientip" target => "geoip" add_field => [ "[geoip][coordinates]","%{[geoip][longitude]}" ] add_field => [ "[geoip][coordinates]","%{[geoip][latitude]}" ] } mutate { convert => [ "[geoip][coordinates]","float" ] } date { match => [ "timestamp","dd/MMM/yyyy:HH:mm:ss Z"] } mutate { remove_field => "timestamp" } } output { elasticsearch { hosts => ["127.0.*.*:9200"] index => "logstash-nginx-access-%{+YYYY.MM.dd}" user => "****" //下文安装kibana会设置 password => "pwd" } stdout { } } 2 . elasticsearch安装 下载地址：https://www.elastic.co/downloads/elasticsearch，安装步骤参见官网 安装完成后，从etc/elasticsearch/ 目录下 vim elasticsearch.yml cluster.name: elk node.name: es2 path.data: /data/elasticsearch（存储目录一定要给elasticsearch账户授权） bootstrap.memory_lock: false bootstrap.system_call_filter: false network.host: *.*.*.*(服务器ip) http.port: 9200 启动服务sudo service elasticsearch start 查看启动日志，或直接查看启动后的进程状态是否成功 elasticsearch (pid 19206) is running. 浏览器输入：http://ip地址:9200/,给出响应结果 3 . kibana安装 下载地址：https://www.elastic.co/downloads/kibana 安装x-pack ,下载地址：https://www.elastic.co/downloads/x-pack 自己安装的在/user/share/ 目录下 从etc/kibana/ 目录下修改kibana.yml文件 sudo vim kibana.yml server.name: "*.*.*.*"// (服务器ip地址) elasticsearch.url: "http://*.*.*.*:9200" elasticsearch.username: "username" elasticsearch.password: "pwd" 增加： tilemap.url: 'http://webrd02.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=7&x={x}&y={y}&z={z}' 配置完成后，三个服务依次启动 elasticsearch–kibana –logstash service elasticsearch start service kibana start initctl start logstash 4. 要外网访问需要配置nginx.conf,访问地址到kibana upstream elk { ip_hash; server 127.0.0.1:5601; } server { listen 80; server_name 域名; server_tokens off; client_body_timeout 5s; client_header_timeout 5s; location / { proxy_pass http://elk/; index index.html index.htm; proxy_redirect off; proxy_http_version 1.1; proxy_set_header Host $http_host; proxy_set_header Upgrade $http_upgrade; proxy_set_header X-Real-IP $remote_addr; } } 5. 配置完成后，重新加载nginx,浏览器输入域名，填写安装x-pack的用户名和密码 6. 登录成功后， 7. 在Configure an index pattern功能下配置：logstash-nginx-access*  8.创建成功后，选择discover模块就能查看到 (责任编辑：IT) |