Collectd , logstash and Elasticsearch

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Install Collectd from opencsw

Modify collectd.conf vi /etc/opt/csw/collectd.conf

Restart or enable collectd

forward the logs to elasticsearch with logstash

Source Monitoring With Collectd And Kibana Collectd plugins https://collectd.org/wiki/index.php/Table_of_Plugins/

Read More

Configuring Elasticsearch betas/topbeats on linux

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

First, lets Download the beats template.

Upload to elastic template

Now, lets install topbeat, add elastic beat repo, and run

Modify /etc/topbeat/topbeat.yml, under elasticsearch add elastic (or logstash) hosts

Start the beat service

Install topbeats dashboards

Finally, create the kibana index, [topbeat]-YYY.MM.DD in the web UI To verify the […]

Read More

Upgrading Elasticsearch To A Newer Version

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

How to upgrading Elasticsearch To A Newer Version The key is exporting the instance. Disable shard movement

Migrate to new version (on linux)

Migrate to new version (on solaris)

Re-enable shared movement, once the cluster is back up

Read More

Kafka To Flume PipeLing To HDFS And Elasticsearch

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

The below post describes how to configure Flum to grab data from a Kafaka topic as the source then send the data to an HDFS target. Pipeline flow

Create a run file

Create configuration files Create environment file

Create configuration file

Start the flume agent Kafka and Flume to HDFS

Read More

Elasticsearch datatypes

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Source https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html

Read More

Logstash Nagios Configuration Grok example (logstash.conf)

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Nagios logstash yml conf file

Read More

elasticserach mappings

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

sql-log mappings

Full template and mapping

Read More

Solaris Elasticsearch Forwarder(Logstash) Setup

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Create elastic data pool

Add elastic user

Install elastic binary’s

Grant access for elastic user

Configre system limits for user elastic /etc/project

add to /opt/elasticsearch/bin/elasticsearch (top)

Configure startup script

Configure elasticsearch.yml for hot warm (SSD) Hot warm config Configure hot template

Add latter the mappings

To […]

Read More