<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Elasticsearch Archives - Linux Windows and android Tutorials</title>
	<atom:link href="https://www.osradar.com/tag/elasticsearch/feed/" rel="self" type="application/rss+xml" />
	<link>https://www.osradar.com</link>
	<description>tutorials and news and Seurity</description>
	<lastBuildDate>Mon, 21 Oct 2019 14:14:11 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=5.8.12</generator>
	<item>
		<title>How to Install ELK Stack on Centos 8?</title>
		<link>https://www.osradar.com/how-to-install-elk-stack-on-centos-8/</link>
					<comments>https://www.osradar.com/how-to-install-elk-stack-on-centos-8/#respond</comments>
		
		<dc:creator><![CDATA[sabi]]></dc:creator>
		<pubDate>Tue, 22 Oct 2019 02:17:25 +0000</pubDate>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Servers]]></category>
		<category><![CDATA[Technology]]></category>
		<category><![CDATA[Tutorials]]></category>
		<category><![CDATA[Unix]]></category>
		<category><![CDATA[Centos 8]]></category>
		<category><![CDATA[Elasticsearch]]></category>
		<category><![CDATA[ELK]]></category>
		<category><![CDATA[Kibana]]></category>
		<category><![CDATA[Logstash]]></category>
		<guid isPermaLink="false">https://www.osradar.com/?p=14775</guid>

					<description><![CDATA[<p>What is ELK?&#160; ELK is the group of three open source projects in Linux.&#160; Elastic Search, Logstash and Kibana respectively. Before going into depth let&#8217;s have a short definition about these: Elasticsearch: This is an open source distribution, reliable, scalable, easy to use and flexible Lucene library based search engine.&#160;It provides multitenant-capable text with an [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/how-to-install-elk-stack-on-centos-8/">How to Install ELK Stack on Centos 8?</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<h3><strong> What is ELK?&nbsp;</strong></h3>



<p>ELK is the group of three open source projects in Linux.&nbsp; Elastic Search, Logstash and Kibana respectively. Before going into depth let&#8217;s have a short definition about these:</p>



<h3><strong>Elasticsearch: </strong></h3>



<p>This is an open source distribution, reliable, scalable, easy to use and flexible Lucene library based search engine.&nbsp;It provides multitenant-capable text with an HTTP web interface.</p>



<h3><strong>Logstash: </strong></h3>



<p>It is an open source tool used to store data, collect information, and store it for further use. Kibana is used to retrieve the logs stored by Logstash.</p>



<h3><strong>Kibana:</strong></h3>



<p> Let&#8217;s you transform your data into your own format or specific shape like charts and graphs in Elasticsearch.&nbsp;</p>



<p>So, in this article we will cover the following :</p>



<ul><li><strong>How to Install Java on Centos 8</strong></li></ul>



<ul><li><strong>How to add ELK repository to Centos 8</strong></li></ul>



<ul><li><strong>How to install and Configure Elasticsearch&nbsp;</strong></li></ul>



<ul><li><strong>How to instaall and configure Kibana on Centos 8</strong></li></ul>



<ul><li><strong>How to install and configure Logstash on Centos 8</strong></li></ul>



<ul><li><strong>How to install other ELK tools(Optional)</strong></li></ul>



<h3><strong>Step 1: Install Java on Centos 8</strong></h3>



<p>Before installing Elasticsearch we must have java installed on our system as Elasticsearch depends on java.&nbsp; So install it before further proceeding.</p>



<p><a href="https://www.osradar.com/install-java-on-rhel-8-centos-8-oracle-linux-8/">How to install Java 11 (OpenJdk 11 on RHEL / Centos 8</a></p>



<h3><strong>Step 2: Add ELK repository to Centos 8</strong></h3>



<p>After installing java, add ELK repository&nbsp; to Centos 8 and run the following command as Sudo.</p>



<h3><strong>For</strong> <strong>Elasticsearch 7.x&nbsp;</strong></h3>



<p class="has-background has-cyan-bluish-gray-background-color">cat &lt;&lt;EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo</p>



<p class="has-background has-cyan-bluish-gray-background-color">[elasticsearch -7.x]</p>



<p class="has-background has-cyan-bluish-gray-background-color">name=Elasticsearch repository for 7.x packages</p>



<p class="has-background has-cyan-bluish-gray-background-color">baseurl=https://artifacts.elasticsearch.co/packages/7.x/yum</p>



<p class="has-background has-cyan-bluish-gray-background-color">gpgcheck=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticse</p>



<p class="has-background has-cyan-bluish-gray-background-color">enabled=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">autorefresh=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">type=rpm-md</p>



<p class="has-background has-cyan-bluish-gray-background-color">EOF</p>



<h3><strong> For Elasticsearch 6.x </strong></h3>



<p class="has-background has-cyan-bluish-gray-background-color">cat &lt;&lt;EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo</p>



<p class="has-background has-cyan-bluish-gray-background-color">[elasticsearch -6.x]</p>



<p class="has-background has-cyan-bluish-gray-background-color">name=Elasticsearch repository for 6.x packages</p>



<p class="has-background has-cyan-bluish-gray-background-color">baseurl=https://artifacts.elasticsearch.co/packages/6.x/yum</p>



<p class="has-background has-cyan-bluish-gray-background-color">gpgcheck=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticse</p>



<p class="has-background has-cyan-bluish-gray-background-color">enabled=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">autorefresh=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">type=rpm-md</p>



<p class="has-background has-cyan-bluish-gray-background-color">EOF</p>



<h3><strong>For</strong> <strong>Elasticsearch 5.x</strong></h3>



<p class="has-background has-cyan-bluish-gray-background-color">cat &lt;&lt;EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo</p>



<p class="has-background has-cyan-bluish-gray-background-color">[elasticsearch -5.x]</p>



<p class="has-background has-cyan-bluish-gray-background-color">name=Elasticsearch repository for 5.x packages</p>



<p class="has-background has-cyan-bluish-gray-background-color">baseurl=https://artifacts.elasticsearch.co/packages/5.x/yum</p>



<p class="has-background has-cyan-bluish-gray-background-color">gpgcheck=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticse</p>



<p class="has-background has-cyan-bluish-gray-background-color">enabled=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">autorefresh=1</p>



<p class="has-background has-cyan-bluish-gray-background-color">type=rpm-md</p>



<p class="has-background has-cyan-bluish-gray-background-color">EOF</p>



<p>After doing so, import GPG key</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo rpm &#8211;import https://artifacts.elastic.co/GPG-KEY-elasticsearch</p>



<p>

Now,&nbsp; clear and update your YUM package index.&nbsp;

</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo yum clean all</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo yum makecache</p>



<div class="wp-block-image"><figure class="aligncenter"><img loading="lazy" width="780" height="563" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/elasticsearch-repo.png" alt="" class="wp-image-14787" srcset="https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-repo.png 780w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-repo-300x217.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-repo-768x554.png 768w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-repo-324x235.png 324w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-repo-696x502.png 696w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-repo-582x420.png 582w" sizes="(max-width: 780px) 100vw, 780px" /></figure></div>



<h3><strong>Step 3: Install and Configure Elasticsearch</strong></h3>



<p> As we have done with Elasticsearch repository and now it&#8217;s ready to use. Now make sure to run this command to install Elasticsearch.</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo yum -y install elasticsearch</p>



<div class="wp-block-image"><figure class="aligncenter"><img loading="lazy" width="799" height="588" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/elasticsearch-installed.png" alt="" class="wp-image-14816" srcset="https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-installed.png 799w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-installed-300x221.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-installed-768x565.png 768w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-installed-80x60.png 80w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-installed-696x512.png 696w, https://www.osradar.com/wp-content/uploads/2019/10/elasticsearch-installed-571x420.png 571w" sizes="(max-width: 799px) 100vw, 799px" /></figure></div>



<p>Double-check that installation completed successfully.&nbsp;</p>



<p class="has-background has-cyan-bluish-gray-background-color">rpm -qi elasticsearch</p>



<div class="wp-block-image"><figure class="aligncenter"><img loading="lazy" width="734" height="503" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/query-es.png" alt="" class="wp-image-14817" srcset="https://www.osradar.com/wp-content/uploads/2019/10/query-es.png 734w, https://www.osradar.com/wp-content/uploads/2019/10/query-es-300x206.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/query-es-100x70.png 100w, https://www.osradar.com/wp-content/uploads/2019/10/query-es-218x150.png 218w, https://www.osradar.com/wp-content/uploads/2019/10/query-es-696x477.png 696w, https://www.osradar.com/wp-content/uploads/2019/10/query-es-613x420.png 613w" sizes="(max-width: 734px) 100vw, 734px" /></figure></div>



<p>Set up the JVM options like memory limits and others according to your own needs. For this edit the following file:</p>



<p>Here we have set up maximum size of total heap space.</p>



<p class="has-background has-cyan-bluish-gray-background-color"> /etc/elasticsearch/jvm.options</p>



<div class="wp-block-image"><figure class="aligncenter"><img loading="lazy" width="716" height="111" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/heapsize.png" alt="" class="wp-image-14818" srcset="https://www.osradar.com/wp-content/uploads/2019/10/heapsize.png 716w, https://www.osradar.com/wp-content/uploads/2019/10/heapsize-300x47.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/heapsize-696x108.png 696w" sizes="(max-width: 716px) 100vw, 716px" /></figure></div>



<p>You can adjust according to your system requirements.&nbsp;</p>



<p>Now,&nbsp; start and enable the Elasticsearch services.</p>



<figure class="wp-block-image"><img loading="lazy" width="737" height="130" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/enable-elasticsearch-services.png" alt="" class="wp-image-14819" srcset="https://www.osradar.com/wp-content/uploads/2019/10/enable-elasticsearch-services.png 737w, https://www.osradar.com/wp-content/uploads/2019/10/enable-elasticsearch-services-300x53.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/enable-elasticsearch-services-696x123.png 696w" sizes="(max-width: 737px) 100vw, 737px" /></figure>



<p>Make sure these are properly working.&nbsp;</p>



<div class="wp-block-image"><figure class="aligncenter"><img loading="lazy" width="756" height="347" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/test-to-verify-elasticsearch.png" alt="" class="wp-image-14820" srcset="https://www.osradar.com/wp-content/uploads/2019/10/test-to-verify-elasticsearch.png 756w, https://www.osradar.com/wp-content/uploads/2019/10/test-to-verify-elasticsearch-300x138.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/test-to-verify-elasticsearch-696x319.png 696w" sizes="(max-width: 756px) 100vw, 756px" /></figure></div>



<p>Let&#8217;s create a test index.</p>



<p class="has-background has-cyan-bluish-gray-background-color">curl -X PUT &#8220;http://127.0.0.1:9200/mytest_index&#8221;</p>



<h3><strong>Step 4: Install / Configure Kibana on Centos 8</strong></h3>



<p>From added Elasticsearch repository download and install kibana.&nbsp;</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo yum -y install kibana</p>



<div class="wp-block-image"><figure class="aligncenter"><img loading="lazy" width="729" height="408" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/installed-kibana.png" alt="" class="wp-image-14823" srcset="https://www.osradar.com/wp-content/uploads/2019/10/installed-kibana.png 729w, https://www.osradar.com/wp-content/uploads/2019/10/installed-kibana-300x168.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/installed-kibana-696x390.png 696w" sizes="(max-width: 729px) 100vw, 729px" /></figure></div>



<p>Configure it after installation completed.&nbsp;</p>



<p class="has-background has-cyan-bluish-gray-background-color"><strong>sudo vim /etc/kibana/kibana.yml </strong><br> server.host: &#8220;<strong>0.0.0.0</strong>&#8220;<br> server.name: &#8220;<strong>kibana.example.com&#8221;</strong><br> elasticsearch.url: &#8220;<strong>http://localhost:9200</strong>&#8220;</p>



<p>Set up other settings to your own requirements and start kibana services.</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo systemctl enable &#8211;now kibana</p>



<p>Visit&nbsp;<a href="http://osradar.com" target="_blank" rel="noreferrer noopener" aria-label="http://ip-address:5601 (opens in a new tab)">http://ip-address:5601</a>&nbsp;to open kibana dashboard&nbsp;</p>



<div class="wp-block-image"><figure class="aligncenter"><img loading="lazy" width="714" height="252" src="//1723336065.rsc.cdn77.org/wp-content/uploads/2019/10/kibana-dashboard.png" alt="" class="wp-image-14822" srcset="https://www.osradar.com/wp-content/uploads/2019/10/kibana-dashboard.png 714w, https://www.osradar.com/wp-content/uploads/2019/10/kibana-dashboard-300x106.png 300w, https://www.osradar.com/wp-content/uploads/2019/10/kibana-dashboard-696x246.png 696w" sizes="(max-width: 714px) 100vw, 714px" /></figure></div>



<p>If you have firewall service active make sure to allow TCP port 5601.</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo firewall-cmd &#8211;add-port=5601/tcp &#8211;permanent<br>
sudo firewall-cmd &#8211;reload</p>



<h3><strong>Step 4: Install / Configure Logstash on Centos 8</strong></h3>



<p>The last step is to install and configure Logstash which will act like a centralized logs server for your client systems and runs an agent like filebeat.</p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo yum -y install logstash</p>



<p>Customize settings under the following directory: <strong>/etc/logstash/conf.d</strong>/ For further information you can check out <a href="https://www.elastic.co/guide/en/logstash/current/index.html">Logstash configuration manual.&nbsp;</a></p>



<h3><strong>Step 5: Install other ELK tools &#8211; (optional)&nbsp;</strong></h3>



<p>Some of these tools help you to work smoothly.</p>



<h4><strong>Filebeat:</strong></h4>



<p>It makes things simple by following lightweight way to forward and centralized logs and files.&nbsp;</p>



<h4><strong>Metricbeat:</strong></h4>



<p>Helps you to send and collect metrics from your systems and services, from CPU to memory,&nbsp; Redis to NGINX,&nbsp; and many more.&nbsp; It&#8217;s also a lightweight way to access system and services statistics.</p>



<h4><strong>Packetbeat:</strong></h4>



<p>Packetbeat provides a lightweight way for Network Data to increase performance.</p>



<h4><strong>Heartbeat:</strong></h4>



<p>Monitors the up time of Services. Helps you to know Availability of services.&nbsp;</p>



<h4><strong>Auditbeat:</strong></h4>



<p>Useful for auditing the activities and processes on your system by users.&nbsp;The tools we have discussed so far can be installed with the give command one time or you can install individually by this command. </p>



<p class="has-background has-cyan-bluish-gray-background-color">sudo yum install filebeat auditbeat metricbeat packetbeat heartbeat-elastic</p>



<p>These add-on tools help you better experience.To configure any tool you can check&nbsp; <a href="https://www.elastic.co/guide/index.html">official ELK stack documentation.</a></p>



<p>Hope you are all done!&nbsp;&nbsp;If have any queries regarding this tutorial leave a comment! </p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/how-to-install-elk-stack-on-centos-8/">How to Install ELK Stack on Centos 8?</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://www.osradar.com/how-to-install-elk-stack-on-centos-8/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Install ElasticSearch via Ansible</title>
		<link>https://www.osradar.com/install-elasticsearch-via-ansible/</link>
					<comments>https://www.osradar.com/install-elasticsearch-via-ansible/#respond</comments>
		
		<dc:creator><![CDATA[osradar_editor]]></dc:creator>
		<pubDate>Mon, 16 Sep 2019 19:41:45 +0000</pubDate>
				<category><![CDATA[Technology]]></category>
		<category><![CDATA[Tutorials]]></category>
		<category><![CDATA[ansible]]></category>
		<category><![CDATA[Elasticsearch]]></category>
		<guid isPermaLink="false">https://www.osradar.com/?p=11307</guid>

					<description><![CDATA[<p>The whole idea of this post is obvious. I have already covered the same objective in a standard fashion but this is about perform the installation via ansible automation. If you don&#8217;t have an understanding on Ansible Directory Structure &#8211; I really recommend to check my other post &#8211;  where I highlighted each key areas [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/install-elasticsearch-via-ansible/">Install ElasticSearch via Ansible</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p>The whole idea of this post is obvious. I have already covered the same objective in a standard fashion but this is about perform the installation via ansible automation. If you don&#8217;t have an understanding on Ansible Directory Structure &#8211; I really recommend to check my other <a href="https://www.osradar.com/getting-understand-ansible-structure/">post</a> &#8211;  where I highlighted each key areas that needed to get familiar.</p>
<h4><strong>Inventory File</strong></h4>
<p>In the example, I am going to deploy Three nodes of ElasticSearch &#8211; one represent MASTER role, while other two as DATA role.</p>
<pre><strong>[elastic_master]</strong>
es-0001 ansible_host=172.17.0.10
<strong>
[elastic_data]</strong>
es-0002 ansible_host=172.17.0.11
es-0003 ansible_host=172.17.0.12
</pre>
<p>&nbsp;</p>
<h4><strong>prod.yaml</strong></h4>
<pre>- hosts: elastic_master <span style="color: #008000;">&lt;------------ following actions will be performed against any host listed in <strong>elastic_master</strong> alias which found to be in inventory file</span>
  remote_user: root <span style="color: #008000;">&lt;---------------- to execute the command as root</span>
  become: true
  pre_tasks:
    - name: "Installing basic packages"
      action: yum <span style="color: #008000;">&lt;---------------- calls the yum module and any key that goes with <strong>with_items</strong> will be installed</span>
              name={{ item }}
              state=installed
      with_items:
        - unzip
      when: ansible_os_family == "RedHat" <span style="color: #008000;">&lt;---------------- a condition such that instruct the ansible <strong>pre_tasks</strong> should only suppose to be executed on a Fedora based distribution</span>
  roles:
   - { role: elastic_master_install }


- hosts: elastic_data
  remote_user: root
  become: true
  pre_tasks:
    - name: "Installing basic packages"
      action: yum
              name={{ item }}
              state=installed
      with_items:
        - unzip
      when: ansible_os_family == "RedHat"
  roles:
   - { role: elastic_data_install }</pre>
<p>&nbsp;</p>
<h4><strong>Default File</strong></h4>
<p>As we already covered this variables will be used when files which are in Jinga format are being copied under the Template DIrectory.</p>
<pre># vim roles/elastic_master_install/defaults</pre>
<pre>cluster_name: clusterName
node_master_true: "true"
node_data_true: "false"
node_ingest_true: "false"
path_to_log: /data/elk/logs
path_to_data: /data/elk/data
http_port: 9200
transport_tcp_port: 9300
discovery_zen_ping_unicast_hosts: '["172.17.0.10"]'

### - jvm config
init_heap_size: "-Xms8g"
max_heap_size: "-Xmx8g"
</pre>
<p>&nbsp;</p>
<p>&nbsp;</p>
<h4><strong>Template file Example</strong></h4>
<p>This is how a basic elasticsearch.yaml looks like in Jinja fromat.</p>
<pre># vim roles/elastic_master_install/templates/elasticsearch.yml.j2</pre>
<pre>cluster.name: {{cluster_name}}
node.name: {{inventory_hostname}}
node.master: {{node_master_true}}
node.data: {{node_data_true}}
node.ingest: {{node_ingest_true}}
path.data: {{path_to_data}}
path.logs: {{path_to_log}}
network.host: {{ansible_host}}
</pre>
<p>&nbsp;</p>
<h4><strong>Task File</strong></h4>
<p>This is where we can define all the task that are part of the respective role, in this case task that needed to execute setting up Elasticsearch</p>
<pre># vim roles/elastic_master_install/tasks/main.yml</pre>
<pre>- name: Creating elk user...
  user:
    name: elk
    comment: "elk User"
    createopt: yes
    opt: /opt/elk/
    uid: 1999
    shell: /bin/bash
  become: true

- name: Copying &amp; untar ElasticSearch5.5..
  unarchive: 
    src: /root/Ansible/ElasticSearch5/roles/elastic_master_install/Files/elasticsearch-5.5.0.tar.gz
    dest: /opt/elk/
    owner: elk
    group: elk
    mode: 0755
  become: true

- name: Creating necessary directories..
  file:
    path: /data/elk/{{ item }}
    state: directory
    owner: elk
    group: elk
    mode: 0775
    recurse: yes
  with_items:
     - [data, logs, run]
  become: true

- name: Copying the main config file...
  template: src=elasticsearch.yml.j2 dest={{elasticsearch_config_dir}}/elasticsearch.yml owner=elk group=elk mode=0644 
  become: true
</pre>
<p>&nbsp;</p>
<p>Please note that I have only added files for Role &#8220;<em>elastic_master_install</em>&#8221; =&gt; <strong>Task</strong>/<strong>Template</strong>/<strong>Default</strong>. However, as in the prod.yaml there is another role called &#8220;<em>elastic_data_install</em>&#8221; which you also need to work on as did in above last three steps.</p>
<p>When you have the Directory Structure ready, you can initiate the Ansible by;</p>
<pre># ansible-playbook -i inventory prod.yaml</pre>
<p>&nbsp;</p>
<p><em><strong>&#8220;I hope this has been informative for you&#8221;</strong></em></p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/install-elasticsearch-via-ansible/">Install ElasticSearch via Ansible</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://www.osradar.com/install-elasticsearch-via-ansible/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>ElasticSearch Alerting</title>
		<link>https://www.osradar.com/elasticsearch-alerting/</link>
					<comments>https://www.osradar.com/elasticsearch-alerting/#respond</comments>
		
		<dc:creator><![CDATA[osradar_editor]]></dc:creator>
		<pubDate>Wed, 30 Jan 2019 22:18:52 +0000</pubDate>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Technology]]></category>
		<category><![CDATA[Elasticsearch]]></category>
		<guid isPermaLink="false">https://www.osradar.com/?p=10397</guid>

					<description><![CDATA[<p>Think about a solution that keeps track incoming documents at Elasticsearch and when certain context appear system itself detect that and letting people aware such occurrence. If that&#8217;s what you need, I can tell, you have visited the correct blog. First take your attention on prerequisites. Elasticsearch environment Email sending application &#8211; I would prefer [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/elasticsearch-alerting/">ElasticSearch Alerting</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p>Think about a solution that keeps track incoming documents at Elasticsearch and when certain context appear system itself detect that and letting people aware such occurrence. If that&#8217;s what you need, I can tell, you have visited the correct blog.</p>
<p>First take your attention on prerequisites.</p>
<ul>
<li>Elasticsearch environment</li>
<li>Email sending application &#8211; I would prefer on Postfix but you can select your own preference</li>
<li>python2.7</li>
</ul>
<p>P<strong>lease consider to read our previous Articles about <a href="https://www.osradar.com/?s=Elastic">Install and Configure Elesticsearch </a></strong></p>
<p>&nbsp;</p>
<p>This whole goal is being achieved by a program called &#8220;ElastAlert&#8221; &#8211; which is a python based program. To tell you what the program does, it simply executes elasticsearch own DSL queries within a configured interval and if the search result is True, send alerts based on email addresses or other supported medium.</p>
<p>&nbsp;</p>
<h3>Getting Started:</h3>
<p>01. Installing python prerequisites</p>
<pre class="lang:sh decode:true">yum install python-devel libevent-devel python-pip git</pre>
<p>&nbsp;</p>
<p>02. Clone Elastalert remote git repository and install the python dependacies via &#8216;pip&#8217; &#8211; a python based package manager</p>
<pre class="lang:sh decode:true ">git clone https://github.com/Yelp/elastalert.git
cd /elastalert
pip install -r requirements.txt</pre>
<p>NOTE that you might find a situation that the program throws exceptional cases while starting up. Don&#8217;t worry, its probably a part of unmet dependencies that python is unable to find in your system.  Going through the error will help to find what packages are missing. Once you do, install them as well via &#8220;pip install &lt;packageName&gt;&#8221;, that will resolve the issue that you unable to start up the program.</p>
<p>&nbsp;</p>
<p>03. As far as configuration goes, there are two aspects.</p>
<ul>
<li><strong>Main configuration file</strong> =&gt; a location where we can define our Elasticsearch URL and other basics.</li>
</ul>
<pre class="lang:sh decode:true"># vim /opt/elastalert/config.yaml

# The Elasticsearch hostname for metadata writeback
es_host: 192.168.0.1

# The Elasticsearch port
es_port: 9200

# When rule match a certain condition, where should the alert go in order to find the SMTP sender
smtp_host: "localhost"
smtp_port: 25
from_addr: 'ElastAlert@example.com'

</pre>
<p>&nbsp;</p>
<ul>
<li><strong>Rules configuration files</strong> =&gt; Inside the download directory you will find a sub directory call &#8220;example_rules&#8221;. This is where you can find some basics example rules which are in yaml format. For now, I won&#8217;t discuss them but at the end of the article, I will share different types of configuration whose got different methodologies to detect events.</li>
</ul>
<p>&nbsp;</p>
<p>04. Starting up a rule. (assume Elastalert parent directory locates in /opt/elastalert/)</p>
<pre class="lang:sh decode:true">python2.7 /opt/elastalert/elastalert.py --verbose --rule/opt/elastalert/exmaple_rules/rule1.yaml</pre>
<p>That will start the process of capturing the given occurrence against what condition we mentioned in the rules1.yaml</p>
<p>&nbsp;</p>
<p>I already mentioned that &#8220;<strong>rules</strong>&#8221; are what define how these alerts are being captured. Further, they have different types of mechanism to detect different methodologies. Below examples are basic types that people often required to detect such incidents.</p>
<p>01. <strong>Frequency Type:</strong> This type of rules match when there are at least certain number of events with in a given time frame.</p>
<ul>
<li>If “<em>http-404</em>” text appears five times within the message flow in last minute..</li>
</ul>
<pre class="lang:sh decode:true">name: rune-1
type: frequency
index: testindex-*
num_events: 5
timeframe:
  minutes: 1
filter:
- term:
    name: "http-404"
alert:
- "email"
email:
- "john@exmaple.com"

</pre>
<p>&nbsp;</p>
<p>02. <strong>Change Type</strong>: Monitor a certain field (target-field) within a message flow and it will be a match if that field changes within the subsequent messages. Important thing here is that in-between two message blocks target-field must change with respect to another field &#8211; can call “reference-field” &#8211; which of course should not change across the target field.</p>
<ul>
<li>John user (reference-field) logs-in www.osradar.com in two different countries (target-filed) with in a day.</li>
</ul>
<pre class="lang:sh decode:true">name: Abnormal behavior about john user
type: change
index: testindex-*
compare_key: countryName
ignore_null: true
query_key: userName
timeframe:
  minutes: 1
filter:
- query:
    query_string:
        query: "siteURL:\"www.osradar.com\""
alert:
- "email"
email:
- "admin@example.com"</pre>
<p>&nbsp;</p>
<p>Now If you note down, both the rules types have email address associated and I know you already guessed that it would be the alert recipient mail id. Yes absolutely. However, one thing is missing in our setup and that would be SMTP agent which suppose to send mails. If you go back and find the main Elastalert configuration file, you even can see there is a line &#8220;<strong>smtp_host</strong>&#8220;. This is what instruct Elastalert to use send mails to. Setting up mail server is another task, but don&#8217;t worry visit link <a href="https://www.osradar.com/setting-up-mail-infrastructure-on-top-of-centos7/">here</a> where you find my other post which guide you to get the job done.</p>
<p>Now, it time to make your won Alerts by building python ElastAlert. Good Luck!</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/elasticsearch-alerting/">ElasticSearch Alerting</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://www.osradar.com/elasticsearch-alerting/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Securing Elasticsearch REST API</title>
		<link>https://www.osradar.com/securing-elasticsearch-rest-api/</link>
					<comments>https://www.osradar.com/securing-elasticsearch-rest-api/#respond</comments>
		
		<dc:creator><![CDATA[osradar_editor]]></dc:creator>
		<pubDate>Mon, 28 Jan 2019 21:43:51 +0000</pubDate>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Technology]]></category>
		<category><![CDATA[Tutorials]]></category>
		<category><![CDATA[Elasticsearch]]></category>
		<guid isPermaLink="false">https://www.osradar.com/?p=9755</guid>

					<description><![CDATA[<p>Why.? In my one of the previous ELK posts, we discovered the build in REST API that support out of the box in Elasticsearch. It communicates via 9200/tcp. When it comes security, though, this API is a weak aspect as it is fully open, leaving whole cluster in vulnerable to outsiders. Whoever the party can [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/securing-elasticsearch-rest-api/">Securing Elasticsearch REST API</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<h3>Why.?</h3>
<p>In my one of the previous ELK <a href="https://www.osradar.com/elasticsearch-dsl-query-examples/">posts</a>, we discovered the build in REST API that support out of the box in Elasticsearch. It communicates via 9200/tcp. When it comes security, though, this API is a weak aspect as it is fully open, leaving whole cluster in vulnerable to outsiders. Whoever the party can access the URL can perform any action without being subjected to Authentication &amp; Authorization. In this section of topic, I am going to cover how we can integrate these two major security aspect into Elasticsearch.</p>



<p>&nbsp;</p>
<h3>How.?</h3>
<p>As the figures shows, port tcp 9200 is fully restricted from its communication to outside. It can only work on nodes which are part of the cluster and else blocked. The type of the mechanism which implemented to restrict internal only communication is via Linux in-build firewall &#8211; firewallD daemon. This still extends port 9200 to transit over Kiana as well as Logstash as they are also part of the ELK stack.</p>



<h4>Extending REST access to outside:</h4>



<p>The Security design is being implemented in such a way that yet it is possible of extending REST access to outside if anyone wishes to do so. However, this time they have to pass user authentication plus authorization, which not only enforce check valid user credentials but also a granular access control over what they execute on the REST. This task purely operate on NginX web-server and &#8220;lua-scripts&#8221;</p>



<h3> </h3>
<h3>FirewallD Configuration:</h3>



<ul>
<li>Across all Elasticsearch/ Logstash/ Kibana nodes, Linux firewall limits port 9200 &amp; 9300 communication to themself. This is accomplished via zone &#8220;drop&#8221; &#8211; which block everything, except the source-address &amp; &#8220;source-ports&#8221; as configured below.</li>
</ul>



<ul>
<li>Note that below X.X.X.X will have to replace with your all ELK stack nodes IP addresses.</li>
</ul>



<pre class="wp-block-preformatted">firewall-cmd --zone=drop --add-source=X.X.X.X --permanent</pre>



<pre class="wp-block-preformatted">firewall-cmd --zone=drop --add-source-port=9200/tcp --permanen</pre>



<pre class="wp-block-preformatted">firewall-cmd --zone=drop --add-source-port=9300/tcp --permanent</pre>



<div class="wp-block-spacer" style="height: 23px;" aria-hidden="true"> </div>



<ul>
<li>Let one Elasticsearch node expose port 8080/TCP via Nginx proxy which enforces authentication + authorization. Zone &#8220;Public&#8221; will be configured for 8080 tcp port expose which will allow any IP addresses to be connect with.</li>
</ul>



<pre class="wp-block-preformatted">firewall-cmd  --zone=public  --add-port=8080/tcp</pre>



<div class="wp-block-spacer" style="height: 50px;" aria-hidden="true"> </div>



<h3 id="mce_60">Nginx Proxy Configuration:</h3>



<div class="wp-block-spacer" style="height: 20px;" aria-hidden="true"> </div>



<p>Following Installation works on CentOS 7 system.</p>



<ul>
<li>Let install required packages to build nginx</li>
</ul>



<pre class="wp-block-preformatted">yum install -y openssl-devel perl-ExtUtils-Embed GeoIP-devel pcre-devel zlib zlib-devel gcc make</pre>



<ul>
<li>Download the nginx source</li>
</ul>



<pre class="wp-block-preformatted">wget https://openresty.org/download/openresty-1.11.2.4.tar.gz</pre>



<ul>
<li>Extract the download tar archive</li>
</ul>



<pre class="wp-block-preformatted">tar -xvf openresty-1.11.2.4.tar.gz</pre>



<ul>
<li>Configuring the environment</li>
</ul>



<pre class="wp-block-preformatted ">./configure --prefix=/opt/nginx --sbin-path=/usr/sbin/nginx --conf-path=/opt/nginx/config/nginx.conf --pid-path=/opt/nginx/nginx.pid --error-log-path=/opt/nginx/logs/error.log --http-log-path=/opt/nginx/logs/access.log --user=nginx --group=nginx --with-luajit --with-http_auth_request_module </pre>



<ul>
<li>Once completed with successful build, now its time to install</li>
</ul>



<pre class="wp-block-preformatted">gmake &amp;&amp; gmake install</pre>



<ul>
<li>Lets create a systemD unit file at =&gt; vim /etc/systemd/system/nginx.service</li>
</ul>



<pre class="wp-block-preformatted">[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/opt/nginx/nginx.pid
ExecStartPre=/usr/sbin/nginx -t
ExecStart=/usr/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target</pre>



<ul>
<li>Let re-load the systemD database</li>
</ul>



<pre class="wp-block-preformatted">systemctl daemon-reload</pre>



<ul>
<li>Edit Nginx the configuration file and have reflect the required instruction =&gt; vim /opt/nginx/config/nginx.conf</li>
</ul>



<pre class="wp-block-preformatted">error_log logs/lua.log notice;
events {
         worker_connections  1024;
 }
 http {
    upstream elasticsearch {
         server 192.168.0.1:9200;
    }
    server {
            listen 8080;
            location / {
                         auth_basic "Protected Elasticsearch";
                         auth_basic_user_file passwords;
                         access_by_lua_file 'authorize.lua';
                         proxy_pass http://elasticsearch;
                         proxy_redirect off;
            }
    }
 }</pre>



<p>The above configuration instruct nginx that</p>



<ul>
<li><strong>auth_basic_user_file</strong> =&gt; perform username/password check up for any communication that goes via 192.168.0.1:9200 which is in this example, Elasticserach node IP address</li>
<li><strong>access_by_lua_file</strong> =&gt; Even after the successful authentication, every users&#8217; HTTP methods will follow up a privilege escalation against privileges that are defined in the lua script which we will be setting up next.</li>
</ul>



<ul>
<li>Finally, make sure lua script is controlling who&#8217;s got which level of HTTP access onto the REST API calls =&gt; vim /home/dlogs/nginx/authorize.lua</li>
</ul>



<pre class="lang:sh decode:true ">local restrictions = {
  all  = {
    ["^/$"]                             = { "HEAD" }
  },
 
  bob = {
    ["^/?[^/]*/?[^/]*/_search"]         = { "VIEW", "GET" },
    ["^/?[^/]*/?[^/]*/_msearch"]        = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/_validate/query"] = { "GET", "POST" }
  },
 
  admin = {
    ["^/?[^/]*/?[^/]*/_bulk"]          = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/_refresh"]       = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/?[^/]*/_create"] = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/?[^/]*/_update"] = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/?.*"]            = { "GET", "POST", "PUT", "DELETE" },
    ["^/?[^/]*/?[^/]*$"]               = { "GET", "POST", "PUT", "DELETE" },
    ["/_aliases"]                      = { "GET", "POST" }
  }
}
 
 
local role = ngx.var.remote_user
 
 
if restrictions[role] == nil then
  ngx.header.content_type = 'text/plain'
  ngx.status = 403
  ngx.say("403 Forbidden: You don't have access to this resource.")
  return ngx.exit(403)
end
 
 
local uri = ngx.var.uri
 
 
local method = ngx.req.get_method()
 
local allowed  = false
 
for path, methods in pairs(restrictions[role]) do
 
 
  local p = string.match(uri, path)
 
  local m = nil
 
 
  for _, _method in pairs(methods) do
    m = m and m or string.match(method, _method)
  end
 
  if p and m then
    allowed = true
  end
end
 
if not allowed then
  ngx.header.content_type = 'text/plain'
  ngx.log(ngx.WARN, "Role ["..role.."] not allowed to access the resource ["..method.." "..uri.."]")
  ngx.status = 403
  ngx.say("403 Forbidden: You don't have access to this resource.")
  return ngx.exit(403)
end</pre>
<p>&nbsp;</p>
<ul>
<li>Start the Nginx service</li>
</ul>
<pre class="wp-block-code ">systemctl start nginx</pre>

<p>&nbsp;</p>
<p>Now, you need to change Rest API url which, for example, were being;</p>
<pre class="lang:sh decode:true ">curl -XGET http://192.168.0.1:9200/testindex/_search </pre>
<p><br />then the new url and its parameter would be;</p>
<pre class="lang:sh decode:true ">curl -XGET --user bob:password http://192.168.0.1:8080/testindex/_search</pre>
<p>&nbsp;</p>
<p><em><strong>&#8220;I hope this has been very informative for you&#8221;</strong></em></p><p>The post <a rel="nofollow" href="https://www.osradar.com/securing-elasticsearch-rest-api/">Securing Elasticsearch REST API</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://www.osradar.com/securing-elasticsearch-rest-api/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Elasticsearch DSL Query Examples</title>
		<link>https://www.osradar.com/elasticsearch-dsl-query-examples/</link>
					<comments>https://www.osradar.com/elasticsearch-dsl-query-examples/#respond</comments>
		
		<dc:creator><![CDATA[osradar_editor]]></dc:creator>
		<pubDate>Fri, 25 Jan 2019 15:02:06 +0000</pubDate>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Tutorials]]></category>
		<category><![CDATA[Elasticsearch]]></category>
		<category><![CDATA[ELK]]></category>
		<guid isPermaLink="false">https://www.osradar.com/?p=9662</guid>

					<description><![CDATA[<p>If you have been reading my previous posts of ELK stack, you should now be comfortable on How to setup Elasaticsearch Node. What options do we have to ingest data into Elasticsearch database. Elasticsearch uses an entity called &#8220;INDEX&#8221; to store data. Every message that&#8217;s gets stored is a &#8220;Document&#8221; at Elasticsearch. And even we [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/elasticsearch-dsl-query-examples/">Elasticsearch DSL Query Examples</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p>If you have been reading my previous <a href="https://www.osradar.com/setting-up-elasticsearch-on-linux">posts</a> of ELK stack, you should now be comfortable on</p>
<ul>
<li>How to setup Elasaticsearch Node.</li>
<li>What options do we have to ingest data into Elasticsearch database.</li>
<li>Elasticsearch uses an entity called &#8220;INDEX&#8221; to store data.</li>
<li>Every message that&#8217;s gets stored is a &#8220;Document&#8221; at Elasticsearch.</li>
<li>And even we know that using Kibana it is possible to view those data back for analysis.</li>
</ul>
<p>In this post, I am going to cover the native Query language that Elasticsearch use to search data. It is DSL (Domain Specific Language).</p>
<p><img loading="lazy" class="alignnone  wp-image-9743" src="https://www.osradar.com/wp-content/uploads/2019/01/query-300x127.png" alt="" width="484" height="205" srcset="https://www.osradar.com/wp-content/uploads/2019/01/query-300x127.png 300w, https://www.osradar.com/wp-content/uploads/2019/01/query.png 668w" sizes="(max-width: 484px) 100vw, 484px" /></p>
<h3>REST API:</h3>
<p>9200/tcp is one of the network communication socket that Elasticsearch use. And when running, Elasticsearch expose its REST API on this port for external communication and that is what we can connect with to perform these DSL queries. This API is build on top of HTTP protocol, so its aware any http calls, for example, GET, POST, PUT..</p>
<p>If you are wondering on which client that will support executing these queries, well, there are 3rd party applications, such as &#8220;Postman&#8221;. However, since we already have Kibana Installed, let&#8217;s check that on how to access the REST API.</p>
<p></p>
<ul>
<li>Since Kibana is aware the Elasticsearch node url, it is not required to mention in the query itself, like if you are working on 3rd party tool.</li>
</ul>
<p></p>
<p>&nbsp;<strong>match_all</strong> Query:</p>
<p>If we don&#8217;t know what to search, but we need to see every document inside of a INDEX, this query comes handy.</p>
<pre class="lang:sh decode:true">GET 192.168.0.1:9200/index_name/_search

{
  "size": 1,
  "query": {
    "match_all": {}
  }
}</pre>
<ul>
<li><strong>index_name</strong> =&gt; on which index that perform the query</li>
<li><strong>_search</strong>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =&gt; one of the API endpoint that elasticsearch exposes. _search of course is what provide the search facility.</li>
<li><strong>{}&nbsp;</strong> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =&gt; everything inside of the curly braces will be HTTP body</li>
<li><strong>size</strong>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;=&gt; number of matched document that need to filter out.</li>
<li><strong>match_all</strong>&nbsp; &nbsp; &nbsp;=&gt; this is a catch all statement where anything inside the &#8220;<em>index_name</em>&#8221; INDEX we would be able to see.</li>
</ul>
<p></p>
<h4><strong>exists</strong> Query:</h4>
<p>We know that Elasticsearch is a Json store, so if we know the exact FieldName that the document we are searching upon, this query comes handy.</p>
<pre class="lang:sh decode:true">GET 192.168.0.1:9200/index_name/_search

{
  "query": {
    "exists": {
      "field": "firstName"
    }
  }
}</pre>
<ul>
<li><strong>field&nbsp;</strong> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =&gt; We are searching any document inside the &#8220;<em>index_name</em>&#8221; INDEX, but each document should have a filed named called &#8220;firstName&#8221;</li>
</ul>
<p></p>
<h4><strong>term</strong> Query:</h4>
<p>This query is best when If we know the exact field name and we also want to limit the search result only on documents that have filed value of exactly what we are searching.</p>
<pre class="lang:sh decode:true">GET 192.168.0.1:9200/index_name/_search

{
  "query": {
    "term": {
      "userName": "john"
    }
  }
}</pre>
<p>Note that this query should be perform on Fields within a document that has single term, not on Fields that has many terms inside of it, for example;<br>&#8220;userName&#8221;: &#8220;don john mcalister&#8221;</p>
<p></p>
<h4><strong>match</strong>&nbsp;Query:</h4>
<p>If &#8220;term&#8221; query is not best for search multiple terms, then which query is best of doing the task. This is where match query comes into play which does full-text search at it&#8217;s best.</p>
<pre class="lang:sh decode:true">GET 192.168.0.1:9200/index_name/_search

{
  "query": {
    "match": {
      "userCommnet": "Sri Lanka beautiful country"
    }
  }
}</pre>
<p>Note that even in the actual document&#8217;s &#8220;<strong><em>userComment</em></strong>&#8220;:&nbsp; contains &#8220;<strong><em>Sri Lanka is one of the best beautiful countries in the world</em></strong>&#8221; the above query get you that document out of the result search result.&nbsp;</p>
<p></p>
<h4><strong>query_string</strong>&nbsp;Query:</h4>
<p>If you are looking at AND/OR operators to match document, this would do it for you.</p>
<pre class="lang:sh decode:true">GET 192.168.0.1:9200/index_name/_search

{
  "query": {
    "query_string": {
      "query": "<strong>city</strong>:(new york) OR (new-york)"
    }
  }
}</pre>
<p>Note that <em>&#8220;<strong>city</strong>&#8221; </em>is the field name &amp; rest will be the search text that we are looking..&nbsp;</p>
<p>For basic operations, these type of queries are best for getting task done, but like any other database query, DSL also has many powerful options that leverage to narrow down the search result as to the best of requirement. Once you get these basics covered, you can comment on more for such advanced queries.&nbsp;</p>
<p><em>&#8220;I hope this has been informative for you.&#8221;</em></p>
<p></p>


<p></p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/elasticsearch-dsl-query-examples/">Elasticsearch DSL Query Examples</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://www.osradar.com/elasticsearch-dsl-query-examples/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>How to install Elasticsearch on CentOS 7?</title>
		<link>https://www.osradar.com/how-to-install-elasticsearch-on-centos-7/</link>
					<comments>https://www.osradar.com/how-to-install-elasticsearch-on-centos-7/#respond</comments>
		
		<dc:creator><![CDATA[angeloma]]></dc:creator>
		<pubDate>Thu, 20 Sep 2018 09:04:02 +0000</pubDate>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Technology]]></category>
		<category><![CDATA[Tutorials]]></category>
		<category><![CDATA[CentOS]]></category>
		<category><![CDATA[data]]></category>
		<category><![CDATA[Elasticsearch]]></category>
		<category><![CDATA[howto]]></category>
		<category><![CDATA[search]]></category>
		<category><![CDATA[server]]></category>
		<category><![CDATA[Tutorial]]></category>
		<guid isPermaLink="false">https://www.osradar.com/?p=5879</guid>

					<description><![CDATA[<p>Linux is a great operating system and many companies rely on it for their business solutions. That is why more and more projects are born to be implemented in them. In this article, I will explain how to install Elasticsearch in CentOS 7. In short, Elastichsearch is a document-oriented search engine that allows us to [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/how-to-install-elasticsearch-on-centos-7/">How to install Elasticsearch on CentOS 7?</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p><strong>Linux</strong> is a great operating system and many companies rely on it for their business solutions. That is why more and <a href="https://www.forbes.com/sites/dharmeshthakker/2018/09/12/another-open-source-ipo-shows-the-market-power-of-free-software/#593505faf6a9">more projects</a> are born to be implemented in them. In this article, I will explain how to install Elasticsearch in CentOS 7.</p>
<p>In short, <a href="https://www.elastic.co/products/elasticsearch">Elastichsearch</a> is a document-oriented search engine that allows us to index a large volume of data to make queries about them later. It&#8217;s open source licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0.html">Apache License</a> 2.0. With this in mind, let&#8217;s install Elasticsearch.</p>
<h2>0. What you need</h2>
<p>To install Elasticsearch on CentOS7, you need in a first place, get <strong>administrator privileges</strong>, to perform the commands correctly.</p>
<p>Then, you need a machine with <strong>CentOS 7</strong> and the possibility of accessing the server through ssh.</p>
<p>It is always useful to have knowledge about the <strong>use of the terminal</strong>.</p>
<h2>1.- Upgrading the System and Installing Java</h2>
<p>We are going to perform the installation on a server, this implies that we must always have updated packages in order to always get the latest security patches.</p>
<p>To become a root user, type in the terminal:</p>
<pre class="">:~$ sudo -i</pre>
<p>After entering the root password, we will be able to upgrade the system.</p>
<pre class="">:~# yum update</pre>
<p><figure id="attachment_5882" aria-describedby="caption-attachment-5882" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5882" src="https://www.osradar.com/wp-content/uploads/2018/09/1-12.png" alt="1.- Upgrading the system" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/1-12.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/1-12-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/1-12-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/1-12-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/1-12-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/1-12-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/1-12-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5882" class="wp-caption-text">1.- Upgrading the system</figcaption></figure></p>
<p>Before starting to install <strong>Elasticsearch,</strong> you must make sure you have Java on your computer as the application requires it.</p>
<pre class="">:~# yum install java-1.8.0-openjdk.x86_64</pre>
<p><figure id="attachment_5883" aria-describedby="caption-attachment-5883" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5883" src="https://www.osradar.com/wp-content/uploads/2018/09/2-11.png" alt="2.- Installing OpenJDK" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/2-11.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/2-11-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/2-11-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/2-11-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/2-11-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/2-11-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/2-11-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5883" class="wp-caption-text">2.- Installing OpenJDK</figcaption></figure></p>
<p>When the installation is finished, you can run the following command to be sure of it.</p>
<pre class="">:~# java --version</pre>
<p><figure id="attachment_5884" aria-describedby="caption-attachment-5884" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5884" src="https://www.osradar.com/wp-content/uploads/2018/09/3-11.png" alt="3.- Showing Java installed version" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/3-11.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/3-11-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/3-11-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/3-11-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/3-11-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/3-11-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/3-11-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5884" class="wp-caption-text">3.- Showing Java installed version</figcaption></figure></p>
<h2>2.- Install Elasticsearch</h2>
<p>A simple and practical way to install Elasticsearch is to do it through your .rpm packages. This is very useful if you need to install it in other distributions like Suse or RHEL.</p>
<p>First, download the packages:</p>
<pre class="">:~# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.1.rpm
:~# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.1.rpm.sha512</pre>
<p><figure id="attachment_5890" aria-describedby="caption-attachment-5890" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5890" src="https://www.osradar.com/wp-content/uploads/2018/09/4-15.png" alt="4.- Downloading the packages" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/4-15.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/4-15-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/4-15-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/4-15-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/4-15-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/4-15-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/4-15-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5890" class="wp-caption-text">4.- Downloading the packages</figcaption></figure></p>
<p>The first is the program itself, the second package is to verify its integrity, something vital because you are doing the installation on a server.</p>
<p>After downloading the packages, it is necessary to check them.</p>
<pre class="">:~# shasum -a 512 -c elasticsearch-6.4.1.rpm.sha512</pre>
<p><figure id="attachment_5891" aria-describedby="caption-attachment-5891" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5891" src="https://www.osradar.com/wp-content/uploads/2018/09/5-12.png" alt="5.- Verifying the packages" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/5-12.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/5-12-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/5-12-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/5-12-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/5-12-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/5-12-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/5-12-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5891" class="wp-caption-text">5.- Verifying the packages</figcaption></figure></p>
<p>And finally, install the packages.</p>
<pre class="">:~# rpm --install elasticsearch-6.4.1.rpm</pre>
<p><figure id="attachment_5892" aria-describedby="caption-attachment-5892" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5892" src="https://www.osradar.com/wp-content/uploads/2018/09/6-11.png" alt="6.- installing elasticsearch" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/6-11.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/6-11-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/6-11-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/6-11-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/6-11-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/6-11-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/6-11-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5892" class="wp-caption-text">6.- installing Elasticsearch</figcaption></figure></p>
<p>Then it&#8217;s your turn to set up the Elasticsearch service in order to enable and initialize it.</p>
<pre class="">:~# systemctl daemon-reload
:~# systemctl enable elasticsearch
:~# systemctl start elasticsearch</pre>
<p><figure id="attachment_5893" aria-describedby="caption-attachment-5893" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5893" src="https://www.osradar.com/wp-content/uploads/2018/09/7-11.png" alt="7.- Enabling Elasticsearch service" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/7-11.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/7-11-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/7-11-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/7-11-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/7-11-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/7-11-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/7-11-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5893" class="wp-caption-text">7.- Enabling Elasticsearch service</figcaption></figure></p>
<h2>3. Check that Elasticsearch is working</h2>
<p>Now that you have installed Elasticsearch, you must check that everything is in order, to do so, run:</p>
<pre class="">:~# lsof -i :9200</pre>
<p>You must remember or know that Elasticsearch uses port 9200 by default. That&#8217;s why the command makes an inspection of the applications that use the mentioned port.</p>
<p>You can also check <code>/var/log/elasticsearch/elasticsearch.log</code> for errors.</p>
<h2>A small configuration and Conclusion</h2>
<p>It is possible to make Elasticsearch listen to a single IP in particular. Open the file <code>/etc/elasticsearch/elasticsearch.yml.</code></p>
<pre class="">:~# nano /etc/elasticsearch/elasticsearch.yml</pre>
<p><figure id="attachment_5894" aria-describedby="caption-attachment-5894" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5894" src="https://www.osradar.com/wp-content/uploads/2018/09/8-9.png" alt="8.- Elasticsearch - listening on particular IPv4" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/8-9.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/8-9-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/8-9-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/8-9-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/8-9-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/8-9-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/8-9-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5894" class="wp-caption-text">8.- Elasticsearch &#8211; listening on particular IPv4</figcaption></figure></p>
<p>You can also further protect the instance by disabling public access. In that same file, add the following at the end.</p>
<pre>network.bind_host: 127.0.0.1</pre>
<p><figure id="attachment_5895" aria-describedby="caption-attachment-5895" style="width: 1366px" class="wp-caption alignnone"><img loading="lazy" class="size-full wp-image-5895" src="https://www.osradar.com/wp-content/uploads/2018/09/9-10.png" alt="9.- Disabling public access to Elasticsearch" width="1366" height="736" srcset="https://www.osradar.com/wp-content/uploads/2018/09/9-10.png 1366w, https://www.osradar.com/wp-content/uploads/2018/09/9-10-300x162.png 300w, https://www.osradar.com/wp-content/uploads/2018/09/9-10-768x414.png 768w, https://www.osradar.com/wp-content/uploads/2018/09/9-10-1024x552.png 1024w, https://www.osradar.com/wp-content/uploads/2018/09/9-10-696x375.png 696w, https://www.osradar.com/wp-content/uploads/2018/09/9-10-1068x575.png 1068w, https://www.osradar.com/wp-content/uploads/2018/09/9-10-780x420.png 780w" sizes="(max-width: 1366px) 100vw, 1366px" /><figcaption id="caption-attachment-5895" class="wp-caption-text">9.- Disabling public access to Elasticsearch</figcaption></figure></p>
<p>Then, restart the service.</p>
<p>As a conclusion, it can be said that Elasticsearch is a solution intended for large data and companies but is still easy to install on a server with CentOS.</p>
<p>We want to know about your experience, have you used elasticsearch?</p>
<p>Please share this article on your social networks.</p>
<p>The post <a rel="nofollow" href="https://www.osradar.com/how-to-install-elasticsearch-on-centos-7/">How to install Elasticsearch on CentOS 7?</a> appeared first on <a rel="nofollow" href="https://www.osradar.com">Linux  Windows and android  Tutorials</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://www.osradar.com/how-to-install-elasticsearch-on-centos-7/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
	</channel>
</rss>
