Dissecting Docker Swarm Elastic

Authored byRaseel B.on Mar 20, 2017 in Topic Technology
Raseel B. Software Architect, Zymr, Inc.
Keyword Cloud

Subscribe to email updates


Docker and Elasticsearch both have the properties and capabilities to function hand-in-hand in an optimal fashion – due to their elasticity. As most developers are aware already, running Elasticsearch clusters on Docker Swarm is relatively difficult due to the fact that the user must specify their IP addresses of ALL master nodes in order to join the cluster. Not only is this is a hassle, but it is also relatively time-consuming. The reason behind this is because Elasticsearch removed their multicast discovery option and incorporating it into a plugin – users must have this plugin in order to benefit from the features. Furthermore, the IP addresses that are required are not visible, for a lack of better words, to the user when one starts Elasticsearch. And Docker service rejects any use of a bridge or host network, leaving the user essentially caught between a rock and a hard place.

With a few workarounds, a user can bypass the difficulties of searching for other nodes through several specification commands. For instance, the developer can specify an Elasticsearch container to ask the node to search for other Elasticsearch nodes. Through this method, the IP addresses returned will be that of a random Elasticsearch node, therefore forming the cluster with little to no work.

While this method is a useful tactic, it does not fully solve the networking problem that many users face. Since Docker does not permit outside connections into overlay networks, due to an initial design limitation, the user will run into a port connection issue. The use of a proxy service to connect an outside network to the Elasticsearch network is a work-around of the design – or bug as many people like to believe. Users will need to utilize a proxy of their choice to discover the target service (through a series of tags). Note that these tags must be set as environment variables in regards to the Docker Swarm service for optimal effectiveness. If the user is not running a DNS configuration in the setup, a public DNS can be used for the virtual hostname. By incorporating these tactics, the user can allow an “outside” network to virtually connect to the Elasticsearch network without having to go to great lengths in order establish a simple connection.

Monitoring an Elasticsearch cluster through an ES image (along with specific plugins installed) will allow the user to check the health of the cluster frequently, or as needed. The idea behind this is that through the use of the resourceful options in Docker as well as a bridge network driver, one can essentially create a test environment for the user to determine what works effectively as opposed to other methods. For example, if an individual were to simulate a multi-node environment on a single server, the user can actually utilize Docker on a single server. Although this presents a formidable challenge to the developer, it is also a blessing in disguise as he or she can find success through a trial-and-error method.



Leave a Reply

Contact Us

Request a Consultation

Smartsourcing: A guide to selecting an Agile Development Partner

Smartsourcing is a brief guide to the world of modern technology partnerships. It was developed through a collaborative effort of top Zymr executives as we uncovered a gap in the market between the perception of what outsourcing used to be, and how leading technology innovators are leveraging this globalized approach to value generation. Read this guide to learn...

  • Key factors to consider for your development strategy
  • Popular destinations with a track record of high quality services
  • Selection criteria to narrow your shortlisted vendors

Get access to Smartsourcing eBook

 30 days 3 Months 1 year Still exploring

Register below to download your free eBook

Register below to download your free White Paper

Register below to download your free Guide

Register below to download your full Case Study

Register below to download your Healthcare Cloud Stack

Register below to download your Microservices eBook

Register below to download your free White Paper