1.1 Register and Discover Services
Consul allows to register other software systems as a service at the Consul catalog. Examples of such services could be: MongoDB services, PostgreSQL services, webservices and so on. Consul itself will not consume the registered service, instead it will act as an information desk for someone/something looking for information to connect to a registered service.
It is assumed, that the functionality of the registered services can be consumed via one or multiple network connections. This means everything that opens a listen socket on a network interface could be registered at the Consul service catalog. The registration is not limited to services that run on one machine/server, instead it is also possible to register a distributed system running on multiple machines (e.g. a horizontally scaled webservice runs on more than one machine). Machines/servers that run services are referred to as "nodes" in the Consul terminology as it is referred to in the distributed system terminology.
Once, one or more services are available in the service catalog, applications can query service information that are required to establish a connection to a specific service instance. In practice, when asking Consul for a service, it will return a list of IP/port tuples that can be used to connect to the service.
Querying service information from the catalog can be done using the RESTfull HTTP interface of consul, or using the DNS interface. The fact that Consul provides a DNS interface for service information retrieval is a powerful feature that enables some sophisticated use cases.
The Consul architecture allows the service catalog to be highly available.
1.2 Store and Find Key-Value Pairs
Consul provides the possibility to store a small amount of data (usually text values). To read a specific stored value later, you must store the value in combination with a key. You (or more explicitly your application) has to remember this key to retrieve the stored value later.
The Consul architecture allows to provide a Key-Value store that is durable, hight available, strong consistent and secure (fine grained access control).
1.3 Observe Service Health
Consul is able to observe whether a node that provides a service is working correctly. In case it is not, consul will remove the node from the service catalog. This ensures that only healthy IP/Port tuples are returned when a service is discovered. In case a service is running on more then one node, the other nodes will still be present in the service catalog.
This feature is realized with an health check application (usually a simple shell script) that is placed on each node. Consul doesn't care about the content of the health check script, it just executes the script periodically and it decides whether a service is healthy based on its exit code. The path to the health check script and the health check interval can be configured using the RESTfull API of Consul or changing a Consul configuration file. Because one node could run more then one service, it is also possible to setup more then one health check on a node.
Consul Use Cases
2.1 A System Operator Sets Up an Internal Dynamic DNS
Using the Consul DNS interface to query connection information from the Consul service catalog makes it easy to use Consul as an internal domain name system.
In case you are using a public domain name (e.g. my-mongodb-database.example.com) that points to an IP address of a service node and this service node gets a new IP Address, you have to tell the public DNS about the IP change. After that update it could take up to a day until each client retrieve the updated IP address (due to caching).
An alternative to the public domain name could be to don't use a domain name and instead using the IP address of the service node. In this case you would configure each application that requires a connection to the service node with an IP address (e.g. you put the IP address into the configuration of your webservice(s), which then will use the IP address to connect to a MongoDB service). In this scenario you would have to reconfigure and restart each application in case of an IP change.
The common practice to solve this problem is to ensure that the IP address doesn't change or to use an internal DNS system that has a low update latency (caching disabled or reduced). Consul provides you the possibility to setup a internal DNS that is always up to date automatically and hight available. For this you just have to setup a consul server cluster and a consul agent on each service node.
2.2 A System Operator Sets Up a Load Balancing
A load balanced system consists of multiple instances of an application. The workload of the system is distributed across all available instances in order to increase the throughput of the system. The distribution of the workload is usually done by a component called "load balancer". A load balancer receives all requests and performs a redirect to one of the available application instances, where the time intensive work is done. In case you want to distribute the load of a stateless application, consul could be a good tools for that. The application has to be stateless because you don't have a chance to ensure that requests from one client always gets redirected to the same application instance.
When registering a service at the consul catalog you have to specify a name of the service and a name for each node that runs an instance of the service. Using the DNS interface, consul provides you two ways to retrieve the IP addresses of the nodes. You could query the IP address of a specific node or you can query the IP addresses of each node running a specific service. To retrieve the IP address of a specific node you would use a domain name that looks something like this:
But the more interesting query method in this use case is the last one, where you query all IP addresses of the nodes running a service. For that you would like to use a domain name like this:
To realize the load balancing you have to configure each client with the last domain type. Whenever a client resolves this domain name it gets a random IP address out of all node addresses. Due to the possibility to define health checks in consul, only healthy nodes will be returned, so that it is ensured that no traffic gets redirected to a broken node.
The drawback of this solution is that you have to ensure that the operating system on each client will contact the consul server when resolving an hostname.