Clustering

We have started our first agent and run a job against it in development mode. This demonstrates the ease of use and the workflow of Nomad, but did not show how this could be extended to a scalable, production-grade configuration. In this step, we will create our first real cluster with multiple nodes.

Starting the Server

The first step is to create the config file for the server. Either download the file from the repository here, or paste this into a file called server.hcl:

echo '# Increase log verbosity
log_level = "DEBUG"

advertise {

  http = "'$(hostname -i)'"
  rpc  = "'$(hostname -i)'"
  serf = "'$(hostname -i)':5648"
}
# Setup data dir
data_dir = "/tmp/nomad"

# Enable the server
server {
    enabled = true

    # Self-elect, should be 3 or 5 for production
    bootstrap_expect = 1
}' | tee server.hcl

This is a fairly minimal server configuration file, but it is enough to start an agent in server only mode and have it elected as a leader. The major change that should be made for production is to run more than one server, and to change the corresponding bootstrap_expect value.

Once the file is created, start the agent in a new tab:

nomad agent -config server.hcl &> nomad.log &

We can see above that client mode is disabled, and that we are only running as the server. This means that this server will manage state and make scheduling decisions but will not run any tasks. Now we need some agents to run tasks!

Starting the Clients

Similar to the server, we must first configure the clients. Either download the configuration for client1 and client2 from the repository here, or paste the following into client.hcl:

echo '# Increase log verbosity
log_level = "DEBUG"

# Setup data dir
data_dir = "/tmp/nomad"

# Enable the client
client {
    enabled = true

    network_interface = "eth0"

    # For demo assume we are talking to server1. For production,
    # this should be like "nomad.service.consul:4647" and a system
    # like Consul used for service discovery.
    servers = ["node1:4647"]
}' | tee client.hcl

nomad agent -config client.hcl &> nomad.log &

In the output we can see the agent is running in client mode only. This agent will be available to run tasks but will not participate in managing the cluster or making scheduling decisions.

Using the node-status command we should see both nodes in the ready state:

nomad node-status

We now have a simple two node cluster running like you would have in your own production datacenter.

Submit a Job

Now that we have a simple cluster, we can use it to schedule a job. We’ll use the example.nomad job file, but verify that the count is still set to 3.

Then, use the run command to submit the job:

nomad init
sed -i 's/count = 1/count = 2/' example.nomad
nomad run example.nomad

We can see in the output that the scheduler assigned the tasks to the client node.

We can again use the status command to verify:

nomad status example

We can see that all our tasks have been allocated and are running. Once we are satisfied that our job is happily running, we can tear it down with nomad stop.

Next Steps

We’ve now concluded the getting started guide, however there are a number of next steps to get started with Nomad.