Vagrant and Ansible are used to manage my development environment which is really wonderful. Recently, I have written a Ansible playbook which would reboot the VM during Vagrant provisioning. Check here to see how Vagrant works together with Ansible. Besides, there are tasks left which are expected to continue after the VM comes back. It is easy to find a solution in Ansible
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
This is the standard way to reboot and wait for a host to come back through polling in Ansible. However, it doesn’t work as expected. Technically, it works with libvirt provider but not the built-in VirtualBox provider.
- Output of Ansible shows that the
wait_fortask is successful
- SSH connection failure occurs in
wait_fortask’s following task
- VirtualBox UI tells that the VM is still booting when
wait_fortask is done
What makes me confused is that VirtualBox tells that VM is still booting but the
wait_for module has finished successfully. Disclose SSH connection failure from following task means that the VM is not ready yet. It seems that something is wrong with
For the first try, the value of
delay option of
wait_for task was set to a longer time like
30 which means Ansible will block for 30 seconds before it starts to check if the host is back or not. Every task got executed successfully and VirtualBox UI showed that the VM had been up. Though every thing seems to be OK, the workaround didn’t make sense because that is not how
wait_for module works. It is supposed to poll the status of the VM and wait until port 22 of the VM is ready. However, the result tell that
wait_for module will return immediately whenever it start to check the status.
Next, enable Ansible’s verbose mode in Vagrant. From the output of
wait_for module we can find
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Wait, why are the values of host and port
2222? Aren’t they supposed to be the ip of VM and 22? The output means that 2 of Ansible’s facts,
ansible_ssh_host, are interpreted unexpectedly. If not specified, Ansible’s inventory file will be generated by Vagrant automatically. The content of inventory file generated by Vagrant is
1 2 3
Because Vagrant use a forwarded port to create SSH connection in VirtualBox, the host will be
127.0.0.1 and the port is a
2222 other than 22.
So, can we instruct Vagrant to generate the inventory file with the VM’s own address and port 22 by disabling port forwarding? Unfortunately, NO! You can disable any port except the forwarded SSH port in VirtualBox. That is because Vagrant require a way to access and configure the VM through SSH and port forwarding is used with VirtualBox provider. The libvirt provider uses private network to provide the SSH connection so the playbook works well.
The reason why
wait_for module is able to finish successfully is that the forwarded port will be created before the VM is ready. Every time
wait_for module starts to check the VM, it finds that the port is ready. In fact, only the host’s port is ready but the guest’s port is not.
Since the root cause has been found, there are several ways to fix it.
- Provide a customized inventory file to Vagrant. This is not a good practice because we should let Vagrant to manage as much stuff as possible.
- Pass the VM’s address and port to the playbook. Not good enough since hardcoded parameters should be avoided.
- Use the
wait_formodule. With the option,
wait_formodule will not just check if the port is ready, it will also try to read some data from the port and search if it matches the
search_regex. After setting the
OpenSSH, every one is happy now.
- Networking is really the must-know knowledge for Vagrant users.
- Ansible’s verbose mode is informative and should be turned on when something is abnormal.
- Let Vagrant to manage the environment as much as possible.