Ansible fetch Module Tutorial + Examples


Percy Grunwald's Profile Picture

Written by Percy Grunwald

— Last Updated February 22, 2019

What does the Ansible fetch module do?

Ansible’s fetch module transfers files from a remote host to the local host. This is the reverse of the copy module.

- name: fetch nginx access log
  fetch:
    src: /var/log/nginx/access.log
    dest: fetched

By default, the fetched files will be stored in the following way:

fetched
├── 123.123.123.123
│   └── var
│       └── log
│           └── nginx
│               └── access.log
└── 234.234.234.234
    └── var
        └── log
            └── nginx
                └── access.log

Ansible fetch Module Video Tutorial

If you prefer watching to reading, here’s a full video tutorial from the TopTechSkills YouTube channel covering a many of the points and examples from this article. Feel free to comment on this article or the video if you have any questions.

▶️ Watch on YouTube

When to use the fetch module vs copy module

The fetch module is used when you want to move files from the remote host to the local host (i.e. fetch a file from the remote host). The copy module is used when you want to put files onto the remote host, or you want to copy files to another location on the remote host.

Where does fetch put fetched files?

The dest parameter of fetch accepts either an absolute or relative path to the destination folder on the local host. Relative paths are relative to the playbook.

# playbook at /path/to/ansible/playbook.yml

- name: fetch nginx access log (absolute)
  fetch:
    src: /var/log/nginx/access.log
    dest: /path/to/ansible/fetched/absolute

- name: fetch nginx access log (relative)
  fetch:
    src: /var/log/nginx/access.log
    dest: fetched/relative

The tasks above examples above will produce the following file structure:

/path/to/ansible/fetched
├── absolute
│   ├── 123.123.123.123
│   │   └── var
│   │       └── log
│   │           └── nginx
│   │               └── access.log
│   └── 234.234.234.234
│       └── var
│           └── log
│               └── nginx
│                   └── access.log
└── relative
    ├── 123.123.123.123
    │   └── var
    │       └── log
    │           └── nginx
    │               └── access.log
    └── 234.234.234.234
        └── var
            └── log
                └── nginx
                    └── access.log

Examples

How to fetch files from remote hosts

Set the src and dest parameters to fetch files from remote hosts. Remember that the src parameter is the file path on the remote host and the dest parameter represents where the fetched files will go on the local host.

- name: fetch nginx access log
  fetch:
    src: /var/log/nginx/access.log
    dest: fetched

The task above will produce a directory structure like this in the working directory:

fetched
├── 123.123.123.123
│   └── var
│       └── log
│           └── nginx
│               └── access.log
└── 234.234.234.234
    └── var
        └── log
            └── nginx
                └── access.log

How to strip parent directories from the file path when fetching files

By default, fetch will store fetched files with the following file structure:

fetched
├── 123.123.123.123
│   └── var
│       └── log
│           └── nginx
│               └── access.log
└── 234.234.234.234
    └── var
        └── log
            └── nginx
                └── access.log

For each host, fetch will put the file in a directory like {{ inventory_hostname }}/path/to/remote/file, which includes the inventory_hostname and the full file path to the file on the remote host.

If the /var/log/nginx is not important to you, you can remove this from the fetched file by using the flat parameter. This will also remove the {{ inventory_hostname }} from the fetched file. I recommend adding it back in like this (note the trailing slash):

- name: fetch nginx access log
  fetch:
    src: /var/log/nginx/access.log
    dest: fetched/{{ inventory_hostname }}/
    flat: true

Producing the following file structure:

fetched
├── 123.123.123.123
│   └── access.log
└── 234.234.234.234
    └── access.log

You could also do something like this:

- name: fetch nginx access log
  fetch:
    src: /var/log/nginx/access.log
    dest: fetched/access.log.{{ inventory_hostname }}
    flat: true

Which will result in this file structure:

fetched
├── access.log.123.123.123.123
└── access.log.234.234.234.234

How to fetch a directory recursively

Fetching a directory from a remote host is not currently supported by the fetch module. You can fetch multiple files with fetch in a loop (see below) or by using the synchronize module.

fatal: [123.123.123.123]: FAILED! => {"changed": false, "file": "/var/log/nginx", "msg": "remote file is a directory, fetch cannot work on directories"}
fatal: [234.234.234.234]: FAILED! => {"changed": false, "file": "/var/log/nginx", "msg": "remote file is a directory, fetch cannot work on directories"}

How to fetch multiple files in a loop

Use the loop keyword and {{ item }} to fetch multiple files.

- name: fetch nginx access and error log
  fetch:
    src: /var/log/nginx/{{ item }}.log
    dest: fetched
  register: fetch_output
  loop:
    - access
    - error
fetched
├── 123.123.123.123
│   └── var
│       └── log
│           └── nginx
│               ├── access.log
│               └── error.log
└── 234.234.234.234
    └── var
        └── log
            └── nginx
                ├── access.log
                └── error.log

How to capture fetch module output

Use the register keyword to capture the output of the fetch module.

- name: fetch nginx access log
  fetch:
    src: /var/log/nginx/access.log
    dest: fetched
  register: fetch_output

- debug: var=fetch_output

The debug task above will output the following:

ok: [123.123.123.123] => {
    "fetch_output": {
        "changed": true,
        "checksum": "6335795a2cbe55cff38d22ec5591ff7777117255",
        "dest": "/path/to/ansible/fetched/123.123.123.123/var/log/nginx/access.log",
        "failed": false,
        "md5sum": "e64e6ce907561ef869c9204ff25d6bfa",
        "remote_checksum": "6335795a2cbe55cff38d22ec5591ff7777117255",
        "remote_md5sum": null
    }
}
ok: [234.234.234.234] => {
    "fetch_output": {
        "changed": true,
        "checksum": "057b6d679c80ca15746fa9e5f40f39d103d86e79",
        "dest": "/path/to/ansible/fetched/234.234.234.234/var/log/nginx/access.log",
        "failed": false,
        "md5sum": "61ef4e5683a40e4ddc3e10b6d8cbd195",
        "remote_checksum": "057b6d679c80ca15746fa9e5f40f39d103d86e79",
        "remote_md5sum": null
    }
}

How to capture fetch module output from a loop

When using the fetch module in a loop, the output of each item in the loop will go into the results key of the registered variable.

- name: fetch nginx access and error log
  fetch:
    src: /var/log/nginx/{{ item }}.log
    dest: fetched
  register: fetch_output
  loop:
    - access
    - error

- debug: var=fetch_output
ok: [123.123.123.123] => {
    "fetch_output": {
        "changed": true,
        "msg": "All items completed",
        "results": [
            {
                ...
                "changed": false,
                "checksum": "6335795a2cbe55cff38d22ec5591ff7777117255",
                "dest": "/path/to/ansible/fetched/123.123.123.123/var/log/nginx/access.log",
                "failed": false,
                "file": "/var/log/nginx/access.log",
                "item": "access",
                "md5sum": "e64e6ce907561ef869c9204ff25d6bfa"
            },
            {
                ...
                "changed": true,
                "checksum": "617ba89c3206c19fdb1fc12bd7f9acd55284884f",
                "dest": "/path/to/ansible/fetched/123.123.123.123/var/log/nginx/error.log",
                "failed": false,
                "item": "error",
                "md5sum": "454fca61de36f4b418e5b0c78ae9a2d8",
                "remote_checksum": "617ba89c3206c19fdb1fc12bd7f9acd55284884f",
                "remote_md5sum": null
            }
        ]
    }
}
ok: [234.234.234.234] => {
    "fetch_output": {
        "changed": true,
        "msg": "All items completed",
        "results": [
            {
                "_ansible_ignore_errors": null,
                "_ansible_item_label": "access",
                "_ansible_item_result": true,
                "_ansible_no_log": false,
                "changed": false,
                "checksum": "057b6d679c80ca15746fa9e5f40f39d103d86e79",
                "dest": "/path/to/ansible/fetched/234.234.234.234/var/log/nginx/access.log",
                "failed": false,
                "file": "/var/log/nginx/access.log",
                "item": "access",
                "md5sum": "61ef4e5683a40e4ddc3e10b6d8cbd195"
            },
            {
                "_ansible_ignore_errors": null,
                "_ansible_item_label": "error",
                "_ansible_item_result": true,
                "_ansible_no_log": false,
                "changed": true,
                "checksum": "fcd31dc1410809efc74ca6cd0808624677abb16b",
                "dest": "/path/to/ansible/fetched/234.234.234.234/var/log/nginx/error.log",
                "failed": false,
                "item": "error",
                "md5sum": "0ebc15483ca8d6d0b0b86d182d4c88e0",
                "remote_checksum": "fcd31dc1410809efc74ca6cd0808624677abb16b",
                "remote_md5sum": null
            }
        ]
    }
}

Further reading

Ansible fetch Module on Ansible Docs

Comments