HTTP Runner (runner = "http")

The native API caller (http runner) allows a module to fetch data securely from external APIs without requiring external scripts, curl, or Python binaries. The rescile engine executes HTTP requests natively and streams the response directly to the target_input or target_asset.

Key Configuration Fields

• url: The target URL. Supports full Tera templating ({{ env.API_TOKEN }}).
• method: Standard HTTP method (GET, POST, PUT, PATCH, DELETE). Defaults to GET.
• inject_headers: A key-value map of custom headers. Sensitive values (like API keys) can be securely injected via templates.
• payload: The request body for POST/PUT requests. Supports Tera templating.
• response_filter: A map allowing you to filter and transform the API response before saving. Supported engines: "jmespath!" or "tera!".
• pagination: A block to automatically handle paginated APIs.
  • strategy: The pagination strategy (e.g., "cursor").
  • cursor_jmespath: JMESPath query to extract the next page token or URL from the response.
  • inject_into: How to apply the extracted cursor for the next request:
    • "url": The next request will use the extracted cursor directly as its full URL (ideal when the API returns a complete next link).
    • "query", "payload", "header": The extracted cursor is made available as the {{ cursor }} variable in the Tera context. You must explicitly reference it in your url, payload, or inject_headers template.
  • max_pages: Integer safety limit to prevent infinite loops.

Examples

Basic GET Request with Headers

[generator.fetch-azure-inventory]
runner = "http"
target_input = "azure_raw.json"
method = "GET"
url = "https://management.azure.com/subscriptions/{{ params.sub_id }}/resources?api-version=2021-04-01"
abort_on_failure = true

[generator.fetch-azure-inventory.inject_headers]
"Authorization" = "Bearer {{ env.AZURE_ACCESS_TOKEN }}"
"Accept" = "application/json"

Dynamic Request Signatures

For APIs that require requests to be cryptographically signed, you can use full Tera templating inside the inject_headers values. You can combine template variables with custom filters like hmac_sha256 and base64_encode to construct complex authorization headers dynamically:

[generator.fetch-secure-api]
runner = "http"
target_input = "secure_data.json"
method = "GET"
url = "https://api.example.com/v1/data"

[generator.fetch-secure-api.inject_headers]
"Content-Type" = "application/json"
"Authorization" = """
{%- set msg = "GET /v1/data" -%}
{%- set sig = msg | hmac_sha256(key=env.API_SECRET, encoding="hex") -%}
CustomAuth credential={{ env.API_KEY }},signature={{ sig -}}
"""

Pagination and Response Filtering

If the API returns paginated results, you can configure rescile to loop through them automatically. The response filter allows you to extract or format exactly the data you need. The results from all pages are concatenated into a single JSON array or object (depending on your target_input format).

Using jmespath! for extraction and filtering: When you only need to extract a specific array or filter elements from the JSON response, jmespath! provides a fast and concise query syntax:

[generator.fetch-azure-servers]
runner = "http"
target_input = "azure_servers.json"
method = "GET"
url = "https://management.azure.com/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01"
response_filter = { "jmespath!" = "data[?type=='microsoft.compute/virtualmachines']" }

Using tera! for complex transformations:

[generator.fetch-netbox-bundles]
description = "Fetches available netbox bundles from Docker Hub"
runner = "http"
target_input = "bundle.json"
method = "GET"
url = "https://hub.docker.com/v2/repositories/netboxcommunity/netbox/tags?page_size=100"
abort_on_failure = true
ttl = "24h"

[generator.fetch-netbox-bundles.pagination]
strategy = "cursor"
cursor_jmespath = "next"
inject_into = "url"
max_pages = 100

[generator.fetch-netbox-bundles.response_filter]
"tera!" = '''
[
{% set first = true %}
{% for res in response.results %}
  {% if res.name is starting_with("v") %}
    {% if not first %},{% endif %}
    {
      "key": "netbox-{{ res.name | trim_start_matches(pat="v") }}",
      "value": {
        "version": "{{ res.name | trim_start_matches(pat="v") }}",
        "nix_hash": "",
        "checksum": ""
      }
    }
    {% set_global first = false %}
  {% endif %}
{% endfor %}
]
'''

Available Template Variables

For url, payload, and inject_headers:

• {{ env.VAR_NAME }}: System environment variables passed to rescile.
• {{ params.PARAM_NAME }}: Module parameters passed via the CLI (--module-params).
• {{ target_asset }} / {{ target_input }}: Absolute path to the target file.
• {{ cursor }}: The extracted cursor from the previous page (when paginating with inject_into != "url").

For response_filter.tera!:

• {{ response }}: The parsed JSON response object from the API.
• Note: Environment and parameter variables are also available here.