Terrashine

Terrashine is a terraform provider mirror1 implementation that works by automatically caching dependencies as providers are requested.

Use cases:

  • Avoid rate-limits when actively developing (github has a 60 request per hour rate limit)
  • Faster downloads of terraform providers, particularly in CI environments.
  • Ensuring that terraform providers don't disappear if the source has been deleted.

Installation

Terrashine is a deployed as a standalone binary. Binary releases for x86-64 are published can be found on the releases page.

Alternatively, the project can be built from source with the following command:

On Debian-based Linux:

sudo apt install musl-tools
rustup target add x86_64-unknown-linux-musl

SQLX_OFFLINE=1 cargo build --release

Once built, the binary can be found at ./target/release/terrashine

See the --help for more information:

./terrashine --help

Client configuration

Once terrashine is all setup, the terraform client needs to be configured to use the mirror. This can be done a terraform configuration file entry.

  • On linux and MacOS, a .terraformrc file should created in the home directory.
  • On Windows terraform.rc file should be created in the %APPDATA% directory.

This file should contain configuration to point terraform at the installed provider mirror.

provider_installation {
  network_mirror {
    url = "https://example.com/mirror/v1/"
  }
}

For more information on the terraform configuration file, see the CLI Configuration File docs from hashicorp.

High availability

Multiple instances of terrashine can be deployed to support high availability. Simply point the instances at the same storage layer.

Metrics

Terrashine supports the /metrics endpoint to export metrics in the prometheus format. This can be ingested via prometheus or any other monitoring tool that understands the prometheus exposition format.

Dependencies

The following components are required to run terrashine

  • PostgreSQL
  • S3-compatible object storage
  • TLS terminating reverse proxy (NGINX, HAProxy etc..)

Notes

1

Terrashine implements the Provider Network Mirror Protocol

Reverse proxy

The terraform provider network mirror protocol requires that the API request be performed over encrypted HTTPS. Terrashine itself does not currently perform TLS termination, a reverse proxy must always be deployed to perform this function for a working setup.

Securing the admin API

Terrashine provides an API endpoint which should be protected by the reverse proxy. Endpoints hosted under the /api/ should be considered privileged and not exposed externally without an authentication layer. Currently, authentication should be implemented by the reverse proxy and is not natively supported by terrashine.

External Caching

Caching is optional however, terrashine sets Cache-Control headers where possible to allow caching by external reverse proxies. If caching is required, this should be achieved by configuring the reverse proxy to cache responses as appropriate. Cache headers are sometimes not set in cases where caching may incorrect behavior by the terraform client. For example: headers are not set in scenarios where caching could result in subsequent requests from the same client seeing inconsistent views of the available packages, resulting in an error when downloading packages.

Example NGINX configuration

Here is an example NGINX configuration that provides TLS termination and caching enabled for a locally deployed terrashine instance.

user  nginx;
worker_processes  auto;

error_log  /dev/stdout notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /dev/stdout  main;
    sendfile        on;
    keepalive_timeout  65;
    proxy_cache_path /tmp keys_zone=mycache:10m;


    server {
        listen              443 ssl;
        server_name         localhost;
        proxy_cache         mycache;
        ssl_certificate     localhost.pem;
        ssl_certificate_key localhost.pem;
        ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers         HIGH:!aNULL:!MD5;
        location / {
            # terrashine
            proxy_pass http://localhost:9543;
        }
        # Deny traffic to the API endpoint
        # This could be protected by basic auth as well
        location /api {
            deny all;
            return 404;
        }
    }
}

Setting up S3-compatible object storage

Terrashine requires an S3 compatible storage to cache terraform providers. It is currently tested against AWS S3 and Minio. To set up AWS S3, please follow the AWS instruction to create a bucket and obtain a set of credentials.

Terrashine requires a bucket to be created and a set of credentials to be available. The AWS Rust SDK is used to authenticate to S3 so the credentials can be provided to the binary using any supported credential provider. Most commonly, this can be provided using the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

Terrashine requires the following actions in the IAM policy:

  • GetObject
  • PutObject

For a non-AWS S3 compatible object storage, see the docker-compose.yml in the repository where an example minio integration is used. In this case, a CLI flag --s3-endpoint can be used to point terrashine at an alternative URL.

PostgreSQL

Terrashine requires postgreSQL to store metadata associated with upstream registries and downloaded terraform providers. Terrashine does not store any terraform providers inside of postgreSQL itself so the requirements are typically fairly light.

Please see postgreSQL's excellent documentation to set up the database.

Database migrations

When upgrading or starting up terrashine for the first time, we need to run database migrations against the database. We can perform the migration with the following command.

terrashine migrate --database-url postgres://postgres:password@localhost/

This command should be executed from a checkout of the git repository associated with the version.

Starting terrashine

At this point, its expected that you now have a postgres database provisioned, an S3 endpoint for object storage and a reverse proxy for TLS termination.

Terrashine is configured via CLI flags and environment variables. For a complete list of environment variables see:

terrashine --help

Example

Here is an example of starting up terrashine using an S3 bucket named terrashine-example-test-1111, with credentials provided as environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. A TLS terminating reverse proxy hosted is on example.com in this setup. Note that the /mirror/v1/ path is required in the URL to allow the backend server to serve up redirects correctly.

AWS_REGION=eu-west-1 AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=xxx RUST_LOG=info  ./terrashine  server --s3-bucket-name terrashine-example-test-1111  --http-redirect-url https://example.com/mirror/v1/

Private Registry Authentication

To insert credentials for private registries, the auth token can be updated with an API call.

curl  -X POST \
    -d '{ "data": { "token": "xxxx"} }' \
    -H 'Content-Type: application/json' \
    https://localhost:9443/api/v1/credentials/example.com

Likewise, to delete a credential, the auth token can be deleted via a DELETE request.

curl  -X DELETE https://localhost:9443/api/v1/credentials/example.com

Mirror refreshing

To update the mirror dependencies, terrashine will refresh the provider metadata against the provider registry. This is triggered by a request for the index listing after the refresh interval has been exceeded. The refresh interval is configurable and defaults to an hour.

The refresh occurs is triggered by the request, however it is performed as a background job so the request that performs the request will return immediately with the stale data. This design prevents outages of the upstream provider registry from affecting the performance of requests for existing mirrored providers. The refresh job independent across multiple instances of terrashine is ran in a highly available environment, so duplicate refresh jobs can occur as the number of nodes increase.

Only the version and provider metadata is updated, the actual provider artifacts are never modified after the initial download.