Skip to content

Vault Proxy Integration#

HashiCorp Vault proxy is a lightweight, client-side daemon designed to simplify interactions with HashiCorp Vault, automating critical tasks like authentication, token renewal, and static secret caching.

What Is HashiCorp Vault Proxy?#

Vault Proxy aims to remove the initial hurdle to adopt Vault by providing a more scalable and simpler way for applications to integrate with Vault. Vault Proxy acts as an API Proxy for Vault, and can optionally allow or force interacting clients to use its automatically authenticated token.

Key Features of Vault Proxy#

  • Automated Authentication (Auto-Auth)
    • Supports multiple authentication methods such as AppRole, Kubernetes, and AWS. This auto-auth feature automatically handles the process of authenticating to Vault, eliminating manual steps.
  • Token and Lease Management and Renewal
    • Once authenticated, Vault Agent continuously monitors token and lease expiration. It automatically renews tokens and leases as needed, ensuring applications maintain uninterrupted access to secrets.
  • Caching
    • Allows client-side caching of responses containing newly created tokens, responses containing leased secrets, and static KV secrets.
  • API Proxy
    • Acts as a proxy for communicating with Vault's API, optionally using (or forcing the use of) the Auto-Auth token.

Please view the Vault Proxy Documentation for more information on features.

Potential Use Cases#

High-Performance Environments#

In systems where high throughput is critical, local caching provided by the Vault Proxy reduces latency by serving secrets directly from cache. This is particularly beneficial in microservices architectures and real-time applications where speed is paramount.

Disaster Recovery and Business Continuity#

By caching secrets locally, the Vault Proxy ensures that in the event of a temporary Vault server outage, applications can still access critical secrets. This capability is vital for maintaining business continuity during maintenance or unexpected disruptions.

QuickStart with Proxy#

Pre-requisites#

  1. Ensure you have installed the Vault Enterprise Binary

Setup#

  1. Export an environmental variable for the Vault CLI to access the Vault server.
Bash
export VAULT_ADDR=<VAULT_ADDRESS>
  1. Export an environmental variable for the Vault CESI Namespace.
Bash
export VAULT_NAMESPACE=admin/<CESI_NAMESPACE>
  1. Login to Vault. Make sure to set the -namespace flag to admin
Bash
vault login -method=saml -namespace=admin

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                hvs.token
token_accessor       accessor
token_duration       1h
token_renewable      true
token_policies       ["default"]
identity_policies    ["default"]
policies             ["default"]
token_meta_role      saml_role

Create a Working Directory#

  1. Create a directory to store configuration and template files.
Bash
mkdir $HOME/vault-test && cd $HOME/vault-test

Create data#

  1. Create a data file
Bash
tee data.json -<<EOF
{
   "organization": "ACME Inc.",
   "customer_id": "ABXX2398YZPIE7391",
   "region": "US-West",
   "zip_code": "94105",
   "type": "premium",
   "contact_email": "james@acme.com",
   "status": "active"
}
EOF
  1. Create the data in a KV v2 Secrets Engine
Bash
vault kv put secret/customers/acme @data.json

Create an AppRole#

Auto-auth authenticates with Vault to get a client token, store, and manage the token lifecycle. Vault Agent can heal from an invalid token error caused by unexpected incidence (for example, Vault admin revoked the client token used by the Vault Agent). We will be using an AppRole to configure the token and to enable this self-healing capability and enable the static secret caching feature.

For easier implementation, AppRoles can also be configured with terraform from our examples repo.

Note

If installing multiple proxies, please try to reduce the number of roles needed for the agent to preserve client count. Vault considers an AppRole to be only 1 client as long as the same Role ID is used.

Please contact the secrets management team for additional questions.

  1. Create a policy for the approle auth method with permission to subscribe to KV event updates with the Vault event notification system. For example, to create a policy that grants access to static secret (KVv1 and KVv2) events, you need permission to subscribe to the events endpoint, as well as the list and subscribe permissions on KV secrets you want to get secrets from.
    1. By subscribing to KV events, that means that Proxy will receive updates as soon as a secret changes, which reduces staleness in the cache. Vault Proxy will only check for a secret update if an event notification indicates that the related secret was updated
Bash
vault policy write proxy -<<EOL
path "sys/events/subscribe/kv*" {
   capabilities = ["read"]
}
path "*" {
   capabilities = ["list", "subscribe"]
   subscribe_event_types = ["kv*"]
}
path "secret/*" {
   capabilities = ["read", "list"]
}
path "sys/leases/*" {
  capabilities = ["create", "update"]
}
path "auth/token/*" {
  capabilities = ["create", "update"]
}
EOL
  1. Ensure tokens also have capabilities-self access to allows tokens to request cached secrets.
    1. Vault tokens receive update permissions by default. If you have modified or removed the default policy, you must explicitly create a policy with the appropriate permissions.
Text Only
path "sys/capabilities-self" {
    capabilities = ["update"]
}
  1. Create a new role within your approle with the associated policy above
Bash
vault write auth/approle/role/proxy policies=proxy token_num_uses=5
  1. Retrieve your role ID and secret ID and store it locally
Bash
vault read -field=role_id auth/approle/role/proxy/role-id > $HOME/vault-test/roleID.txt
vault write -f -field=secret_id auth/approle/role/proxy/secret-id > $HOME/vault-test/secretID.txt

Note

A role ID and valid secret ID will be needed anytime the vault agent restarts. See this example for an automated way of generating a new secret ID for restarts.

Start a Vault Proxy#

Note

If you are having issues with the Vault Proxy, please make sure the $VAULT_NAMESPACE environmental variable has been correctly set

You will be reading the secrets created above with the Vault Agent while using an AppRole for auto authentication

  1. Create a Vault Agent configuration file
Bash
tee proxy-config.hcl -<<EOF
pid_file = "./pidfile"
vault {
   address = $VAULT_ADDR
   namespace = $VAULT_NAMESPACE
}

auto_auth {
   method {
      type = "approle"
      config = {
         role_id_file_path = "/home/$HOME/vault-test/roleID.txt"
         secret_id_file_path = "/home/$HOME/vault-test/secretID.txt"
      }
   }
   sink "file" {
      config = {
            path = "/home/$HOME/vault-token-via-agent"
      }
   }
}

// Address in which the proxy will listen on
listener "tcp" {
   address     = "127.0.0.1:8200"
   tls_disable = true
}

api_proxy {
   // if use_auto_auth_token is enabled, auto_auth stanza must be configured
   use_auto_auth_token = true
   enforce_consistency = "always"
}

cache {
   // Requires Vault Enterprise 1.16 or later
   cache_static_secrets = true
   static_secret_token_capability_refresh_interval = "5m" //Determines how often Proxy needs to verify and update cached tokens and secrets
}
EOF
  1. Start the Vault Proxy with the log-level set to debug. Take note of the Api Address from the proxy configuration
Bash
vault proxy -config=proxy-config.hcl -log-level=debug

Example output:

Text Only
==> Vault Proxy started! Log data will stream in below:

==> Vault Proxy configuration:

           Api Address 1: http://127.0.0.1:8200
                     Cgo: disabled
               Log Level: debug
                 Version: Vault v1.18.4+ent, built 2025-01-29T14:07:00Z
             Version Sha: a7b6963bc01c138f0b8e24163597817234b8d517

2025-02-25T14:41:40.240-0600 [INFO]  proxy.sink.file: creating file sink
2025-02-25T14:41:40.241-0600 [INFO]  proxy.sink.file: file sink configured: path=/home/$HOME/vault-token-via-agent mode=-rw-r----- owner=1036104 group=1036104
2025-02-25T14:41:40.241-0600 [INFO]  proxy.cache: cache configured: cache_static_secrets=true disable_caching_dynamic_secrets=false
2025-02-25T14:41:40.242-0600 [DEBUG] proxy.apiproxy: configuring inmem auto-auth sink
2025-02-25T14:41:40.243-0600 [DEBUG] proxy: would have sent systemd notification (systemd not present): notification=READY=1
2025-02-25T14:41:40.243-0600 [INFO]  proxy.cache.staticsecretcacheupdater: starting static secret cache updater subsystem
2025-02-25T14:41:40.243-0600 [INFO]  proxy.auth.handler: starting auth handler
2025-02-25T14:41:40.243-0600 [INFO]  proxy.auth.handler: authenticating
2025-02-25T14:41:40.243-0600 [INFO]  proxy.sink.server: starting sink server
2025-02-25T14:41:40.291-0600 [INFO]  proxy.auth.handler: authentication successful, sending token to sinks
2025-02-25T14:41:40.292-0600 [INFO]  proxy.sink.file: token written: path=/home/$HOME/vault-token-via-agent
2025-02-25T14:41:40.293-0600 [DEBUG] proxy.cache.leasecache: storing auto-auth token into the cache
2025-02-25T14:41:40.293-0600 [INFO]  proxy.auth.handler: starting renewal process
2025-02-25T14:41:40.334-0600 [INFO]  proxy.auth.handler: renewed auth token
2025-02-25T14:41:40.426-0600 [DEBUG] proxy.cache.staticsecretcacheupdater: starting pre-event stream update of static secrets
2025-02-25T14:41:40.427-0600 [DEBUG] proxy.cache.staticsecretcacheupdater: finished pre-event stream update of static secrets
...snip...

Self-Healing Tokens#

As we configured the AppRole above, it should now automatically refresh its token in an event that the TTL expires or the token is revoked. This can be viewed in the logs.

Note

A role ID and valid secret ID will be needed anytime the vault agent restarts. See this example for an automated way of generating a new secret ID for restarts.

Text Only
...snip...
[INFO]  proxy.apiproxy: proxy received an invalid token error
[INFO]  proxy.auth.handler: invalid token found, re-authenticating
[INFO]  proxy.auth.handler: authenticating
[INFO]  proxy.auth.handler: authentication successful, sending token to sinks
[INFO]  proxy.auth.handler: starting renewal process
[INFO]  proxy.sink.file: token written: path=/user/vault-token-via-agent
[INFO]  proxy.auth.handler: renewed auth token

Cache Static Secrets#

  1. In a separate terminal or window, export the environmental variables for VAULT_ADDR. Use the API Address that was configured for you for the VAULT_ADDR
Bash
export VAULT_ADDR=<API_ADDRESS>
  1. Run a command to fetch the sample data.
Bash
vault kv get secret/customers/acme

======= Secret Path =======
secret/data/customers/acme

======= Metadata =======
Key                Value
---                -----
created_time       2025-02-14T17:46:02.756842519Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

======== Data ========
Key              Value
---              -----
contact_email    james@acme.com
customer_id      ABXX2398YZPIE7391
organization     ACME Inc.
region           US-West
status           active
type             premium
zip_code         94105
  1. Go back to the window where the proxy was deployed and view the logs. The logs should specify that the response for this command has been stored in the cache.

Example output:

Text Only
[INFO]  proxy.apiproxy: received request: method=GET path=/v1/secret/data/customers/acme
[DEBUG] proxy.cache.leasecache: forwarding request from cache: method=GET path=/v1/secret/data/customers/acme
[INFO]  proxy.apiproxy: forwarding request to Vault: method=GET path=/v1/secret/data/customers/acme
[DEBUG] proxy.apiproxy.client: performing request: method=GET url=https://VAULT_ADDRESS:8200/v1/secret/data/customers/acme
[DEBUG] proxy.cache.leasecache: storing static secret response into the cache: path=admin/CESI_UNIT/secret/data/customers/acme id=5b8b353a20d950abe9643fe141eb571ac83ecfb818940acd7271bac252d9a61b

  1. Run the vault kv get secret/customers/acme again. The vault proxy should now fetch the cached secrets which can be seen in the logs.
Text Only
[INFO]  proxy.apiproxy: received request: method=GET path=/v1/secret/data/customers/acme
[DEBUG] proxy.cache.leasecache: returning cached static secret response: id=5b8b353a20d950abe9643fe141eb571ac83ecfb818940acd7271bac252d9a61b path=admin/CESI_UNIT/secret/data/customers/acme
  1. After the period of time specified in the static_secret_token_capability_refresh_interval parameter within the cache stanza of the Proxy config, the cache will evict it's contents containing tokens and secrets. Secrets are not renewed and are newly cached during each call to read the secrets.
Text Only
[DEBUG] proxy.cache.staticsecretcapabilitymanager: successfully evicted capabilities index from cache: index.ID=7b1ffbb375f7104b1ada51236306a7399c5dd0a9332bfbc3dd08c7fb81e4d2ac

Running Vault Proxy as a RHEL9 systemctl service#

If you would like to run the vault proxy as a systemctl service in the background, see this example service file. This service file also re-generates the secretID on restart (see ExecStartPre). The following policy addition is needed for the approle to create new secret-id's. For more information or an ansible role to setup the vault proxy, see https://github.com/umn-secm/hcp-vault-ansible-examples/blob/main/deploy_vault_proxy.yml.

Bash
path "auth/approle/role/proxy/secret-id" {
  capabilities = ["create", "update"]
}
Bash
tee /etc/systemd/system/vault-proxy.service -<<EOF
[Unit]
Description="HashiCorp Vault Proxy"
Documentation="https://developer.hashicorp.com/vault/docs"

[Service]
User=vault
Group=vault
SecureBits=keep-caps
AmbientCapabilities=CAP_IPC_LOCK
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
NoNewPrivileges=yes
Environment="VAULT_ADDR=https://hcp-vault-private-vault-fc507e0d.5d5b1f21.z1.hashicorp.cloud:8200/"
Environment="VAULT_NAMESPACE=admin/jst"
ExecStartPre=-/bin/bash -c '[ ! -f /etc/vault.d/secretID ] || [ ! -s /etc/vault.d/secretID ] && VAULT_TOKEN=$(cat /etc/vault.d/vault-token-via-agent) vault write -f -field=secret_id auth/approle/role/proxy/secret-id > /etc/vault.d/secretID'
ExecStart=/bin/vault proxy -non-interactive -config=/etc/vault.d/proxy-config.hcl -log-level=debug -log-file=/etc/vault.d/log/
ExecReload=/bin/kill --signal HUP
KillMode=process
KillSignal=SIGINT

[Install]
WantedBy=multi-user.target
EOF

References#

Vault Proxy and Agent Quick Start Guide

What is Vault Proxy?

Vault Proxy Static Secret Caching Guide