Skip to navigation

Rotate On-Prem Agent credentials and certificates

View as Markdown

Rotate the org access secret, a service account password, a secrets manager value, or a TLS certificate on the Moveworks On-Prem Agent. This page is for administrators with shell access to the agent host.

Before you begin

  • Agent host: the server or VM that runs the agent’s Docker or Podman container. SSH in and run every command there, from the folder that contains setup_agent.sh. To find it, run sudo find / -name setup_agent.sh 2>/dev/null.

  • Agent group: group ID 17540. The agent runs as the moveworks user in this group inside the container, so its files must belong to this group. The host needs no group of that name.

  • Agent files: in the moveworks subfolder of the install folder:

    On the hostIn the containerHolds
    moveworks/conf//home/moveworks/agent/conf/agent_config.yml
    moveworks/certs//home/moveworks/agent/certs/TLS certificates and agent_key.pem
    moveworks/logs//var/log/moveworks/Agent log files

    Edit files at the host path. In agent_config.yml, certificate paths use the container path.

  • Backup: copy the config and certificates first:

    sudo cp -a moveworks/conf/agent_config.yml moveworks/conf/agent_config.yml.bak
    sudo cp -a moveworks/certs moveworks/certs.bak

Commands shown with sudo apply to Docker deployments. Omit sudo when using Podman.

Don’t delete or re-create moveworks/certs/. Its agent_key.pem decrypts every encrypted_value; lose it and you must re-enter every secret.

Find the setting to change

What you are rotatingSetting in agent_config.ymlSection
Moveworks org access secretmoveworks_config.moveworks_access_secretRotate the Moveworks org access secret
LDAP or Active Directory service account passwordldap_config.ldap_service_password, or ldap_forest_config.<domain>.ldap_service_passwordRotate a service account password
REST service account password or tokenThe secret under rest_configs.<SERVICE>.header_decoratorsRotate a service account password
A secret stored in AWS Secrets Manager or Azure Key Vaultaws_entry or azure_entryRotate a secret held in a secrets manager
Certificate for the connection to Moveworks, such as a TLS-inspecting proxy’s CAmoveworks_config.path_to_certRotate a certificate
Certificate for LDAP over SSL or StartTLSldap_config.path_to_cert, or ldap_forest_config.<domain>.path_to_certRotate a certificate
Certificate for an on-prem REST systemrest_configs.<SERVICE>.ca_cert_pathRotate a certificate

To see the current settings, run sudo cat moveworks/conf/agent_config.yml. Field definitions are in the Configuration File Reference.

Rotate the Moveworks org access secret

  1. In Moveworks Setup > Core Platform > On-Premise Agents, select Generate Secret and save the new secret.
  2. Run sudo ./setup_agent.sh --reconfigure, select [2] Moveworks Config, and update moveworks_access_secret.
  3. Select [8] Save & Exit. On agents earlier than 2.11.0, it’s [6]. The configurator requires agent 2.10.4 or later; on older agents, edit the file as in the next section.
  4. Restart the agent.

The org access secret expires if it isn’t used within 30 days of generation. If the agent shows as Inactive or logs a 401 error, rotate the secret first.

Rotate a service account password

The agent encrypts plain-text secrets in agent_config.yml each time it starts, so you can replace a secret in the file directly.

  1. Stop the agent with sudo ./setup_agent.sh --stop, then open the config with sudo vi moveworks/conf/agent_config.yml.

  2. Find the secret you’re rotating. Change its encrypted_value key to value, and replace the encrypted string with the new password:

    ldap_config:
    ldap_service_password:
    value: your-new-password
  3. Save and exit (in vi, press Esc and type :wq).

  4. Restart the agent.

  5. Confirm the agent re-encrypted the secret: sudo grep -n -A2 password moveworks/conf/agent_config.yml shows encrypted_value. If it still shows value, the agent didn’t start; check the logs as in Confirm it works.

You can also change these passwords with sudo ./setup_agent.sh --reconfigure, under [3] LDAP Configs or [4] REST Configs.

Rotate a secret held in a secrets manager

If the setting holds an aws_entry or azure_entry, agent_config.yml doesn’t change.

  1. Update the value under the same secret_name, in the console or with the CLI:

    aws secretsmanager put-secret-value --secret-id moveworks_agent/<secret-name> --secret-string '<new-value>'

    The CLI saves the value in your shell history; use the console to avoid that.

  2. Restart the agent so it reads the new value.

Rotate a certificate

  1. Find the current file: sudo grep -nE 'path_to_cert|ca_cert_path' moveworks/conf/agent_config.yml. On the host, /home/moveworks/agent/certs/ldap_cert.pem is moveworks/certs/ldap_cert.pem.
  2. Get the new certificate as a PEM file. For LDAP, follow Configure an LDAP SSL certificate for the On-Prem Agent.
  3. Stop the agent with sudo ./setup_agent.sh --stop.
  4. Copy the file to the host (scp ldap_cert.pem <user>@<agent-host>:~). Replace the old file and give the agent group read access:
    sudo cp ~/ldap_cert.pem moveworks/certs/ldap_cert.pem
    sudo chgrp 17540 moveworks/certs/ldap_cert.pem
    sudo chmod 640 moveworks/certs/ldap_cert.pem
    For a new file name, edit the path with sudo vi moveworks/conf/agent_config.yml or sudo ./setup_agent.sh --reconfigure:
    path_to_cert: /home/moveworks/agent/certs/<new-file>.pem
  5. Restart the agent.

How each setting uses the file:

  • moveworks_config.path_to_cert adds to the certificates the agent already trusts. Separate several paths with commas. A missing file stops the agent with error in bondhttp: cert load for path.
  • LDAP path_to_cert and REST ca_cert_path replace the trusted certificates for that connection. Include every CA certificate in the server’s chain. A non-PEM file loads as empty and fails with x509: certificate signed by unknown authority.

Include the issuing CA certificates, not only the server’s own certificate, or you must replace the file each time the server renews.

Restart the agent and check folder permissions

On every start, the agent reads its certificates and rewrites agent_config.yml to encrypt value entries. If the agent group can’t write the config, the agent exits with open /home/moveworks/agent/conf/agent_config.yml: permission denied and the container keeps restarting.

  1. Stop the agent and validate the host:

    sudo ./setup_agent.sh --stop
    sudo ./setup_agent.sh --validate

    --validate checks the OS, the container runtime, folder ownership (you and the agent group), user lingering, and a Moveworks URL you enter. It stops at the first failure.

  2. If it reports is not owned by the current user or is not owned by the agent group, fix the folders and validate again:

    sudo ./setup_agent.sh --permissions

    This is the same as sudo chgrp -R 17540 moveworks, sudo chmod -R ug+rwx moveworks, and sudo chmod g+s moveworks.

  3. Start the agent with sudo ./setup_agent.sh --start. Each run starts one container; repeat it for each container the host had.

--validate doesn’t check individual files, so a file copied in with scp can still be unreadable after it passes.

Fix permissions on a single file

To fix individual files, give the agent group (17540) access. Run from the folder that contains setup_agent.sh:

sudo chgrp 17540 moveworks/conf/agent_config.yml moveworks/certs/<cert-file>.pem
sudo chmod 660 moveworks/conf/agent_config.yml
sudo chmod 640 moveworks/certs/<cert-file>.pem

The config needs 660 because the agent rewrites it on every start; certificates need only 640 (or 644). To copy a replaced file’s permissions instead, run sudo chown --reference=<old-file> <new-file> and sudo chmod --reference=<old-file> <new-file>. On Podman, also run sudo chown <agent-user> <file>.

Don’t use 600: it removes the group access the agent needs. Don’t use 777: it lets every account on the host read your passwords.

Confirm it works

  1. Check that the container is running: sudo docker ps, or podman ps for Podman.
  2. Check the logs for errors: sudo grep -hE 'x509|401|denied|LDAP Result' moveworks/logs/*.INFO.log | tail -20. To watch live, run sudo tail -f moveworks/logs/*.INFO.log.
  3. In Moveworks Setup > Core Platform > On-Premise Agents, confirm the agent is Active. It shows Healthy once Moveworks sends it requests. See On-Prem Agent Operations and Health.

Troubleshoot credential and certificate rotation

Restart the agent after each fix.

  • open .../agent_config.yml: permission denied, or the container keeps restarting: run sudo chgrp 17540 moveworks/conf/agent_config.yml && sudo chmod 660 moveworks/conf/agent_config.yml.
  • TLS cert path: open ...: permission denied: run sudo chgrp 17540 moveworks/certs/<file>.pem && sudo chmod 640 moveworks/certs/<file>.pem.
  • TLS cert path: open ...: no such file or directory: the configured path names no file in moveworks/certs/. Compare sudo grep -n cert moveworks/conf/agent_config.yml with sudo ls moveworks/certs/.
  • x509 errors after a certificate change: see Troubleshoot LDAP SSL certificate errors.
  • No secrets decrypt: agent_key.pem was lost or replaced. Restore it with sudo cp -a moveworks/certs.bak/agent_key.pem moveworks/certs/. Without a backup, re-enter each secret: in agent_config.yml, rename every encrypted_value key to value and set it to the secret, as in Rotate a service account password. The agent encrypts them on its next start.