I’ve seen this question on one of the forums I participate and wanted to document it a bit, with not too much specifics to give some clues about the process and things to avid.

Relocating a server usually it’s just a matter of installing the new one, adding the services, software etc.

This operation can be made in an easier way if the deployment is automated.

For example, I do use a common set of playbooks in Ansible to configure, install, etc all my servers in a similar way, also creating helper commands I can use for doing some standard operations and monitoring.

In the case of validators, you cannot just copy the files and start the service as there are some risks involved:

  • A validator cannot double sign (same key signs the same block), this causes in most networks the validator to be tombstoned (🪦), so the validator cannot recover.
  • A validator must provide a high uptime to avoid slashing, that is, getting a loss of the tokens delegated to them for not helping the network be secured.

In order to avoid that, the best option is to have a new server installed, configured and running until it completely syncs the chain… depending on the chain time running and validator settings about pruning, it might take more or less time to get it done.

Avoid double-signing

From this point, have extra caution, your node might perform double signing and be tombstoned.

Setup the new validator following the project documentation:

  • restore account instead of creating new one
  • sync the chain
  • etc.

Once the new validator is up and synced, there’s a missing and important key file to copy over: .$chain/config/priv_validator_key.json, but first, read the next steps for safety.

Make really sure to:

  • stop the previous validator server, disable autostart of daemons, monitoring that might re-run it, etc
    • Double check on the chain explorer that the validator is no longer signing blocks.
    • As a security measure, you can move the .$chain/ folder to a new name, to make sure that by any chance the validator can start again.
  • copy the .$chain/data/priv_validator_state.json to the remote new server, in this way, the validator knows the last block it saw.
  • you can additionally rsync the changes from the data folder from the old validator to the data folder of the new one.

Once the file .$chain/config/priv_validator_key.json has been put in place on your new validator, and the service started, the validator will quickly start syncing the blocks that might have changed during the downtime and start signing again.

As final steps:

  • Check on the explorer that the validator is signing
  • Reenable monitoring scripts on the new validator and check that everything is reported fine.
  • Nuke the old server
  • If you provide services over the validator name (rpc, etc), make sure to update DNS to point to the new one

Good luck!