In order to backup TestCaddy, you need to backup the main TestCaddyMaster database on your SQL server and all the Product Databases you have created.
You can also backup the ini file on each box where TestCaddy is installed, but this only stores the SQL connection details (and a few defaults) which are fairly easy to enter again if needed.
You can backup your SQL databases in any standard way. Typically you can do this via Microsoft SQL Management Studio (R) or automate it with a batch file and sql script file run by a scheduled task. Details on these can be found at this Microsoft link,
A simple batch file, TestCaddyBackup.bat might look like this:
sqlcmd -S localhost\TestCaddy -U sa -P TestCaddy1 -i TestCaddyBackup.sql
And the TestCaddyBackup.sql file which backs up the TestCaddyMaster database as well as a product database called MyProductA might be,

BACKUP DATABASE TestCaddyMaster TO DISK='c:\TestCaddyMaster.dat'
BACKUP DATABASE MyProductA TO DISK='c:\MyProductA.dat'