Automate the Syncing of Files and Code to Remote Servers

by Blake Edwards


Posted Fri Jan 04 2019 16:54:14 GMT+0000 (UTC)


Tech stack
Ubuntu 16.04 (remote machine)
Windows, Mac, or Linux (local machine)

Ever find yourself excessively dragging and dropping files to your remote server through tools like FileZilla? Here is an easy solution to all of your drag and drop problems!

Overview
  • Install Dropbox on remote machine
    • This example will use Ubuntu
  • Install dropbox on your local machine
  • Setup directory syncing

Install Dropbox on remote machine
Great tutorials for both graphical and command line installations of dropbox for Ubuntu.

Install dropbox on your local machine
Go to this link and download the respective Dropbox version for your operating system. This website should detect your operating system by default. Follow the install instructions for your operating system. 

After the installation, open the installed Dropbox application, create an account, or login to your existing Dropbox account to begin syncing your files to your remote server.

Setup Directory Syncing
In its current state, Dropbox will sync files to a Dropbox created directory. If you want to learn how to sync a dropbox directory to another local directory on your remote machine follow the directions below. If you are satisfied with the your files syncing to the Dropbox directory on your remote machine then you are done! 

This next part of the tutorial will show you how to continuously sync two local (on the same system) directories. If you are interested in learning how to simply sync two directories one time, this tutorial will help too! The first part will go over how to sync directories and the second we will go over how to make the syncing continuous.

Requirements
Ubuntu 16.04 or above
Rsync version 3.1.1 or above

Assumptions
Basic command line knowledge: moving through a file system, finding the current working directory, and listing directory contents. If you do not have this basic knowledge please learn / become familiar with the basics and come back. It will ultimately benefit you in the long run and will especially help you with this tutorial! 

Personal note: At one point I had no idea how to move through a file system without a graphical user interface (GUI) such as Finder or File Explorer. However, I took the time to learn some of the basics and it has helped me tremendously with everything I work on!

Table of Contents
  • Find the two local directories you want to sync
  • Sync the directories using rsync 
  • Setup continuous syncing

Find the two local directories you want to sync
Directory one:   /root/my/directory/sync-from
Directory two:   /root/my/directory/sync-to

Sync the directories using rsync 

command-line$rsync -avu —delete "/root/my/directory/sync-from” “/root/my/directory/sync-to"

Rsync command arguments explained . . .
-a 
Preserve filesystem attributes
-v 
Run verbosely
-u
Skip syncing newer files in the directory that you are syncing to
—delete
Delete the files in the directory you are syncing to that do not exist in the directory you are syncing from
Where I got this answer: https://unix.stackexchange.com/questions/203846/how-to-sync-two-folders-with-command-line-tools

To find other rsync options:
command-line$ rsync -h


Setup continuous syncing
We will be using a system service to continuously sync files from one directory to another (even after the machine is rebooted).
Helpful source: https://linuxconfig.org/how-to-automatically-execute-shell-script-at-startup-boot-on-systemd-linux and https://unix.stackexchange.com/questions/198444/run-script-every-30-min-with-systemd


If you enjoyed this tutorial and found it helpful please share it with a friend!

~ Blake Edwards