This is the 14th day of my participation in the First Challenge 2022

Like Git, DVC allows collaboration in a distributed environment. We can easily import all the data files and directories and matching source code exactly the same into any machine. All you need to do is set up a remote repository for your DVC project and push the data there so that others can access it. DVC currently supports Amazon S3, Microsoft Azure Blob Storage, Google Drive, Google Cloud Storage, SSH, HDFS, and other remote Storage addresses, and the list continues to grow. (For a complete list and configuration instructions, see DVC Remote Add.)

For example, let’s set up S3 remote storage for the DVC project and push or pull it.

Create S3 buckets

If none is available in your S3 account, follow the instructions in creating a bucket. As an advanced alternative, you can use the AWS S3 MB command instead.

Set up the DVC remote storage

To actually configure S3 remote storage in your project, provide the BUCKET URL used to store the data to the DVC remote add command. Such as:

$ dvc remote add -d myremote s3://mybucket/path

Setting 'myremote' as a default remote.
Copy the code

The -d (–default) parameter sets myRemote as the default remote storage for the project. This adds myremote to your.dvc/config. The configuration file now has a remote section:

['remote "myremote"']
url = s3://mybucket/path

[core]
remote = myremote
Copy the code

DVC remote Modify provides multiple options for configuring S3 buckets.

Next, we commit the changes and push the code to a Git remote repository:

$ git add .dvc/config
$ git push
Copy the code

Upload data and code

After data is added to a project using DVC run or other commands, it is stored in the local cache. Upload it to remote storage using the DVC push command:

dvc push
Copy the code

Download the code

Download the code and DVC metadata files from your Git server using regular Git commands.

For example, use git clone for a project that is not locally available:

$ git clone https://github.com/example/project.git
$ cd myproject
Copy the code

For local existing projects, use Git pull:

$ git pull
Copy the code

Download the data

To download data files for your project, run DVC pull:

$ dvc pull
Copy the code

DVC pull will download the missing data file from the default remote storage configured in the.dvc/config file.