Vivek Shukla - Docker

docker buildx: Multi-platform Builds

2025-04-23T00:00:00+00:00

Tested on: Ubuntu 22.04 LTS
Docker Version: 28.1.1
Docker Buildx Version: v0.23.0

Documentation: Multi-platform Builds</a>

# check if buildx is installed</span></span>
docker</span> buildx version</span></span></code></pre># installing and registering QEMU</span></span>
docker</span> run</span> --privileged --rm</span> tonistiigi/binfmt</span> --install</span> all</span></span></code></pre># create new build that runs in docker container</span></span>
docker</span> buildx create</span> --name</span> mybuilder</span> --driver</span> docker-container</span></span></code></pre># switching to our builder</span></span>
docker</span> buildx use mybuilder</span></span></code></pre># build and pushing image directly to image store (make sure you are logged in)</span></span>
docker</span> buildx build</span> --platform</span> linux/amd64,linux/arm64</span> -t</span> <</span>usernam</span>e</span>></span>/</span><</span>image_nam</span>e</span>></span>:</span><</span>ta</span>g</span>></span> .</span> --push</span></span></code></pre>



Docker Image Publishing
2025-04-23T00:00:00+00:00
Login</strong></p>
docker</span> login</span> --username</span> <</span>usernam</span>e</span>></span> --password-stdin</span></span></code></pre>
Build</strong></p>
docker</span> build .</span> -t</span> <</span>image_nam</span>e</span>></span>:</span><</span>ta</span>g</span>></span></span></code></pre>
Tag</strong></p>
docker</span> tag</span> <</span>image_nam</span>e</span>></span>:</span><</span>ta</span>g</span>> <</span>usernam</span>e</span>></span>/</span><</span>image_nam</span>e</span>></span>:</span><</span>ta</span>g</span>></span></span></code></pre>
Push</strong></p>
docker</span> push</span> <</span>usernam</span>e</span>></span>/</span><</span>image_nam</span>e</span>></span>:</span><</span>ta</span>g</span>></span></span></code></pre>


Plausible Docker Volume Backup
2025-03-18T00:00:00+00:00
#!/bin/bash</span></span>
</span>
home</span>=</span>"/home/<update>/"</span></span>
dest</span>=</span>"<update>"</span></span>
bucket</span>=</span>"<update>"</span></span>
pg_container_id</span>=</span>"<update>"</span></span>
ch_container_id</span>=</span>"<update>"</span></span>
passphrase</span>=</span>"<update>"</span></span>
</span>
date</span>=</span>$(</span>date</span> +"%Y%m%d%H%M%S"</span>)</span></span>
pl_repo</span>=</span>$home</span>"/"</span>$date</span>"_plausible.tar.gz"</span></span>
pl_repo_gpg</span>=</span>$pl_repo</span>".gpg"</span></span>
pg_data</span>=</span>$home</span>"/"</span>$date</span>"_db-data.tar.gz"</span></span>
pg_data_gpg</span>=</span>$pg_data</span>".gpg"</span></span>
ch_data</span>=</span>$home</span>"/"</span>$date</span>"_event-data.tar.gz"</span></span>
ch_data_gpg</span>=</span>$ch_data</span>".gpg"</span></span>
</span>
# stopping postgres and clickhouse containers before backup</span></span>
docker</span> container stop</span> $pg_container_id $ch_container_id</span></span>
</span>
# creating gzip file</span></span>
tar</span> --exclude=images --exclude-vcs -czf</span> $pl_repo</span> -C</span> $home</span> plausible</span></span>
sudo</span> tar</span> -czf</span> $pg_data</span> -C</span> /var/lib/docker/volumes/ plausible_db-data</span></span>
sudo</span> tar</span> -czf</span> $ch_data</span> -C</span> /var/lib/docker/volumes/ plausible_event-data</span></span>
</span>
# starting containers</span></span>
docker</span> container start</span> $pg_container_id $ch_container_id</span></span>
</span>
# encrypting</span></span>
echo</span> $passphrase</span> |</span> gpg</span> --batch --yes --passphrase-fd</span> 0</span> -o</span> $pl_repo_gpg</span> --symmetric</span> $pl_repo</span></span>
echo</span> $passphrase</span> |</span> gpg</span> --batch --yes --passphrase-fd</span> 0</span> -o</span> $pg_data_gpg</span> --symmetric</span> $pg_data</span></span>
echo</span> $passphrase</span> |</span> gpg</span> --batch --yes --passphrase-fd</span> 0</span> -o</span> $ch_data_gpg</span> --symmetric</span> $ch_data</span></span>
</span>
# saving to s3</span></span>
rclone</span> copy</span> $pl_repo_gpg $dest</span>:</span>$bucket</span></span>
rclone</span> copy</span> $pg_data_gpg $dest</span>:</span>$bucket</span></span>
rclone</span> copy</span> $ch_data_gpg $dest</span>:</span>$bucket</span></span>
</span>
# deleting local copies</span></span>
rm</span> -f</span> $pl_repo $pg_data $ch_data</span></span>
rm</span> -f</span> $pl_repo_gpg $pg_data_gpg $ch_data_gpg</span></span></code></pre>

Automate Postgres Database Backup and Upload to S3 2023-10-20T00:00:00+00:00 Database is a very critical system for an application. To prepare for any disaster we all must take regular backups of our database. In this tutorial we will create a bash script to automate postgres database backup from a docker container</a> and native postgres installation then upload to S3 based storage system. 🔗</a>What we will cover</h2> Write db backup automation script for docker and non-docker based system </li> Upload backup file to AWS S3 </li> Setup the cron job for daily backup </li> </ol> 🔗</a>Steps</h2> Move to your project folder, where docker-compose.yml is. </li> Take the database backup in container </li> Copy the backup file from container filesystem to our desired location </li> Delete the backup file inside the container </li> Upload backup file to S3 </li> Delete the local backup file </li> </ol> Code snippet below covers step 1 to 4. But it is not optimal, later we will make modification so that we can easily automate it and make it re-usable. cd project_dir docker compose exec db_service_name sh -c "pg_dump -U db_user db_name > filename" docker compose cp db_service_name:filename destination/filename docker compose exec db_service_name rm filename</code></pre>🔗</a>Upload to S3</h2> To upload file to S3, we need to have aws cli installed, you can follow this guide from AWS docs to install awscli</a>. After installing awscli, you need to run aws configure</code> and provide your key id and key secret. You can use any S3 based storage system here. aws s3 cp filename s3://your-bucket</code></pre> If you are using some other S3 based storage system then you’ll have to provide custom --endpoint-url</code> argument here, make sure you read the respective docs for this. 🔗</a>Full Script</h2> We have laid out all the steps and code to accomplish that, now let’s modify in such a way that we can easily automate it and make it reusable. The filename is created based on the current datetime of the system, so that we can avoid ambiguity and exactly know how old is the backup file. 🔗</a>For docker compose</h3> #!/bin/bash ############## ✅ Update These Values ################## directory="your-project-dir" home="/home/<user>" db_service_name="db" db_user="db_user" db_name="db_name" ##################################################### ############ 🔴 CHANGE WITH CAUTION 🔴 ############# date=$(date +"%Y-%m-%d-%H:%M:%S") filename=$db_name"_"$date".sql" cd $home cd $directory docker compose exec $db_service_name sh -c "pg_dump -U $db_user $db_name > $filename" docker compose cp $db_service_name:$filename $home"/"$filename docker compose exec $db_service_name rm $filename cd $home ##################################################### ##################################################### ############## ✅ CAN CHANGE BELOW ################## aws s3 cp $filename s3://your-bucket rm $filename</code></pre> At the top, you’ve to update variable values based on your project and directory structure, and the main code will use that to backup the database. At the end of script, we have aws command to copy the backup file to S3, here you need to update bucket name and if you are using other S3 storage provider (other than AWS), then --endpoint-url</code> argument with appropriate value. 🔗</a>For Native Postgres Installation</h3> #!/bin/bash ############## ✅ Update These Values ################## home="/home/<user>" db_user="db_user" db_name="db_name" ##################################################### ############ 🔴 CHANGE WITH CAUTION 🔴 ############## date=$(date +"%Y-%m-%d-%H:%M:%S") filename=$db_name"_"$date".sql" ch $home pg_dump -U $db_user $db_name > $filename ##################################################### ############## ✅ CAN CHANGE BELOW ################## aws s3 cp $filename s3://urlify-db-backup rm $filename</code></pre> When running pg_dump in the native installation, there are two ways postgres will try to authenticate the user. Peer based authentication</a>: It will check if the current user (system logged in user) is same as user passed to pg_dump. </li> If peer auth fails then postgres looks for .pgpass</code> file in home directory for the password. </li> If both fails then it might ask for password. </li> </ol> So, make sure you use 1st method and run the script on the same user as the one passed to pg_dump. 🔗</a>Setup Cron Job</h2> To automate backup and upload of our database, we will use crontab. It comes built-in with almost all the Linux distros and are very easy to setup. Give your script executable permission: chmod +x db_backup.sh</code></pre> Open crontab: crontab -e</code></pre> This will open up crontab in your prefered choice of editor. Use the below statement to run the backup script everyday at 12 PM. 0 12 * * * /home/<user>/db_backup.sh >> cron.log 2>&1</code></pre> Create cron.log file touch cron.log</code></pre> Use full path if cron.log is not in current user’s home folder. How To Take Postgres Database Backup From Docker Container 2023-10-18T00:00:00+00:00 To take database backup in Postgres we use pg_dump. pg_dump</code> is a utility provided by PostgreSQL to create the database backup</a>. We are gonna use docker exec</code> and cp</code> command along with pg_dump to take Postgres database backup from docker container. 🔗</a>pg_dump</h2> Let’s take a look at how to take backup with pg_dump</code> command: pg_dump -U your_db_user -d your_db_name > db_backup.sql</code></pre> This is straight forward, pass your db username, database name that you want to take backup of and the backup filename which would be created. 🔗</a>docker exec</h2> Let’s do the same thing on Docker, for that we will use docker exec</a>: docker exec container_name_or_id sh -c "pg_dump -U your_db_user -d your_db_name > db_backup.sql"</code></pre> Make sure that the specified container is running, since docker exec</code> command executes on the running container. 🔗</a>docker cp</h2> The docker exec command created the database backup but it’s inside the container file system. So now we need to copy that file from container to our file system. For that purpose docker gives us docker cp</a>, which we can use to copy to/from the container’s filesystem. docker cp container_name_or_id:db_backup.sql your_destination</code></pre> Now, we have copied our database backup file to our filesystem, it’s time to delete the one from our container. To delete backup file from container do this: docker exec container_name_or_id rm db_backup.sql 🔗</a>All Commands For docker</h2> docker exec container_name_or_id sh -c "pg_dump -U your_db_user -d your_db_name > db_backup.sql" docker cp container_name_or_id:db_backup.sql your_destination docker exec container_name_or_id rm db_backup.sql</code></pre>🔗</a>docker compose</h2> You can run the same commands just use docker compose</code> instead of just docker</code>, and replace container name/id with service name. docker compose exec service_name sh -c "pg_dump -U your_db_user -d your_db_name > db_backup.sql" docker compose cp service_name:db_backup.sql your_destination docker compose exec service_name rm db_backup.sql</code></pre>

Vivek Shukla - Docker

docker buildx: Multi-platform Builds

Docker Image Publishing

Plausible Docker Volume Backup

Automate Postgres Database Backup and Upload to S3

🔗</a>What we will cover</h2> Write db backup automation script for docker and non-docker based system</p> </li> Upload backup file to AWS S3</p> </li>

How To Take Postgres Database Backup From Docker Container