Vivek Shukla - Tutorial

Automate Postgres Database Backup and Upload to S3

2023-10-20T00:00:00+00:00

Database is a very critical system for an application. To prepare for any disaster we all must take regular backups of our database.

In this tutorial we will create a bash script to automate postgres database backup from a docker container</a> and native postgres installation then upload to S3 based storage system.

🔗</a>What we will cover</h2>

Write db backup automation script for docker and non-docker based system </li>

Upload backup file to AWS S3 </li>

Setup the cron job for daily backup </li> </ol>
🔗</a>Steps</h2>

Move to your project folder, where docker-compose.yml is. </li>

Take the database backup in container </li>

Copy the backup file from container filesystem to our desired location </li>

Delete the backup file inside the container </li>

Upload backup file to S3 </li>

Delete the local backup file </li> </ol>
Code snippet below covers step 1 to 4. But it is not optimal, later we will make modification so that we can easily automate it and make it re-usable.
`cd project_dir docker compose exec db_service_name sh -c "pg_dump -U db_user db_name > filename" docker compose cp db_service_name:filename destination/filename docker compose exec db_service_name rm filename</code></pre>`
🔗</a>Upload to S3</h2>
To upload file to S3, we need to have aws cli installed, you can follow this guide from AWS docs to install awscli</a>.
After installing awscli, you need to run `aws configure</code> and provide your key id and key secret. You can use any S3 based storage system here.`
`aws s3 cp filename s3://your-bucket</code></pre> If you are using some other S3 based storage system then you’ll have to provide custom --endpoint-url</code> argument here, make sure you read the respective docs for this.`
🔗</a>Full Script</h2> We have laid out all the steps and code to accomplish that, now let’s modify in such a way that we can easily automate it and make it reusable. The filename is created based on the current datetime of the system, so that we can avoid ambiguity and exactly know how old is the backup file. 🔗</a>For docker compose</h3> #!/bin/bash ############## ✅ Update These Values ################## directory="your-project-dir" home="/home/<user>" db_service_name="db" db_user="db_user" db_name="db_name" ##################################################### ############ 🔴 CHANGE WITH CAUTION 🔴 ############# date=$(date +"%Y-%m-%d-%H:%M:%S") filename=$db_name"_"$date".sql" cd $home cd $directory docker compose exec $db_service_name sh -c "pg_dump -U $db_user $db_name > $filename" docker compose cp $db_service_name:$filename $home"/"$filename docker compose exec $db_service_name rm $filename cd $home ##################################################### ##################################################### ############## ✅ CAN CHANGE BELOW ################## aws s3 cp $filename s3://your-bucket rm $filename</code></pre> At the top, you’ve to update variable values based on your project and directory structure, and the main code will use that to backup the database. At the end of script, we have aws command to copy the backup file to S3, here you need to update bucket name and if you are using other S3 storage provider (other than AWS), then --endpoint-url</code> argument with appropriate value. 🔗</a>For Native Postgres Installation</h3> #!/bin/bash ############## ✅ Update These Values ################## home="/home/<user>" db_user="db_user" db_name="db_name" ##################################################### ############ 🔴 CHANGE WITH CAUTION 🔴 ############## date=$(date +"%Y-%m-%d-%H:%M:%S") filename=$db_name"_"$date".sql" ch $home pg_dump -U $db_user $db_name > $filename ##################################################### ############## ✅ CAN CHANGE BELOW ################## aws s3 cp $filename s3://urlify-db-backup rm $filename</code></pre> When running pg_dump in the native installation, there are two ways postgres will try to authenticate the user. Peer based authentication</a>: It will check if the current user (system logged in user) is same as user passed to pg_dump. </li> If peer auth fails then postgres looks for .pgpass</code> file in home directory for the password. </li> If both fails then it might ask for password. </li> </ol> So, make sure you use 1st method and run the script on the same user as the one passed to pg_dump. 🔗</a>Setup Cron Job</h2> To automate backup and upload of our database, we will use crontab. It comes built-in with almost all the Linux distros and are very easy to setup. Give your script executable permission: chmod +x db_backup.sh</code></pre> Open crontab: crontab -e</code></pre> This will open up crontab in your prefered choice of editor. Use the below statement to run the backup script everyday at 12 PM. 0 12 * * * /home/<user>/db_backup.sh >> cron.log 2>&1</code></pre> Create cron.log file touch cron.log</code></pre> Use full path if cron.log is not in current user’s home folder.



How To Take Postgres Database Backup From Docker Container
2023-10-18T00:00:00+00:00
To take database backup in Postgres we use pg_dump. pg_dump</code> is a utility provided by PostgreSQL to create the database backup</a>. We are gonna use docker exec</code> and cp</code> command along with pg_dump to take Postgres database backup from docker container.</p>
🔗</a>pg_dump</h2>
Let’s take a look at how to take backup with pg_dump</code> command:</p>
pg_dump</span> -U</span> your_db_user</span> -d</span> your_db_name</span> ></span> db_backup.sql</span></span></code></pre>
This is straight forward, pass your db username, database name that you want to take backup of and the backup filename which would be created.</p>
🔗</a>docker exec</h2>
Let’s do the same thing on Docker, for that we will use docker exec</a>:</p>
docker</span> exec</span> container_name_or_id sh</span> -</span>c</span> "pg_dump -U your_db_user -d your_db_name > db_backup.sql"</span></span></code></pre>
Make sure that the specified container is running, since docker exec</code> command executes on the running container.</p>
🔗</a>docker cp</h2>
The docker exec command created the database backup but it’s inside the container file system. So now we need to copy that file from container to our file system.</p>
For that purpose docker gives us docker cp</a>, which we can use to copy to/from the container’s filesystem.</p>
docker</span> cp container_name_or_id:db_backup.sql your_destination</span></span></code></pre>
Now, we have copied our database backup file to our filesystem, it’s time to delete the one from our container. To delete backup file from container do this:</p>
docker exec container_name_or_id rm db_backup.sql</p>
🔗</a>All Commands For docker</h2>
docker</span> exec container_name_or_id sh</span> -c</span> "pg_dump -U your_db_user -d your_db_name > db_backup.sql"</span></span>
</span>
docker</span> cp container_name_or_id:db_backup.sql your_destination</span></span>
</span>
docker</span> exec container_name_or_id rm db_backup.sql</span></span></code></pre>🔗</a>docker compose</h2>
You can run the same commands just use docker compose</code> instead of just docker</code>, and replace container name/id with service name.</p>
docker</span> compose exec service_name sh</span> -c</span> "pg_dump -U your_db_user -d your_db_name > db_backup.sql"</span></span>
</span>
docker</span> compose cp service_name:db_backup.sql your_destination</span></span>
</span>
docker</span> compose exec service_name rm db_backup.sql</span></span></code></pre>


How To Store and Serve Private Media Files in Django
2023-09-14T00:00:00+00:00
In this blog post we will see how to use AWS S3</a> to store and serve private media files in django. S3 provides a layer of security by making object private and accessible to those with the valid presigned url</a> and is only valid for however long owner decides.</p>
We will continue from our last blog post on How To Store Django Static and Media Files in AWS S3</a>. In this post we will only discuss parts which involves setting up private media.</p>
🔗</a>Storage Backend</h2>
Like we had MediaStorage</code> for user uploaded public media files, we will need to create a new storage backend class for Private media files.</p>
In your_project/custom_storage.py</code></p>
class</span> PrivateMediaStorage</span>(</span>S3Boto3Storage</span>):</span></span>
    location</span> =</span> "private"</span></span>
    default_acl</span> =</span> "private"</span></span>
    file_overwrite</span> =</span> False</span></span>
    custom_domain</span> =</span> False</span></span></code></pre>
location</strong>: This is the directory in the s3 bucket where all the private files will go. You can change the name as per your need.</p>
default_acl</strong>: ACL</a> stands for Access Control List, this enables us to manage access to bucket and objects. Here we are passing private</code> as we want the objects to be private.</p>
file_overwrite</strong>: We do not want files to be over-written if they have the same name.</p>
custom_domain</strong>: Earlier we’ve set AWS_S3_CUSTOM_DOMAIN</code> in our settings for static and media url generation, however here we want to generate a signed url so we need to set custom domain to False.</p>
🔗</a>Model</h2>
It is better to explicitly define the private media storage class in model fields so that we do not unnecessarily complicate other file fields which uses public MediaStorage.</p>
Therefore we need to explicitly pass the PrivateMediaStorage</code> to the model field to store it as private media and generate signed url when we serve.</p>
In your_project/your_app/models.py</code></p>
class</span> PrivateFile</span>(</span>models</span>.</span>Model</span>):</span></span>
    file</span> =</span> models</span>.</span>FileField</span>(storage</span>=</span>PrivateMediaStorage</span>()</span>,</span> upload_to</span>=</span>"files"</span>)</span></span>
    uploaded_on</span> =</span> models</span>.</span>DateTimeField</span>(auto_now_add</span>=</span>True</span>)</span></span>
</span>
    def</span> __str__</span>(</span>self</span>):</span></span>
        return</span> str</span>(</span>self</span>.</span>uploaded_on</span>.</span>date</span>())</span></span></code></pre>
file</strong>: This is the field which we are using to point to our private media. You can see we are passing our custom storage class to models.FileField which will now store all our uploads with the settings that we defined for PrivateMediaStorage</code> class.</p>
🔗</a>Conclusion</h2>
You can use the file field as you usually do, now all the uploaded files will be private. While calling field you will get a presigned url which will allow users to access the file.</p>
🔗</a>Things to keep in mind</h2>

FileField and ImageField in django only stores the location of the file and is created as varchar in database which has max_length of 100. So, if your file location is long then you may want to increase it’s allowed length by passing value to max_length</code>.</p>
FileField</a></p>
</blockquote>

The presigned url is only valid for 1 hour by default, you can change this by setting a different value for querystring_expire</code> (in seconds) in PrivateMediaStorage class.</p>
</blockquote>


How To Store Django Static and Media Files in AWS S3
2023-09-06T00:00:00+00:00
In this guide we will use django-storages package to manage our static and media files. Let’s walkthrough the whole process on how to store django static and media files in aws s3.</p>
S3 is a cloud based object storage service provided by AWS (Amazon Web Services), it stands for Simple Storage Service (S3).</p>
Using AWS S3 to serve Static and Media files in Django makes sense since Django is not good at serving those and S3 provides a lot more feature around these.</p>
🔗</a>S3 Bucket</h2>
🔗</a>Create a bucket</h3>
Let’s create a S3 Bucket where our files will be stored. Few things to keep in mind:</p>


ACL must be enabled, so that we can control object permission from our Django</p>
</li>

Uncheck all “Block Public Access settings for this bucket” settings and check the acknowledge box</p>
</li>
</ol>

  
      Create S3 bucket
    </figcaption></figure>
🔗</a>CORS</h3>
After bucket is created successfully, now we need to update the CORS settings of the bucket so that if content is being served to different host then your browser might block it.</p>


Open the bucket</p>
</li>

Go to the Permissions</strong> tab</p>
</li>

At the end of the page there will be CORS settings</p>
</li>

Put the JSON given below, make sure to put the host which will use the S3 files, you can put multiples.</p>
</li>
</ul>
[</span></span>
  {</span></span>
    "AllowedHeaders"</span>:</span> [</span>"*"</span>]</span>,</span></span>
    "AllowedMethods"</span>:</span> [</span>"HEAD"</span>,</span> "GET"</span>]</span>,</span></span>
    "AllowedOrigins"</span>:</span> [</span>"yourdomain.com"</span>]</span>,</span></span>
    "ExposeHeaders"</span>:</span> []</span></span>
  }</span></span>
]</span></span></code></pre>🔗</a>IAM User</h2>
An IAM user allow us to access AWS resources programmatically. We need IAM credentials to upload the files to S3.</p>
🔗</a>Create IAM User</h3>


Go to IAM page in your AWS Console.</p>
</li>

Go to Users</strong> then Add users</strong>.</p>
</li>

Fill out the name that you want to give this user.</p>
</li>

Go ahead and create the user, you can skip the permission.</p>
</li>
</ol>

  
      Create IAM user
    </figcaption></figure>
🔗</a>Setting up Permission</h3>


Open IAM user which you intend on using.</p>
</li>

Make sure you are on Permissions</strong> (default) tab.</p>
</li>

From “Add permissions” dropdown select “Create inline policy”.</p>
</li>

Select JSON</strong> policy editor instead of Visual and paste below JSON. Make sure to update the bucket name.</p>
</li>

Give the policy appropriate name and save the changes.</p>
</li>
</ol>
{</span></span>
  "Version"</span>:</span> "2012-10-17"</span>,</span></span>
  "Statement"</span>:</span> [</span></span>
    {</span></span>
      "Effect"</span>:</span> "Allow"</span>,</span></span>
      "Action"</span>:</span> "s3:*"</span>,</span></span>
      "Resource"</span>:</span> [</span>"arn:aws:s3:::your-bucket-name"</span>,</span> "arn:aws:s3:::your-bucket-name/*"</span>]</span></span>
    }</span></span>
  ]</span></span>
}</span></span></code></pre>🔗</a>Get the access key</h3>
To use the IAM user in our django project we will need to generate access key.</p>


Open user page, go to Security credentials</strong> tab</p>
</li>

Scroll down to Access keys</strong> section and click on Create access key</strong> button</p>
</li>

Select “Local code” from the use case menu</p>
</li>
</ol>

  
      IAM user access key
    </figcaption></figure>
🔗</a>Setting Up Django</h2>
🔗</a>Installing necessary packages</h3>
We will be using django-storages</a> package to manage our connection with AWS S3 and to upload our files. We will also need boto3</a> package to connect to AWS, which django-storages uses.</p>
pip install boto3 django-storages</p>
🔗</a>Storage Backend</h3>
django-storages provides S3Boto3Storage</code> class for media storage and S3StaticStorage</code> for static files.</p>
We will subclass S3Boto3Storage</code> to create our custom storage class so that we can customize certain things.</p>
In your_project/custom_storage.py</code></p>
from</span> django</span>.</span>conf</span> import</span> settings</span></span>
from</span> storages</span>.</span>backends</span>.</span>s3boto3</span> import</span> S3Boto3Storage</span></span>
</span>
class</span> StaticStorage</span>(</span>S3Boto3Storage</span>):</span></span>
    location</span> =</span> settings</span>.</span>STATICFILES_LOCATION</span></span>
    default_acl</span> =</span> "public-read"</span></span>
    querystring_auth</span> =</span> False</span></span>
</span>
class</span> MediaStorage</span>(</span>S3Boto3Storage</span>):</span></span>
    location</span> =</span> settings</span>.</span>MEDIAFILES_LOCATION</span></span>
    default_acl</span> =</span> "public-read"</span></span>
    file_overwrite</span> =</span> False</span></span></code></pre>🔗</a>Update settings.py</h3>
We also need to update our settings.py file with media and static files settings:</p>
STATICFILES_DIRS</span> =</span> [</span></span>
    BASE_DIR</span> /</span> "static_assets"</span>,</span></span>
]</span></span>
</span>
AWS_S3_ACCESS_KEY_ID</span> =</span> "your_aws_key"</span></span>
AWS_S3_SECRET_ACCESS_KEY</span> =</span> "your_aws_secret"</span></span>
AWS_S3_REGION_NAME</span> =</span> "ap-south-1"</span>  # change to your region</span></span>
AWS_STORAGE_BUCKET_NAME</span> =</span> "your_bucket_name"</span></span>
AWS_DEFAULT_ACL</span> =</span> None</span></span>
</span>
AWS_S3_CUSTOM_DOMAIN</span> =</span> f</span>"</span>{AWS_STORAGE_BUCKET_NAME}</span>.s3.amazonaws.com"</span></span>
</span>
# static files</span></span>
STATICFILES_LOCATION</span> =</span> "static"</span></span>
STATIC_URL</span> =</span> f</span>"https://</span>{AWS_S3_CUSTOM_DOMAIN}</span>/</span>{STATICFILES_LOCATION}</span>/"</span></span>
</span>
# media files</span></span>
MEDIAFILES_LOCATION</span> =</span> "media"</span></span>
MEDIA_URL</span> =</span> f</span>"https://</span>{AWS_S3_CUSTOM_DOMAIN}</span>/</span>{MEDIAFILES_LOCATION}</span>/"</span></span>
</span>
# storage backends</span></span>
STORAGES</span> =</span> {</span></span>
    "default"</span>:</span> {</span>"BACKEND"</span>:</span> "your_project.custom_storage.MediaStorage"</span>}</span>,</span>  # for media</span></span>
    "staticfiles"</span>:</span> {</span>"BACKEND"</span>:</span> "your_project.custom_storage.StaticStorage"</span>}</span>,</span></span>
}</span></span>
AWS_S3_OBJECT_PARAMETERS</span> =</span> {</span></span>
    "CacheControl"</span>:</span> "max-age=2592000"</span>,</span></span>
}</span></span></code></pre>
We have done all the setup for our static and media files to be uploaded to our S3 bucket. You can test it out in your django project. You can also checkout the tutorial on how to create file upload api using django rest framework</a>.</p>


Django Rest Framework File Upload API
2023-06-21T00:00:00+00:00
Uploading file via API is a very common requirement. In this guide we will learn to build Django Rest Framework</a> File Upload API.</p>
🔗</a>Setup</h2>


Django: 4.2</p>
</li>

Django Rest Framework: 3.14.0</p>
</li>

Project Name: fileupload</code></p>
</li>

App Name: api</code></p>
</li>
</ul>
🔗</a>Add your app in INSTALLED_APPS</h3>
In fileupload/settings.py</code></p>
INSTALLED_APPS</span> =</span> [</span></span>
    ...</span>.</span>.</span></span>
    'rest_framework'</span>,</span></span>
    'api'</span>,</span> # your django app</span></span>
]</span></span></code></pre>🔗</a>Update MEDIA path in settings.py</h3>
We need to set the path where we want the uploaded files to be stored.</p>
In settings.py</code></p>
MEDIA_URL</span> =</span> '/media/'</span></span>
MEDIA_ROOT</span> =</span> BASE_DIR</span> /</span> 'media'</span></span></code></pre>🔗</a>Update project’s urls.py file</h3>
For Django to be able to serve media files we need to enable this using static</code> helper function that Django provides. This only works in DEBUG mode and only if folder location is local. source</a></p>
In fileupload/urls.py</code></p>
from</span> django</span>.</span>conf</span> import</span> settings</span></span>
from</span> django</span>.</span>conf</span>.</span>urls</span>.</span>static</span> import</span> static</span></span>
from</span> django</span>.</span>contrib</span> import</span> admin</span></span>
from</span> django</span>.</span>urls</span> import</span> include</span>,</span> path</span></span>
</span>
urlpatterns</span> =</span> [</span></span>
    path</span>(</span>'admin/'</span>,</span> admin</span>.</span>site</span>.</span>urls)</span>,</span></span>
    path</span>(</span>'api/'</span>,</span> include</span>(</span>'api.urls'</span>,</span> namespace</span>=</span>'api'</span>))</span>,</span></span>
]</span></span>
</span>
urlpatterns</span> +=</span> static</span>(settings</span>.</span>MEDIA_URL</span>,</span> document_root</span>=</span>settings</span>.</span>MEDIA_ROOT</span>)</span></span></code></pre>🔗</a>Model</h2>
Let’s create a Model where we store the file path (as file</code>) and file upload time (as uploaded_on</code>). So the uploaded file will be stored in media</code> folder and file</code> field will contain it’s path.</p>
In your api/models.py</code></p>
class</span> UploadedFile</span>(</span>models</span>.</span>Model</span>):</span></span>
    file</span> =</span> models</span>.</span>FileField</span>()</span></span>
    uploaded_on</span> =</span> models</span>.</span>DateTimeField</span>(auto_now_add</span>=</span>True</span>)</span></span>
</span>
    def</span> __str__</span>(</span>self</span>):</span></span>
        return</span> self</span>.</span>uploaded_on</span>.</span>date</span>()</span></span></code></pre>
Run the migration</p>
python</span> manage.py makemigrations</span></span>
python</span> manage.py migrate</span></span></code></pre>🔗</a>Serializer</h2>
Creating a ModelSerializer for our Model UploadedFile</code>. ModelSerializer creates serializer fields based on Model passed and comes with buil-in save</code> method that saves passed data into database.</p>
In your api/serializers.py</code></p>
from</span> rest_framework</span> import</span> serializers</span></span>
from</span> .</span>models</span> import</span> UploadedFile</span></span>
</span>
class</span> FileUploadSerializer</span>(</span>serializers</span>.</span>ModelSerializer</span>):</span></span>
    class</span> Meta</span>:</span></span>
        model</span> =</span> UploadedFile</span></span>
        fields</span> =</span> (</span>'file'</span>,</span> 'uploaded_on'</span>,</span>)</span></span></code></pre>🔗</a>View</h2>
Now we create the view class FileUploadAPIView</code> (sub-class of APIView) to save the uploaded file. Few things to note:</p>


Generally the JSONParser</code> is set as the default parser in DRF so that we can send the request in JSON format. However it is not advised/convenient to upload file in JSON.</p>
</li>

form-data</code> is the best way to upload the file, to enable this we need to pass MultiPartParser</code> and FormParser</code> as our parser_classes</code> in our view.</p>
</li>

Only MultiPartParser</code> can be passed to enable file uploading but as per DRF API Guide on MultiPartParser</a>, we should use both parser classes to fully support HTML form data.</p>
</li>
</ul>
In api/views.py</code></p>
from</span> rest_framework</span> import</span> status</span></span>
from</span> rest_framework</span>.</span>parsers</span> import</span> FormParser</span>,</span> MultiPartParser</span></span>
from</span> rest_framework</span>.</span>response</span> import</span> Response</span></span>
from</span> rest_framework</span>.</span>views</span> import</span> APIView</span></span>
</span>
from</span> .</span>serializers</span> import</span> FileUploadSerializer</span></span>
</span>
class</span> FileUploadAPIView</span>(</span>APIView</span>):</span></span>
    parser_classes</span> =</span> (MultiPartParser</span>,</span> FormParser)</span></span>
    serializer_class</span> =</span> FileUploadSerializer</span></span>
</span>
    def</span> post</span>(</span>self</span>,</span> request</span>,</span> *</span>args</span>,</span> **</span>kwargs</span>):</span></span>
        serializer</span> =</span> self</span>.</span>serializer_class</span>(data</span>=</span>request</span>.</span>data)</span></span>
        if</span> serializer</span>.</span>is_valid</span>()</span>:</span></span>
            # you can access the file like this from serializer</span></span>
            # uploaded_file = serializer.validated_data["file"]</span></span>
            serializer</span>.</span>save</span>()</span></span>
            return</span> Response</span>(</span></span>
                serializer</span>.</span>data</span>,</span></span>
                status</span>=</span>status</span>.</span>HTTP_201_CREATED</span></span>
            )</span></span>
</span>
        return</span> Response</span>(</span></span>
            serializer</span>.</span>errors</span>,</span></span>
            status</span>=</span>status</span>.</span>HTTP_400_BAD_REQUEST</span></span>
        )</span></span></code></pre>🔗</a>Urls</h2>
Let’s connect our view in urls to be able to receive requests on it.</p>
In api/urls.py</code></p>
from</span> django</span>.</span>urls</span> import</span> path</span></span>
from</span> .</span>views</span> import</span> FileUploadAPIView</span>,</span> ImageUploadAPIView</span></span>
</span>
app_name</span> =</span> 'api'</span></span>
</span>
urlpatterns</span> =</span> [</span></span>
    path</span>(</span>'upload-file/'</span>,</span> FileUploadAPIView</span>.</span>as_view</span>()</span>,</span> name</span>=</span>'upload-file'</span>)</span>,</span></span>
]</span></span></code></pre>🔗</a>Testing API Endpoint</h2>
Start the server</p>
python</span> manage.py runserver</span></span></code></pre>🔗</a>Postman</h3>

  
      File upload using postman
    </figcaption></figure>


Method: POST (as per our view)</p>
</li>

URL: http://localhost:8000/api/upload-file/</p>
</li>

Body Type: form-data</p>
</li>

Values to be passed</p>

file as KEY (change the type to File in dropdown), Browse and select a file for VALUE</li>
</ul>
</li>
</ul>
🔗</a>cURL</h3>
curl</span> --location --request</span> POST 'http://localhost:8000/api/upload-file/'</span> --form</span> 'file=@"/path/to/yourfile.pdf"'</span></span></code></pre>🔗</a>NodeJS - Axios</h3>
var</span> axios</span> =</span> require</span>(</span>'axios'</span>)</span>;</span></span>
var</span> FormData</span> =</span> require</span>(</span>'form-data'</span>)</span>;</span></span>
var</span> fs</span> =</span> require</span>(</span>'fs'</span>)</span>;</span></span>
var</span> data</span> = new</span> FormData</span>()</span>;</span></span>
data</span>.</span>append</span>(</span>'file'</span>,</span> fs</span>.</span>createReadStream</span>(</span>'/path/to/yourfile.pdf'</span>))</span>;</span></span>
</span>
var</span> config</span> =</span> {</span></span>
  method</span>:</span> 'post'</span>,</span></span>
  url</span>:</span> 'http://localhost:8000/api/upload-file/'</span>,</span></span>
  headers</span>:</span> {</span></span>
    ...</span>data</span>.</span>getHeaders</span>()</span>,</span></span>
  }</span>,</span></span>
  data</span>:</span> data</span>,</span></span>
}</span>;</span></span>
</span>
axios</span>(config)</span></span>
  .</span>then</span>(</span>function</span> (</span>response</span>) {</span></span>
    console</span>.</span>log</span>(JSON</span>.</span>stringify</span>(response</span>.</span>data))</span>;</span></span>
  })</span></span>
  .</span>catch</span>(</span>function</span> (</span>error</span>) {</span></span>
    console</span>.</span>log</span>(error)</span>;</span></span>
  })</span>;</span></span></code></pre>
Checkout this guide to learn how to store static and media files in AWS S3</a></p>


Backup and Restore PostgreSQL database dump in Ubuntu
2023-06-20T00:00:00+00:00
Learn how to backup and restore PostgreSQL</a> database dump in Ubuntu (Linux) and Mac.</p>
🔗</a>Taking backup/dump of the database:</h2>
pg_dump</span> -U</span> db_user database_name</span> ></span> database_dump_name.sql</span></span></code></pre>🔗</a>Restoring backup dump</h2>
🔗</a>Creating new user</h3>
CREATE</span> USER your_user_name WITH PASSWORD 'your_user_password'</span>;</span></span></code></pre>🔗</a>Creating new database</h3>
It is better to switch to new user and create the new database, this way you won’t have to give permission explicitly.</p>
CREATE</span> DATABASE your_db_name</span>;</span></span></code></pre>
But for some reason you didn’t/couldn’t create the database with new user then go to the last section which has all the commands to give user full access to the database.</p>
🔗</a>Restore using psql</h3>
psql</span> -U</span> your_user_name</span> -d</span> your_db_name</span> <</span> dump_name.sql</span></span></code></pre>🔗</a>Give Full Access to Database</h2>
If you haven’t created database with your new user then you’ll have to give it full permission so that it can perform the operations.</p>
GRANT</span> ALL PRIVILEGES ON DATABASE your_db_name TO your_user_name</span>;</span></span>
</span>
# change owner</span></span>
ALTER</span> DATABASE your_db_name OWNER TO your_user_name</span>;</span></span>
</span>
# allow user to connect</span></span>
GRANT</span> CONNECT ON DATABASE your_db_name TO your_user_name</span>;</span></span>
</span>
# grant public schema access</span></span>
GRANT</span> USAGE ON SCHEMA public TO your_user_name</span>;</span></span>
</span>
# grant access to all tables and sequences in schma public</span></span>
GRANT</span> ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO your_user_name</span>;</span></span>
GRANT</span> ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO your_user_name</span>;</span></span></code></pre>

Reference: https://www.postgresql.org/docs/current/backup.html</a></p>
</blockquote>

Vivek Shukla - Tutorial

Automate Postgres Database Backup and Upload to S3

🔗</a>What we will cover</h2> Write db backup automation script for docker and non-docker based system</p> </li> Upload backup file to AWS S3</p> </li>

How To Take Postgres Database Backup From Docker Container

How To Store and Serve Private Media Files in Django

How To Store Django Static and Media Files in AWS S3

🔗</a>S3 Bucket</h2> 🔗</a>Create a bucket</h3> Let’s create a S3 Bucket where our files will be stored. Few things to keep in mind:</p> ACL must be enabled, so that we can control object permission from our Django</p> </li>

🔗</a>Create a bucket</h3> Let’s create a S3 Bucket where our files will be stored. Few things to keep in mind:</p> ACL must be enabled, so that we can control object permission from our Django</p> </li>

🔗</a>Create IAM User</h3> Go to IAM page in your AWS Console.</p> </li> Go to Users</strong> then Add users</strong>.</p> </li> Fill out the name that you want to give this user.</p> </li>

🔗</a>Get the access key</h3> To use the IAM user in our django project we will need to generate access key.</p> Open user page, go to Security credentials</strong> tab</p> </li> Scroll down to Access keys</strong> section and click on Create access key</strong> button</p> </li>

Django Rest Framework File Upload API

🔗</a>Setup</h2> Django: 4.2</p> </li> Django Rest Framework: 3.14.0</p> </li> Project Name: fileupload</code></p> </li>

🔗</a>Postman</h3> File upload using postman </figcaption></figure> Method: POST (as per our view)</p> </li> URL: http://localhost:8000/api/upload-file/</p> </li> Body Type: form-data</p> </li>

Backup and Restore PostgreSQL database dump in Ubuntu