Vivek Shukla - PostgreSQL

PostgreSQL: Make Create/Drop Operations Infallible

2026-04-25T00:00:00+00:00

While creating a database table, column or index, if it already exist, postgres will throw the error and abort the whole operation. Using IF EXISTS</code> and IF NOT EXISTS</code> is a way to make them infallible.</p>

🔗</a>Creating and Dropping Entire Tables</h3>
Safe Create:</strong></p>
CREATE TABLE</span> IF</span> NOT EXISTS</span> users (</span></span>
    id UUID </span>PRIMARY KEY</span>,</span></span>
    email </span>VARCHAR</span>(</span>255</span>)</span> NOT NULL</span></span>
);</span></span></code></pre>
Safe Drop:</strong></p>
DROP TABLE IF EXISTS</span> users;</span></span></code></pre>🔗</a>Adding and Removing Columns</h3>
Safe Add Column:</strong></p>
ALTER TABLE</span> users </span></span>
ADD</span> COLUMN </span>IF NOT EXISTS</span> phone_number </span>VARCHAR</span>(</span>20</span>);</span></span></code></pre>
Safe Drop Column:</strong></p>
ALTER TABLE</span> users </span></span>
DROP</span> COLUMN </span>IF EXISTS</span> phone_number;</span></span></code></pre>🔗</a>Dropping Constraints (Easy)</h3>
Safe Drop Constraint:</strong></p>
ALTER TABLE</span> users </span></span>
DROP CONSTRAINT IF EXISTS</span> unique_email_constraint;</span></span></code></pre>🔗</a>Adding Constraints (The Tricky One)</h3>
To make adding a constraint truly infallible, we have to use a tiny bit of PostgreSQL logic called a DO</code> blocIt checks the database records first before trying to add the rule.</p>
Safe Add Constraint (using a DO block):</strong></p>
DO $$ </span></span>
BEGIN</span></span>
    -- Check if the constraint name already exists</span></span>
    IF NOT EXISTS</span> (</span></span>
        SELECT</span> 1</span> </span></span>
        FROM</span> pg_constraint </span></span>
        WHERE</span> conname </span>=</span> 'unique_email_constraint'</span></span>
    ) </span>THEN</span></span>
        -- If it does not exist, add it</span></span>
        ALTER TABLE</span> users </span></span>
        ADD CONSTRAINT</span> unique_email_constraint </span>UNIQUE</span> (email);</span></span>
    END IF</span>;</span></span>
END</span> $$;</span></span></code></pre>🔗</a>Custom Types and Enums (Bonus)</h3>
Safe Drop Type:</strong></p>
DROP TYPE IF EXISTS</span> job_status_enum;</span></span></code></pre>
Safe Create Type:</strong> (Postgres does not have CREATE TYPE IF NOT EXISTS</code>, so we use another DO</code> block)</em></p>
DO $$ </span></span>
BEGIN</span></span>
    IF NOT EXISTS</span> (</span>SELECT</span> 1</span> FROM</span> pg_type </span>WHERE</span> typname </span>=</span> 'job_status_enum'</span>) </span>THEN</span></span>
        CREATE TYPE</span> job_status_enum</span> AS</span> ENUM (</span>'submitted'</span>, </span>'pending'</span>, </span>'completed'</span>);</span></span>
    END IF</span>;</span></span>
END</span> $$;</span></span></code></pre>



How I use Postgres DB as Key Value Store in Rust with SQLx
2025-09-02T00:00:00+00:00
For my smaller application (or personal) or when I’m starting out a new app, I would like to keep the dependencies of my program to minimum. Since I’m going to need a DB anyway, so instead of using Redis and adding one more component to my tech stack, I would like to use Postgres</a> as my key value store.</p>
Prematurely thinking about scaling your system when you don’t even know if it would be successful enough to actually need all those bells and whistle is sure way to extend the deadline and procrastinate.</p>
🔗</a>Overview</h2>
We are going to create a trait</code> called PgStore</code>, which will implement all the functions to set, get, pop and delete the value.</p>
Then you can implement this trait for your own types. Your types must derive serde’s Serialize</code> and Deserialize</code> for this to work.</p>
🔗</a>Dependencies</h2>
You need the following dependencies in your Cargo.toml</code></p>
chrono</span> =</span> { version</span> =</span> "0.4"</span>,</span> features</span> =</span> [</span>"serde"</span>] }</span></span>
serde</span> =</span> { version</span> =</span> "1.0"</span>,</span> features</span> =</span> [</span>"derive"</span>] }</span></span>
serde_json</span> =</span> { version</span> =</span> "1.0"</span> }</span></span>
sqlx</span> =</span> { version</span> =</span> "0.8"</span>,</span> features</span> =</span> [</span></span>
    "runtime-tokio"</span>,</span></span>
    "tls-rustls-ring-native-roots"</span>,</span></span>
    "migrate"</span>,</span></span>
    "postgres"</span>,</span></span>
    "macros"</span>,</span></span>
    "chrono"</span>,</span></span>
] }</span></span></code></pre>🔗</a>Creating static PgPool connection</h2>
I don’t like to pass around DB connection across functions or threads or tasks. Instead I like to use tokio’s OnceCell</code> which is a thread safe Cell which can be written to only once, it’s thread safe which means we can use async code in it.</p>
use</span> sqlx</span>::</span>PgPool</span>;</span></span>
use</span> tokio</span>::</span>sync</span>::</span>OnceCell</span>;</span></span>
</span>
static</span> PG_POOL</span>:</span> OnceCell</span><</span>PgPool</span>></span> =</span> OnceCell</span>::</span>const_new</span>();</span></span>
</span>
pub async fn</span> pg_pool</span>()</span> -> &</span>'</span>static PgPool</span> {</span></span>
    PG_POOL</span></span>
        .</span>get_or_init</span>(</span>async</span> ||</span> {</span></span>
            PgPool</span>::</span>connect</span>(</span>std</span>::</span>env</span>(</span>"DATABASE_URL"</span>)</span>.</span>expect</span>(</span>"DATABASE_URL is not set"</span>))</span></span>
                .</span>await</span></span>
                .</span>expect</span>(</span>"❌ PgPool connection failed"</span>)</span></span>
        })</span></span>
        .</span>await</span></span>
}</span></span></code></pre>
Now we can just use pg_pool().await</code> to fetch the db pool connection wherever we need.</p>
🔗</a>DB Migration</h2>
We are creating a new table kv_store</code> in our Postgres DB. If you notice we are using UNLOGGED</code> keyword to create the table, this tells postgres to not write WAL (write-ahead log) for kv_store</code> table, which improves the performance at the expanse of data durability.</p>
CREATE</span> UNLOGGED </span>TABLE</span> "kv_store"</span> (</span></span>
    "key"</span> VARCHAR</span>(</span>2048</span>)</span> PRIMARY KEY</span>,</span></span>
    "value"</span> TEXT NOT NULL</span>,</span></span>
    "expires"</span> TIMESTAMPTZ NOT NULL</span></span>
);</span></span></code></pre>🔗</a>PgStore Trait</h2>
We are creating PgStore</code> trait which will have methods to store, retrieve and delete the value in the kv_store</code> table.</p>
use</span> chrono</span>::</span>{</span>DateTime</span>,</span> Utc</span>};</span></span>
use</span> serde</span>::</span>Serialize</span>;</span></span>
use</span> serde</span>::</span>de</span>::</span>DeserializeOwned</span>;</span></span>
use</span> std</span>::</span>time</span>::</span>Duration</span>;</span></span>
</span>
#[allow(async_fn_in_trait)]</span></span>
pub trait</span> PgStore</span>:</span> Serialize</span> +</span> DeserializeOwned</span> {</span></span>
    const</span> EXPIRE_IN</span>:</span> usize</span>;</span> // seconds</span></span>
</span>
    fn</span> key_format</span>(key</span>:</span> String</span>)</span> -></span> String</span>;</span></span>
</span>
    fn</span> get_expire_time</span>()</span> -></span> DateTime</span><</span>Utc</span>> {</span></span>
        Utc</span>::</span>now</span>()</span> +</span> Duration</span>::</span>from_secs</span>(</span>Self</span>::</span>EXPIRE_IN</span> as</span> u64</span>)</span></span>
    }</span></span>
</span>
    async fn</span> set_ex</span>(</span>&</span>self</span>, key</span>:</span> String</span>)</span> -></span> Result</span><(),</span> String</span>> {</span></span>
        let</span> value</span> =</span></span>
            serde_json</span>::</span>to_string</span>(</span>&</span>self</span>)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to serialize value."</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        sqlx</span>::</span>query!</span>(</span></span>
            r#"INSERT INTO kv_store ("key", "value", expires) VALUES ($1, $2, $3)</span></span>
            ON CONFLICT ("key") DO UPDATE SET "value" = $2, expires = $3"#</span>,</span></span>
            Self</span>::</span>key_format</span>(key),</span></span>
            value,</span></span>
            Self</span>::</span>get_expire_time</span>(),</span></span>
        )</span></span>
        .</span>execute</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to set value in kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        Ok</span>(())</span></span>
    }</span></span>
</span>
    // it will just fetch the value if it exists</span></span>
    async fn</span> get</span>(key</span>:</span> String</span>)</span> -></span> Result</span><</span>Self</span>,</span> String</span>> {</span></span>
        let</span> record</span> =</span> sqlx</span>::</span>query!</span>(</span></span>
            r#"SELECT "value" FROM kv_store WHERE "key" = $1"#</span>,</span></span>
            Self</span>::</span>key_format</span>(key)</span></span>
        )</span></span>
        .</span>fetch_one</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to get value from kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        serde_json</span>::</span>from_str</span>(</span>&</span>record</span>.</span>value)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to deserialize value"</span>.</span>to_string</span>())</span></span>
    }</span></span>
</span>
    // this will fetch the value while extending the expire time</span></span>
    async fn</span> get_ex</span>(key</span>:</span> String</span>)</span> -></span> Result</span><</span>Self</span>,</span> String</span>> {</span></span>
        let</span> record</span> =</span> sqlx</span>::</span>query!</span>(</span></span>
            r#"UPDATE kv_store SET expires=$2 WHERE "key"=$1 RETURNING "value""#</span>,</span></span>
            Self</span>::</span>key_format</span>(key),</span></span>
            Self</span>::</span>get_expire_time</span>(),</span></span>
        )</span></span>
        .</span>fetch_one</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to get value from kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        serde_json</span>::</span>from_str</span>(</span>&</span>record</span>.</span>value)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to deserialize value"</span>.</span>to_string</span>())</span></span>
    }</span></span>
</span>
    // it will fetch the value while deleting the key</span></span>
    async fn</span> get_del</span>(key</span>:</span> String</span>)</span> -></span> Result</span><</span>Self</span>,</span> String</span>> {</span></span>
        let</span> record</span> =</span> sqlx</span>::</span>query!</span>(</span></span>
            r#"DELETE FROM kv_store WHERE key=$1 RETURNING "value""#</span>,</span></span>
            Self</span>::</span>key_format</span>(key)</span></span>
        )</span></span>
        .</span>fetch_one</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to get value from kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        serde_json</span>::</span>from_str</span>(</span>&</span>record</span>.</span>value)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to deserialize value"</span>.</span>to_string</span>())</span></span>
    }</span></span>
</span>
    async fn</span> del</span>(key</span>:</span> String</span>)</span> -></span> Result</span><(),</span> String</span>> {</span></span>
        sqlx</span>::</span>query!</span>(</span></span>
            r#"DELETE FROM kv_store WHERE key=$1"#</span>,</span></span>
            Self</span>::</span>key_format</span>(key)</span></span>
        )</span></span>
        .</span>execute</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to delete value"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        Ok</span>(())</span></span>
    }</span></span>
}</span></span></code></pre>
By defining our trait like PgStore: Serialize + DeserializeOwned</code>, we are telling that whichever type is going to implement PgStore</code> trait it must also have Serialize</code> and DeserializeOwned</code> implemented.</p>
Following are the brief explaination of PgStore</code>’s attributes:</p>
EXPIRE_IN:</strong> This will be used to set the to be expired in seconds. It must be implemented by the Type you are using it for.</p>
key_format:</strong> This function needs to be implemented. It will be used as a way to generate key.</p>
set_ex:</strong> This method will store the type against the passed key in DB, overwriting value if it already exists.</p>
get:</strong> It will fetch the value for a given key, error will be returned if it does not exist.</p>
get_ex:</strong> Fetches the value while resetting the expire time.</p>
get_del:</strong> Fetch the value while deleting the key. It’s more like pop.</p>
del:</strong> Just deletes the value. Doesn’t return error even if key doesn’t exist.</p>
🔗</a>Cleaning up expired keys</h2>
Then we have a helper function pgstore_cleanup</code> which will cleanup expired keys every 60 seconds. You must spawn to run this function in the background for all the time your program is running.</p>
pub async fn</span> pgstore_cleanup</span>() {</span></span>
    loop</span> {</span></span>
        // tracing::info!("🟡 PgStore cleanup is running...");</span></span>
        sqlx</span>::</span>query!</span>(</span>r#"DELETE FROM kv_store WHERE expires < NOW()"#</span>)</span></span>
            .</span>execute</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
            .</span>await</span></span>
            .</span>ok</span>();</span></span>
        tokio</span>::</span>time</span>::</span>sleep</span>(</span>Duration</span>::</span>from_secs</span>(</span>60</span>))</span>.</span>await</span>;</span></span>
    }</span></span>
}</span></span></code></pre>🔗</a>Usage</h2>
#[derive(serde</span>::</span>Serialize</span>, serde</span>::</span>Deserialize</span>)]</span></span>
pub struct</span> Game</span> {</span></span>
    pub</span> id</span>:</span> usize</span>,</span></span>
    pub</span> name</span>:</span> String</span>,</span></span>
    pub</span> genre</span>:</span> String</span>,</span></span>
}</span></span>
</span>
impl</span> PgStore</span> for</span> Game</span> {</span></span>
    const</span> EXPIRE_IN</span>:</span> usize</span> =</span> 3600</span>;</span></span>
</span>
    fn</span> key_format</span>(key</span>:</span> String</span>)</span> -></span> String</span> {</span></span>
        format!</span>(</span>"game-{}"</span>, key)</span></span>
    }</span></span>
}</span></span>
</span>
#[tokio</span>::</span>main]</span></span>
async fn</span> main</span>() {</span></span>
    // make sure to run this to delete expired keys regularly</span></span>
    tokio</span>::</span>spawn</span>(</span>pgstore_cleanup</span>());</span></span>
</span>
    let</span> game</span> =</span> Game</span> {</span></span>
        id</span>:</span> 1</span>,</span></span>
        name</span>:</span> "Skyrim"</span>.</span>to_string</span>(),</span></span>
        genre</span>:</span> "rpg"</span>.</span>to_string</span>(),</span></span>
    };</span></span>
</span>
    // storing value in kv_store</span></span>
    // it will be store with key: "game-1"</span></span>
    let</span> _</span> =</span> game</span>.</span>set_ex</span>(game</span>.</span>id</span>.</span>to_string</span>())</span>.</span>await</span>;</span></span>
</span>
    // fetching value</span></span>
    if let</span> Ok</span>(v)</span> =</span> Game</span>::</span>get</span>(</span>1</span>.</span>to_string</span>())</span>.</span>await</span> {</span></span>
        println!</span>(</span>"fetched game with name: {}"</span>, v</span>.</span>name);</span></span>
    }</span></span>
</span>
    // deleting key</span></span>
    let</span> _</span> =</span> Game</span>::</span>del</span>(</span>1</span>.</span>to_string</span>())</span>.</span>await</span>;</span></span>
}</span></span></code></pre>
Similarly you can do this with your own types, as long as they derive serde’s Serialize and Deserialize.</p>


Self Host PostgreSQL With TLS
2025-02-03T00:00:00+00:00
In this tutorial we will go through how to self-host PostgreSQL Database and setup TLS to securely connect.</p>
🔗</a>Installing PostgreSQL</h2>
Postgres APT repository configuration:</p>
sudo</span> apt install</span> -y</span> postgresql-common</span></span>
sudo</span> /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh</span></span></code></pre>
Install PostgreSQL 16</p>
sudo</span> apt install postgresql-16</span></span></code></pre>
After installing PostgreSQL, it creates a new user postgres</code>, which is the superuser of the database. We can switch to postgres</code> user in ubuntu to access postgresql command prompt:</p>
sudo</span> -i -u</span> postgres</span></span></code></pre>
Now to create new user with passowrd and with privilege to create new db. It will ask you to enter your password, which will not be logged into history hence this is the recommended way to set password.</p>
createuser</span> your_db_user</span> --pwprompt --createdb</span></span></code></pre>
Now we can exit from postgres</code> account</p>
exit</span></span></code></pre>
We will add new user in our Ubuntu with the same name as the new user we created in postgresql. (Optional) Add -m</code> to create home directory for the user.</p>
sudo</span> useradd</span> -s</span> /bin/bash your_db_user</span></span></code></pre>
Accessing psql as the new user we just created. Since we do not have any database associated with our user therefore we are using ‘-d postgres’ to use postgres db for now.</p>
sudo</span> -u</span> your_db_user psql</span> -U</span> your_db_user</span> -d</span> postgres</span></span></code></pre>
Create your database.</p>
CREATE</span> DATABASE your_db_name</span>;</span></span></code></pre>
This is how you can connect with your db if not already connected.</p>
\c</span> your_db_name</span>;</span></span></code></pre>🔗</a>Setting Up SSL</h2>
It’s important that we secure our communication with the database while accessing it remotely. We are going to setup SSL certificate using certbot</code> from LetsEncrypt.</p>
Install certbot</p>
sudo</span> snap install core</span>;</span> sudo</span> snap refresh core</span></span>
sudo</span> apt-get remove certbot</span></span>
sudo</span> snap install</span> --classic</span> certbot</span></span>
sudo</span> ln</span> -s</span> /snap/bin/certbot /usr/bin/certbot</span></span></code></pre>
Obtain certificate</p>
sudo</span> certbot certonly</span> --standalone -d</span> psql.example.com</span></span></code></pre>
Find PostgreSQL data directory</p>
sudo</span> -u</span> postgres psql</span> -U</span> postgres</span> -c</span> 'SHOW data_directory'</span></span></code></pre>
Create a renewal hook file</p>
sudo</span> nano /etc/letsencrypt/renewal-hooks/deploy/postgresql.deploy</span></span></code></pre>
Paste the following</p>
umask 0177</span></span>
DOMAIN=psql.example.com</span></span>
DATA_DIR=/var/lib/postgresql/16/main</span></span>
cp /etc/letsencrypt/live/$DOMAIN/fullchain.pem $DATA_DIR/server.crt</span></span>
cp /etc/letsencrypt/live/$DOMAIN/privkey.pem $DATA_DIR/server.key</span></span>
chown postgres:postgres $DATA_DIR/server.crt $DATA_DIR/server.key</span></span></code></pre>
Save and exit the file.</p>
Give file executable permission</p>
sudo</span> chmod +x /etc/letsencrypt/renewal-hooks/deploy/postgresql.deploy</span></span></code></pre>
Get the path of PostgreSQL configuration file</p>
sudo</span> -u</span> postgres psql</span> -U</span> postgres</span> -c</span> 'SHOW config_file'</span></span></code></pre>
Open the file in edit mode and update the SSL section of the file as below:</p>
sudo</span> nano /etc/postgresql/16/main/postgresql.conf</span></span></code></pre>ssl = on</span></span>
ssl_cert_file = 'server.crt'</span></span>
ssl_key_file = 'server.key'</span></span>
ssl_prefer_server_ciphers = on</span></span></code></pre>
Locate the Connection Settings section and verify the listen_address is to * for all addresses. Make sure the line is not commented out.</p>
listen_addresses = '*'</span></span></code></pre>
Edit pg_hba.conf</code> file which is in the same dir as config file:</p>
sudo</span> nano /etc/postgresql/16/main/pg_hba.conf</span></span></code></pre>
Add the following line to enable connection over ssl for any user and database:</p>
# TYPE DATABASE USER ADDRESS METHOD</span></span>
</span>
hostssl all all 0.0.0.0/0 scram-sha-256</span></span>
hostssl all all ::/0 scram-sha-256</span></span></code></pre>

Although in above config it’s shown to allow connection from all IPv4 and IPv6 addresses, however from the security point of view only allow the ip address of your connecting machine (client).</p>
</blockquote>
Renew the SSL certificate</p>
sudo</span> certbot renew</span> --force-renewal</span></span></code></pre>
Restart PostgreSQL</p>
sudo</span> service postgresql restart</span></span></code></pre>
Connect to your db remotely</p>
psql</span> -d</span> "dbname=your_db_name sslmode=require"</span> -h</span> psql.example.com</span> -U</span> your_db_user</span></span></code></pre>


Automate Postgres Database Backup and Upload to S3
2023-10-20T00:00:00+00:00
Database is a very critical system for an application. To prepare for any disaster we all must take regular backups of our database.</p>
In this tutorial we will create a bash script to automate postgres database backup from a docker container</a> and native postgres installation then upload to S3 based storage system.</p>
🔗</a>What we will cover</h2>


Write db backup automation script for docker and non-docker based system</p>
</li>

Upload backup file to AWS S3</p>
</li>

Setup the cron job for daily backup</p>
</li>
</ol>
🔗</a>Steps</h2>


Move to your project folder, where docker-compose.yml is.</p>
</li>

Take the database backup in container</p>
</li>

Copy the backup file from container filesystem to our desired location</p>
</li>

Delete the backup file inside the container</p>
</li>

Upload backup file to S3</p>
</li>

Delete the local backup file</p>
</li>
</ol>
Code snippet below covers step 1 to 4. But it is not optimal, later we will make modification so that we can easily automate it and make it re-usable.</p>
cd</span> project_dir</span></span>
</span>
docker</span> compose exec db_service_name sh</span> -c</span> "pg_dump -U db_user db_name > filename"</span></span>
</span>
docker</span> compose cp db_service_name:filename destination/filename</span></span>
</span>
docker</span> compose exec db_service_name rm filename</span></span></code></pre>🔗</a>Upload to S3</h2>
To upload file to S3, we need to have aws cli installed, you can follow this guide from AWS docs to install awscli</a>.</p>
After installing awscli, you need to run aws configure</code> and provide your key id and key secret. You can use any S3 based storage system here.</p>
aws s3 cp filename s3</span>:</span>//</span>your</span>-</span>bucket</span></span></code></pre>
If you are using some other S3 based storage system then you’ll have to provide custom --endpoint-url</code> argument here, make sure you read the respective docs for this.</p>
🔗</a>Full Script</h2>
We have laid out all the steps and code to accomplish that, now let’s modify in such a way that we can easily automate it and make it reusable.</p>
The filename is created based on the current datetime of the system, so that we can avoid ambiguity and exactly know how old is the backup file.</p>
🔗</a>For docker compose</h3>
#!/bin/bash</span></span>
</span>
############## ✅ Update These Values ##################</span></span>
directory</span>=</span>"your-project-dir"</span></span>
home</span>=</span>"/home/<user>"</span></span>
</span>
db_service_name</span>=</span>"db"</span></span>
db_user</span>=</span>"db_user"</span></span>
db_name</span>=</span>"db_name"</span></span>
</span>
#####################################################</span></span>
############ 🔴 CHANGE WITH CAUTION 🔴 #############</span></span>
date</span>=</span>$(</span>date</span> +"%Y-%m-%d-%H:%M:%S"</span>)</span></span>
filename</span>=</span>$db_name</span>"_"</span>$date</span>".sql"</span></span>
</span>
cd</span> $home</span></span>
cd</span> $directory</span></span>
docker</span> compose exec</span> $db_service_name</span> sh</span> -c</span> "pg_dump -U </span>$db_user $db_name</span> > </span>$filename</span>"</span></span>
</span>
docker</span> compose cp</span> $db_service_name</span>:</span>$filename $home</span>"/"</span>$filename</span></span>
docker</span> compose exec</span> $db_service_name</span> rm</span> $filename</span></span>
</span>
cd</span> $home</span></span>
#####################################################</span></span>
#####################################################</span></span>
</span>
############## ✅ CAN CHANGE BELOW ##################</span></span>
aws</span> s3 cp</span> $filename</span> s3://your-bucket</span></span>
</span>
rm</span> $filename</span></span></code></pre>
At the top, you’ve to update variable values based on your project and directory structure, and the main code will use that to backup the database.</p>
At the end of script, we have aws command to copy the backup file to S3, here you need to update bucket name and if you are using other S3 storage provider (other than AWS), then --endpoint-url</code> argument with appropriate value.</p>
🔗</a>For Native Postgres Installation</h3>
#!/bin/bash</span></span>
</span>
############## ✅ Update These Values ##################</span></span>
home</span>=</span>"/home/<user>"</span></span>
</span>
db_user</span>=</span>"db_user"</span></span>
db_name</span>=</span>"db_name"</span></span>
</span>
#####################################################</span></span>
############ 🔴 CHANGE WITH CAUTION 🔴 ##############</span></span>
date</span>=</span>$(</span>date</span> +"%Y-%m-%d-%H:%M:%S"</span>)</span></span>
filename</span>=</span>$db_name</span>"_"</span>$date</span>".sql"</span></span>
</span>
ch</span> $home</span></span>
pg_dump</span> -U</span> $db_user $db_name</span> ></span> $filename</span></span>
</span>
#####################################################</span></span>
</span>
############## ✅ CAN CHANGE BELOW ##################</span></span>
aws</span> s3 cp</span> $filename</span> s3://urlify-db-backup</span></span>
</span>
rm</span> $filename</span></span></code></pre>
When running pg_dump in the native installation, there are two ways postgres will try to authenticate the user.</p>


Peer based authentication</a>: It will check if the current user (system logged in user) is same as user passed to pg_dump.</p>
</li>

If peer auth fails then postgres looks for .pgpass</code> file in home directory for the password.</p>
</li>

If both fails then it might ask for password.</p>
</li>
</ol>
So, make sure you use 1st method and run the script on the same user as the one passed to pg_dump.</p>
🔗</a>Setup Cron Job</h2>
To automate backup and upload of our database, we will use crontab. It comes built-in with almost all the Linux distros and are very easy to setup.</p>
Give your script executable permission:</p>
chmod</span> +x db_backup.sh</span></span></code></pre>
Open crontab:</p>
crontab</span> -e</span></span></code></pre>
This will open up crontab in your prefered choice of editor. Use the below statement to run the backup script everyday at 12 PM.</p>
0 12 * * * /home/<user>/db_backup.sh >> cron.log 2>&1</span></span></code></pre>
Create cron.log file</p>
touch</span> cron.log</span></span></code></pre>
Use full path if cron.log is not in current user’s home folder.</p>


How To Take Postgres Database Backup From Docker Container
2023-10-18T00:00:00+00:00
To take database backup in Postgres we use pg_dump. pg_dump</code> is a utility provided by PostgreSQL to create the database backup</a>. We are gonna use docker exec</code> and cp</code> command along with pg_dump to take Postgres database backup from docker container.</p>
🔗</a>pg_dump</h2>
Let’s take a look at how to take backup with pg_dump</code> command:</p>
pg_dump</span> -U</span> your_db_user</span> -d</span> your_db_name</span> ></span> db_backup.sql</span></span></code></pre>
This is straight forward, pass your db username, database name that you want to take backup of and the backup filename which would be created.</p>
🔗</a>docker exec</h2>
Let’s do the same thing on Docker, for that we will use docker exec</a>:</p>
docker</span> exec</span> container_name_or_id sh</span> -</span>c</span> "pg_dump -U your_db_user -d your_db_name > db_backup.sql"</span></span></code></pre>
Make sure that the specified container is running, since docker exec</code> command executes on the running container.</p>
🔗</a>docker cp</h2>
The docker exec command created the database backup but it’s inside the container file system. So now we need to copy that file from container to our file system.</p>
For that purpose docker gives us docker cp</a>, which we can use to copy to/from the container’s filesystem.</p>
docker</span> cp container_name_or_id:db_backup.sql your_destination</span></span></code></pre>
Now, we have copied our database backup file to our filesystem, it’s time to delete the one from our container. To delete backup file from container do this:</p>
docker exec container_name_or_id rm db_backup.sql</p>
🔗</a>All Commands For docker</h2>
docker</span> exec container_name_or_id sh</span> -c</span> "pg_dump -U your_db_user -d your_db_name > db_backup.sql"</span></span>
</span>
docker</span> cp container_name_or_id:db_backup.sql your_destination</span></span>
</span>
docker</span> exec container_name_or_id rm db_backup.sql</span></span></code></pre>🔗</a>docker compose</h2>
You can run the same commands just use docker compose</code> instead of just docker</code>, and replace container name/id with service name.</p>
docker</span> compose exec service_name sh</span> -c</span> "pg_dump -U your_db_user -d your_db_name > db_backup.sql"</span></span>
</span>
docker</span> compose cp service_name:db_backup.sql your_destination</span></span>
</span>
docker</span> compose exec service_name rm db_backup.sql</span></span></code></pre>


Backup and Restore PostgreSQL database dump in Ubuntu
2023-06-20T00:00:00+00:00
Learn how to backup and restore PostgreSQL</a> database dump in Ubuntu (Linux) and Mac.</p>
🔗</a>Taking backup/dump of the database:</h2>
pg_dump</span> -U</span> db_user database_name</span> ></span> database_dump_name.sql</span></span></code></pre>🔗</a>Restoring backup dump</h2>
🔗</a>Creating new user</h3>
CREATE</span> USER your_user_name WITH PASSWORD 'your_user_password'</span>;</span></span></code></pre>🔗</a>Creating new database</h3>
It is better to switch to new user and create the new database, this way you won’t have to give permission explicitly.</p>
CREATE</span> DATABASE your_db_name</span>;</span></span></code></pre>
But for some reason you didn’t/couldn’t create the database with new user then go to the last section which has all the commands to give user full access to the database.</p>
🔗</a>Restore using psql</h3>
psql</span> -U</span> your_user_name</span> -d</span> your_db_name</span> <</span> dump_name.sql</span></span></code></pre>🔗</a>Give Full Access to Database</h2>
If you haven’t created database with your new user then you’ll have to give it full permission so that it can perform the operations.</p>
GRANT</span> ALL PRIVILEGES ON DATABASE your_db_name TO your_user_name</span>;</span></span>
</span>
# change owner</span></span>
ALTER</span> DATABASE your_db_name OWNER TO your_user_name</span>;</span></span>
</span>
# allow user to connect</span></span>
GRANT</span> CONNECT ON DATABASE your_db_name TO your_user_name</span>;</span></span>
</span>
# grant public schema access</span></span>
GRANT</span> USAGE ON SCHEMA public TO your_user_name</span>;</span></span>
</span>
# grant access to all tables and sequences in schma public</span></span>
GRANT</span> ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO your_user_name</span>;</span></span>
GRANT</span> ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO your_user_name</span>;</span></span></code></pre>

Reference: https://www.postgresql.org/docs/current/backup.html</a></p>
</blockquote>

Vivek Shukla - PostgreSQL

PostgreSQL: Make Create/Drop Operations Infallible

How I use Postgres DB as Key Value Store in Rust with SQLx

Self Host PostgreSQL With TLS

Automate Postgres Database Backup and Upload to S3

🔗</a>What we will cover</h2> Write db backup automation script for docker and non-docker based system</p> </li> Upload backup file to AWS S3</p> </li>

How To Take Postgres Database Backup From Docker Container

Backup and Restore PostgreSQL database dump in Ubuntu