Vivek Shukla - Programming

Setup Redis TLS With Self-signed Certificate

2025-09-08T00:00:00+00:00

This method is only recommended when connecting to redis server via local network.</p> </blockquote>

Generate Cert and Key file for self signed TLS, then move both these files to redis directory and set the proper permissions. Delete the Key file and copy the cert file to the client machine which will connect to the redis-server.</p>

cd</span> ~</span></span>
</span>
openssl</span> req</span> -x509 -nodes -days</span> 3650</span> -newkey</span> rsa:2048</span> \</span></span>
  -keyout</span> redis_tls.key</span> -out</span> redis_tls.crt</span> \</span></span>
  -subj</span> "/CN=YourOrg"</span></span>
</span>
sudo</span> cp redis_tls.key redis_tls.crt /etc/redis/</span> &&</span> rm</span> redis_tls.key</span></span>
sudo</span> chmod</span> 600</span> /etc/redis/redis_tls.key</span></span>
sudo</span> chmod</span> 644</span> /etc/redis/redis_tls.crt</span></span>
sudo</span> chown redis:redis /etc/redis/redis_tls.key</span></span>
sudo</span> chown redis:redis /etc/redis/redis_tls.crt</span></span></code></pre>🔗</a>Update Redis Configuration File</h2>
This command will append the value to redis.conf</code> (redis configuration file). Make sure to update the password.</p>
It will disable the non-tls support and enable the tls connection at port 6379</code> with previously generated key and cert files.</p>
sudo</span> tee</span> -a</span> /etc/redis/redis.conf</span> <<</span> EOF</span></span>
bind * -::*</span></span>
port 0</span></span>
tls-port 6379</span></span>
tls-cert-file /etc/redis/redis_tls.crt</span></span>
tls-key-file /etc/redis/redis_tls.key</span></span>
tls-auth-clients no</span></span>
tls-prefer-server-ciphers yes</span></span>
requirepass YOUR_PASSWORD_HERE</span></span>
EOF</span></span></code></pre>
Restart and enable redis</p>
sudo</span> systemctl restart redis-server</span></span>
sudo</span> systemctl enable redis-server</span></span></code></pre>
Check status with</p>
sudo</span> systemctl status redis-server</span></span></code></pre>
If any error then check journal (log)</p>
sudo</span> journalctl</span> -u</span> redis-server</span> -f</span></span></code></pre>🔗</a>Add Redis Certificate to Trusted CA Store in Client Machine</h2>
# Copy your certificate to CA certificates directory</span></span>
sudo</span> mv redis_tls.crt /usr/local/share/ca-certificates/redis_tls.crt</span></span>
</span>
# Update CA certificates</span></span>
sudo</span> update-ca-certificates</span></span></code></pre>🔗</a>Redis Connection URL</h2>
Since we want to connect to redis using TLS connection, so the protocol in the url will be rediss</code> (with double “ss”).</p>
rediss://:<your_password_here>@<redis_server_ip>:6379/0#insecure</span></span></code></pre>


How I Use Cargo Workspace to structure my Rust projects
2025-09-03T00:00:00+00:00

I’ve learn to use Cargo Workspace from Olivia’s post in bluesky</a>, which you should checkout. This article is a way of self-documenting since I’ve come back to it more often.</p>
</blockquote>
🔗</a>What is a Cargo Workspace</h2>
A cargo workspace is collection of one or more library or binary crates. Cargo workspace let’s you break your rust project into multiple different crates (or packages), which may aid you in better project structure and management.</p>
🔗</a>Why use Cargo Workspace</h2>

Your rust project has grown big and is hard to manage</li>
Multiple people work on a project, so breaking your project into multiple crates can help with better ownership and less conflict among developers</li>
You want to reduce your project’s compile time</li>
</ul>

Read this (external link): Cutting Down Rust Compile Times From 30 to 2 Minutes With One Thousand Crates</a></p>
</blockquote>
🔗</a>Creating a Workspace</h2>
Create a new workspace directory</p>
mkdir</span> runner</span></span>
cd</span> runner</span></span></code></pre>
Now create Cargo.toml</code> with following content</p>
[</span>workspace</span>]</span></span>
resolver</span> =</span> "3"</span></span></code></pre>
Here we are using resolver</code> version 3, which is the latest version and comes in Rust 2024 edition (Rust 1.84+). Resolver tells cargo how to resolve the dependencies.</p>
🔗</a>Create a binary crate</h3>
We are going to use cargo new</code> command to create a binary crate. When no argument is passed, cargo new command will create binary crate by default, you can use --lib</code> as argument to create library crate or --bin</code> to explicit that you want binary crate.</p>
cargo</span> new run</span></span></code></pre>
Now your workspace will look like this:</p>
.</span></span>
├── Cargo.toml</span></span>
└── run</span></span>
    ├── Cargo.toml</span></span>
    └── src</span></span>
        └── main.rs</span></span></code></pre>
If you look at your workspace’s Cargo.toml</code> you will notice that cargo has already added the newly added crate run</code> against the members</code> key.</p>
[</span>workspace</span>]</span></span>
resolver</span> =</span> "3"</span></span>
members</span> =</span> [</span>"run"</span>]</span></span></code></pre>🔗</a>Adding lib crates</h3>
Let’s add two lib crates</p>
cargo</span> new participant</span> --lib</span></span>
cargo</span> new track</span> --lib</span></span></code></pre>
Updated structure:</p>
.</span></span>
├── Cargo.lock</span></span>
├── Cargo.toml</span></span>
├── participant</span></span>
│   ├── Cargo.toml</span></span>
│   └── src</span></span>
│       └── lib.rs</span></span>
├── run</span></span>
│   ├── Cargo.toml</span></span>
│   └── src</span></span>
│       └── main.rs</span></span>
└── track</span></span>
    ├── Cargo.toml</span></span>
    └── src</span></span>
        └── lib.rs</span></span></code></pre>🔗</a>Adding dependencies</h3>
If you try to simply add a dependency like this</p>
cargo</span> add uuid</span> --features</span> v4</span></span></code></pre>
‼️ it will fail and you will see this error</p>
error: `cargo add` could not determine which package to modify. Use the `--package` option to specify a package. </span></span>
available packages: participant, run, track</span></span></code></pre>
Since inside our Cargo workspace we have 3 packages, therefore cargo is unable to determine to which package dependency belongs to. ✅ We can solve this by using --package</code> arg.</p>
cargo</span> add uuid</span> --features</span> v4</span> --package</span> participant</span></span></code></pre>🔗</a>Dependency Resolution</h3>
Since in cargo workspace we can have multiple crates, which means we have to add dependency to each crate separately and in that sometimes there may be a version mismatch.</p>
Like with uuid</code> in participant</code> crate, it is using version 1.18.1</code> and now you if you try to add uuid</code> of different version in run</code> crate, for example if we want to stick to only 1.17</code> version of the uuid</code> crate:</p>
cargo</span> add uuid@=</span>1.17</span> --features</span> v4</span> --package</span> run</span></span></code></pre>
But wait, we got error</p>
error: failed to select a version for `uuid`.</span></span>
    ... required by package `run v0.1.0 (/home/vivek/Documents/code/hands-on/runner/run)`</span></span>
versions that meet the requirements `=1.17` are: 1.17.0</span></span>
</span>
all possible versions conflict with previously selected packages.</span></span>
</span>
  previously selected package `uuid v1.18.1`</span></span>
    ... which satisfies dependency `uuid = "^1.18.1"` of package `participant v0.1.0 (/home/vivek/Documents/code/hands-on/runner/participant)`</span></span>
</span>
failed to select a version for `uuid` which could resolve this conflict</span></span></code></pre>
It’s because a Cargo workspace can only have one Cargo.lock</code> file, which is shared between all the members of the workspace, so it can only have one version of the dependency across the workspace. So instead of pinning to =1.17</code> version of uuid, we had used just 1.17</code>, cargo would have choosen uuid@1.18.1</code> since that was already being used by participant</code> crate.</p>

In order to resolve the dependency across workspace, cargo will try to find the most suitable version of a crate, mostly by downgrading to the appropriate version.</p>
</blockquote>
🔗</a>Better way to declare dependency</h3>
It’s very normal to have common dependencies across crates, that is why Cargo Workspace let us declare dependencies in workspace’s Cargo.toml</code> and then you can re-use the same dependency inside of your crate.</p>
# Cargo.toml</span></span>
</span>
[</span>workspace</span>.</span>dependencies</span>]</span></span>
uuid</span> =</span> { version</span> =</span> "=1.17"</span>,</span> features</span> =</span> [</span>"v4"</span>] }</span></span></code></pre>
And then in your crates you can use the dependency with:</p>
# participant/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
uuid</span> =</span> { workspace</span> =</span> true</span> }</span></span>
</span>
# run/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
uuid</span> =</span> { workspace</span> =</span> true</span> }</span></span></code></pre>🔗</a>Importing members as dependencies</h3>
A common way to import a crate (or package) inside another crate is like this:</p>
# run/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
participant</span> =</span> { path</span> =</span> "../participant"</span> }</span></span>
track</span> =</span> { path</span> =</span> "../track"</span> }</span></span>
</span>
# track/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
participant</span> =</span> { path</span> =</span> "../participant"</span> }</span></span></code></pre>
But now if you had to do this for each of the crate in your project and sometime there can be a lot of crates. Also the path is relative to the current Cargo.toml file, so if you dare re-structure your workspace, it can backfire and you will have another mess to deal with.</p>
However we can use the same way of delcaring the dependency in the workspace’s Cargo.toml</code> and then just use it inside the members by telling cargo to use workspace’s dependency.</p>
# Cargo.toml</span></span>
[</span>workspace</span>]</span></span>
resolver</span> =</span> "3"</span></span>
members</span> =</span> [</span>"participant"</span>,</span> "run"</span>,</span> "track"</span>]</span></span>
</span>
[</span>workspace</span>.</span>dependencies</span>]</span></span>
participant</span> =</span> { path</span> =</span> "participant"</span> }</span></span>
track</span> =</span> { path</span> =</span> "track"</span> }</span></span>
</span>
# run/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
participant</span> =</span> { workspace</span> =</span> true</span> }</span></span>
track</span> =</span> { workspace</span> =</span> true</span> }</span></span>
</span>
# track/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
participant</span> =</span> { workspace</span> =</span> true</span> }</span></span></code></pre>🔗</a>Multiple Binaries</h3>
When you have multiple binaries in your workspace, cargo run</code> won’t work since it doesn’t know which binary crate to run. So instead explicitly pass the crate name:</p>
cargo</span> run</span> --package</span> my_bin_crate</span></span></code></pre>🔗</a>Cheat sheet</h2>
Create workspace:</strong></p>
# Cargo.toml</span></span>
[</span>workspace</span>]</span></span>
resolver</span> =</span> "3"</span></span></code></pre>
Adding bin and lib to workspace:</strong></p>
# creates binary crate</span></span>
cargo</span> new my_bin</span></span>
</span>
# crates lib crate</span></span>
cargo</span> new my_lib</span> --lib</span></span></code></pre>
Importing dependencies:</strong></p>
# Add dependencies in workspace's Cargo.toml</span></span>
[</span>workspace</span>.</span>dependencies</span>]</span></span>
uuid</span> =</span> { version</span> =</span> "=1.17"</span>,</span> features</span> =</span> [</span>"v4"</span>] }</span></span>
</span>
# import in members like this: my_lib/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
uuid</span> =</span> { workspace</span> =</span> true</span> }</span></span></code></pre>
Import members like this:</strong></p>
# Cargo.toml</span></span>
[</span>workspace</span>]</span></span>
resolver</span> =</span> "3"</span></span>
members</span> =</span> [</span>"my_bin"</span>,</span> "my_lib"</span>]</span></span>
</span>
[</span>workspace</span>.</span>dependencies</span>]</span></span>
my_lib</span> =</span> { path</span> =</span> "my_lib"</span> }</span></span>
</span>
# my_bin/Cargo.toml</span></span>
[</span>dependencies</span>]</span></span>
my_lib</span> =</span> { workspace</span> =</span> true</span> }</span></span></code></pre>
Other commands:</strong></p>
# run if multiple binaries exist</span></span>
cargo</span> run</span> --package</span> my_bin</span></span>
</span>
# build specific binary</span></span>
cargo</span> build</span> --package</span> my_bin</span></span>
</span>
# build all</span></span>
cargo</span> build</span> --all</span></span>
</span>
# test all</span></span>
cargo</span> test</span> --all</span></span></code></pre>


How I use Postgres DB as Key Value Store in Rust with SQLx
2025-09-02T00:00:00+00:00
For my smaller application (or personal) or when I’m starting out a new app, I would like to keep the dependencies of my program to minimum. Since I’m going to need a DB anyway, so instead of using Redis and adding one more component to my tech stack, I would like to use Postgres</a> as my key value store.</p>
Prematurely thinking about scaling your system when you don’t even know if it would be successful enough to actually need all those bells and whistle is sure way to extend the deadline and procrastinate.</p>
🔗</a>Overview</h2>
We are going to create a trait</code> called PgStore</code>, which will implement all the functions to set, get, pop and delete the value.</p>
Then you can implement this trait for your own types. Your types must derive serde’s Serialize</code> and Deserialize</code> for this to work.</p>
🔗</a>Dependencies</h2>
You need the following dependencies in your Cargo.toml</code></p>
chrono</span> =</span> { version</span> =</span> "0.4"</span>,</span> features</span> =</span> [</span>"serde"</span>] }</span></span>
serde</span> =</span> { version</span> =</span> "1.0"</span>,</span> features</span> =</span> [</span>"derive"</span>] }</span></span>
serde_json</span> =</span> { version</span> =</span> "1.0"</span> }</span></span>
sqlx</span> =</span> { version</span> =</span> "0.8"</span>,</span> features</span> =</span> [</span></span>
    "runtime-tokio"</span>,</span></span>
    "tls-rustls-ring-native-roots"</span>,</span></span>
    "migrate"</span>,</span></span>
    "postgres"</span>,</span></span>
    "macros"</span>,</span></span>
    "chrono"</span>,</span></span>
] }</span></span></code></pre>🔗</a>Creating static PgPool connection</h2>
I don’t like to pass around DB connection across functions or threads or tasks. Instead I like to use tokio’s OnceCell</code> which is a thread safe Cell which can be written to only once, it’s thread safe which means we can use async code in it.</p>
use</span> sqlx</span>::</span>PgPool</span>;</span></span>
use</span> tokio</span>::</span>sync</span>::</span>OnceCell</span>;</span></span>
</span>
static</span> PG_POOL</span>:</span> OnceCell</span><</span>PgPool</span>></span> =</span> OnceCell</span>::</span>const_new</span>();</span></span>
</span>
pub async fn</span> pg_pool</span>()</span> -> &</span>'</span>static PgPool</span> {</span></span>
    PG_POOL</span></span>
        .</span>get_or_init</span>(</span>async</span> ||</span> {</span></span>
            PgPool</span>::</span>connect</span>(</span>std</span>::</span>env</span>(</span>"DATABASE_URL"</span>)</span>.</span>expect</span>(</span>"DATABASE_URL is not set"</span>))</span></span>
                .</span>await</span></span>
                .</span>expect</span>(</span>"❌ PgPool connection failed"</span>)</span></span>
        })</span></span>
        .</span>await</span></span>
}</span></span></code></pre>
Now we can just use pg_pool().await</code> to fetch the db pool connection wherever we need.</p>
🔗</a>DB Migration</h2>
We are creating a new table kv_store</code> in our Postgres DB. If you notice we are using UNLOGGED</code> keyword to create the table, this tells postgres to not write WAL (write-ahead log) for kv_store</code> table, which improves the performance at the expanse of data durability.</p>
CREATE</span> UNLOGGED </span>TABLE</span> "kv_store"</span> (</span></span>
    "key"</span> VARCHAR</span>(</span>2048</span>)</span> PRIMARY KEY</span>,</span></span>
    "value"</span> TEXT NOT NULL</span>,</span></span>
    "expires"</span> TIMESTAMPTZ NOT NULL</span></span>
);</span></span></code></pre>🔗</a>PgStore Trait</h2>
We are creating PgStore</code> trait which will have methods to store, retrieve and delete the value in the kv_store</code> table.</p>
use</span> chrono</span>::</span>{</span>DateTime</span>,</span> Utc</span>};</span></span>
use</span> serde</span>::</span>Serialize</span>;</span></span>
use</span> serde</span>::</span>de</span>::</span>DeserializeOwned</span>;</span></span>
use</span> std</span>::</span>time</span>::</span>Duration</span>;</span></span>
</span>
#[allow(async_fn_in_trait)]</span></span>
pub trait</span> PgStore</span>:</span> Serialize</span> +</span> DeserializeOwned</span> {</span></span>
    const</span> EXPIRE_IN</span>:</span> usize</span>;</span> // seconds</span></span>
</span>
    fn</span> key_format</span>(key</span>:</span> String</span>)</span> -></span> String</span>;</span></span>
</span>
    fn</span> get_expire_time</span>()</span> -></span> DateTime</span><</span>Utc</span>> {</span></span>
        Utc</span>::</span>now</span>()</span> +</span> Duration</span>::</span>from_secs</span>(</span>Self</span>::</span>EXPIRE_IN</span> as</span> u64</span>)</span></span>
    }</span></span>
</span>
    async fn</span> set_ex</span>(</span>&</span>self</span>, key</span>:</span> String</span>)</span> -></span> Result</span><(),</span> String</span>> {</span></span>
        let</span> value</span> =</span></span>
            serde_json</span>::</span>to_string</span>(</span>&</span>self</span>)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to serialize value."</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        sqlx</span>::</span>query!</span>(</span></span>
            r#"INSERT INTO kv_store ("key", "value", expires) VALUES ($1, $2, $3)</span></span>
            ON CONFLICT ("key") DO UPDATE SET "value" = $2, expires = $3"#</span>,</span></span>
            Self</span>::</span>key_format</span>(key),</span></span>
            value,</span></span>
            Self</span>::</span>get_expire_time</span>(),</span></span>
        )</span></span>
        .</span>execute</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to set value in kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        Ok</span>(())</span></span>
    }</span></span>
</span>
    // it will just fetch the value if it exists</span></span>
    async fn</span> get</span>(key</span>:</span> String</span>)</span> -></span> Result</span><</span>Self</span>,</span> String</span>> {</span></span>
        let</span> record</span> =</span> sqlx</span>::</span>query!</span>(</span></span>
            r#"SELECT "value" FROM kv_store WHERE "key" = $1"#</span>,</span></span>
            Self</span>::</span>key_format</span>(key)</span></span>
        )</span></span>
        .</span>fetch_one</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to get value from kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        serde_json</span>::</span>from_str</span>(</span>&</span>record</span>.</span>value)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to deserialize value"</span>.</span>to_string</span>())</span></span>
    }</span></span>
</span>
    // this will fetch the value while extending the expire time</span></span>
    async fn</span> get_ex</span>(key</span>:</span> String</span>)</span> -></span> Result</span><</span>Self</span>,</span> String</span>> {</span></span>
        let</span> record</span> =</span> sqlx</span>::</span>query!</span>(</span></span>
            r#"UPDATE kv_store SET expires=$2 WHERE "key"=$1 RETURNING "value""#</span>,</span></span>
            Self</span>::</span>key_format</span>(key),</span></span>
            Self</span>::</span>get_expire_time</span>(),</span></span>
        )</span></span>
        .</span>fetch_one</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to get value from kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        serde_json</span>::</span>from_str</span>(</span>&</span>record</span>.</span>value)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to deserialize value"</span>.</span>to_string</span>())</span></span>
    }</span></span>
</span>
    // it will fetch the value while deleting the key</span></span>
    async fn</span> get_del</span>(key</span>:</span> String</span>)</span> -></span> Result</span><</span>Self</span>,</span> String</span>> {</span></span>
        let</span> record</span> =</span> sqlx</span>::</span>query!</span>(</span></span>
            r#"DELETE FROM kv_store WHERE key=$1 RETURNING "value""#</span>,</span></span>
            Self</span>::</span>key_format</span>(key)</span></span>
        )</span></span>
        .</span>fetch_one</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to get value from kv_store"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        serde_json</span>::</span>from_str</span>(</span>&</span>record</span>.</span>value)</span>.</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to deserialize value"</span>.</span>to_string</span>())</span></span>
    }</span></span>
</span>
    async fn</span> del</span>(key</span>:</span> String</span>)</span> -></span> Result</span><(),</span> String</span>> {</span></span>
        sqlx</span>::</span>query!</span>(</span></span>
            r#"DELETE FROM kv_store WHERE key=$1"#</span>,</span></span>
            Self</span>::</span>key_format</span>(key)</span></span>
        )</span></span>
        .</span>execute</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>map_err</span>(</span>|</span>_</span>|</span> "Failed to delete value"</span>.</span>to_string</span>())</span>?</span>;</span></span>
</span>
        Ok</span>(())</span></span>
    }</span></span>
}</span></span></code></pre>
By defining our trait like PgStore: Serialize + DeserializeOwned</code>, we are telling that whichever type is going to implement PgStore</code> trait it must also have Serialize</code> and DeserializeOwned</code> implemented.</p>
Following are the brief explaination of PgStore</code>’s attributes:</p>
EXPIRE_IN:</strong> This will be used to set the to be expired in seconds. It must be implemented by the Type you are using it for.</p>
key_format:</strong> This function needs to be implemented. It will be used as a way to generate key.</p>
set_ex:</strong> This method will store the type against the passed key in DB, overwriting value if it already exists.</p>
get:</strong> It will fetch the value for a given key, error will be returned if it does not exist.</p>
get_ex:</strong> Fetches the value while resetting the expire time.</p>
get_del:</strong> Fetch the value while deleting the key. It’s more like pop.</p>
del:</strong> Just deletes the value. Doesn’t return error even if key doesn’t exist.</p>
🔗</a>Cleaning up expired keys</h2>
Then we have a helper function pgstore_cleanup</code> which will cleanup expired keys every 60 seconds. You must spawn to run this function in the background for all the time your program is running.</p>
pub async fn</span> pgstore_cleanup</span>() {</span></span>
    loop</span> {</span></span>
        // tracing::info!("🟡 PgStore cleanup is running...");</span></span>
        sqlx</span>::</span>query!</span>(</span>r#"DELETE FROM kv_store WHERE expires < NOW()"#</span>)</span></span>
            .</span>execute</span>(</span>db</span>::</span>pg_pool</span>()</span>.</span>await</span>)</span></span>
            .</span>await</span></span>
            .</span>ok</span>();</span></span>
        tokio</span>::</span>time</span>::</span>sleep</span>(</span>Duration</span>::</span>from_secs</span>(</span>60</span>))</span>.</span>await</span>;</span></span>
    }</span></span>
}</span></span></code></pre>🔗</a>Usage</h2>
#[derive(serde</span>::</span>Serialize</span>, serde</span>::</span>Deserialize</span>)]</span></span>
pub struct</span> Game</span> {</span></span>
    pub</span> id</span>:</span> usize</span>,</span></span>
    pub</span> name</span>:</span> String</span>,</span></span>
    pub</span> genre</span>:</span> String</span>,</span></span>
}</span></span>
</span>
impl</span> PgStore</span> for</span> Game</span> {</span></span>
    const</span> EXPIRE_IN</span>:</span> usize</span> =</span> 3600</span>;</span></span>
</span>
    fn</span> key_format</span>(key</span>:</span> String</span>)</span> -></span> String</span> {</span></span>
        format!</span>(</span>"game-{}"</span>, key)</span></span>
    }</span></span>
}</span></span>
</span>
#[tokio</span>::</span>main]</span></span>
async fn</span> main</span>() {</span></span>
    // make sure to run this to delete expired keys regularly</span></span>
    tokio</span>::</span>spawn</span>(</span>pgstore_cleanup</span>());</span></span>
</span>
    let</span> game</span> =</span> Game</span> {</span></span>
        id</span>:</span> 1</span>,</span></span>
        name</span>:</span> "Skyrim"</span>.</span>to_string</span>(),</span></span>
        genre</span>:</span> "rpg"</span>.</span>to_string</span>(),</span></span>
    };</span></span>
</span>
    // storing value in kv_store</span></span>
    // it will be store with key: "game-1"</span></span>
    let</span> _</span> =</span> game</span>.</span>set_ex</span>(game</span>.</span>id</span>.</span>to_string</span>())</span>.</span>await</span>;</span></span>
</span>
    // fetching value</span></span>
    if let</span> Ok</span>(v)</span> =</span> Game</span>::</span>get</span>(</span>1</span>.</span>to_string</span>())</span>.</span>await</span> {</span></span>
        println!</span>(</span>"fetched game with name: {}"</span>, v</span>.</span>name);</span></span>
    }</span></span>
</span>
    // deleting key</span></span>
    let</span> _</span> =</span> Game</span>::</span>del</span>(</span>1</span>.</span>to_string</span>())</span>.</span>await</span>;</span></span>
}</span></span></code></pre>
Similarly you can do this with your own types, as long as they derive serde’s Serialize and Deserialize.</p>


Minify HTML in Axum Middleware
2025-09-01T00:00:00+00:00
Minifying HTML is one of the simplest way to save bandwidth and decrease the time in transit. We can easily lower page size by 30-50% by minifying HTML.</p>
We will be using minify-html</a> crate for this. With the use of Axum middleware, minification will be applied to all the eligible routes.</p>
Using axum middleware for HTML minification is template agnostic which means it will work with any template engine such as Askama, Tera, Maud, etc., as long as response type is HTML, it should work fine.</p>
Cargo.toml</strong></p>
Add minify-html</code> to your Cargo.toml.</p>
...</span></span>
minify</span>-</span>html</span> =</span> "0.16.4"</span></span>
...</span></span></code></pre>🔗</a>Basic Usage of minify-html</h2>
minify-html provides minify</code> function which accepts array of bytes and reference to configuration struct which tells it how to minify.</p>
let</span> cfg</span> =</span> minify_html</span>::</span>Cfg</span> {</span></span>
        keep_closing_tags</span>:</span> true</span>,</span></span>
        keep_html_and_head_opening_tags</span>:</span> true</span>,</span></span>
        keep_comments</span>:</span> false</span>,</span></span>
        minify_doctype</span>:</span> false</span>,</span></span>
        minify_css</span>:</span> true</span>,</span></span>
        minify_js</span>:</span> true</span>,</span></span>
        ..</span>Default</span>::</span>default</span>()</span></span>
    };</span></span>
</span>
let mut</span> code</span>: &</span>[</span>u8</span>]</span> =</span> b"<p>  Hello, world!  </p>"</span>;</span></span>
let</span> minified</span> =</span> minify_html</span>::</span>minify</span>(</span>&</span>code,</span> &</span>cfg);</span></span>
assert_eq!</span>(minified,</span> b"<p>Hello, world!</p>"</span>.</span>to_vec</span>());</span></span></code></pre>🔗</a>Creating Axum Middleware</h2>
We are using Axum’s axum::middleware::map_response</code> middleware to modify the response on the fly by passing an async function.</p>
Steps to be performed by middleware:</strong></p>

Only move forward if the Content-Type</code> header of the reponse is text/html</code></li>
Use into_parts</code> method of response to consume the response body give out parts (headers, status code, etc.) and body (actual html content)</li>
Using to_bytes</code> function we collect and convert axum::body::Body</code> type into Bytes</code></li>
Then using minify_html::minify</code> function we minify the html</li>
Finally build the new response using Response::from_parts</code> with the parts that we extracted earlier and new body (minified html)</li>
</ol>
use</span> std</span>::</span>sync</span>::</span>LazyLock</span>;</span></span>
use</span> axum</span>::</span>{</span></span>
    body</span>::</span>{</span>Body</span>, to_bytes},</span></span>
    http</span>::</span>header,</span></span>
    response</span>::</span>Response</span>,</span></span>
};</span></span>
</span>
static</span> MINIFY_CFG</span>:</span> LazyLock</span><minify_html</span>::</span>Cfg</span>></span> =</span> LazyLock</span>::</span>new</span>(</span>||</span> minify_html</span>::</span>Cfg</span> {</span></span>
    keep_closing_tags</span>:</span> true</span>,</span></span>
    keep_html_and_head_opening_tags</span>:</span> true</span>,</span></span>
    minify_doctype</span>:</span> false</span>,</span></span>
    minify_css</span>:</span> true</span>,</span></span>
    minify_js</span>:</span> true</span>,</span></span>
    ..</span>Default</span>::</span>default</span>()</span></span>
});</span></span>
</span>
pub async fn</span> minify_html_response</span>(response</span>:</span> Response</span><</span>Body</span>>)</span> -></span> Response</span><</span>Body</span>> {</span></span>
    let</span> content_type</span> =</span> response</span></span>
        .</span>headers</span>()</span></span>
        .</span>get</span>(</span>header</span>::</span>CONTENT_TYPE</span>)</span></span>
        .</span>map</span>(</span>|</span>h</span>|</span> h</span>.</span>to_str</span>()</span>.</span>unwrap_or_default</span>())</span></span>
        .</span>unwrap_or_default</span>();</span></span>
</span>
    if</span> content_type</span>.</span>contains</span>(</span>"text/html"</span>) {</span></span>
        let</span> (parts, body)</span> =</span> response</span>.</span>into_parts</span>();</span></span>
        let</span> bytes</span> =</span> to_bytes</span>(body,</span> usize</span>::</span>MAX</span>)</span>.</span>await</span>.</span>unwrap_or_default</span>();</span></span>
        let</span> minified</span> =</span> minify_html</span>::</span>minify</span>(</span>&</span>bytes,</span> &</span>MINIFY_CFG</span>);</span></span>
        let</span> new_response</span> =</span> Response</span>::</span>from_parts</span>(parts,</span> Body</span>::</span>from</span>(minified));</span></span>
        return</span> new_response;</span></span>
    }</span></span>
</span>
    response</span></span>
}</span></span></code></pre>
Note: We are using LazyLock</code> to initialize minify_html::Cfg</code> only once, since we are going to need the reference to it for each minify call.</p>
Plugging into the Router:</strong></p>
let</span> router</span> =</span> axum</span>::</span>Router</span>::</span>new</span>()</span></span>
    .</span>route</span>(</span>"/"</span>,</span> get</span>(index_page))</span></span>
    .</span>layer</span>(</span>axum</span>::</span>middleware</span>::</span>map_response</span>(minify_html_response));</span></span></code></pre>🔗</a>Benchmark</h2>
I’ve done very basic benchmarking on this</p>
Overhead:</strong> The overhead of extracting response body and minifying adds around 600</code> to 800</code> µs.</p>
Size Reduction:</strong> This will be varied for everyone and depends on the content they serve. However, I’ve consistently seen at least 30% reduction in HTML document size.</p>


Using SQLite Database in Rust with Sqlx
2025-04-13T00:00:00+00:00
SQLite is self-contained, file based relational database. If you have a simple use case and want to avoid complex databases then SQLite is one of the best choice.</p>
We will use sqlx</code> crate</a> to interact with SQLite database.</p>
Why Sqlx?</p>

Async support</li>
Supports migrations</li>
Compile time type safety</li>
Supports SQLite, Postgres and MySQL</li>
</ul>
🔗</a>Setup</h2>
cargo</span> init sqlite-demo</span></span></code></pre>🔗</a>Cargo.toml</h3>
Add the following dependencies to your Cargo.toml</code> file. We are using tokio runtime</a> and sqlite features of sqlx</code>.</p>
[</span>package</span>]</span></span>
name</span> =</span> "sqlite-demo"</span></span>
version</span> =</span> "0.1.0"</span></span>
edition</span> =</span> "2024"</span></span>
</span>
[</span>dependencies</span>]</span></span>
sqlx</span> =</span> { version</span> =</span> "0.8"</span>,</span> features</span> =</span> [</span></span>
    "runtime-tokio"</span>,</span></span>
    "sqlite"</span>,</span></span>
    "migrate"</span>,</span></span>
] }</span></span>
tokio</span> =</span> { version</span> =</span> "1.44.2"</span>,</span> features</span> =</span> [</span>"full"</span>] }</span></span></code></pre>🔗</a>Connect to SQLite Database</h2>
Here we are using SqliteConnectOptions</code> to connect to SQLite database. We are also creating the database file if it doesn’t exist. We are using SqliteJournalMode::Delete</code> to delete the journal file after commit so that we will only have one file in the database directory.</p>
Since we want file based database, we are using sqlite://main.db</code> as connection string. For memory based database, we can use sqlite::memory:</code> as connection string.</p>
use</span> sqlx</span>::</span>{</span>ConnectOptions</span>,</span> Connection</span>,</span> sqlite</span>::</span>SqliteConnectOptions</span>};</span></span>
use</span> std</span>::</span>str</span>::</span>FromStr</span>;</span></span>
</span>
#[tokio</span>::</span>main]</span></span>
async fn</span> main</span>() {</span></span>
    let mut</span> conn</span> =</span> SqliteConnectOptions</span>::</span>from_str</span>(</span>"sqlite://main.db"</span>)</span></span>
        .</span>unwrap</span>()</span></span>
        .</span>create_if_missing</span>(</span>true</span>)</span></span>
        .</span>journal_mode</span>(</span>sqlx</span>::</span>sqlite</span>::</span>SqliteJournalMode</span>::</span>Delete</span>)</span></span>
        .</span>connect</span>()</span></span>
        .</span>await</span></span>
        .</span>expect</span>(</span>"Failed to connect to database"</span>);</span></span>
</span>
    // Create the users table if it doesn't exist</span></span>
    sqlx</span>::</span>query</span>(</span></span>
        "CREATE TABLE IF NOT EXISTS users (</span></span>
            id INTEGER PRIMARY KEY AUTOINCREMENT,</span></span>
            name TEXT NOT NULL,</span></span>
            age INTEGER</span></span>
        )"</span>,</span></span>
    )</span></span>
    .</span>execute</span>(</span>&</span>mut</span> conn)</span></span>
    .</span>await</span></span>
    .</span>unwrap</span>();</span></span>
</span>
    let</span> _</span> =</span> conn</span>.</span>close</span>()</span>.</span>await</span>;</span></span>
}</span></span></code></pre>🔗</a>Creating Connection Pool</h2>
We will use SqlitePoolOptions</code></a> to create a connection pool.</p>
We will create a helper function db_conn</code> to create a connection pool. This function will return a connection pool that can be used to interact with the database.</p>
use</span> sqlx</span>::</span>{</span></span>
    Pool</span>,</span> Sqlite</span>,</span></span>
    sqlite</span>::</span>{</span>SqliteConnectOptions</span>,</span> SqlitePoolOptions</span>},</span></span>
};</span></span>
use</span> std</span>::</span>{</span>str</span>::</span>FromStr</span>,</span> time</span>::</span>Duration</span>};</span></span>
</span>
#[tokio</span>::</span>main]</span></span>
async fn</span> main</span>() {</span></span>
    let</span> conn</span> =</span> db_conn</span>()</span>.</span>await</span>;</span></span>
</span>
    // Create the users table if it doesn't exist</span></span>
    sqlx</span>::</span>query</span>(</span></span>
        "CREATE TABLE IF NOT EXISTS users (</span></span>
            id INTEGER PRIMARY KEY AUTOINCREMENT,</span></span>
            name TEXT NOT NULL,</span></span>
            age INTEGER</span></span>
        )"</span>,</span></span>
    )</span></span>
    .</span>execute</span>(</span>&</span>conn)</span></span>
    .</span>await</span></span>
    .</span>unwrap</span>();</span></span>
</span>
    let</span> _</span> =</span> conn</span>.</span>close</span>()</span>.</span>await</span>;</span></span>
}</span></span>
</span>
async fn</span> db_conn</span>()</span> -></span> Pool</span><</span>Sqlite</span>> {</span></span>
    let</span> db_url</span> =</span> "sqlite://main.db"</span>;</span></span>
</span>
    SqlitePoolOptions</span>::</span>new</span>()</span></span>
        .</span>max_connections</span>(</span>20</span>)</span></span>
        .</span>idle_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>60</span>))</span></span>
        .</span>acquire_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>5</span>))</span></span>
        .</span>connect_with</span>(</span></span>
            SqliteConnectOptions</span>::</span>from_str</span>(db_url)</span></span>
                .</span>unwrap</span>()</span></span>
                .</span>create_if_missing</span>(</span>true</span>)</span></span>
                .</span>journal_mode</span>(</span>sqlx</span>::</span>sqlite</span>::</span>SqliteJournalMode</span>::</span>Delete</span>),</span></span>
        )</span></span>
        .</span>await</span></span>
        .</span>unwrap</span>()</span></span>
}</span></span></code></pre>🔗</a>Using OnceCell to create a connection pool</h2>
We can use OnceCell</code></a> to create a connection pool. This will ensure that the connection pool is created only once and can be shared across multiple threads. So that we won’t to pass connection pool to every function, we can just use this helper function to get the connection pool and it will re-use the same one.</p>
use</span> sqlx</span>::</span>{</span></span>
    Pool</span>,</span> Sqlite</span>,</span></span>
    sqlite</span>::</span>{</span>SqliteConnectOptions</span>,</span> SqlitePoolOptions</span>},</span></span>
};</span></span>
use</span> std</span>::</span>{</span>str</span>::</span>FromStr</span>,</span> time</span>::</span>Duration</span>};</span></span>
use</span> tokio</span>::</span>sync</span>::</span>OnceCell</span>;</span></span>
</span>
static</span> DB_CONN</span>:</span> OnceCell</span><</span>Pool</span><</span>Sqlite</span>>></span> =</span> OnceCell</span>::</span>const_new</span>();</span></span>
</span>
async fn</span> db_conn</span>()</span> -></span> Pool</span><</span>Sqlite</span>> {</span></span>
    let</span> db_url</span> =</span> "sqlite://main.db"</span>;</span></span>
</span>
    DB_CONN</span></span>
        .</span>get_or_init</span>(</span>||</span> async</span> {</span></span>
            SqlitePoolOptions</span>::</span>new</span>()</span></span>
                .</span>max_connections</span>(</span>20</span>)</span></span>
                .</span>idle_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>60</span>))</span></span>
                .</span>acquire_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>5</span>))</span></span>
                .</span>connect_with</span>(</span></span>
                    SqliteConnectOptions</span>::</span>from_str</span>(db_url)</span></span>
                        .</span>unwrap</span>()</span></span>
                        .</span>create_if_missing</span>(</span>true</span>)</span></span>
                        .</span>journal_mode</span>(</span>sqlx</span>::</span>sqlite</span>::</span>SqliteJournalMode</span>::</span>Delete</span>),</span></span>
                )</span></span>
                .</span>await</span></span>
                .</span>unwrap</span>()</span></span>
        })</span></span>
        .</span>await</span></span>
        .</span>clone</span>()</span></span>
}</span></span></code></pre>🔗</a>Handling Database Migrations</h2>
We will use sqlx-cli</code></a> to handle migrations.</p>
cargo</span> install sqlx-cli</span></span></code></pre>
Create migration file for users table.</p>
sqlx</span> migrate add users</span></span></code></pre>
This will create a migration file in migrations</code> directory with empty migration file for users table. Add the following content to the newly created migration file.</p>
-- Add migration script here</span></span>
CREATE TABLE</span> IF</span> NOT EXISTS</span> users (</span></span>
    id </span>INTEGER PRIMARY KEY</span> AUTOINCREMENT,</span></span>
    name TEXT NOT NULL</span>,</span></span>
    age </span>INTEGER</span></span>
);</span></span></code></pre>
Embed the migration file in the executable.</p>
use</span> sqlx</span>::</span>{</span></span>
    Pool</span>,</span> Sqlite</span>,</span></span>
    sqlite</span>::</span>{</span>SqliteConnectOptions</span>,</span> SqlitePoolOptions</span>},</span></span>
};</span></span>
use</span> std</span>::</span>{</span>str</span>::</span>FromStr</span>,</span> time</span>::</span>Duration</span>};</span></span>
use</span> tokio</span>::</span>sync</span>::</span>OnceCell</span>;</span></span>
</span>
#[tokio</span>::</span>main]</span></span>
async fn</span> main</span>() {</span></span>
    // this will make sure migrations are added to the executable</span></span>
    sqlx</span>::</span>migrate!</span>()</span></span>
        .</span>run</span>(</span>&</span>db_conn</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>expect</span>(</span>"Migrations went wrong :("</span>);</span></span>
}</span></span>
</span>
static</span> DB_CONN</span>:</span> OnceCell</span><</span>Pool</span><</span>Sqlite</span>>></span> =</span> OnceCell</span>::</span>const_new</span>();</span></span>
</span>
async fn</span> db_conn</span>()</span> -></span> Pool</span><</span>Sqlite</span>> {</span></span>
    let</span> db_url</span> =</span> "sqlite://main.db"</span>;</span></span>
</span>
    DB_CONN</span></span>
        .</span>get_or_init</span>(</span>||</span> async</span> {</span></span>
            SqlitePoolOptions</span>::</span>new</span>()</span></span>
                .</span>max_connections</span>(</span>20</span>)</span></span>
                .</span>idle_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>60</span>))</span></span>
                .</span>acquire_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>5</span>))</span></span>
                .</span>connect_with</span>(</span></span>
                    SqliteConnectOptions</span>::</span>from_str</span>(db_url)</span></span>
                        .</span>unwrap</span>()</span></span>
                        .</span>create_if_missing</span>(</span>true</span>)</span></span>
                        .</span>journal_mode</span>(</span>sqlx</span>::</span>sqlite</span>::</span>SqliteJournalMode</span>::</span>Delete</span>),</span></span>
                )</span></span>
                .</span>await</span></span>
                .</span>unwrap</span>()</span></span>
        })</span></span>
        .</span>await</span></span>
        .</span>clone</span>()</span></span>
}</span></span></code></pre>🔗</a>Full Code with Example Queries</h2>
main.rs</strong></p>
use</span> sqlx</span>::</span>{</span></span>
    Pool</span>,</span> Row</span>,</span> Sqlite</span>,</span></span>
    prelude</span>::</span>FromRow</span>,</span></span>
    sqlite</span>::</span>{</span>SqliteConnectOptions</span>,</span> SqlitePoolOptions</span>},</span></span>
};</span></span>
use</span> std</span>::</span>{</span>str</span>::</span>FromStr</span>,</span> time</span>::</span>Duration</span>};</span></span>
use</span> tokio</span>::</span>sync</span>::</span>OnceCell</span>;</span></span>
</span>
#[tokio</span>::</span>main]</span></span>
async fn</span> main</span>() {</span></span>
    sqlx</span>::</span>migrate!</span>()</span></span>
        .</span>run</span>(</span>&</span>db_conn</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>expect</span>(</span>"❌ Migrations went wrong :("</span>);</span></span>
</span>
    sqlx</span>::</span>query</span>(</span>"INSERT INTO users (name, age) VALUES (?, ?)"</span>)</span> // Use ? for placeholders</span></span>
        .</span>bind</span>(</span>"Alice"</span>)</span></span>
        .</span>bind</span>(</span>30</span>)</span></span>
        .</span>execute</span>(</span>&</span>db_conn</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>unwrap</span>();</span></span>
</span>
    let</span> query</span> =</span> sqlx</span>::</span>query</span>(</span>"SELECT count(*) AS count FROM users"</span>)</span></span>
        .</span>fetch_one</span>(</span>&</span>db_conn</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>unwrap</span>();</span></span>
</span>
    println!</span>(</span>"Count: {:?}"</span>, query</span>.</span>get</span>::</span><</span>i64</span>,</span> &</span>str</span>>(</span>"count"</span>));</span></span>
</span>
    let</span> users</span> =</span> sqlx</span>::</span>query_as</span>::</span><_,</span> User</span>>(</span>"SELECT id, name, age FROM users"</span>)</span></span>
        .</span>fetch_all</span>(</span>&</span>db_conn</span>()</span>.</span>await</span>)</span></span>
        .</span>await</span></span>
        .</span>unwrap</span>();</span></span>
</span>
    for</span> user</span> in</span> users {</span></span>
        println!</span>(</span>"Found user: {:?}"</span>, user);</span></span>
    }</span></span>
}</span></span>
</span>
static</span> DB_CONN</span>:</span> OnceCell</span><</span>Pool</span><</span>Sqlite</span>>></span> =</span> OnceCell</span>::</span>const_new</span>();</span></span>
</span>
async fn</span> db_conn</span>()</span> -></span> Pool</span><</span>Sqlite</span>> {</span></span>
    let</span> db_url</span> =</span> "sqlite://main.db"</span>;</span></span>
</span>
    DB_CONN</span></span>
        .</span>get_or_init</span>(</span>||</span> async</span> {</span></span>
            SqlitePoolOptions</span>::</span>new</span>()</span></span>
                .</span>max_connections</span>(</span>20</span>)</span></span>
                .</span>idle_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>60</span>))</span></span>
                .</span>acquire_timeout</span>(</span>Duration</span>::</span>from_secs</span>(</span>5</span>))</span></span>
                .</span>connect_with</span>(</span></span>
                    SqliteConnectOptions</span>::</span>from_str</span>(db_url)</span></span>
                        .</span>unwrap</span>()</span></span>
                        .</span>create_if_missing</span>(</span>true</span>)</span></span>
                        .</span>journal_mode</span>(</span>sqlx</span>::</span>sqlite</span>::</span>SqliteJournalMode</span>::</span>Delete</span>),</span></span>
                )</span></span>
                .</span>await</span></span>
                .</span>unwrap</span>()</span></span>
        })</span></span>
        .</span>await</span></span>
        .</span>clone</span>()</span></span>
}</span></span>
</span>
#[derive(</span>Debug</span>,</span> FromRow</span>)]</span></span>
struct</span> User</span> {</span></span>
    id</span>:</span> i64</span>,</span></span>
    name</span>:</span> String</span>,</span></span>
    age</span>:</span> Option</span><</span>i32</span>>,</span></span>
}</span></span></code></pre>


Self Host PostgreSQL With TLS
2025-02-03T00:00:00+00:00
In this tutorial we will go through how to self-host PostgreSQL Database and setup TLS to securely connect.</p>
🔗</a>Installing PostgreSQL</h2>
Postgres APT repository configuration:</p>
sudo</span> apt install</span> -y</span> postgresql-common</span></span>
sudo</span> /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh</span></span></code></pre>
Install PostgreSQL 16</p>
sudo</span> apt install postgresql-16</span></span></code></pre>
After installing PostgreSQL, it creates a new user postgres</code>, which is the superuser of the database. We can switch to postgres</code> user in ubuntu to access postgresql command prompt:</p>
sudo</span> -i -u</span> postgres</span></span></code></pre>
Now to create new user with passowrd and with privilege to create new db. It will ask you to enter your password, which will not be logged into history hence this is the recommended way to set password.</p>
createuser</span> your_db_user</span> --pwprompt --createdb</span></span></code></pre>
Now we can exit from postgres</code> account</p>
exit</span></span></code></pre>
We will add new user in our Ubuntu with the same name as the new user we created in postgresql. (Optional) Add -m</code> to create home directory for the user.</p>
sudo</span> useradd</span> -s</span> /bin/bash your_db_user</span></span></code></pre>
Accessing psql as the new user we just created. Since we do not have any database associated with our user therefore we are using ‘-d postgres’ to use postgres db for now.</p>
sudo</span> -u</span> your_db_user psql</span> -U</span> your_db_user</span> -d</span> postgres</span></span></code></pre>
Create your database.</p>
CREATE</span> DATABASE your_db_name</span>;</span></span></code></pre>
This is how you can connect with your db if not already connected.</p>
\c</span> your_db_name</span>;</span></span></code></pre>🔗</a>Setting Up SSL</h2>
It’s important that we secure our communication with the database while accessing it remotely. We are going to setup SSL certificate using certbot</code> from LetsEncrypt.</p>
Install certbot</p>
sudo</span> snap install core</span>;</span> sudo</span> snap refresh core</span></span>
sudo</span> apt-get remove certbot</span></span>
sudo</span> snap install</span> --classic</span> certbot</span></span>
sudo</span> ln</span> -s</span> /snap/bin/certbot /usr/bin/certbot</span></span></code></pre>
Obtain certificate</p>
sudo</span> certbot certonly</span> --standalone -d</span> psql.example.com</span></span></code></pre>
Find PostgreSQL data directory</p>
sudo</span> -u</span> postgres psql</span> -U</span> postgres</span> -c</span> 'SHOW data_directory'</span></span></code></pre>
Create a renewal hook file</p>
sudo</span> nano /etc/letsencrypt/renewal-hooks/deploy/postgresql.deploy</span></span></code></pre>
Paste the following</p>
umask 0177</span></span>
DOMAIN=psql.example.com</span></span>
DATA_DIR=/var/lib/postgresql/16/main</span></span>
cp /etc/letsencrypt/live/$DOMAIN/fullchain.pem $DATA_DIR/server.crt</span></span>
cp /etc/letsencrypt/live/$DOMAIN/privkey.pem $DATA_DIR/server.key</span></span>
chown postgres:postgres $DATA_DIR/server.crt $DATA_DIR/server.key</span></span></code></pre>
Save and exit the file.</p>
Give file executable permission</p>
sudo</span> chmod +x /etc/letsencrypt/renewal-hooks/deploy/postgresql.deploy</span></span></code></pre>
Get the path of PostgreSQL configuration file</p>
sudo</span> -u</span> postgres psql</span> -U</span> postgres</span> -c</span> 'SHOW config_file'</span></span></code></pre>
Open the file in edit mode and update the SSL section of the file as below:</p>
sudo</span> nano /etc/postgresql/16/main/postgresql.conf</span></span></code></pre>ssl = on</span></span>
ssl_cert_file = 'server.crt'</span></span>
ssl_key_file = 'server.key'</span></span>
ssl_prefer_server_ciphers = on</span></span></code></pre>
Locate the Connection Settings section and verify the listen_address is to * for all addresses. Make sure the line is not commented out.</p>
listen_addresses = '*'</span></span></code></pre>
Edit pg_hba.conf</code> file which is in the same dir as config file:</p>
sudo</span> nano /etc/postgresql/16/main/pg_hba.conf</span></span></code></pre>
Add the following line to enable connection over ssl for any user and database:</p>
# TYPE DATABASE USER ADDRESS METHOD</span></span>
</span>
hostssl all all 0.0.0.0/0 scram-sha-256</span></span>
hostssl all all ::/0 scram-sha-256</span></span></code></pre>

Although in above config it’s shown to allow connection from all IPv4 and IPv6 addresses, however from the security point of view only allow the ip address of your connecting machine (client).</p>
</blockquote>
Renew the SSL certificate</p>
sudo</span> certbot renew</span> --force-renewal</span></span></code></pre>
Restart PostgreSQL</p>
sudo</span> service postgresql restart</span></span></code></pre>
Connect to your db remotely</p>
psql</span> -d</span> "dbname=your_db_name sslmode=require"</span> -h</span> psql.example.com</span> -U</span> your_db_user</span></span></code></pre>


AES Encryption in Rust
2024-03-22T00:00:00+00:00
What we will cover</strong></p>

Brief explanation of AES encryption</li>
AES-256 encryption and decryption</li>
Full Rust code for AES-256 encryption</li>
</ul>
🔗</a>Introduction</h2>
AES stands for Advanced Encryption Standard. It is a type of Symmetric key encryption algorithm, which means the same key is used for encryption and decryption.</p>
In this tutorial we will use aes-gcm</a> crate from Rust Crypto</a>.</p>
There are 3 type of AES encryption: AES-128, AES-192 and AES-256, which represents encryption Key</em> of length 128 bit (16 byte), 192 bit (24 byte) and 256 bit (32 byte) respectively.</p>
Both are pretty secure and hard to crack but for obvious reasons AES-256 is the most secure one.</p>
AES encryption requires key and nonce (for gcm variant) to encrypt/decrypt the data.</p>
🔗</a>Key</h2>
Key means encryption key, which is used to encrypt and decrypt the data. Length of the key matters, AES-128 requires key of 16 byte and AES-256 requires key of 32 byte.</p>
For the simplicity from now on we will focus on AES-256, but the core concept is similar even for AES-128.</p>
Key is byte array of length 32, where each element is of 1 byte. We can either generate the key using aes-gcm crate itself or we can use our own key. It’s prefferable to use our own key, so let’s see it in action:</p>
Create new crypto</em> project and add aes-gcm crate.</p>
cargo</span> new crypto</span></span>
cargo</span> add aes-gcm</span></span></code></pre>
main.rs</em></p>
use</span> aes_gcm</span>::</span>{</span></span>
    aead</span>::</span>{</span>Aead</span>,</span> AeadCore</span>,</span> KeyInit</span>,</span> OsRng</span>},</span></span>
    Aes256Gcm</span>,</span> Key</span>,</span> Nonce</span></span>
};</span></span>
</span>
fn</span> main</span>() {</span></span>
    let</span> key_str</span> =</span> "thiskeystrmustbe32charlongtowork"</span>;</span></span>
    let</span> key</span> =</span> Key</span>::</span><</span>Aes256Gcm</span>></span>::</span>from_slice</span>(key_str</span>.</span>as_bytes</span>());</span></span>
}</span></span></code></pre>
It’s important that the string/str used for key must be 32 char long, since AES-256 requires key length of 32 byte.</p>
Note: If you want to use key string of any length (less or more than 32), you can use Key Derivation Function to generate secure 32 byte long key. Argon2, PBKDF2 are some of the fitting choices for this.</p>
🔗</a>Nonce</h2>
Nonce means “number used only once”. It is an array of random numbers (type: u8) that is unique to each operation. For AES encryption the required size of Nonce is 12 byte.</p>
For each encryption operation the same key can be used but the Nonce must be unique</u>, so we generate Nonce each time we encrypt data.</p>
Since nonce is required to decrypt the data, it is stored along with the encrypted data itself.</p>
let</span> nonce</span> =</span> Aes256Gcm</span>::</span>generate_nonce</span>(</span>&</span>mut</span> OsRng</span>);</span></span></code></pre>🔗</a>Encryption</h2>
We will create encrypt</code> function which will take key_str and plaintext string. This will make our code more readable and easy to use.</p>
use</span> aes_gcm</span>::</span>{</span></span>
    aead</span>::</span>{</span>Aead</span>,</span> AeadCore</span>,</span> KeyInit</span>,</span> OsRng</span>},</span></span>
    Aes256Gcm</span>,</span> Key</span>,</span> Nonce</span></span>
};</span></span>
</span>
fn</span> main</span>() {</span></span>
    let</span> plaintext</span> =</span> "vivekshuk.la"</span>.</span>to_string</span>();</span></span>
</span>
    let</span> key_str</span> =</span> "thiskeystrmustbe32charlongtowork"</span>.</span>to_string</span>();</span></span>
</span>
    let</span> encrypted_data</span> =</span> encrypt</span>(key_str, plaintext);</span></span>
}</span></span>
</span>
fn</span> encrypt</span>(key_str</span>:</span> String</span>, plaintext</span>:</span> String</span>)</span> -></span> Vec</span><</span>u8</span>> {</span></span>
    let</span> key</span> =</span> Key</span>::</span><</span>Aes256Gcm</span>></span>::</span>from_slice</span>(key_str</span>.</span>as_bytes</span>());</span></span>
    let</span> nonce</span> =</span> Aes256Gcm</span>::</span>generate_nonce</span>(</span>&</span>mut</span> OsRng</span>);</span></span>
</span>
    let</span> cipher</span> =</span> Aes256Gcm</span>::</span>new</span>(key);</span></span>
</span>
    let</span> ciphered_data</span> =</span> cipher</span>.</span>encrypt</span>(</span>&</span>nonce, plaintext</span>.</span>as_bytes</span>())</span></span>
        .</span>expect</span>(</span>"failed to encrypt"</span>);</span></span>
</span>
    // combining nonce and encrypted data together</span></span>
    // for storage purpose</span></span>
    let mut</span> encrypted_data</span>:</span> Vec</span><</span>u8</span>></span> =</span> nonce</span>.</span>to_vec</span>();</span></span>
    encrypted_data</span>.</span>extend_from_slice</span>(</span>&</span>ciphered_data);</span></span>
</span>
    encrypted_data</span></span>
}</span></span></code></pre>
Here we are:</p>

creating instance of Aes256Gcm</code> with our encryption key, which will be used for encryption.</li>
we need to convert our plaintext from string to byte array for the encrypt</code> method.</li>
combine nonce and ciphered_data together</li>
</ul>
🔗</a>Decryption</h2>
Here first we will separate out nonce and ciphered data by splitting encrypted data vector at 12th position (because we used nonce of length 12).</p>
Then we create Nonce instance from our nonce array that we got.</p>
After decrypting we will get byte array which we can pass to String::from_utf8</code> function to create our original string value back.</p>
fn</span> decrypt</span>(key_str</span>:</span> String</span>, encrypted_data</span>:</span> Vec</span><</span>u8</span>>)</span> -></span> String</span> {</span></span>
    let</span> key</span> =</span> Key</span>::</span><</span>Aes256Gcm</span>></span>::</span>from_slice</span>(key_str</span>.</span>as_bytes</span>());</span></span>
</span>
    let</span> (nonce_arr, ciphered_data)</span> =</span> encrypted_data</span>.</span>split_at</span>(</span>12</span>);</span></span>
    let</span> nonce</span> =</span> Nonce</span>::</span>from_slice</span>(nonce_arr);</span></span>
</span>
    let</span> cipher</span> =</span> Aes256Gcm</span>::</span>new</span>(key);</span></span>
</span>
    let</span> plaintext</span> =</span> cipher</span>.</span>decrypt</span>(nonce, ciphered_data)</span></span>
        .</span>expect</span>(</span>"failed to decrypt data"</span>);</span></span>
</span>
    String</span>::</span>from_utf8</span>(plaintext)</span></span>
        .</span>expect</span>(</span>"failed to convert vector of bytes to string"</span>)</span></span>
}</span></span></code></pre>🔗</a>Storage</h2>
There is one issue you might encounter that the encrypt function will give out Vector of numbers (u8), which is not easy to store. Of course you might convert it to string by joining them with comma but it’s not good solution.</p>
To solve this we can convert our vector of numbers to hex string, this will make storage and operation much easier for us.</p>
Let’s install hex</code> crate for this.</p>
cargo</span> add hex</span></span></code></pre>
hex</code> crate provides encode</code> (vec to string) and decode</code> (string to vec) function for easy conversion.</p>
let</span> encoded_string</span> =</span> hex</span>::</span>encode</span>(encrypted_data);</span></span>
</span>
let</span> decoded_vec</span> =</span> hex</span>::</span>decode</span>(encoded_string)</span></span>
    .</span>expect</span>(</span>"failed to decode hex string into vec"</span>);</span></span></code></pre>🔗</a>Full Code</h2>
In this final version we have made few changes to incorporate hex</code> conversion within the encrypt</code> and decrypt</code> function itself.</p>
use</span> aes_gcm</span>::</span>{</span></span>
    aead</span>::</span>{</span>Aead</span>,</span> AeadCore</span>,</span> KeyInit</span>,</span> OsRng</span>},</span></span>
    Aes256Gcm</span>,</span> Key</span>,</span> Nonce</span></span>
};</span></span>
</span>
fn</span> main</span>() {</span></span>
    let</span> plaintext</span> =</span> "vivekshuk.la"</span>.</span>to_string</span>();</span></span>
</span>
    let</span> key_str</span> =</span> "thiskeystrmustbe32charlongtowork"</span>.</span>to_string</span>();</span></span>
</span>
    let</span> encrypted_data</span> =</span> encrypt</span>(key_str</span>.</span>clone</span>(), plaintext);</span></span>
</span>
    println!</span>(</span>"encrypted_data: {:?}"</span>, encrypted_data</span>.</span>clone</span>());</span></span>
</span>
    let</span> original</span> =</span> decrypt</span>(key_str, encrypted_data);</span></span>
</span>
    println!</span>(</span>"original: {:?}"</span>, original);</span></span>
}</span></span>
</span>
fn</span> encrypt</span>(key_str</span>:</span> String</span>, plaintext</span>:</span> String</span>)</span> -></span> String</span> {</span></span>
    let</span> key</span> =</span> Key</span>::</span><</span>Aes256Gcm</span>></span>::</span>from_slice</span>(key_str</span>.</span>as_bytes</span>());</span></span>
    let</span> nonce</span> =</span> Aes256Gcm</span>::</span>generate_nonce</span>(</span>&</span>mut</span> OsRng</span>);</span></span>
</span>
    let</span> cipher</span> =</span> Aes256Gcm</span>::</span>new</span>(key);</span></span>
</span>
    let</span> ciphered_data</span> =</span> cipher</span>.</span>encrypt</span>(</span>&</span>nonce, plaintext</span>.</span>as_bytes</span>())</span></span>
        .</span>expect</span>(</span>"failed to encrypt"</span>);</span></span>
</span>
    // combining nonce and encrypted data together</span></span>
    // for storage purpose</span></span>
    let mut</span> encrypted_data</span>:</span> Vec</span><</span>u8</span>></span> =</span> nonce</span>.</span>to_vec</span>();</span></span>
    encrypted_data</span>.</span>extend_from_slice</span>(</span>&</span>ciphered_data);</span></span>
</span>
    hex</span>::</span>encode</span>(encrypted_data)</span></span>
}</span></span>
</span>
fn</span> decrypt</span>(key_str</span>:</span> String</span>, encrypted_data</span>:</span> String</span>)</span> -></span> String</span> {</span></span>
    let</span> encrypted_data</span> =</span> hex</span>::</span>decode</span>(encrypted_data)</span></span>
        .</span>expect</span>(</span>"failed to decode hex string into vec"</span>);</span></span>
</span>
    let</span> key</span> =</span> Key</span>::</span><</span>Aes256Gcm</span>></span>::</span>from_slice</span>(key_str</span>.</span>as_bytes</span>());</span></span>
</span>
    let</span> (nonce_arr, ciphered_data)</span> =</span> encrypted_data</span>.</span>split_at</span>(</span>12</span>);</span></span>
    let</span> nonce</span> =</span> Nonce</span>::</span>from_slice</span>(nonce_arr);</span></span>
</span>
    let</span> cipher</span> =</span> Aes256Gcm</span>::</span>new</span>(key);</span></span>
</span>
    let</span> plaintext</span> =</span> cipher</span>.</span>decrypt</span>(nonce, ciphered_data)</span></span>
        .</span>expect</span>(</span>"failed to decrypt data"</span>);</span></span>
</span>
    String</span>::</span>from_utf8</span>(plaintext)</span></span>
        .</span>expect</span>(</span>"failed to convert vector of bytes to string"</span>)</span></span>
}</span></span></code></pre>


Python Poetry Cheat Sheet
2023-10-30T00:00:00+00:00
Poetry is an amazing tool for dependancy management in Python, it solves a lot of problems of pip. Here in this Poetry Cheat Sheet we have useful commands that we may need frequently.</p>
🔗</a>Initializing</h2>
🔗</a>New Project</h3>
poetry</span> new poetry-demo</span></span></code></pre>🔗</a>In Pre-existing Directory</h3>
cd</span> pre-existing-project</span></span>
poetry</span> init</span></span></code></pre>🔗</a>Environments</h2>
🔗</a>Displaying the environment information</h3>
poetry</span> env info</span></span></code></pre>🔗</a>Using your own virtual environment</h3>
Keep virtualenv in project root</strong></p>
More on this here</a>.</p>
This will also let VS Code discover your porject’s virtual env.</p>
poetry</span> config virtualenvs.in-project</span> true</span> --local</span></span></code></pre>
External virtual environment management</strong></p>
Poetry will detect and respect an existing virtual environment that has been externally activated. This is a powerful mechanism that is intended to be an alternative to Poetry’s built-in, simplified environment management.</p>
You can use pyenv</a> to create external virtual environment.</p>
🔗</a>Switching between environments</h3>
poetry</span> env use /full/path/to/python</span></span></code></pre>poetry</span> env use</span> 3.7</span></span></code></pre>poetry</span> env use system</span></span></code></pre>🔗</a>Listing the environments associated with the project</h3>
poetry</span> env list</span></span></code></pre>🔗</a>Deleting the environments</h3>
poetry</span> env remove /full/path/to/python</span></span>
poetry</span> env remove python3.7</span></span>
poetry</span> env remove</span> 3.7</span></span>
poetry</span> env remove test-O3eWbxRl-py3.7</span></span></code></pre>
Use the --all</code></strong> option to delete all virtual environments at once.</p>
poetry</span> env remove</span> --all</span></span></code></pre>🔗</a>Dependencies</h2>
🔗</a>Add new dependency</h3>
poetry</span> add</span> <</span>package nam</span>e</span>></span></span></code></pre>🔗</a>Removing dependency</h3>
poetry</span> remove</span> <</span>package nam</span>e</span>></span></span></code></pre>🔗</a>Installing dependencies</h3>
poetry</span> install</span></span></code></pre>🔗</a>Updating dependencies to their latest versions</h3>
This will fetch the latest matching versions (according to your pyproject.toml</code></strong> file) and update the lock file with the new versions. (This is equivalent to deleting the poetry.lock</code></strong> file and running install</code></strong> again.)</p>
poetry</span> update</span></span></code></pre>🔗</a>Adding a dependency to a group</h3>
poetry</span> add pytest</span> --group</span> test</span></span></code></pre>poetry</span> add isort</span> --group</span> dev</span></span></code></pre>🔗</a>Installing group dependencies</h3>
without</code></p>
poetry</span> install</span> --without</span> test, docs</span></span></code></pre>
with</code></p>
poetry</span> install</span> --with</span> docs</span></span></code></pre>
only</code></p>
poetry</span> install</span> --only</span> main</span></span></code></pre>🔗</a>Removing dependencies from a group</h3>
poetry</span> remove mkdocs</span> --group</span> docs</span></span></code></pre>🔗</a>Synchronizing dependencies</h3>
Poetry supports what’s called dependency synchronization. Dependency synchronization ensures that the locked dependencies in the poetry.lock</code></strong> file are the only ones present in the environment, removing anything that’s not necessary.</p>
poetry</span> install</span> --sync</span></span></code></pre>poetry</span> install</span> --without</span> dev</span> --sync</span></span>
poetry</span> install</span> --with</span> docs</span> --sync</span></span>
poetry</span> install</span> --only</span> dev</span></span></code></pre>🔗</a>Export</h2>
poetry</span> export</span> -f</span> requirements.txt</span> --output</span> requirements.txt</span></span></code></pre>


--format (-f)</code></strong>: The format to export to (default: requirements.txt</code></strong>). Currently, only constraints.txt</code></strong> and requirements.txt</code></strong> are supported.</p>
</li>

--output (-o)</code></strong>: The name of the output file. If omitted, print to standard output.</p>
</li>

--extras (-E)</code></strong>: Extra sets of dependencies to include.</p>
</li>

--without</code></strong>: The dependency groups to ignore.</p>
</li>

--with</code></strong>: The optional dependency groups to include.</p>
</li>

--only</code></strong>: The only dependency groups to include.</p>
</li>

--without-hashes</code></strong>: Exclude hashes from the exported file.</p>
</li>

--without-urls</code></strong>: Exclude source repository urls from the exported file.</p>
</li>

--with-credentials</code></strong>: Include credentials for extra indices.</p>
</li>
</ul>
🔗</a>Packaging</h2>
🔗</a>Build</h3>
poetry</span> build</span></span></code></pre>🔗</a>Publish</h3>
poetry</span> publish</span></span></code></pre>🔗</a>Configuration</h2>
You can find all the configuration options here</a>.</p>


pyenv Cheat Sheet
2023-10-22T00:00:00+00:00
Here is the pyenv cheat sheet of all the basic commands list.</p>
🔗</a>What is pyenv</h2>
pyenv is Python</a> version management tool, it lets you easily switch between multiple versions of Python.</p>
🔗</a>What it does</h2>


Lets you change the global Python version on a per-user basis.</p>
</li>

Provides support for per-project Python versions.</p>
</li>

Allows you to override the Python version with an environment variable.</p>
</li>
</ul>
🔗</a>List available python versions to install</h2>
This will list all the available version of python</p>
pyenv</span> install</span> --list</span></span></code></pre>
You can filter it using grep. Here we are filtering list by python version:</p>
pyenv</span> install</span> --list</span> |</span> grep</span> " 3\.11"</span></span></code></pre>🔗</a>Installing Python Version</h2>
pyenv</span> install</span> <</span>versio</span>n</span>></span></span></code></pre>
add -v</code> flag to see install progress output</p>
🔗</a>List Installed Python Versions</h2>
pyenv</span> versions</span></span></code></pre>
Astrick (*) indicate current active version.</p>
🔗</a>See List Of All pyenv Commands</h2>
pyenv</span> commands</span></span></code></pre>🔗</a>Activating Python Version</h2>
🔗</a>global</h3>


Sets the given version as global python version.</p>
</li>

It overrides system’s default python version.</p>
</li>
</ul>
pyenv</span> global</span> <</span>versio</span>n</span>></span></span></code></pre>🔗</a>local</h3>


Use it to set directory / project specific python version.</p>
</li>

It can override python version set by global</code></p>
</li>
</ul>
pyenv</span> local</span> <</span>versio</span>n</span>></span></span></code></pre>🔗</a>shell</h3>


Sets shell specific python version.</p>
</li>

The python version stays active until shell has been closed or by using --unset</code> flag.</p>
</li>

It can override python version set by local</code> and global</code></p>
</li>
</ul>
pyenv</span> shell</span> <</span>versio</span>n</span>></span></span></code></pre>🔗</a>Virtual Environments</h2>
🔗</a>Create Virtual Environments</h3>
pyenv</span> virtualenv</span> <</span>versio</span>n</span>> <</span>env_nam</span>e</span>></span></span></code></pre>🔗</a>Set virtual environment as default for dir/project</h3>
pyenv</span> local</span> <</span>env_nam</span>e</span>></span></span></code></pre>🔗</a>Activate Virtual Environment</h3>
pyenv</span> activate</span> <</span>env_nam</span>e</span>></span></span></code></pre>🔗</a>Deactivate Virtual Environment</h3>
pyenv</span> deactivate</span></span></code></pre>🔗</a>Recommended Reading Materials</h2>


Official Page of pyenv</a></p>
</li>

Real Python: Managing Multiple Python Versions With pyenv</a></p>
</li>
</ul>


Automate Postgres Database Backup and Upload to S3
2023-10-20T00:00:00+00:00
Database is a very critical system for an application. To prepare for any disaster we all must take regular backups of our database.</p>
In this tutorial we will create a bash script to automate postgres database backup from a docker container</a> and native postgres installation then upload to S3 based storage system.</p>
🔗</a>What we will cover</h2>


Write db backup automation script for docker and non-docker based system</p>
</li>

Upload backup file to AWS S3</p>
</li>

Setup the cron job for daily backup</p>
</li>
</ol>
🔗</a>Steps</h2>


Move to your project folder, where docker-compose.yml is.</p>
</li>

Take the database backup in container</p>
</li>

Copy the backup file from container filesystem to our desired location</p>
</li>

Delete the backup file inside the container</p>
</li>

Upload backup file to S3</p>
</li>

Delete the local backup file</p>
</li>
</ol>
Code snippet below covers step 1 to 4. But it is not optimal, later we will make modification so that we can easily automate it and make it re-usable.</p>
cd</span> project_dir</span></span>
</span>
docker</span> compose exec db_service_name sh</span> -c</span> "pg_dump -U db_user db_name > filename"</span></span>
</span>
docker</span> compose cp db_service_name:filename destination/filename</span></span>
</span>
docker</span> compose exec db_service_name rm filename</span></span></code></pre>🔗</a>Upload to S3</h2>
To upload file to S3, we need to have aws cli installed, you can follow this guide from AWS docs to install awscli</a>.</p>
After installing awscli, you need to run aws configure</code> and provide your key id and key secret. You can use any S3 based storage system here.</p>
aws s3 cp filename s3</span>:</span>//</span>your</span>-</span>bucket</span></span></code></pre>
If you are using some other S3 based storage system then you’ll have to provide custom --endpoint-url</code> argument here, make sure you read the respective docs for this.</p>
🔗</a>Full Script</h2>
We have laid out all the steps and code to accomplish that, now let’s modify in such a way that we can easily automate it and make it reusable.</p>
The filename is created based on the current datetime of the system, so that we can avoid ambiguity and exactly know how old is the backup file.</p>
🔗</a>For docker compose</h3>
#!/bin/bash</span></span>
</span>
############## ✅ Update These Values ##################</span></span>
directory</span>=</span>"your-project-dir"</span></span>
home</span>=</span>"/home/<user>"</span></span>
</span>
db_service_name</span>=</span>"db"</span></span>
db_user</span>=</span>"db_user"</span></span>
db_name</span>=</span>"db_name"</span></span>
</span>
#####################################################</span></span>
############ 🔴 CHANGE WITH CAUTION 🔴 #############</span></span>
date</span>=</span>$(</span>date</span> +"%Y-%m-%d-%H:%M:%S"</span>)</span></span>
filename</span>=</span>$db_name</span>"_"</span>$date</span>".sql"</span></span>
</span>
cd</span> $home</span></span>
cd</span> $directory</span></span>
docker</span> compose exec</span> $db_service_name</span> sh</span> -c</span> "pg_dump -U </span>$db_user $db_name</span> > </span>$filename</span>"</span></span>
</span>
docker</span> compose cp</span> $db_service_name</span>:</span>$filename $home</span>"/"</span>$filename</span></span>
docker</span> compose exec</span> $db_service_name</span> rm</span> $filename</span></span>
</span>
cd</span> $home</span></span>
#####################################################</span></span>
#####################################################</span></span>
</span>
############## ✅ CAN CHANGE BELOW ##################</span></span>
aws</span> s3 cp</span> $filename</span> s3://your-bucket</span></span>
</span>
rm</span> $filename</span></span></code></pre>
At the top, you’ve to update variable values based on your project and directory structure, and the main code will use that to backup the database.</p>
At the end of script, we have aws command to copy the backup file to S3, here you need to update bucket name and if you are using other S3 storage provider (other than AWS), then --endpoint-url</code> argument with appropriate value.</p>
🔗</a>For Native Postgres Installation</h3>
#!/bin/bash</span></span>
</span>
############## ✅ Update These Values ##################</span></span>
home</span>=</span>"/home/<user>"</span></span>
</span>
db_user</span>=</span>"db_user"</span></span>
db_name</span>=</span>"db_name"</span></span>
</span>
#####################################################</span></span>
############ 🔴 CHANGE WITH CAUTION 🔴 ##############</span></span>
date</span>=</span>$(</span>date</span> +"%Y-%m-%d-%H:%M:%S"</span>)</span></span>
filename</span>=</span>$db_name</span>"_"</span>$date</span>".sql"</span></span>
</span>
ch</span> $home</span></span>
pg_dump</span> -U</span> $db_user $db_name</span> ></span> $filename</span></span>
</span>
#####################################################</span></span>
</span>
############## ✅ CAN CHANGE BELOW ##################</span></span>
aws</span> s3 cp</span> $filename</span> s3://urlify-db-backup</span></span>
</span>
rm</span> $filename</span></span></code></pre>
When running pg_dump in the native installation, there are two ways postgres will try to authenticate the user.</p>


Peer based authentication</a>: It will check if the current user (system logged in user) is same as user passed to pg_dump.</p>
</li>

If peer auth fails then postgres looks for .pgpass</code> file in home directory for the password.</p>
</li>

If both fails then it might ask for password.</p>
</li>
</ol>
So, make sure you use 1st method and run the script on the same user as the one passed to pg_dump.</p>
🔗</a>Setup Cron Job</h2>
To automate backup and upload of our database, we will use crontab. It comes built-in with almost all the Linux distros and are very easy to setup.</p>
Give your script executable permission:</p>
chmod</span> +x db_backup.sh</span></span></code></pre>
Open crontab:</p>
crontab</span> -e</span></span></code></pre>
This will open up crontab in your prefered choice of editor. Use the below statement to run the backup script everyday at 12 PM.</p>
0 12 * * * /home/<user>/db_backup.sh >> cron.log 2>&1</span></span></code></pre>
Create cron.log file</p>
touch</span> cron.log</span></span></code></pre>
Use full path if cron.log is not in current user’s home folder.</p>

`🔗</a>Packaging</h2> 🔗</a>Build</h3> poetry</span> build</span></span></code></pre>🔗</a>Publish</h3> poetry</span> publish</span></span></code></pre>🔗</a>Configuration</h2> You can find all the configuration options here</a>.</p>`

🔗</a>Build</h3>
`poetry</span> build</span></span></code></pre>`
🔗</a>Publish</h3>
`poetry</span> publish</span></span></code></pre>`
🔗</a>Configuration</h2>
You can find all the configuration options here</a>.</p>

🔗</a>Publish</h3>
`poetry</span> publish</span></span></code></pre>`
🔗</a>Configuration</h2>
You can find all the configuration options here</a>.</p>

🔗</a>Configuration</h2>
You can find all the configuration options here</a>.</p>

🔗</a>What is pyenv</h2>
pyenv is Python</a> version management tool, it lets you easily switch between multiple versions of Python.</p>

🔗</a>Deactivate Virtual Environment</h3>
`pyenv</span> deactivate</span></span></code></pre>`
🔗</a>Recommended Reading Materials</h2>

Official Page of pyenv</a></p> </li>

Real Python: Managing Multiple Python Versions With pyenv</a></p> </li> </ul>

🔗</a>Recommended Reading Materials</h2>

Official Page of pyenv</a></p> </li>

Real Python: Managing Multiple Python Versions With pyenv</a></p> </li> </ul>

Vivek Shukla - Programming

Setup Redis TLS With Self-signed Certificate

🔗</a>Redis Connection URL</h2>
Since we want to connect to redis using TLS connection, so the protocol in the url will be `rediss</code> (with double “ss”).</p>`
`rediss://:<your_password_here>@<redis_server_ip>:6379/0#insecure</span></span></code></pre>`

How I Use Cargo Workspace to structure my Rust projects

How I use Postgres DB as Key Value Store in Rust with SQLx

Minify HTML in Axum Middleware

Using SQLite Database in Rust with Sqlx

Self Host PostgreSQL With TLS

AES Encryption in Rust

Python Poetry Cheat Sheet

pyenv Cheat Sheet

Automate Postgres Database Backup and Upload to S3

Vivek Shukla - Programming

Setup Redis TLS With Self-signed Certificate

How I Use Cargo Workspace to structure my Rust projects

🔗</a>Why use Cargo Workspace</h2> Your rust project has grown big and is hard to manage</li> Multiple people work on a project, so breaking your project into multiple crates can help with better ownership and less conflict among developers</li>

How I use Postgres DB as Key Value Store in Rust with SQLx

Minify HTML in Axum Middleware

Using SQLite Database in Rust with Sqlx

Self Host PostgreSQL With TLS

AES Encryption in Rust

Python Poetry Cheat Sheet

pyenv Cheat Sheet

🔗</a>What it does</h2> Lets you change the global Python version on a per-user basis.</p> </li> Provides support for per-project Python versions.</p> </li>

🔗</a>See List Of All pyenv Commands</h2> pyenv</span> commands</span></span></code></pre>🔗</a>Activating Python Version</h2> 🔗</a>global</h3> Sets the given version as global python version.</p> </li>

🔗</a>Activating Python Version</h2> 🔗</a>global</h3> Sets the given version as global python version.</p> </li>

🔗</a>global</h3> Sets the given version as global python version.</p> </li>

🔗</a>local</h3> Use it to set directory / project specific python version.</p> </li>

🔗</a>shell</h3> Sets shell specific python version.</p> </li> The python version stays active until shell has been closed or by using --unset</code> flag.</p> </li>

Automate Postgres Database Backup and Upload to S3