Building IPO System at Paytm Money7 min read

Building IPO System at Paytm Money7 min read

Paytm Money has been a leader and pioneer in providing access to ease-of-use investment products to help its customers learn, invest and achieve their financial goals. In this journey, Paytm Money launched a platform to make IPO investment simple, transparent and reliable. IPO as a feature is available on Paytm Money Web, Android and iOS applications as well as on the Paytm App.

What is an IPO?

In simple terms, an IPO or Initial Public Offering can be defined as the process where a private company decides to go public by getting itself listed on the stock exchanges like BSE or NSE. 

IPO helps the company to raise capital through issue of new shares in the primary market. 

IPO can be bought during the issue dates defined by exchange and later on a given listing day. The company gets listed on the stock exchange and becomes a publicly-traded company and its shares can be traded in the open market.

Now, let us dive into the design considerations and challenges faced when we engineered this system from scratch. It took several iterations for us to come up with the following architecture due to constraints from the Exchange legacy systems.

Architecture

Core components & their responsibilities

1. User Facing Servers: These  servers are responsible for hosting a set of user facing CRUD APIs which help users to navigate through different types of IPOs (current / past / upcoming / announced), View IPO details, Apply for IPO, Edit IPO, View IPO applications and their details.
2. Internal server: This is the heart of the IPO system. It deals with the exchange API integration, processing of the IPO applications submitted by users, submitting applications to the exchange with constant TPS, updating IPO application status and details, handling fallback flows for any exchange downtime through offline file submission flow, CRUD APIs for IPO details, callback request processing and facilitating application sync with exchange. 

• API Gateway: API Gateway acts as an API manager which is the entry point for callback requests arriving from the exchange and handles routing of the requests to the Lambda service.
• AWS Lambda: AWS Lambda  functions as an authorizer for the API gateway to control access to the APIs. The Lambda function authenticates the request using the auth token present in the request header.
• AWS SQS:  Once the request is authenticated, depending on the type of the payload 

(payment or DP verification) json message is sent to their respective FIFO queues which is then processed in the internal server to update the application status of IPO applications.

• MySQL DB ( master-reader ):  MySQL DB is used to store the IPO related data and with master-reader setup. All the read heavy requests are served from the reader and the write transactions are handled by the master DB and replication keeps them in sync. 
• Redis: Redis here is majorly used as a distributed lock facilitator with a short TTL period. It is looked up by each system which decides to update the IPO application and after the transaction is committed to DB, the lock is released or gets removed automatically  after TTL expiry.

Technical Challenges & Solutions Adopted

• Developing High Throughput user facing APIs

Being a read-heavy application, using MySQL DB with master-replica configuration proved to be a good choice to easily scale, stabilize the system and distribute read and write requests. User facing APIs have been developed to provide low latency and high throughput by using in memory caching techniques, writing optimal MySQL queries  & serving read requests from reader DB helped us to serve the requests in ~5ms response time for most of the APIs. If we face any failure in the system while updating cache or a cache miss, we can quickly change the properties file to start pulling data from DB for the APIs.

• Managing Submission & Processing of IPO applications-

In order to make the system faster and loosely coupled, we decided to make the entire sub system processing asynchronous. It gave us the power to divide different tasks, manage the load and scale the systems independently and efficiently.  Asynchronous processing of IPO applications helps us to support our IPO platform’s best features such as Pre Open Application and AMO Application. 

User facing servers can be scaled horizontally to support increasing traffic. Internal Server also, can be scaled vertically on demand and is a fully configurable server which can be controlled through an internal application by the operations team with minimum config change. 

• Pre Open IPO Applications : Users can apply for the IPO before IPO start Date and we will submit on exchange on IPO start date. 
• AMO Application Users can apply after IPO closes for the day ( 10 am – 5 Pm), Exchange is not accepting IPO applications after IPO close time. We allow users to apply. We take it as an AMO Application and submit it on the next IPO day. 
• Concurrent IPO Applications Update Problem

Entire IPO system revolves around creating and updating IPO applications and thus applications are bound to be updated by multiple sub systems. To avoid concurrent updation issues , we managed to implement distributed locking using Redis and this locking mechanism helps us to acquire locks before each update to the application by different modules. Lock has a short TTL period to avoid locking for an indefinite amount of time.

• Exchange TPS Limit

Exchange supports X number of transactions per second (TPS) (TPS). The challenge was to respect the TPS limit. We came up with a solution to maintain  a counter based on current day second in our system using locks. If the current day second is a new second, its default value will be zero and calling the exchange API will increment that counter. If Counter reaches the X number, the next API call will wait till the next second starts.

• Exchange Callback Integration

One of the major  challenges in this integration has been that there is no SLA for callback response from the exchange. We were unsure of the number of  requests and the  time when they could arrive. So, to tackle this problem we chose AWS API Gateway, Lambda and SQS. We need these only during IPO duration plus n days so they seem to be a cost effective choice. Additionally, they scale really well and provide on demand compute power. API gateway was the best choice for us to provide our public endpoint to exchange to make callback requests. It also helps us to manage our downstream AWS resources from being targeted by a DDoS attack. While Lambda empowered us with the easy setup of request authentication, we also used SQS as it adds resilience to message delivery and also throttles the rate of messages to the consumer, helping smooth out traffic bursts. 

1. Delay in Callbacks & Application Status Updates

Callbacks from the exchanges could take a few minutes to few hours, few days or no callbacks and this made us think of a solution for making  the application status update faster for our customers who have invested with us. The solution was to build a cron process which could sync the status of the application at exchange with the one in our DB. This synchronization is done through making an API call for the applications with specific statuses  and fetching the payment or DP verification status updates ( which usually come from the callbacks) and updating them in DB. This led to lower customer escalations regarding slower status updates and customers raising questions on mandate approval notifications and other application queries.

• Fallback mechanisms in case of exchange downtime

There is a chance that the exchange APIs are down. To handle this situation, we have built a  capability  to use the exchange’s  legacy flow. ie. 

1. Generate and upload application files in batches  to exchange. 
2. Download response files from exchange and upload that in our system.

Some Interesting Insights –

With this system, we were able to contribute significantly to the growth of Paytm Money. Here are few observations since its launch:

1. We were first in the industry to support “Pre-Open” and AMO IPO Applications, where you can submit applications before the IPO opens.
2. Zomato IPO was the first IPO with Pre Open feature and it garnered around 37K applications before IPO start date.
3. We received around 56K applications for Paytm IPO and close to 59K for Zomato and the number of successful applications are only increasing with every IPO depending on the company.
4. We have so far processed around 1.3 Mil applications successfully.
5. Processed around 6 Mil callback requests so far.
6. On an average IPO Apply Time is 20 ms. 

We hope after reading this, you might have got a better understanding of how we at Paytm Money built our IPO system and designed it to be robust and reliable to ensure our customers have a seamless experience. 

Wondering how you can apply to IPO? Check out our blog here – https://www.paytmmoney.com/blog/apply-for-ipo/

Team Members: Ganesh Kumar, Rahul Chudasama, V Ramya, Barnik Banerjee, Aman Yadav

Blog Contribution: Rahul Chudasama, V Ramya