Table of contents
502 Bad Gateway Timeout error can be due to the fact that the server is overloaded with requests and it is not able to process them completely, which in-turn results in the timeout from the server side. This happens when you try to retrieve large amount of data from the DB without any mechanism to process the data. This is a very common issue in the software world and it can cause heavy amount of losses in terms of revenue, packets, etc.
Pagination
Let us say that one of the APIs from your application returns 502 Bad Gateway, when you try to retrieve 300k elements from a GET call. You have an option to limit the amount of data that you retrieve in a single page, and you do not need to process the whole chunk of data, but rather small chunks in several pages. This is termed as Pagination.
How does Pagination work?
There are two attributes that you use in pagination :
Limit : You specify the limit of elements that you want to see in a single page
Cursor : You specify the end of the page so that it can be used to partition the data accordingly and return only those chunks of data in a single page
In the API call, you specify these two parameters in the payload and therefore, you will get flawless execution.
How does Pagination solve the problem?
Since, you are using limit and cursor, as explained earlier, you will get small chunks of data and the server overload does not occur. This may not be a viable solution if you have insane amount of data in the Database, and limiting the amount of data might be a problem and can cause performance issues for your APIs. However, there are other solutions that will be discussed in this article, which can be used to solve the performance issues in that case.
Database Indexing
Indexing improves performance by reducing the number of data retrievals in a query. It is a mechanism that is used to quickly locate and access data from the database, therefore, solving any performance issues on the way. In above example, we had discussed the problems of performance in case of insane amount of data and most of it can be solved using the indexing mechanism.
How does DB Indexing work?
Let us say that your API is trying to fetch from a table and that causes the 502 error. Indexing is applied on the table, where indices are added as extra attribute, that consist of data references. These data references hold the actual values of the columns in the table. We will introduce the pointer mechanism, where these pointers hold the addresses of the block, and key value can be found.
How does DB indexing solve the problem?
Every time you query an API with DB Indexing code present, it uses the pointers to locate the actual data stored in the DB, so it does not actually retrieve the large chunks of data, but rather it locates the data using the indices. This solves the performance issue of retrieving insane amount of data at the same time. This is a mechanism that will definitely solve the 502 Gateway problem, and can improve the performance of your APIs
Cache Mechanism
If your database consists of insane amount of data, then you can introduce the cache mechanism that will solve the performance issue. Let us say your GET API call is returning the 502 error, you will use the POST call or a batch job, to create that insane amount of data. This POST call will first add the data to the cache and every time you try to retrieve it, you can hit the cache, rather than hitting the database.
Cache Invalidation
When you hit the cache, it will return that chunk of data, but how long does the data be stored in the cache? This is where you have a timer and cache invalidation flag that will help you in the process. This timer is set off and when the time ends, it will send the small chunks of data to the database and you can make sure it is consistent throughout the process. The flag will convey if the cache is invalidated or not, for that particular GET call that you do to retrieve the data.
This is another way to solve the 502 Bad Gateway issue. Although, this may not be an exact solution in all cases, so depending upon your use-case you can choose any of these mechanisms explained in the article and you may have to do some trial/error for your cause.
Rate-limiting
API rate-limiting ensures the stability and performance of the APIs. This is the final solution listed in this article to solve the 502 error. It is implemented by limiting how many requests can be made at a point of time.
Unlike pagination, where we set the limit for the data, here we set the limit for the number of requests in the API calls so that the GET does not fail while trying to retrieve insane amount of data. You can also avoid malicious attacks and downtime problems.
Performance of the API is key when it comes to real world scenarios. Take e-commerce companies as example, they will be looking to save on costs for managing their API interface. If there is any performance issue, they will lose millions of dollars and customers.
Size of the Requests
First and foremost, you need to monitor the traffic of your application. Before setting the request limit, you need to see how much traffic your application can handle in a small amount of time. In order to implement that, you can introduce some load testing scenarios and check the load balanced by your application for the requests it will receive from the GET call.
User Activity
Another way to monitor traffic is by keeping an eye on the user-activity. You need to come up with metrics similar to Grafana Dashboards that determines the amount of requests and response time of your APIs. This will help in setting the limit for the number of requests you can send in a short period of time.
API timeouts
You can voluntarily set the timeouts, rather than the API returning the 502 Bad Gateway error. Let me explain why these timeouts are necessary, if your application needs to be flawless, then instead of limiting the number of requests, you can set a timeout, where the API will send an indication to the user, stating that the limit has been reached and user has to wait to retrieve the remaining set of elements. This will provide good user experience and the user can be aware of how much your application can manage and process.
Conclusion
Proper use of these mechanisms completely depends on your use-case and dataset. All the solutions described in the article may not solve the 502 Bad Gateway for your requirement as it depends on how much data is there in the DB, how much is the server overloaded and if any firewall is blocking the access. Depending on these cases, you can pick and choose the correct solution. Before attempting trial/error scenarios, please go through these mechanisms in detail and pick the winner.
Image Credit
<a href="https://www.freepik.com/free-vector/memory-storage-concept-illustration_10117876.htm#fromView=search&page=1&position=30&uuid=db411ade-f727-4dba-b31a-97335ee61538">Image by storyset on Freepik</a>