Micro-caching with Nginx and Memcached
Introduction
Our web site had performance issues in the Christmas season. Our web servers used a lot of CPU so we had to add a new node to process the request to Web, API, Backend hosts. The overflow was caused by the integration of new partners that has been using heavily our API and the huge amount of orders in our database.
We can improve the code, or course. But there are other options with some extra benefits.
Resume
In this document I am going to test how much the performance of the app improvements using caching on the second of the resource heavier in the app: /api/v1/places
. The first places is to /orders. I am going to test two way to cache the response: nginx disk cache and memcached.
Theory
On the one hand there is Memcached that is an external service to the web server so that it could have latency, and as is very knowing one is vary expensive in money and time. However, in this testing the memcached service run in the same machine, so the latency is present but little.
On other hand, has the nginx’s disk cache and it is well know the access to disk is always more expensive than access to memory. This could be a critical point to choose memcached instead nginx. However if nginx is running on a Linux OS these files are loaded in memory for fast access. Therefore, nginx’s disk cache should be faster than memcached.
Specifications
The tests are going run locally with the app in production mode. The local machine has a core i5 with 8gb in ram and a SSD. Also, it has the same services (mysql, memcached and nginx) and database that production (in the time to run this tests).
The software of testing will be Apache Benchmark. The command and its configuration is the following
ab -t 30 -c 10 https://api.local.clickbus.com.mx/api/v1/places
Some issues before the tests. The implementation was straightforward to nginx’s disc caching. However to memcached was necessary research to enabled. The issue was that it requires the application saves the response. Then, an extra work is required to use memcached as cache of responses of nginx.
Results
Cached system | Response by seconds (mean) | Improvement (increase/original*100) % |
---|---|---|
Without | 1.68 | 0 |
Nginx | 355 | 15,078% |
Memcached | 151 | 9,363% |
It is clear that nginx’s disc cache has the better performance and we expected memcached improves the performance however the latency was a critical point, though, the tests and all the services involved are in the same machine, in production where the memcached service and nginx are in different machines the lantecy will be more big.
Conclusion
Using micro-caching improves the performance in a brutal way, we achieved process over 150 request without matter that cache we used. However, the nginx’s cache gives us more performance without touch the code of the app and if a new response has to be cached then only a new rule has to be added to nginx instead of wait a release and deploys the changes of the app to save the response.
References
[1] Facebook Cookbook: Building Applications to Grow Your Facebook Empire, October 2008 Jay Goldman, chapter Advanced Caching with Nginx and memcached [2] https://www.nginx.com/blog/benefits-of-microcaching-nginx/ 2016-12 [3] http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_path 2016-12
Appendix
Wihtout any cache
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking api.local.clickbus.com.mx (be patient)
Finished 54 requests
Server Software: nginx/1.10.0
Server Hostname: api.local.clickbus.com.mx
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /api/v1/places
Document Length: 672864 bytes
Concurrency Level: 10
Time taken for tests: 32.090 seconds
Complete requests: 54
Failed requests: 0
Total transferred: 36364194 bytes
HTML transferred: 36334656 bytes
Requests per second: 1.68 [#/sec] (mean)
Time per request: 5942.524 [ms] (mean)
Time per request: 594.252 [ms] (mean, across all concurrent requests)
Transfer rate: 1106.65 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 3 10 12.2 4 37
Processing: 1750 5013 1255.5 4740 8661
Waiting: 1748 5010 1254.7 4739 8659
Total: 1784 5023 1252.4 4744 8665
Percentage of the requests served within a certain time (ms)
50% 4744
66% 5101
75% 5299
80% 5853
90% 6768
95% 7616
98% 7620
99% 8665
100% 8665 (longest request)
Using NGINX cache
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking api.local.clickbus.com.mx (be patient)
Completed 5000 requests
Completed 10000 requests
Finished 10655 requests
Server Software: nginx/1.10.0
Server Hostname: api.local.clickbus.com.mx
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /api/v1/places
Document Length: 672864 bytes
Concurrency Level: 10
Time taken for tests: 30.003 seconds
Complete requests: 10655
Failed requests: 0
Total transferred: 7179313708 bytes
HTML transferred: 7173981708 bytes
Requests per second: 355.14 [#/sec] (mean)
Time per request: 28.158 [ms] (mean)
Time per request: 2.816 [ms] (mean, across all concurrent requests)
Transfer rate: 233681.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 3 21 7.3 24 39
Processing: 3 7 4.4 4 32
Waiting: 0 3 1.5 3 25
Total: 6 28 6.0 29 64
Percentage of the requests served within a certain time (ms)
50% 29
66% 29
75% 30
80% 31
90% 36
95% 37
98% 41
99% 44
100% 64 (longest request)
Using MEMCACHED
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking api.local.clickbus.com.mx (be patient)
Finished 4547 requests
Server Software: nginx/1.10.0
Server Hostname: api.local.clickbus.com.mx
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /api/v1/places
Document Length: 672864 bytes
Concurrency Level: 10
Time taken for tests: 30.012 seconds
Complete requests: 4547
Failed requests: 0
Total transferred: 3062114512 bytes
HTML transferred: 3061077112 bytes
Requests per second: 151.51 [#/sec] (mean)
Time per request: 66.004 [ms] (mean)
Time per request: 6.600 [ms] (mean, across all concurrent requests)
Transfer rate: 99637.96 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 24 17.5 22 135
Processing: 10 42 19.6 38 151
Waiting: 0 22 18.7 21 132
Total: 14 66 25.4 64 200
Percentage of the requests served within a certain time (ms)
50% 64
66% 74
75% 81
80% 85
90% 99
95% 110
98% 127
99% 137
100% 200 (longest request)