This week I am totally ripping off an email that Nimesh Chakravarthi sent out to his team about an operational win. I mean, just shameless plagiarism. So… here it is in its mostly-unadulterated form (with minor edits for formatting and to include attribution).
I wanted to send out a report on a change that Lenny Dong made and also demoed last Friday because we have seen some outstanding results from it.
The change he made optimizes on a nasty request pattern in voyager-api when it communicates with the service responsible for providing information about member restrictions. We were seeing a giant fanout of hundreds of these requests during some calls to the conversation list. Lenny managed to batch these into one single request, and we saw the following improvements:
~90% decrease in downstream traffic to filterRestrictedMembers, which is a ~60k decrease at peak (inGraph):
Close to a 100ms decrease in p95 latency of the conversation list (inGraph). 100% ramp was at 2:56pm on 2/4:
These types of changes highlight the amazing craftsmanship that our team is capable of. Huge shout out to Lenny for getting this out! Can’t wait to see more improvements like this!