Ratelimiter4s : A functional rate limiting library

Photo by Ksenia Kudelkina on Unsplash

Rate limiting is a common feature that come across these days. With the upsurge of cloud based api’s , rate limiting has become a de-facto standard of protecting the resources and controlling requests.

What is rate limiting ?

Rate limiting is used to control the amount of incoming and outgoing traffic to or from a network. For example, let’s say you are using a particular service’s API that is configured to allow 100 requests/minute. If the number of requests you make exceeds that limit, then an error will be triggered. The reasoning behind implementing rate limits is to allow for a better flow of data and to increase security by mitigating attacks such as DDoS.

There are some pretty good libraries out there that provide some kind of support to add rate limiting capabilities to your API’s. Resilience4j is one such library which in inspired from Netflix Hystrix and supports Java8.

I was inspired from Resilience4j to write a rate limiting api in Scala. However the motive was not to built the rate limiting capabilities from scratch. That would be reinventing the wheel. The motivation was to support Scala based effect while providing rate limiting capabilities.

ratelimter4s is lightweight library i came up with. It is designed for Scala and provides wrappers to enhance any Function with rate limiting capabilities. The wrappers are available in 3 flavours

• FRateLimiter : Rate limited method returns a Either type
• CatsRateLimiter : Rate limited method returns a EitherT type
• ZIORateLimiter : Rate limited method returns a Task type

Lets check out FRateLimiter in action :

Consider a public service which takes a character name and returns an Artist.

trait ArtistService {
  def getArtist(character: String): Artist 
}

Rules dictate that calling rate for getArtist to be not higher than 3 requests/second.We can define a RateLimiter config defined like this

val threePerSecondConfig: RateLimiterConfig = RateLimiterConfig.custom()
                      .limitRefreshPeriod(Duration.ofSeconds(1))
                      .limitForPeriod(3)
                      .timeoutDuration(Duration.ofMillis(25))
                      .build()
val threePerSecondLimiter = RateLimiter.of("3/second", threePerSecondConfig)

The details of the limiter config are described in detail here

We can use this config and add rate-limiting capability to our service. We would need to import the FRateLimiter class that provides the method limit.

import ratelimiter4s.core.FRateLimiter._

val rateLimitedService = limit(getArtist _, threePerSecondLimiter)

Lets checkout a rate limited service in action

//first attempt
rateLimitedService("Sheldon") == Right(Artist("Jim Parsons"))
rateLimitedService("Penny")   == Right(Artist("Kaley Cuoco"))
rateLimitedService("Leonard") == Right(Artist("Johnny Galecki"))
//fourth attempt will fail with a  RequestNotPermitted 
rateLimitedService("Kripke") == Left(RequestNotPermitted)

And that is it! We have achieved the objective of limiting the requests to 3/second.

Pure and Impure

A pure scala function is side-effect free, plus the result does not depend on anything other than its inputs. Example :

def greeter(guest: String) : String = s"Hello $guest"

FunctionNLimiter provides a pure apply for every rate-limited function. The return type of pure is Either[RequestNotPermitted,A].

def greeter(guest: String) : String = s"Hello $guest"

Decorating greeter with a rate limiter

val rateLimitedGreeter = limit(getArtist _, rateLimiter)

We can call pure on the rate limited method.

val result: Either[RequestNotPermitted, String] = rateLimitedGreeter.pure("World")

Any exceptions thrown by the original greeter method are unhandled. So any instances of Left can only have a RequestNotPermitted type inside.

However many useful methods that we would like to rate limit interact with outside world, mutate state and would not qualify as pure functions. Such a rate limited method could throw exceptions which are caused by the actual business logic.

Consider the following example as a tweak to the original greeter.

def greeter(guest: String) : String = {
  
  if(!guestList.contains(guest)) throw new UninvitedGuestException(guest)
  
  s"Hello $guest"
}

Decorating the impure greeter with a rate limiter would remain the same.

val rateLimitedGreeter = limit(getArtist _, rateLimiter)

We should call an apply on the rate limited method instead of pure. This means that we are also expecting greeterto fail for reasons other than rate limitations.

val result: Either[Throwable, String] = rateLimitedGreeter("World")

The Left is of Throwable type in this case which is self-explanatory.

ratelimiter4s also has support for creating cats-io and zio effect types which are enhancements on the top of the basic scala Either effect type. Detailed documentation about its usage can be read on the github site.

I would be more than happy to receive feedback on the library and what other features can be supported.

Happy coding!