This article is participating in “Java Theme Month – Java Debug Notes Event”, see < Event link > for more details.

Question: How do you come up with a Java actuator with throttling/throughput control?

I am looking for a Java executor that allows me to specify throttling/throughput/speed limits, for example, that can handle no more than 100 tasks a second — if more tasks are submitted, they should be queued for later execution. The main purpose of this is to avoid restrictions when accessing foreign apis or servers.

I wonder if base Java (I doubt it, because I checked) or some other reliable place (like apachecommons) offers this feature, or if I have to write it myself. Preferably light weight. I don’t mind writing my own, but if there’s a “standard” version out there somewhere, I’d at least like to see it.

Answer a

See Guavas’ RateLimiter:

Rate limiter. Conceptually, a rate limiter distributes licenses at a configurable rate. If needed, each acquire() blocks until a license is available, and then it is acquired. Once a permit is obtained, it is not required to be issued. Rate limiters are typically used to limit the rate at which certain physical or logical resources are accessed. This contrasts with a semaphore that limits the number of concurrent accesses rather than the rate (note that concurrency and rate are closely related, for example, see Little’s law).

It is thread-safe, but still Beta. It might be worth a try.

You must wrap every call to Executor in a rate limiter. For a cleaner solution, you can create some kind of wrapper for the ExecutorService.

From the javadoc:

 final RateLimiter rateLimiter = RateLimiter.create(2.0); // rate is "2 permits per second"
  void submitTasks(List<Runnable> tasks, Executor executor) {
    for (Runnable task : tasks) {
      rateLimiter.acquire(); // may waitexecutor.execute(task); }}Copy the code

Answer two

Javaex Ecutor does not provide such a limit, only a limit on the number of threads, which is not what you want.

In general, however, the executor does not restrict the location of such operations, but should control when the thread tries to call the external server. You can do this, for example, by having the thread add a waiting limit signal before submitting the request.

Calling thread:

public void run(a) {
  // ...
  requestLimiter.acquire();
  connection.send();
  // ...
 }
Copy the code

At the same time, you can schedule threads to release acquired resources periodically (e.g., every 60 seconds) :

 public void run(a) {
  // ...
  requestLimiter.drainPermits();  // make sure not more than max are released by draining the Semaphore empty
  requestLimiter.release(MAX_NUM_REQUESTS);
  // ...
 }
Copy the code

The article translated from Stack Overflow:stackoverflow.com/questions/1…