Requirements
functional:
- allowRequest(request)
non functiona:
- low latency
- accurate
- scalable
highly availablesince we can assume that default is allow the request if the throttling service is not available.
request processing
token bucket algorithm
one of the popular algorithm for rate-limiting algorithm.
interfaces and classes
- JobSceduler: fetch rules from RuleService periodically
- RulesCache: store token bucket objects
- ClientIdentifier: identify client form the request
- RateLimiter: provide
allowRequest()api
Distributed world
as we move to distributed world, we need to share local remaining tokens to other hosts.
message boarcasting
possible approaches:
- Tell everyone everything: not scalable, message grows quadratically
- gossip communication
- distributed cache cluster
- coordination service: one host takes coordination leader role. Paxos, Raft
- random leader selection
TCP VS UDP
TCP: slow, accuracy, order guaranteed
UDP: fast, not reliable, order not guaranteed
How to integrate all this with the service?
final look
