Kafka vs REST
Some notes on the Kafka (or event driven architecture in general) vs REST.
Kafka
Good:
- Decoupling of logic between applications. You just publish what you do, instead of having to explicitly call or ask other services to do things.
- Asynchronous
- Requests are not lost if there is a spike and you can’t handle it at that moment, Kafka acts as a buffer / storage and these requests can be processed later when more resources are available
- Has persistence (can set the retention period)
- Easier to test a new MVP, you can just start a consumer locally and consume from the necessary topics, you don’t need some other application to call you
Bad:
- You can only be asynchronous, can’t be synchronous if you want to
- As a result of the above, you can’t do error handling. Sometimes we need to explicitly handle errors in the previous application, e.g. an application A that calls a data layer that writes to database - sometimes you need to know if the write succeeded or failed with a certain error, or tweak retry configurations based on database write failures - this is application logic that belongs in application A.
REST
Good:
- Can be tweaked to be synchronous, partially synchronous, or asynchronous (synchronous by default though, see below)
- Instant feedback, can be easily tested (e.g. with Postman), don’t need to build another helper test tool to see whether kafka message was processed, and whether the entry was written to database
Bad:
- Synchronous by default, needs extra work or implement polling to have asynchronous behaviour.
- Can have problems if server cannot handle the amount of requests at a point in time. But this can be mitigated if the infrastructure has autoscaling in place
- Tight coupling between microservices