| ▲ | Joeri 4 days ago |
| Kafka isn’t a queue, it’s a distributed log. A partitioned topic can take very large volumes of message writes, persist them indefinitely, deliver them to any subscriber in-order and at-least-once (even for subscribers added after the message was published), and do all of that distributed and HA. If you need all those things, there just are not a lot of options. |
|
| ▲ | diarrhea 3 days ago | parent | next [-] |
| Perhaps the best terse summary of Kafka I have come across yet. |
|
| ▲ | holografix 3 days ago | parent | prev | next [-] |
| Finally a valuable answer thank you |
|
| ▲ | HarHarVeryFunny 3 days ago | parent | prev | next [-] |
| Why do you say log rather than just publish and subscribe? |
| |
| ▲ | exitb 3 days ago | parent | next [-] | | Clients don’t have to subscribe to latest messages, but rather can request any available offset range. | |
| ▲ | halifaxbeard 3 days ago | parent | prev | next [-] | | the log stays on kafka for replay until your per log retention settings delete it | |
| ▲ | HarHarVeryFunny 3 days ago | parent | prev [-] | | The way people choose to use feedback on HN never fails to suprise me - we've got a generally intelligent user base here, but the most common type of feedback voting isn't because something is wrong but rather a childish "I don't like it - I want to suppress this comment". In this case it's something different - this was an honest question, and received two useful replies, so why downvote?! The mental model of people using Kafka is useful to know - in this case the published data being more log-like than stream-like since it's retained per a TTL policy, with each "subscriber" having their own controllable read index. | | |
|
|
| ▲ | edem 3 days ago | parent | prev | next [-] |
| What do you think about Temporal? |
| |
| ▲ | cyberpunk 3 days ago | parent [-] | | Okay for small numbers of high value jobs (e.g uber trips or food deliveries etc), prohibitively expensive for anything you need even a few k/sec of. |
|
|
| ▲ | mannycalavera42 3 days ago | parent | prev [-] |
| HERO |
