Close

ZeroMQ notes

Write down any known technical requirements in terms of throughput, latency, reliability, and so on.

OS can handle 4-5K threads

The client waits for the server to reply with state, and meanwhile queues all updates. It does this simply by not reading them: ZeroMQ keeps them queued on the socket queue.

When someone asks for “reliability”, ask them to list the failures they want to handle

“credit-based flow control” effectively removes the need for high-water marks, and any risk of memory overflow

SOCKETS:
REQ
– adds empty frame in front of a new message
– REQ socket in ZeroMQ cannot be really used in real-world environments, as they get stuck if message is lost due to service failure or similar. Users have to use XREQ instead and implement the request re-trying themselves. With nanomsg, the re-try functionality is built into REQ socket
REP
DEALER
– PUSH and DEALER rotate messages to “one of many”
– PUSH and DEALER block if there is no peer ready to receive a message
– keeps messages in internal queue
ROUTER
– needs the address of the peer before it can send it a message
– keeps conterparty’s info
– discards messages for unknown peers
– ZMQ_HWM option action: Drop
DEALER and ROUTER sockets are fully asynchronous
SUB
– cannot talk back to PUB
PUB
– sends each message to “all of many”
XSUB
XPUB
– does not report duplicate subscriptions
PULL
PUSH
– PUSH and DEALER rotate messages to “one of many”
– PUSH and DEALER block if there is no peer ready to receive a message
– keeps messages in internal queue
PAIR
– does not reconnect if the peer disappears and comes back

PUB-SUB
– The ZeroMQ pub-sub pattern will lose messages arbitrarily when a subscriber is connecting, when a network failure occurs, or just if the subscriber or network can’t keep up with the publisher.

ZMQ patterns:

PUB-SUB:
Espresso – proxy between subs and pubs; may cache last value
Suicidal Snail – subscriber kills itself if recv message timestamp differnece with the current timestamp is greater than some reasonable value
Black Box – high speed subscriber that distributes messages to workers (multithreaded implementation)
Clone – replicate some state to every client via single (double) server