Kafka subscriptions

[strictlyskyler]

While publishing to a kafka subscription works just fine, and allows the character set kafka allows, subscribing to them doesn't work.

Examples of topics which publish successfully, which aren't subscribed successfully:

foo.bar
foo_bar
_foo

These are all valid topics for kafka, and even work with straightup kafka-node, the dependent library used by ascoltatori. Ascoltatori fails silently when subscribing to such topics, however.

[mcollina]

Let's ask @fullergalway, who originally contributed that piece.

Ascoltatori is messing with the topics. What it should not do is failing silently.

Would you like to send a PR?

[fullergalway]

Hi,

Kafka doesn't allow "/" in topics, so the ascoltatori uses "_" as separator. Following the ascoltatori pattern you should be able to pub/sub succesfully to topics foo/bar, /foo, and /foo/bar. In Kafka these will show as topics foo_bar, _foo, and _foo_bar.

Kakfa won't let you have both topics foo.bar and foo_bar (at least my kafka v0.9.0.1 won't). I already have topic foo.bar, and here's what happens:

kafka/bin/kafka-topics.sh --zookeeper localhost --topic foo_bar --create --partitions 1 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Error while executing topic command : Topic "foo_bar" collides with existing topics: foo.bar

I hope this helps.

[flaviostutz]

"_" is a common character in mqtt topic names (snake case alert!)... should we change this character to something like ".", which is permitted?