Neo4j apoc.periodic.repeat doesn't work with streams.publish

Neo4j Graph Platform Procedures & APOC
1

Does anyone ever used Neo4j apoc.periodic.repeat function to publish message to kafka? Here is what I have now:

I have a working query to send message to my kafka topic:

CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")

However, when I used apoc repeat function:

CALL apoc.periodic.repeat('kafka-only',
    'CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")', 5)

Nothing happens..

I have checked the log/debug log and can't get any hint. Can anyone help me?

This is my configuration:

kafka.zookeeper.connect=localhost:2181
kafka.bootstrap.servers=localhost:9092
streams.procedures.enabled=true
streams.source.enabled=true
streams.source.schema.polling.interval=10000
dbms.security.procedures.unrestricted=apoc.*

Please provide the following information if you ran into a more serious issue:

  • neo4j version: neo4j-community-3.4.17
  • desktop version : MacOS 10.12.6
  • which plugins / extensions / procedures do you use : apoc-3.4.0.7-all.jar and neo4j-streams-3.4.4.jar
  • neo4j.log and debug.log : No exceptions and no logs
1 Like
2

Can you be more specific on nothing happens? You don't see the messages going to Kafka, or you don't get any results from APOC?

What happens if you call streams.publish() by itself without the APOC?

3

When I use APOC procedure then I don't see the messages getting publish to Kafka topic.
If i call streams.publish() without APOC in that case message is getting published in Kafka topic.

4

Could you try to reproduce with a 3.5.12 setup and appropriate more current apoc + streams versions?

1 Like
5

Yes, I tried to reproduce with 3.5.12 version and it has same issue.
streams.publish without apoc producer is publishing message in Kafka topic.

CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")

Below query with apoc job management procedure is not publishing message in Kafka topic.

CALL apoc.periodic.repeat('kafka-only',
    'CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")', 5)
6

Thanks @kavitakjava please try with the following jar:

I look forward to your feedback.
Thanks a lot
Andrea

1 Like
7

Hi Andrea @conker84,

The jar which you provided worked with neo4j-community-3.5.13 and i can see messages published every 5 seconds in the Kafka topic !!

The jar from Release 3.5.4 · neo4j-contrib/neo4j-streams · GitHub did not worked.

Seems like you have some fixes in ur jar.
Can this be available on github?

Your fix worked! Thank you so much for your help!

Best,
Kavita

8

Yes I fixed the bug in that jar. I'm preparing a PR so it'll be available with the next release.
Thanks a lot for pointing this out!

1 Like
9

Awesome!
I'm glad that my use case helped to brought up this issue !

Will this fix available for previous releases e.g. 3.4.17 so that we might now have to upgrade the version in production?

1 Like
10

@kavitakjava sorry for the delay, do you think it's an option upgrade Neo4j to v 3.5.13?
Please lemme know otherwise we have to find an alternative to support you.
Thanks a lot

11

Hi @conker84,

Has this issue fixed got in the latest release? We can upgrade to the latest release in prod.
Thank you!

Best,
Kavita

12

HI @kavitakjava we did a new release yesterday that contains the fix:

2 Likes
13

Awesome!!
Thank you so much @conker84

2 Likes