Design Problems in Sink Connector

Integrations Stream Processing
#1

Im trying to use sink connectotr to sync data from mongodb.

I receive relationship event as,

"fullDocument": {
    "_id": {
      "$oid": "6112376c9ade02082f500711"
    },
    "_relatedFromId": "6112376b9ade02082f5006f6",
    "_relatedFromType": "PERSON",
    "_relatedToIds": [
      "6112376c9ade02082f500700",
      "5312475c8ade04052e300919"
    ],
    "_relationshipType": "AUTHORED",
    "_relatedToType": "BOOK"
  },

Here, I have few problems using it.

CYPHER_TEMPLATE strategy:

  • I couldn't specify the node type dynamically. Is there way to specify node type dynamically ?
  • I couldn't handle all the operations. Is there any switch case kind of logic?

CUD strategy:
Looks like I can handle multiple operations. But,

  • I need to transform the original mongo document.
  • It doesn't do node create/merge and relationship with just one event. I need to create three events two of them for creating nodes and another one for creating relationship. Is there a way to achive this just one event?
  • It accepts one relationship at time. ids just accepts one. My use case is that I will have a node will have more than a relationship in mongo document. How can I handle with one event to create a many relationships for a FROM node?
#2

Cypher

  • for dynamic node labels you can use apoc and the apoc.create.node() procedure where you can pass in the node label(s) as parameter
  • you can use WHERE filter for alternative executions in subqueries see

CUD

  • transforming the event in kafka is usually advisable
  • yes you can do three merges in one cypher strategy merge node1, merge node2, merge relationship