We are experiencing unresponsiveness issue while loading data into neo4j for complete delta cycle, mostly on the step of node deletion and relationship creation.
Our Environment Configuration can be described as:
GCP Compute Engine Machine 16vCPU 64GB RAM
Operating System Ubuntu 18.04LTS
Neo4J Community edition.
Node in Neo4j: 500,000,000
The Neo4J Database just go into infinite state of processing for that particular deletion or relationship creation query. Metrics indicates 40% RAM usage but CPU shoots upto 80%.
If the compute engine instance is restarted and job is submitted again, it works like a charm. No, simultaneous queries were executed during this scenario. Before every data load, we are clearing database cache but this does not solves the problem.
Is there a way I can configure my database where it's response is consistent and Query response time always remain same. If it takes 5 minutes to execute, it must take 5 minutes every time. Caching can speed up the things but I don't understand how restarting VM instance or database engine speeds up the process. Is it due to any kind of deadlocks ?
Can anyone help in this case.