In the presentation I showed versions of the queries that had `LIMIT`

in them and did not use apoc.periodic.iterate only because they were nicer to show visually, but in building the graph I certainly used the periodic.iterate versions all the time, as you suspected.

The symptoms you describe sound like it is likely you are running out of memory. I know I needed to tweak memory settings to make the most of my RAM, but also the apoc.periodic.iterate settings were important to get the best performance and memory usage. I don't have records of the exact tweaking I did, but I do have a copy of the notes I took for the queries I ran:

Here are the queries relevant to building the routing graph:

```
//
// Identify (:OSMNode) instances that are intersections (connected INDIRECTLY to more than one (:OSMWayNode) and on ways or relations that are also streets.
//
MATCH (n:OSMNode)
WHERE size((n)<-[:NODE]-(:OSMWayNode)-[:NEXT]-(:OSMWayNode)) > 2
AND NOT (n:Intersection)
WITH n LIMIT 100
MATCH (n)<-[:NODE]-(wn:OSMWayNode), (wn)<-[:NEXT*0..100]-(wx),
(wx)<-[:FIRST_NODE]-(w:OSMWay)-[:TAGS]->(wt:OSMTags)
WHERE exists(wt.highway) AND NOT n:Intersection
SET n:Intersection
RETURN COUNT(*);
// Periodic iterate
CALL apoc.periodic.iterate(
'MATCH (n:OSMNode) WHERE NOT (n:Intersection)
AND size((n)<-[:NODE]-(:OSMWayNode)-[:NEXT]-(:OSMWayNode)) > 2 RETURN n',
'MATCH (n)<-[:NODE]-(wn:OSMWayNode), (wn)<-[:NEXT*0..100]-(wx),
(wx)<-[:FIRST_NODE]-(w:OSMWay)-[:TAGS]->(wt:OSMTags)
WHERE exists(wt.highway) AND NOT n:Intersection
SET n:Intersection',
{batchSize:10000, parallel:true});
MATCH (i:OSMNode) RETURN 'OSM Nodes' AS type, count(i)
UNION
MATCH (i:OSMPathNode) RETURN 'Nodes on paths' AS type, count(i)
UNION
MATCH (i:PointOfInterest) RETURN 'Points of interest' AS type, count(i)
UNION
MATCH (i:Intersection) RETURN 'Intersections' AS type, count(i);
// Produced 50k intersections in 185s for NY
// US-NE took 45 minutes to produce 789505
// San Francisco took 16s to process nodes Intersections
// San Francisco
//ββββββββββββββββββββββ€βββββββββββ
//β"type" β"count(i)"β
//ββββββββββββββββββββββͺβββββββββββ‘
//β"OSM Nodes" β2880804 β
//ββββββββββββββββββββββΌβββββββββββ€
//β"Nodes on paths" β235730 β
//ββββββββββββββββββββββΌβββββββββββ€
//β"Points of interest"β3124 β
//ββββββββββββββββββββββΌβββββββββββ€
//β"Intersections" β53744 β
//ββββββββββββββββββββββ΄βββββββββββ
//
// Find and connect intersections into routes
//
MATCH (x:Intersection) WITH x LIMIT 100
CALL spatial.osm.routeIntersection(x,true,false,false)
YIELD fromNode, toNode, fromRel, toRel, distance, length, count
WITH fromNode, toNode, fromRel, toRel, distance, length, count
MERGE (fromNode)-[r:ROUTE {fromRel:id(fromRel),toRel:id(toRel)}]->(toNode)
ON CREATE SET r.distance = distance, r.length = length, r.count = count
RETURN COUNT(*);
// With Periodic Iterate:
CALL apoc.periodic.iterate(
'MATCH (x:Intersection) RETURN x',
'CALL spatial.osm.routeIntersection(x,true,false,false)
YIELD fromNode, toNode, fromRel, toRel, distance, length, count
WITH fromNode, toNode, fromRel, toRel, distance, length, count
MERGE (fromNode)-[r:ROUTE {fromRel:id(fromRel),toRel:id(toRel)}]->(toNode)
ON CREATE SET r.distance = distance, r.length = length, r.count = count
RETURN count(*)',
{batchSize:100, parallel:false});
// San Francisco took 103s to perform 54k committed operations
// If there are errors, repeat with smaller batch size to better cope with StackOverFlow
CALL apoc.periodic.iterate(
'MATCH (x:Intersection) WHERE NOT (x)-[:ROUTE]->() RETURN x',
'CALL spatial.osm.routeIntersection(x,true,false,false)
YIELD fromNode, toNode, fromRel, toRel, distance, length, count
WITH fromNode, toNode, fromRel, toRel, distance, length, count
MERGE (fromNode)-[r:ROUTE {fromRel:id(fromRel),toRel:id(toRel)}]->(toNode)
ON CREATE SET r.distance = distance, r.length = length, r.count = count
RETURN count(*)',
{batchSize:10, parallel:false});
// Now find Routable nodes from the PointOfInterest search and link them to the route map
MATCH (x:Routable:OSMNode)
WHERE NOT (x)-[:ROUTE]->(:Intersection) WITH x LIMIT 100
CALL spatial.osm.routeIntersection(x,true,false,false)
YIELD fromNode, toNode, fromRel, toRel, distance, length, count
WITH fromNode, toNode, fromRel, toRel, distance, length, count
MERGE (fromNode)-[r:ROUTE {fromRel:id(fromRel),toRel:id(toRel)}]->(toNode)
ON CREATE SET r.distance = distance, r.length = length, r.count = count
RETURN COUNT(*);
// With periodic iterate
CALL apoc.periodic.iterate(
'MATCH (x:Routable:OSMNode)
WHERE NOT (x)-[:ROUTE]->(:Intersection) RETURN x',
'CALL spatial.osm.routeIntersection(x,true,false,false)
YIELD fromNode, toNode, fromRel, toRel, distance, length, count
WITH fromNode, toNode, fromRel, toRel, distance, length, count
MERGE (fromNode)-[r:ROUTE {fromRel:id(fromRel),toRel:id(toRel)}]->(toNode)
ON CREATE SET r.distance = distance, r.length = length, r.count = count
RETURN count(*)',
{batchSize:10, parallel:false});
// SF took 16s to do 1538 committed operations
// The algorithm makes self relationships, so delete with
MATCH (a:Intersection)-[r:ROUTE]->(a) DELETE r RETURN COUNT(*);
// SF had a 402 self relationships
// Now to get an idea of the distribution of route distances
MATCH (a:Intersection)-[r:ROUTE]->() RETURN 'All routes' AS type, COUNT(*) AS count
UNION
MATCH (a:Intersection)-[r:ROUTE]->() WHERE r.distance > 25 RETURN '>25m' AS type, COUNT(*) AS count
UNION
MATCH (a:Intersection)-[r:ROUTE]->() WHERE r.distance > 50 RETURN '>50m' AS type, COUNT(*) AS count
UNION
MATCH (a:Intersection)-[r:ROUTE]->() WHERE r.distance > 100 RETURN '>100m' AS type, COUNT(*) AS count
UNION
MATCH (a:Intersection)-[r:ROUTE]->() WHERE r.distance > 250 RETURN '>250m' AS type, COUNT(*) AS count
UNION
MATCH (a:Intersection)-[r:ROUTE]->() WHERE r.distance > 500 RETURN '>500m' AS type, COUNT(*) AS count
UNION
MATCH (a:Intersection)-[r:ROUTE]->() WHERE r.distance > 5000 RETURN '>5000m' AS type, COUNT(*) AS count;
// SF
//ββββββββββββββ€ββββββββ
//β"type" β"count"β
//ββββββββββββββͺββββββββ‘
//β"All routes"β86315 β
//ββββββββββββββΌββββββββ€
//β">25m" β55662 β
//ββββββββββββββΌββββββββ€
//β">50m" β40227 β
//ββββββββββββββΌββββββββ€
//β">100m" β18992 β
//ββββββββββββββΌββββββββ€
//β">250m" β3976 β
//ββββββββββββββΌββββββββ€
//β">500m" β1174 β
//ββββββββββββββΌββββββββ€
//β">5000m" β59 β
//ββββββββββββββ΄ββββββββ
// To improve inner-city routing we can optionally remove some of the longer ones which might be falsely detected
MATCH (a:Intersection)-[r:ROUTE]->() WHERE r.distance > 500 DELETE r RETURN COUNT(*);
```