Following up on a problem I posted on the Slack spatial channel on 24th of August. I’m trying to remove some old legacy spatial indexes that I’m no longer using. I’ve tried using the neo4j-shell, but they don’t remove. @craig.taverner suggested that I try deleting them from java using the index manager.
I’ve tried that, but they still don’t delete. The lucine indexes are definitely created by neo4j-spatial, not SDN, as was suggested before. The lucene indexes delete, but the spatial indexes don’t.
def uplift_remove_unused_indexes() {
def expected = i(GenericGraphService).fetchAll(FeatureSet).spatialIdx
def manager = i(GraphDatabaseService).index()
def i = manager.nodeIndexNames()
i.each { idx ->
if (!expected.find { idx.startsWith(it) }) {
println "Deleting index $idx"
def index = manager.forNodes(idx)
index.delete()
}
}
}
Running it a few times you can see that the indexes are still registered. The lucene indexes are deleted ok, but the spatial indexes backing them are not. (After the second attempt to delete them, the lucine indexes are automatically recreated by spatial)
DB UPLIFT 106: TDB-716/2018-08-20: Remove old indexes
Deleting index wpd
Deleting index locations
Deleting index spatial__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index landreg__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index blpu__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index locations__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index test
Deleting index junctionLocations__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index property__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index blpu2
Deleting index features__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index routes__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index junctionLocations
Deleting index spatial
Deleting index wpd__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index blpu
Deleting index spatial2__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index routeSpatial__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index test__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index network
Deleting index spatial2
Deleting index addressbase
Deleting index features
Deleting index property
Deleting index routes
Deleting index gps
Deleting index postcode__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index nga__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index topography__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index nga
Deleting index postcode
Deleting index routeSpatial
Deleting index network__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index landreg
Deleting index topography
Deleting index blpu2__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index addressbase__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index gps__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
DB UPLIFT 107: TDB-716/2018-08-20: Remove old indexes
Deleting index wpd
Deleting index locations
Deleting index test
Deleting index blpu2
Deleting index junctionLocations
Deleting index spatial
Deleting index blpu
Deleting index network
Deleting index spatial2
Deleting index addressbase
Deleting index features
Deleting index property
Deleting index routes
Deleting index gps
Deleting index nga
Deleting index postcode
Deleting index routeSpatial
Deleting index landreg
Deleting index topography
DB UPLIFT 108: TDB-716/2018-08-20: Remove old indexes
Deleting index wpd
Deleting index locations
Deleting index spatial__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index landreg__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index blpu__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index locations__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index test
Deleting index property__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index junctionLocations__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index blpu2
Deleting index features__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index routes__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index junctionLocations
Deleting index spatial
Deleting index wpd__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index blpu
Deleting index routeSpatial__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index spatial2__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index test__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index network
Deleting index spatial2
Deleting index addressbase
Deleting index features
Deleting index property
Deleting index routes
Deleting index gps
Deleting index postcode__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index nga__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index topography__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index nga
Deleting index postcode
Deleting index routeSpatial
Deleting index network__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index blpu2__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index landreg
Deleting index topography
Deleting index addressbase__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Deleting index gps__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
This is with neo4j-spatial (0.15-neo4j-2.3.4)
Perhaps there was a bug regarding the deletion of indexes in that version?
neo4j-sh (?)$ index --indexes
Node indexes:
wpd
locations
bng_landreg__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
bng_spatial__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
test
bng_mastermap
bng_postcode
blpu2
junctionLocations
spatial
bng_blpu__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
blpu
bng_postcode__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
bng_network__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
bng_spatial
bng_blpu
network
bng_nga__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
spatial2
addressbase
bng_mastermap__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
features
property
routes
gps
nga__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
geotile__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
nga
bng_nga
postcode
routeSpatial
bng_wpd
landreg
topography
bng_landreg
bng_network
geotile
bng_wpd__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Relationship indexes:
__rel_types__
neo4j-sh (?)$ index --get-config wpd
{
"provider": "spatial",
"wkt": "wkt"
}
neo4j-sh (?)$ index --delete wpd
neo4j-sh (?)$ index --indexes
Node indexes:
wpd
locations
bng_landreg__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
bng_spatial__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
test
bng_mastermap
bng_postcode
blpu2
junctionLocations
spatial
wpd__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
bng_blpu__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
blpu
bng_postcode__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
bng_network__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
bng_spatial
bng_blpu
network
bng_nga__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
spatial2
addressbase
bng_mastermap__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
features
property
routes
gps
nga__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
geotile__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
nga
bng_nga
postcode
routeSpatial
bng_wpd
landreg
topography
bng_landreg
bng_network
geotile
bng_wpd__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__
Relationship indexes:
__rel_types__
neo4j-sh (?)$
The spatial plugin definitely appears to have code to delete indexes:
SpatialDatabaseService:
public class LayerNodeIndex implements Index<Node> {
[cut]
public void delete() {
this.spatialDB.deleteLayer(this.layer.getName(), new NullListener());
}
}
Maybe it’s buggy.
Forgetting the lucine indexes for a second, the legacy indexes are just standard neo4j indexes with a spatial provider. As far as I know spatial doesn’t have any special state files, it just used layer nodes attached to the ReferenceNode (in this version). So it should be possible to just delete the indexes through Neo4j and cleanup the nodes by hand later.
So, I’m unsure as to why deleting the indexes doesn’t work.
If you’ve got any thoughts they would be gratefully received .