How to boost Lucene queries with field values?

Hello,

I am curious how to boost full-text queries with document (node) values. We do get a field from our source systems that contain a weight that we shall use as multiplicator.

Lucene allows to influence the scores via boosting. This could be done via index or query time boosting. But my problem is that this shall not be done via a flat factor, but by a custom weight that is individual per document.

I have found this blog post:

In there you can find how to boost with a flat value. Unfortunately that did not help me with my issue.

When doing fulltext index queries via CALL db.index.fulltext.queryNodes(<indexName>, <luceneQuery>) YIELD node, score RETURN node.title, score
you can pass in a lucene query as second argument. For details on lucene query syntax see org.apache.lucene.queryparser.classic (Lucene 5.5.5 API).

Assume you have two properties title and description and you want to search for liar boosting results on title you could do

call db.index.fulltext.queryNodes(myindex, "title:liar^4 OR description:liar") yield node, score return node, score

Thank you Stefan, this is what I meant when: I found out how to boost it with a flat value :slight_smile:

We do get for each record an individual weight that we shall take into account. So image a document has three attributes

  • title
  • description
  • weight

Document-1 would have the following values

  • title: Not telling lies
  • description: Trust the author
  • weight: 1

Document-2 would have the following values

  • title: Another title
  • description: Some great description
  • weight: 3

I would like to perform this

call db.index.fulltext.queryNodes(myindex, "title:liar^weight OR description:liar") yield node, score return node, score

so you want the weight property being a dynamic boost factor on a per-record level?

1 Like

Exactly :slight_smile:

Lucene does support that, see org.apache.lucene.search (Lucene 5.5.5 API) and Field (Lucene 5.5.5 API)

However that is currently not used by Neo4j fulltext indexing and would require a change in Neo4j itself. Maybe create a github issue for this and reference this thread there.

1 Like

index the weight field and in query list all the weight like below
call db.index.fulltext.queryNodes(myindex, "(title:liar OR description:liar) AND (weight:3^3 OR weight:2^2) weight:1^1") yield node, score return node, score

Old question, I know...
I stumbled on this blog post, Exploring the full-text search index in Neo4j on a movies dataset, where it's written:

let’s combine the rating-boosting and the time decay effect in a single query. And just so you know you can always manipulate results later with cypher, let’s add a filter that will return only thrillers.

WITH apoc.text.join([x in range(0,10) | 
"string_release_date:" + toString((date().year — x)) + "^" +   
  toString(10-x)]," ") as time_decay
CALL db.index.fulltext.queryNodes("MovieIndex", "title:dream string_rating:[50 TO 99]^2 "+ time_decay) YIELD node, score
// filter only thrillers
MATCH (node)-[:HAS_TAG]->(:MovieTag{id:'thriller'})
RETURN node.title as title,score
LIMIT 5

It looks like the author is using a node property as a multiplier for boost.