Elasticsearch - How to normalize score when combining regular query and function_score?

In Elasticsearch, when combining standard queries with function_score queries, a common challenge arises: how to balance the relative importance of the standard query and the function score? To address this, we can use a normalization method to ensure scores are reasonably distributed.

Step 1: Performing a Standard Query for Search

First, we need to define a standard query to search for documents meeting basic criteria. For example, consider searching for products containing specific keywords within a product database.

json
{
  "query": {
    "match": {
      "description": "smartphone"
    }
  }
}

Step 2: Applying function_score

Next, we use function_score to adjust the scores of these search results. This can be achieved in various ways, such as increasing the score based on certain field values (e.g., user ratings, sales, etc.).

json
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "description": "smartphone"
        }
      },
      "functions": [
        {
          "field_value_factor": {
            "field": "sales",
            "factor": 1.2,
            "modifier": "sqrt",
            "missing": 1
          }
        }
      ],
      "score_mode": "multiply"
    }
  }
}

In this example, we apply a weighted factor based on sales to the base score of each document, using a square root modifier to reduce the extreme impact of high sales on the score.

Step 3: Normalizing Scores

The most critical step is normalizing scores. Since different functions can lead to scores with widely varying ranges, we need a method to normalize these scores. Elasticsearch provides several options, such as min, max, avg, etc., but often custom scripts are required for precise control over score normalization.

json
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "description": "smartphone"
        }
      },
      "functions": [
        {
          "field_value_factor": {
            "field": "sales",
            "factor": 1.2,
            "modifier": "sqrt",
            "missing": 1
          }
        }
      ],
      "score_mode": "multiply",
      "boost_mode": "replace",
      "script_score": {
        "script": {
          "source": "Math.log(1 + _score * params.factor)",
          "params": {
            "factor": 10
          }
        }
      }
    }
  }
}

Here, we use a custom script to adjust the final score. This script takes the original score (computed by function_score) and applies a logarithmic function to reduce the impact of high scores, while adjusting the sensitivity of the score through the factor parameter.

Conclusion

This approach allows us to combine basic queries with function_score while ensuring the reasonableness and applicability of scores through normalization and custom scripts. Such queries not only filter documents based on basic matching criteria but also adjust document scores according to business requirements, achieving more refined sorting of search results.