跳到主要内容

Decompounder

decompounder filter 基于指定的词典将复合词分解为单个组件，使搜索复合术语的部分变得更容易。此过滤器对处理德语等语言特别有用，这些语言经常使用复合词。

配置​

decompounder filter 是 Milvus 中的自定义 filter。要使用它，请在 filter 配置中指定 "type": "decompounder"，并使用 word_list 参数提供词典。

PythonJavaNodeJSGocURL
analyzer_params = {
    "tokenizer": "standard",
    "filter":[{
        "type": "decompounder", # Specifies the filter type as decompounder
        "word_list": ["dampf", "schiff", "fahrt", "brot", "backen", "automat"],
    }],
}
Map<String, Object> analyzerParams = new HashMap<>();
analyzerParams.put("tokenizer", "standard");
analyzerParams.put("filter",
        Collections.singletonList(
                new HashMap<String, Object>() {{
                    put("type", "decompounder");
                    put("word_list", Arrays.asList("dampf", "schiff", "fahrt", "brot", "backen", "automat"));
                }}
        )
);
const analyzer_params = {
    "tokenizer": "standard",
    "filter":[{
        "type": "decompounder", // Specifies the filter type as decompounder
        "word_list": ["dampf", "schiff", "fahrt", "brot", "backen", "automat"],
    }],
};
analyzerParams = map[string]any{"tokenizer": "standard",
    "filter": []any{map[string]any{
        "type":       "decompounder",
        "word_list": []string{"dampf", "schiff", "fahrt", "brot", "backen", "automat"},
    }}}
# restful
analyzerParams='{
  "tokenizer": "standard",
  "filter": [
    {
      "type": "decompounder",
      "word_list": [
        "dampf",
        "schiff",
        "fahrt",
        "brot",
        "backen",
        "automat"
      ]
    }
  ]
}'

decompounder filter 接受以下可配置参数：

参数

描述

word_list

用于分解的单词列表。该 filter 使用这些单词尝试将复合词分解为其组成部分。

decompounder filter 对 tokenizer 生成的 term 进行操作，因此必须与 tokenizer 结合使用。

定义 analyzer_params 后，您可以在定义 collection schema 时将其应用于 VARCHAR field。这允许 Milvus 使用指定的 analyzer 处理该 field 中的文本，以实现高效的分词和过滤。有关详细信息，请参阅 示例使用。

示例​

在将 analyzer 配置应用到您的 collection schema 之前，使用 run_analyzer 方法验证其行为。

Analyzer 配置​

PythonJavaNodeJSGocURL
analyzer_params = {
    "tokenizer": "standard",
    "filter":[{
        "type": "decompounder", # Specifies the filter type as decompounder
        "word_list": ["dampf", "schiff", "fahrt", "brot", "backen", "automat"],
    }],
}
Map<String, Object> analyzerParams = new HashMap<>();
analyzerParams.put("tokenizer", "standard");
analyzerParams.put("filter",
        Collections.singletonList(
                new HashMap<String, Object>() {{
                    put("type", "decompounder");
                    put("word_list", Arrays.asList("dampf", "schiff", "fahrt", "brot", "backen", "automat"));
                }}
        )
);
// javascript
analyzerParams = map[string]any{"tokenizer": "standard",
    "filter": []any{map[string]any{
        "type":       "decompounder",
        "word_list": []string{"dampf", "schiff", "fahrt", "brot", "backen", "automat"},
    }}}
# restful
analyzerParams='{
  "tokenizer": "standard",
  "filter": [
    {
      "type": "decompounder",
      "word_list": [
        "dampf",
        "schiff",
        "fahrt",
        "brot",
        "backen",
        "automat"
      ]
    }
  ]
}'

使用 run_analyzer 验证​

PythonJavaNodeJSGocURL
# Sample text to analyze
sample_text = "dampfschifffahrt brotbackautomat"

# Run the standard analyzer with the defined configuration
result = MilvusClient.run_analyzer(sample_text, analyzer_params)
print(result)
// java
// javascript
// go
# restful

预期输出​

['schul', 'schulhof', 'hof']