ES Aggs count distinct group by聚合排序查询

1. kibana query hits限制了10000条

添加
“track_total_hits”: true
query:

2. 查询返回特定字段

“_source”:[“includes”:[“oid”,“seq”,“ts”]]

3. 查询默认只返回10条数据

“size”: 100

4. sort排序

5. 分页from,size

6. aggs聚合

如果aggs，fiter，sort的字段是text，则解决方法1需要写成user_id.keyword，方法2：set fielddata=true不建议此方法；

text默认分词了，并未建索引，不允许进行聚合，排序,过滤

keyword类型 默认不分词的数据类型，常常被用来过滤、排序和聚合。
text类型 在存储数据的时候会默认进行分词，不能用来过滤、排序和聚合等操作。
使用聚合查询时不能使用分词，因此字段需要设置为type = FieldType.Keyword

相当于select distinct(user_id) from xxx where user_id_type=3

GET idx_test_query/_search
{"_source": ["id","goodsName"],"query": {"term": {"user_id_type": 3}},"collapse": {"field": "user_id"},"sort": [{"_score": "desc"}],"from": 0,"size": 100
}

报错

当user_id为text时，上述语句会报错org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=illegal_argument_exception, reason=Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [idtest] in order to load field data by uninverting the inverted index. Note that this can use significant memory.]

解决办法1

{"_source": ["id","user_id"],"query": {"term": {"user_id_type": 3}},"collapse": {"field": "user_id.keyword"},"sort":{},"from": 0,"size": 100
}

解决办法2 不推荐

在[user_id]上设置fielddata=true，以便通过取消反转索引来加载字段数据
请注意，这可能会占用大量内存

POST /idx_test_query/_mapping
{"properties": {"user_id": {"type": "text","fielddata": true}}
}

https://blog.csdn.net/zhuchunyan_aijia/article/details/122901327
https://blog.csdn.net/weixin_45204847/article/details/117423652

参考

• 查询query，range，指定source
• http://www.noobyard.com/article/p-cuojtyia-bp.html
• index.max_result_window
• Elasticsearch7.x聚合查询发生异常Text fields are not optimised
• https://cloud.tencent.com/developer/article/2024062