概述

继续跟中华石杉老师学习ES，第28篇

课程地址： /view/55

之前大家会发现，我们全部是用英文在学习，其实我们更习惯用中文做搜索。

英文默认的standard分词器没有办法对中文进行合理分词的，只是将每个中文字符一个一个的切割开来，比如说中国人 --> 中国人

在搜索引擎领域，比较成熟和流行的，就是ik分词器

举个简单的例子：中国人很喜欢吃油条,不全，但是意思已经表达的很清楚了。

standard：中国人很喜欢吃油条ik：中国人很喜欢吃油条

Github

/medcl/elasticsearch-analysis-ik

IK安装

根据github上的指导

第一种安装方法：

访问 /medcl/elasticsearch-analysis-ik/releases ，找到对应版本的ik插件安装包

在es安装目录 plugins 下新建目录 ik

将 zip包解压到 ik目录下 ,我这里用的是6.4.1 ，所以我下载对应的版本的ik

重启es ，观察启动日志loaded plugin [analysis-ik]

ik分词器基础知识

ik有两种analyzer，可根据自己的需要自己选择，但是一般是选用ik_max_word

ik_max_word

ik_max_word: 会将文本做最细粒度的拆分，比如会将“中华人民共和国国歌”拆分为“中华人民共和国,中华人民,中华,华人,人民共和国,人民,人,民,共和国,共和,和,国国,国歌”，会穷尽各种可能的组合；

ik_smart

ik_smart: 会做最粗粒度的拆分，比如会将“中华人民共和国国歌”拆分为“中华人民共和国,国歌”。比如这个时候搜索“共和国” --> 中华人民共和国和国歌，搜到吗？？？？显然不能搜索的到。根据场景合理选择。

验证ik分词器

新建个索引

PUT /artisan {"mappings": {"my_type": {"properties": {"text": {"type": "text","analyzer": "ik_max_word"}}}}}

查看分词

GET /artisan/_analyze{"text": "小工匠跟中华石杉老师学习分布式搜索引擎elasticsearch","analyzer": "ik_max_word"}

{"tokens": [{"token": "小工","start_offset": 0,"end_offset": 2,"type": "CN_WORD","position": 0},{"token": "工匠","start_offset": 1,"end_offset": 3,"type": "CN_WORD","position": 1},{"token": "跟","start_offset": 3,"end_offset": 4,"type": "CN_CHAR","position": 2},{"token": "中华","start_offset": 4,"end_offset": 6,"type": "CN_WORD","position": 3},{"token": "石","start_offset": 6,"end_offset": 7,"type": "CN_CHAR","position": 4},{"token": "杉","start_offset": 7,"end_offset": 8,"type": "CN_CHAR","position": 5},{"token": "老师","start_offset": 8,"end_offset": 10,"type": "CN_WORD","position": 6},{"token": "学习","start_offset": 10,"end_offset": 12,"type": "CN_WORD","position": 7},{"token": "分布式","start_offset": 12,"end_offset": 15,"type": "CN_WORD","position": 8},{"token": "分布","start_offset": 12,"end_offset": 14,"type": "CN_WORD","position": 9},{"token": "式","start_offset": 14,"end_offset": 15,"type": "CN_CHAR","position": 10},{"token": "搜索引擎","start_offset": 15,"end_offset": 19,"type": "CN_WORD","position": 11},{"token": "搜索","start_offset": 15,"end_offset": 17,"type": "CN_WORD","position": 12},{"token": "索引","start_offset": 16,"end_offset": 18,"type": "CN_WORD","position": 13},{"token": "引擎","start_offset": 17,"end_offset": 19,"type": "CN_WORD","position": 14},{"token": "elasticsearch","start_offset": 19,"end_offset": 32,"type": "ENGLISH","position": 15}]}

说明ik分词器安装成功。

示例

造点数据

POST /artisan/my_type/_bulk{"index": {"_id": "1"} }{"text": "男子偷上万元发红包求交女友被抓获时仍然单身" }{"index": {"_id": "2"} }{"text": "16岁少女为结婚“变”22岁 7年后想离婚被法院拒绝" }{"index": {"_id": "3"} }{"text": "深圳女孩骑车逆行撞奔驰遭索赔被吓哭(图)" }{"index": {"_id": "4"} }{"text": "女人对护肤品比对男票好？网友神怼" }{"index": {"_id": "5"} }{"text": "为什么国内的街道招牌用的都是红黄配？" }

查询

GET /artisan/my_type/_search {"query": {"match": {"text": "16岁少女结婚好还是单身好？"}}}

{"took": 50,"timed_out": false,"_shards": {"total": 5,"successful": 5,"skipped": 0,"failed": 0},"hits": {"total": 3,"max_score": 2.8514345,"hits": [{"_index": "artisan","_type": "my_type","_id": "2","_score": 2.8514345,"_source": {"text": "16岁少女为结婚“变”22岁 7年后想离婚被法院拒绝"}},{"_index": "artisan","_type": "my_type","_id": "4","_score": 1.4914938,"_source": {"text": "女人对护肤品比对男票好？网友神怼"}},{"_index": "artisan","_type": "my_type","_id": "1","_score": 0.2876821,"_source": {"text": "男子偷上万元发红包求交女友被抓获时仍然单身"}}]}}