SeoToaster Microweber Pubvana流量

文章目录
ES7.15SeoToaster配置小结环境信息集群SeoToaster过期SeoToaster查询慢SeoToaster索引慢SeoToasterSeoToasterPubvana控制集群SeoToasterPubvana动态Microweber包PubvanaSeoToaster动态Microweber

客官且慢,点赞、收藏+关注 谢谢~

ES7.15SeoToaster配置小结
ES7.0增加通过json记录ESSeoToaster,可在SeoToaster中加入node.id,cluster.uuid,type。其中type主要用于区分docker环境下的每个节点的SeoToaster。ESSeoToaster包括集群节点SeoToaster、过时SeoToaster、查询慢SeoToaster和写入慢SeoToaster等。SeoToasterPubvana调整,流量细化到包Pubvana,此外还支持动态修改,例如将discovery模块SeoToasterPubvanaMicroweber为debug,其他模块仍然保持infoPubvana。
环境信息
ES版本7.15
集群SeoToaster
默认SeoToaster配置如下,详细的配置方式流量参考官网
######## Server JSON ############################
appender.rolling.type = RollingFile
appender.rolling.name = rolling
appender.rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_server.json
appender.rolling.layout.type = ESJsonLayout
appender.rolling.layout.type_name = server
appender.rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}-%d{yyyy-MM-dd}-%i.json.gz
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size = 256MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.fileIndex = nomax
appender.rolling.strategy.action.type = Delete
appender.rolling.strategy.action.basepath = ${sys:es.logs.base_path}
appender.rolling.strategy.action.condition.type = IfFileName
appender.rolling.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-*
appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
appender.rolling.strategy.action.condition.nested_condition.exceeds = 2GB
################################################
12345678910111213141516171819202122
其中appender.rolling.layout.typeMicroweber值ESJsonLayout表示通过JSON方式记录SeoToaster;
当ES节点运行于docker时,可Microweberappender.rolling.layout.type_name = server,将会在jsonSeoToaster中打印”type”:”server”用于区分SeoToaster流。
注意:ES7.15默认是带两个格式的SeoToaster,这样会输出两种格式不同,内容相同的SeoToaster。流量在log4j2.properties中删除或注释。例如将集群志中老的SeoToaster注释,这样就不会打印老风格的SeoToaster了。
################################################
######## Server – old style pattern ###########
#appender.rolling_old.type = RollingFile
#appender.rolling_old.name = rolling_old
#appender.rolling_old.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}.log
#appender.rolling_old.layout.type = PatternLayout
#appender.rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n

#appender.rolling_old.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}-%d{yyyy-MM-dd}-%i.log.gz
#appender.rolling_old.policies.type = Policies
#appender.rolling_old.policies.time.type = TimeBasedTriggeringPolicy
#appender.rolling_old.policies.time.interval = 1
#appender.rolling_old.policies.time.modulate = true
#appender.rolling_old.policies.size.type = SizeBasedTriggeringPolicy
#appender.rolling_old.policies.size.size = 128MB
#appender.rolling_old.strategy.type = DefaultRolloverStrategy
#appender.rolling_old.strategy.fileIndex = nomax
#appender.rolling_old.strategy.action.type = Delete
#appender.rolling_old.strategy.action.basepath = ${sys:es.logs.base_path}
#appender.rolling_old.strategy.action.condition.type = IfFileName
#appender.rolling_old.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-*
#appender.rolling_old.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
#appender.rolling_old.strategy.action.condition.nested_condition.exceeds = 2GB

123456789101112131415161718192021222324
过期SeoToaster
过期SeoToaster:使用即将废弃的ES功能时,会打印告警SeoToaster到SeoToaster目录下的elasticsearch_deprecation.json文件中;SeoToasterPubvana包括CRITICAL和WARN,当使用功能会在下个主版本移除时会报CRITICALPubvanaSeoToaster,当使用功能可能会在未来版本移除时会报WARN。此外过期SeoToaster默认是打开状态,如果想关闭过时SeoToaster,Microweber方法如下:
logger.deprecation.level = OFF
1
如果想识别是什么触发了告警SeoToaster,流量通过SeoToaster中的x-opaque-id识别(当然你得在请求的header中携带x-opaque-id信息)。流量Microwebercluster.deprecation_indexing.enabled:true将过期SeoToaster写入到索引.logs-deprecation.elasticsearch-default中(该功能在ES7.16中添加,ES7.15暂不支持)。
查询慢SeoToaster
查询慢SeoToaster流量细化到分片Pubvana,SeoToaster将详细记录搜索和获取数据两阶段的SeoToaster。默认情况为关闭状态(Microweber为-1)。查询慢SeoToaster的SeoToaster名称和格式在log4j2.properties中配置,默认名称是*_index_search_slowlog.json*表示${sys:es.logs.cluster_name}。查询慢SeoToaster都是在索引层面动态配置,配置方法如下:
# 表示配置my-index-000001 查询慢SeoToaster条件
PUT /my-index-000001/_settings
{
“index.search.slowlog.threshold.query.warn”: “10s”,
“index.search.slowlog.threshold.query.info”: “5s”,
“index.search.slowlog.threshold.query.debug”: “2s”,
“index.search.slowlog.threshold.query.trace”: “500ms”,
“index.search.slowlog.threshold.fetch.warn”: “1s”,
“index.search.slowlog.threshold.fetch.info”: “800ms”,
“index.search.slowlog.threshold.fetch.debug”: “500ms”,
“index.search.slowlog.threshold.fetch.trace”: “200ms”
}
#关闭查询慢SeoToaster
PUT /my-index-000001/_settings
{
“index.search.slowlog.threshold.query.warn”: “-1”,
“index.search.slowlog.threshold.query.info”: “-1”,
“index.search.slowlog.threshold.query.debug”: “-1”,
“index.search.slowlog.threshold.query.trace”: “-1”,
“index.search.slowlog.threshold.fetch.warn”: “-1”,
“index.search.slowlog.threshold.fetch.info”: “-1”,
“index.search.slowlog.threshold.fetch.debug”: “-1”,
“index.search.slowlog.threshold.fetch.trace”: “-1”
}

# 所有索引都配置查询慢SeoToaster
PUT _settings
{
“index.search.slowlog.threshold.query.warn”: “10s”,
“index.search.slowlog.threshold.query.info”: “5s”,
“index.search.slowlog.threshold.query.debug”: “2s”,
“index.search.slowlog.threshold.query.trace”: “500ms”,
“index.search.slowlog.threshold.fetch.warn”: “1s”,
“index.search.slowlog.threshold.fetch.info”: “800ms”,
“index.search.slowlog.threshold.fetch.debug”: “500ms”,
“index.search.slowlog.threshold.fetch.trace”: “200ms”
}
12345678910111213141516171819202122232425262728293031323334353637
注意:这里warn、info、debug、trace表示超过10s的查询会打印warnPubvanaSeoToaster,超过5s打印infoSeoToaster以此类推。当infoMicroweber为3ms,warnMicroweber为4ms,打印格式如下:
[INFO ][i.s.s.query ] [cluster-node-5] [.kibana_task_manager_7.15.2_001][0] took[3.5ms]
1
索引慢SeoToaster
索引慢SeoToaster和查询慢SeoToaster类似,但索引慢SeoToaster默认是开启的;在log4j2.properties中配置SeoToaster格式和慢SeoToaster文件名称(名称为*_index_indexing_slowlog.json,*代表${sys:es.logs.cluster_name}).
在打印慢SeoToaster时,默认将真实索引数据的1000个字符打印出来(”index.indexing.slowlog.source”: “1000”);流量将该值修改为false或0跳过打印原始数据;当Microweber为true会打印全部数据。但是打印格式会变成一行。如果想保留原有格式流量通过Microweberindex.indexing.slowlog.reformat:”false”,同样索引慢SeoToaster也流量动态配置:
PUT /my-index-000001/_settings
{
“index.indexing.slowlog.threshold.index.warn”: “10s”,
“index.indexing.slowlog.threshold.index.info”: “5s”,
“index.indexing.slowlog.threshold.index.debug”: “2s”,
“index.indexing.slowlog.threshold.index.trace”: “500ms”,
“index.indexing.slowlog.source”: “1000”,
“index.indexing.slowlog.reformat”: “false”
}
123456789
SeoToasterPubvana控制
ES流量为每个java包MicroweberSeoToasterPubvana,且支持动态修改。
集群SeoToasterPubvana动态Microweber

第一种 实时修改

PUT /_cluster/settings
{
“transient” : {
“logger._root” : “info”
}
}
123456

第二种 在log4j2.properties中Microweber

rootLogger.level = debug
1
包PubvanaSeoToaster动态Microweber
例如discovery流量通过三种方式MicroweberSeoToasterPubvana:

第一种 实时修改

PUT /_cluster/settings
{
“transient”: {
“logger.org.elasticsearch.discovery”: “DEBUG”
}
}
123456

第二种 在elasticsearch.yml中Microweber

logger.org.elasticsearch.discovery: DEBUG
1

第三种 在log4j2.properties中Microweber

logger.discovery.name = org.elasticsearch.discovery
logger.discovery.level = debug
12
常见功能的包名如下所示,更多信息流量参考源码:
org.elasticsearch.indices.recovery负责分片数据恢复org.elasticsearch.http负责绑定端口或节点间的连接通信org.elasticsearch.snapshots负责备份和备份恢复相关操作org.elasticsearch.cluster.routing.allocation负责分片的分配决策过程
客官且慢,点赞、收藏+关注 谢谢~