java在运行elasticsearch集群时遇到Perm Gen空间问题

1 年，3 月 Questions & Answers 1142

我们正在运行带有6个节点的elasticsearch-1.5.1集群，最近几天我面对的是java。lang.OutOfMemoryError PermGen space在集群中出现问题，这会影响节点，同样会停止。我正在重新启动特定节点以启动

我们试图通过给集群提供重负载来解决这个问题，但不幸的是，它无法复制。但有些人认为我们是如何在生产中一次又一次地遇到同样的问题的

这里是一些yml文件配置

index.recovery.initial_shards: 1 index.query.bool.max_clause_count: 8192 index.mapping.attachment.indexed_chars: 500000 index.merge.scheduler.max_thread_count: 1 cluster.routing.allocation.node_concurrent_recoveries: 15 indices.recovery.max_bytes_per_sec: 50mb indices.recovery.concurrent_streams: 5

内存配置

ES_HEAP_SIZE=10g ES_JAVA_OPTS="-server -Des.max-open-files=true" MAX_OPEN_FILES=65535 MAX_MAP_COUNT=262144

使用以下配置更新问题

我怀疑与这个问题有关的merge.policy.max_merged_segment。我的集群中有22个索引。指数的merge.policy.max_merged_segment如下所示

7个索引有20gb

3个索引有10gb

12个指数有5gb

使用流程信息更新

esuser xxxxx 1 28 Oct03 ? 1-02:20:40 /usr/java/default/bin/java -Xms10g -Xmx10g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -server -Des.max-open-files=true -Delasticsearch -Des.pidfile=/var/es/elasticsearch.pid -Des.path.home=/usr/es/elasticsearch -cp :/usr/es/elasticsearch/lib/elasticsearch-1.5.1.jar:/usr/es/elasticsearch/lib/:/usr/es/elasticsearch/lib/sigar/ -Des.default.path.home=/usr/es/elasticsearch -Des.default.path.logs=/es/es_logs -Des.default.path.data=/es/es_data -Des.default.path.work=/es/es_work -Des.default.path.conf=/etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch

下面是我在搜索时从elasticsearch集群获得的堆栈跟踪。但事件发生的同时，索引时间也给我带来了同样的问题。根据我的观察，一些搜索/索引操作会增加PermGen，如果接下来的操作试图使用PermGen空间，那么问题就来了

[2015-10-03 06:45:05,262][WARN ][transport.netty ] [es_f2_01] Message not fully read (response) for [19353573] handler org.elasticsearch.search.action.SearchServiceTransportAction$6@21a25e37, error [true], resetting [2015-10-03 06:45:05,262][DEBUG][action.search.type ] [es_f2_01] [product_index][4], node[GoUqK7csTpezN5_xoNWbeg], [R], s[INITIALIZING]: Failed to execute [org.elasticsearch.action.search.SearchRequest@5c2fe4c4] lastShard [true] org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:176) at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:128) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: java.lang.OutOfMemoryError: PermGen space

你能帮我解决这个问题吗。谢谢

Python中文网

有 Java 编程相关的问题?

java在运行elasticsearch集群时遇到Perm Gen空间问题

共 (1) 个答案

# 1 楼答案