kafka 8和内存没有足够的内存供Java运行时环境继续运行

Question

我使用的DigiOcean实例有512兆的ram，我用kafka得到下面的错误。 我不是一个精通java的开发人员。如何调整kafka以利用少量的ram。这是一个开发服务器。我不想为一台更大的机器每小时付更多的钱

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# //hs_err_pid6500.log
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000bad30000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)

Answer 1

添加环境变量JAVA_OPTS

在Linux中，编辑~/。bashrc文件，并更好地源代码

export JAVA_OPTS="$JAVA_OPTS -Xms128M -Xmx128M"

在widows中，添加系统变量

JAVA_OPTS "-Xms128M -Xmx128M"

然后重新启动

如果它不工作，许多文件应该被更改，e.x.bin/kafka服务器启动。sh，配置/服务器。属性。减少这些文件中的数字

Answer 2

地区：热点/gc

概要

Crashes due to failure to allocate large pages.

On Linux, failures when allocating large pages can lead to crashes. When running JDK 7u51 or later versions, the issue can be recognized in two ways:

    Before the crash happens, one or more lines similar to the following example will have been printed to the log:

    os::commit_memory(0x00000006b1600000, 352321536, 2097152, 0) failed;
    error='Cannot allocate memory' (errno=12); Cannot allocate large pages, 
    falling back to regular pages

    If a file named hs_err is generated, it will contain a line similar to the following example:

    Large page allocation failures have occurred 3 times

The problem can be avoided by running with large page support turned off, for example, by passing the "-XX:-UseLargePages" option to the java binary.

Answer 3

在您的环境路径中，在JDK32上将java路径改为JDK64。这解决了我的问题，当时我有64gb的ram，使用32位Jvm将路径更改为JDK64（64位Jvm）解决了这个问题

Answer 4

您可以通过编辑kafka-server-start.sh、zookeeper-server-start.sh等来调整JVM堆大小：

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"

-Xms参数指定最小堆大小。要使服务器至少启动，请尝试将其更改为使用更少的内存。假设您只有512M，您也应该更改最大堆大小（-Xmx）：

export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

我不确定默认配置中kafka的最小内存需求是什么-也许您需要调整kafka中的消息大小以使其运行

Answer 5

在运行Kafka的Docker容器中，here的答案也可以用来解决相同的问题

只需将以下ENV变量添加到docker compose文件（或运行脚本）：

KAFKA_HEAP_OPTS: "-Xmx512M -Xms256M"

当然，您应该微调主机资源可用性上的值