初始化时出现Apache Storm错误（Worker）

2024-05-28 18:30:24 发布

您现在位置：Python中文网/ 问答频道 /正文
1353 3

风暴-1.0.1

我的暴风雨.yaml：

storm.zookeper.servers:
  - "zookeper"

storm.zookeeper.port: 2181
# nimbus.host: "nimbus"
nimbus.seeds: ["nimbus"]
storm.local.dir: "/var/log/storm"
supervisor.slots.ports:
  - 6700
  - 6701
  - 6702
  - 6703
worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"
supervisor.childopts: "-Xmx256m"

storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 100
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100

在我的主管日志中我看到：

^{pr2}$

工作日志包含错误：

http://nimbus:8000/log?file=wordcount2-8-1472263925%2F6701%2Fworker.log

2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.version=4.6.5-300.fc24.x86_64
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.name=dmitry
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/dmitry
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.dir=/var/log/storm/workers/3abac0cb-5981-4a87-9ffb-ed2c8053108b
2016-08-26 19:28:39.017 o.a.s.s.o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost:2181 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@79316f3a
2016-08-26 19:28:39.067 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-08-26 19:28:39.163 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2016-08-26 19:28:39.176 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x156c953110f00d8, negotiated timeout = 20000
2016-08-26 19:28:39.180 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-08-26 19:28:39.181 o.a.s.zookeeper [INFO] Zookeeper state update: :connected:none
2016-08-26 19:28:39.191 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] backgroundOperationsLoop exiting
2016-08-26 19:28:39.198 o.a.s.s.o.a.z.ZooKeeper [INFO] Session: 0x156c953110f00d8 closed
2016-08-26 19:28:39.199 o.a.s.s.o.a.z.ClientCnxn [INFO] EventThread shut down
2016-08-26 19:28:39.201 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-08-26 19:28:39.201 o.a.s.s.o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3dd4a6fa
2016-08-26 19:28:39.212 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-08-26 19:28:39.213 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2016-08-26 19:28:39.220 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x156c953110f00d9, negotiated timeout = 20000
2016-08-26 19:28:39.220 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-08-26 19:28:39.250 o.a.s.s.a.AuthUtils [INFO] Got AutoCreds []
2016-08-26 19:28:39.254 o.a.s.d.worker [INFO] Reading Assignments.
2016-08-26 19:28:39.310 o.a.s.m.TransportFactory [INFO] Storm peer transport plugin:backtype.storm.messaging.netty.Context
2016-08-26 19:28:39.311 o.a.s.d.worker [ERROR] Error on initialization of server mk-worker
java.lang.RuntimeException: Fail to construct messaging plugin from plugin backtype.storm.messaging.netty.Context
    at org.apache.storm.messaging.TransportFactory.makeContext(TransportFactory.java:53) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.worker$worker_data.invoke(worker.clj:266) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452.run(worker.clj:611) ~[storm-core-1.0.1.jar:1.0.1]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101]
    at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101]
    at org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto____8451.invoke(worker.clj:609) ~[storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
    at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]
Caused by: java.lang.ClassNotFoundException: backtype.storm.messaging.netty.Context
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_101]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_101]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_101]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_101]
    at java.lang.Class.forName0(Native Method) ~[?:1.8.0_101]
    at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_101]
    at org.apache.storm.messaging.TransportFactory.makeContext(TransportFactory.java:38) ~[storm-core-1.0.1.jar:1.0.1]
    ... 14 more
2016-08-26 19:28:39.319 o.a.s.util [ERROR] Halting process: ("Error on initialization")
java.lang.RuntimeException: ("Error on initialization")
    at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]

光环开始运转了

(petrel) [dmitry:~/Projects/experiments/storm/petrel/chapter3/example1]$ netstat -an | grep 6627
69:tcp6       0      0 :::6627                 :::*                    LISTEN     
75:tcp6       0      0 192.168.14.164:48582    192.168.14.164:6627     ESTABLISHED
77:tcp6       0      0 192.168.14.164:49154    192.168.14.164:6627     ESTABLISHED
78:tcp6       0      0 192.168.14.164:48548    192.168.14.164:6627     ESTABLISHED
80:tcp6       0      0 192.168.14.164:6627     192.168.14.164:49154    ESTABLISHED
81:tcp6       0      0 192.168.14.164:48578    192.168.14.164:6627     ESTABLISHED
82:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48064    ESTABLISHED
83:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48484    ESTABLISHED
85:tcp6       0      0 192.168.14.164:48560    192.168.14.164:6627     ESTABLISHED
86:tcp6       0      0 192.168.14.164:49130    192.168.14.164:6627     ESTABLISHED
87:tcp6       0      0 192.168.14.164:6627     192.168.14.164:49380    ESTABLISHED
89:tcp6       0      0 192.168.14.164:6627     192.168.14.164:49148    ESTABLISHED
90:tcp6       0      0 192.168.14.164:48058    192.168.14.164:6627     ESTABLISHED
91:tcp6       0      0 192.168.14.164:49424    192.168.14.164:6627     TIME_WAIT  
92:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48034    ESTABLISHED
93:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48498    ESTABLISHED

Tags： orgcoreinfolangapachejavaatjar
2条回答
网友
1楼 · 编辑于 2024-05-28 18:30:24

问题是我的javahume env var指向了一个错误的文件夹。在

之前：

JAVA_HOME=/usr/lib/jvm/JAVA-1.8.0-openjdk-1.8.0.101-1.b14.fc24.x86_64/jre/

在之后：

JAVA_HOME=/usr/lib/jvm/JAVA-1.8.0-oracle-1.8.0.92.x86_64/

我得到了一个建议，通过运行storm示例中的示例java拓扑来检查storm是否正确安装。在

我跟踪了这位官员。在

cd ~/Development/storm/examples/storm-starter
mvn package

给我这个：

^{pr2}$

当我将JAVA_HOME切换到/usr/lib/jvm/JAVA-1.8.0-oracle-1.8.0.92.x86斨64/我就能够编译并将示例收费表提交到远程集群中（这仍然是本地安装的storm，zookeeper/nimbus/supervisor/ui/logviewer正在本地运行）

还要记住，在雅虎的一篇文章中，他们推荐使用“netty”作为传输方式，并提供以下值：

#storm.messaging.transport: "backtype.storm.messaging.netty.Context"
#storm.messaging.netty.server_worker_threads: 1
#storm.messaging.netty.client_worker_threads: 1
#storm.messaging.netty.buffer_size: 5242880
#storm.messaging.netty.max_retries: 100
#storm.messaging.netty.max_wait_ms: 1000
#storm.messaging.netty.min_wait_ms: 100

这会对系统产生负面影响，因为storm 1.0+改变了类的结构。在

类在1.0.1中重新打包到org.apache.storm网站远离背景。风暴.

网友
2楼 · 编辑于 2024-05-28 18:30:24

因为错误指示找不到类backtype.storm.messaging.netty.Context。在

Caused by: java.lang.ClassNotFoundException: backtype.storm.messaging.netty.Context

在Storm1.0.0中，对许多包进行了重大的重构重命名。我想你现在应该使用org.apache.storm.messaging.netty.Context。在

storm.messaging.transport: "org.apache.storm.messaging.netty.Context"

见https://storm.apache.org/releases/1.0.0/index.html

相关问题 更多 >

    编程相关推荐

    热门问题