`
cloudeagle_bupt
  • 浏览: 541215 次
文章分类
社区版块
存档分类
最新评论

Hama0.7.1 的坑

阅读更多

1. 在GraphJobRunner中的loadVertices函数中,增加打印读取的顶点,就可以避免如下问题:


owest notify.
16/03/26 20:57:27 DEBUG sync.ZooKeeperSyncClientImpl: leaveBarrier() at superstep: 3 taskid:attempt_201603262059_0001_000004_0 lowest notify other nodes.
16/03/26 20:57:27 DEBUG sync.ZooKeeperSyncClientImpl: leaveBarrier() !!! checking znodes contnains /ready node or not: at superstep:3 znode:[ready]
16/03/26 20:57:27 DEBUG sync.ZooKeeperSyncClientImpl: leaveBarrier() at superstep:3 znode size: (0) znodes:[]
16/03/26 20:57:27 DEBUG ipc.Client: IPC Client (591203588) connection to localhost/127.0.0.1:50001 from liuqiang2 sending #67
16/03/26 20:57:27 DEBUG ipc.Client: IPC Client (591203588) connection to localhost/127.0.0.1:50001 from liuqiang2 got value #67
16/03/26 20:57:27 DEBUG ipc.RPC: Call: statusUpdate 2
16/03/26 20:57:27 INFO graph.GraphJobRunner: Total time spent for broadcasting aggregation values: 118 ms
16/03/26 20:57:27 ERROR bsp.BSPTask: Error running bsp setup and bsp function.
java.lang.NullPointerException
	at org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputStream.java:63)
	at org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
	at org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
	at org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.java:368)
	at org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:278)
	at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:188)
	at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
	at org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
	at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
16/03/26 20:57:27 DEBUG bsp.Counters: Adding TASK_OUTPUT_RECORDS

原因我也不知道,但是就是可以! 怀疑是concurrencyHashMap的并发问题,真TM神奇。。。

我打印了消息值后发现被发送过来的消息ID在原task的MapVertices中确实没有,因此处理消息时取值为空报错,怀疑一种可能是读的时候没有读进来,另外的一种可能是并发重分区的时候数据没有同步好,导致该点未被读入。


检查应该是并发重分区时的问题,可能没有同步好。。。。




2. zookeeper的同步问题,记得随时:

bin/zookeepers.sh ~/zookeeper-3.4.5/bin/zkCli.sh -server 127.0.0.1:21810

删除/bsp下面的所有数据,否则会有稀奇古怪的问题。。。。




分享到:
评论

相关推荐

Global site tag (gtag.js) - Google Analytics