hadoop0.23 编译

Yarn编译指南

首先下载hadoop0.23版本

svn checkout http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.23.0-rc1

2. 进入release-0.23.0-rc0目录下能看到INSTALL.TXT文件,这里有编译hadoop 0.23的教程.

编译前的准备:.

a) * UnixSystem

b) * JDK1.6

c) * Maven3.0

d) *Forrest 0.8 (if generating docs)

e) *Findbugs 1.3.9 (if running findbugs)

f) *ProtocolBuffer 2.4.1+ (for MapReduce)

g) *Autotools (if compiling native code)

h) *Internet connection for first build (to fetch all Maven and Hadoopdependencies)

可以根据需要安装全部或部分的工具,然后把它们加入到$PATH中. 这里介绍一下

ProtocolBuffer的安装方法:下载2.4.1版本后解压,进入目录,运行如下命令即可.

1、./configure（注：默认可能会安装在/usr/local目录下，可以加--prefix=/usr来指定安装到/usr/lib下，可以免去路径的设置）

2、make

3、make check

4、make install

ProtocolBuffer要添加环境变量: export LD_LIBRARY_PATH=/usr/local/lib

出现问题:

save java.io.IOException: error=13,Permissiondenied

chmod777 BLAAAA …..

编译: mvn package -Pdist -DskipTests –Dtar

[INFO] Executed tasks
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [3.373s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [0.034s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [1.046s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.128s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [2.941s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [0.777s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [0.825s]
[INFO] Apache Hadoop Common .............................. SUCCESS [28.432s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.004s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [10.670s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.003s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.005s]
[INFO] hadoop-yarn-api ................................... SUCCESS [14.801s]
[INFO] hadoop-yarn-common ................................ SUCCESS [4.097s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.005s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [4.820s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [4.134s]

[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [0.286s]

[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [5.655s]

[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.368s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.006s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [0.831s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.212s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.007s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [1.532s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [10.000s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [0.172s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [1.347s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [0.230s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [0.261s]
[INFO] hadoop-mapreduce .................................. SUCCESS [4.987s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.003s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [7.688s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:52.011s
[INFO] Finished at: Fri Sep 07 13:18:27 CST 2012
[INFO] Final Memory: 75M/218M