Yarn编译指南
首先下载hadoop0.23版本
svn checkout
http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.23.0-rc1
2. 进入release-0.23.0-rc0目录下能看到INSTALL.TXT文件,这里有编译hadoop 0.23的教程.
编译前的准备:.
a) * UnixSystem
b) * JDK1.6
c) * Maven3.0
d) *Forrest 0.8 (if generating docs)
e) *Findbugs 1.3.9 (if running findbugs)
f) *ProtocolBuffer 2.4.1+ (for MapReduce)
g) *Autotools (if compiling native code)
h) *Internet connection for first build (to fetch all Maven and Hadoopdependencies)
可以根据需要安装全部或部分的工具,然后把它们加入到$PATH中. 这里介绍一下
ProtocolBuffer的安装方法:下载2.4.1版本后解压,进入目录,运行如下命令即可.
1、./configure(注:默认可能会安装在/usr/local目录下,可以加--prefix=/usr来指定安装到/usr/lib下,可以免去路径的设置)
2、make
3、make check
4、make install
ProtocolBuffer要添加环境变量: export LD_LIBRARY_PATH=/usr/local/lib
出现问题:
save java.io.IOException: error=13,Permissiondenied
chmod777 BLAAAA …..
编译: mvn package -Pdist -DskipTests –Dtar
[INFO] Executed tasks
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [3.373s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [0.034s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [1.046s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.128s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [2.941s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [0.777s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [0.825s]
[INFO] Apache Hadoop Common .............................. SUCCESS [28.432s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.004s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [10.670s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.003s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.005s]
[INFO] hadoop-yarn-api ................................... SUCCESS [14.801s]
[INFO] hadoop-yarn-common ................................ SUCCESS [4.097s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.005s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [4.820s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [4.134s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [0.286s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [5.655s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.368s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.006s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [0.831s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.212s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.007s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [1.532s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [10.000s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [0.172s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [1.347s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [0.230s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [0.261s]
[INFO] hadoop-mapreduce .................................. SUCCESS [4.987s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.003s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [7.688s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:52.011s
[INFO] Finished at: Fri Sep 07 13:18:27 CST 2012
[INFO] Final Memory: 75M/218M
分享到:
相关推荐
hadoop0.23,绿色包,免安装!
hadoop cdh 编译所需下载的所有软件,很全的
hadoop源码编译所需要的maven仓库,省去编译时下载时间。
hadoop windows 编译版 bin 和 lib 有需要自行下载.....
Hadoop 编译 64 位本地库 方法 Hadoop 编译 64 位本地库 方法Hadoop 编译 64 位本地库 方法
Hadoop源码编译
hadoop源码进行编译,需要预安装的一些软件包,包含protobuf,snappy,findbugs,hadoop源码包,jdk1.7,maven等。
编译环境hadoop2.5.0 ,snappy1.1.3,linux Centos 6.4。 注意不支持CDH版本,CDH版本会报错 用法:替换native 文件夹即可
自己在hadoop官网下载的hadoop源码,然后再Centos7.4上面编译通过的,自己的集群就是用的这个版本配置的
win 7 64上编译 Hadoop 2.7.3 源码 的真实经历。
5.1 前期准备工作 5.2 jar 包安装 5.3 编译源码 5.4 常见的问题及解决方案 2.7.0 版本的问题汇总帖子 http://www.tuicoo
sqoop-1.4.6.bin__hadoop-0.23.tar.gz
hadoop2.6.0编译的全部工具 包括ant maven findbugs protobuf hadoop-2.6.0-src
里面是 protobuf-2.5.0.tar.gz apache-ant-1.9.9-bin.tar.gz 和meavn
Hadoop源码编译.pdf
hadoop2.4.1编译后的snappy包,可以部署到hbase9.8.3和hadoop2.4.1
hadoop2.9.2编译的windows下的dll和exe,本人自己项目中也在使用
Hadoop2.7.3编译,包括所有工具包,照着做就行,亲测可行
hadoop官方文件是没有编译的,直接使用会提示缺少hadoop.dll,该资源是编译后产生的bin文件,替换Hadoop2.74原始bin文件即可
centos7.4在VMware详细安装文档,以及hadoop2.7.6编译环境安装和编译。