Mac hadoop 运行权威指南天气例子

1.修改环境变量 ~/bash_profile

export ANDROID_HOME=~/Library/Android/sdk
# added by Anaconda3 5.1.0 installer
export PATH="/Users/walle/anaconda3/bin:$PATH"
# hadoop 
export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.0/libexec 
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

for f in $HADOOP_HOME/share/hadoop/common/hadoop-*.jar;do
     export CLASSPATH=$CLASSPATH:$f
done
for f in $HADOOP_HOME/share/hadoop/hdfs/hadoop-*.jar;do
     export CLASSPATH=$CLASSPATH:$f
done
for f in $HADOOP_HOME/share/hadoop/mapreduce/hadoop-*.jar;do
     export CLASSPATH=$CLASSPATH:$f
done
for f in $HADOOP_HOME/share/hadoop/yarn/hadoop-*.jar;do
     export CLASSPATH=$CLASSPATH:$f
done
export CLASSPATH=$CLASSPATH:$HADOOP_HOME/share/common/lib:$HADOOP_HOME/share/hdfs/lib:$HADOOP_HOME/share/mapreduce/lib:$HADOOP_HOME/share/tools/lib:$HADOOP_HOME/share/yarn/lib

2.编译java 为jar文件

javac *.java
jar -cvf MaxTemperature.jar .

在src目录执行上面2行代码，得到MaxTemperature.jar

3. 拷贝文件到hfs

#创建几个目录
hadoop fs -mkdir /user/
hadoop fs -mkdir /user/walle
#拷贝文件
hadoop fs -put ./input/1901 /user/walle
hadoop fs -put ./input/1902 /user/walle

4.执行mapreduce

hadoop jar ./src/MaxTemperature.jar MaxTemperature /user/walle /user/walle/output

5.查看结果

hadoop fs -cat /user/walle/output/part-r-00000

http://www.waitingfy.com/archives/4001

hadoop | RSS 2.0 | Respond | Trackback |