Hadoop 2.7.2 Dockerfile完整脚本

古城微笑少年丶 2023-09-28 20:43 10阅读 0赞

## 下载地址 ##

[https://github.com/kiwenlau/hadoop-cluster-docker][https_github.com_kiwenlau_hadoop-cluster-docker]

## 目录结构 ##

![在这里插入图片描述][90465af855de4bc09b36f6f40db014ba.png]

## Dockerfile ##

FROM ubuntu:14.04
    
    MAINTAINER KiwenLau <kiwenlau@gmail.com>
    
    WORKDIR /root
    
    # install openssh-server, openjdk and wget
    RUN apt-get update && apt-get install -y openssh-server openjdk-7-jdk wget
    
    # install hadoop 2.7.2
    RUN wget https://github.com/kiwenlau/compile-hadoop/releases/download/2.7.2/hadoop-2.7.2.tar.gz && \
        tar -xzvf hadoop-2.7.2.tar.gz && \
        mv hadoop-2.7.2 /usr/local/hadoop && \
        rm hadoop-2.7.2.tar.gz
    
    # set environment variable
    ENV JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 
    ENV HADOOP_HOME=/usr/local/hadoop 
    ENV PATH=$PATH:/usr/local/hadoop/bin:/usr/local/hadoop/sbin 
    
    # ssh without key
    RUN ssh-keygen -t rsa -f ~/.ssh/id_rsa -P '' && \
        cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    
    RUN mkdir -p ~/hdfs/namenode && \ 
        mkdir -p ~/hdfs/datanode && \
        mkdir $HADOOP_HOME/logs
    
    COPY config/* /tmp/
    
    RUN mv /tmp/ssh_config ~/.ssh/config && \
        mv /tmp/hadoop-env.sh /usr/local/hadoop/etc/hadoop/hadoop-env.sh && \
        mv /tmp/hdfs-site.xml $HADOOP_HOME/etc/hadoop/hdfs-site.xml && \ 
        mv /tmp/core-site.xml $HADOOP_HOME/etc/hadoop/core-site.xml && \
        mv /tmp/mapred-site.xml $HADOOP_HOME/etc/hadoop/mapred-site.xml && \
        mv /tmp/yarn-site.xml $HADOOP_HOME/etc/hadoop/yarn-site.xml && \
        mv /tmp/slaves $HADOOP_HOME/etc/hadoop/slaves && \
        mv /tmp/start-hadoop.sh ~/start-hadoop.sh && \
        mv /tmp/run-wordcount.sh ~/run-wordcount.sh
    
    RUN chmod +x ~/start-hadoop.sh && \
        chmod +x ~/run-wordcount.sh && \
        chmod +x $HADOOP_HOME/sbin/start-dfs.sh && \
        chmod +x $HADOOP_HOME/sbin/start-yarn.sh 
    
    # format namenode
    RUN /usr/local/hadoop/bin/hdfs namenode -format
    
    CMD [ "sh", "-c", "service ssh start; bash"]

--------------------

[参考链接：https://github.com/kiwenlau/hadoop-cluster-docker][https_github.com_kiwenlau_hadoop-cluster-docker]

[https_github.com_kiwenlau_hadoop-cluster-docker]: https://github.com/kiwenlau/hadoop-cluster-docker
[90465af855de4bc09b36f6f40db014ba.png]: https://img-blog.csdnimg.cn/90465af855de4bc09b36f6f40db014ba.png