docker系列3：docker搭建CDH集群[单机单节点]

电玩女神 2022-04-03 08:38 989阅读 0赞

官方文档1：[https://www.cloudera.com/documentation/enterprise/5-13-x/topics/quickstart\_docker\_container.html\#importing\_quickstart\_image][https_www.cloudera.com_documentation_enterprise_5-13-x_topics_quickstart_docker_container.html_importing_quickstart_image]  
官方文档2：[https://hub.docker.com/r/cloudera/quickstart][https_hub.docker.com_r_cloudera_quickstart]

*  1, 使用docker 拉取cdh的docker镜像
 *  2, 运行容器
 *  3, 进入容器启动cloudera服务
 *  4, 通过web访问： [http://quickstart.cloudera:7180][http_quickstart.cloudera_7180]

<table> 
 <thead> 
  <tr> 
   <th>服务名称</th> 
   <th>账户</th> 
   <th>密码</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>7180-web控制台</td> 
   <td>cloudera</td> 
   <td>cloudera</td> 
  </tr> 
  <tr> 
   <td>hue</td> 
   <td>cloudera</td> 
   <td>cloudera</td> 
  </tr> 
  <tr> 
   <td>mysql</td> 
   <td>root</td> 
   <td>cloudera</td> 
  </tr> 
 </tbody> 
</table>

# 拉取cdh的docker镜像， 运行容器 #

###1, 使用docker 拉取cdh的docker镜像， 运行容器
    root@wang-pc:/var/lib/docker# docker pull cloudera/quickstart
    
    root@wang-pc:/var/lib/docker# docker run \
     -id \
     --hostname=quickstart.cloudera \
    --privileged=true  \
     -p 8020:8020 -p 7180:7180 -p 21050:21050 -p 50070:50070 -p 50075:50075 \
     -p 50010:50010 -p 50020:50020 -p 8890:8890 -p 60010:60010 -p 10002:10002  \
     -p 25010:25010 -p 25020:25020 -p 18088:18088 -p 8088:8088 -p 19888:19888 \
     -p 7187:7187 -p 11000:11000 -t -p 8888:8888 \
     --name=mycdh3 \
     cloudera/quickstart /usr/bin/docker-quickstart 
    d805d3c51b3f4c41c68fe2f3ba923f7e710b7f3c987b990b63a8fda9ee7bec4a
    
    root@wang-pc:/var/lib/docker# docker ps
    CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NAMES
    d805d3c51b3f        cloudera/quickstart   "/usr/bin/docker-q..."   5 seconds ago       Up 3 seconds        0.0.0.0:7180->7180/tcp, 0.0.0.0:7187->7187/tcp, 0.0.0.0:8020->8020/tcp, 0.0.0.0:8088->8088/tcp, 0.0.0.0:8888->8888/tcp, 0.0.0.0:8890->8890/tcp, 0.0.0.0:10002->10002/tcp, 0.0.0.0:11000->11000/tcp, 0.0.0.0:18088->18088/tcp, 0.0.0.0:19888->19888/tcp, 0.0.0.0:21050->21050/tcp, 0.0.0.0:25010->25010/tcp, 0.0.0.0:25020->25020/tcp, 0.0.0.0:50010->50010/tcp, 0.0.0.0:50020->50020/tcp, 0.0.0.0:50070->50070/tcp, 0.0.0.0:50075->50075/tcp, 0.0.0.0:60010->60010/tcp   mycdh3

# 1， cloudera-manager启动脚本分析 #

/home/cloudera/cloudera-manager：

# cm server启动
    log 'Starting Cloudera Manager server...'
    service cloudera-scm-server start 2>&1 > /dev/null
    # cm agent启动
    log 'Starting Cloudera Manager agent...'
    service cloudera-scm-agent start 2>&1 > /dev/null

# 2， 进入容器： 启动cdh服务 #

###, 运行容器, 启动cloudera
    root@wang-pc:/var/lib/docker# docker exec -it mycdh3 bash
    [root@quickstart /]# ls
    bin  boot  dev  etc  home  lib  lib64  lost+found  media  mnt  opt  packer-files  proc  root  sbin  selinux  srv  sys  tmp  usr  var
    [root@quickstart /]# /home/cloudera/cloudera-manager --enterprise
    
    [QuickStart] Shutting down CDH services via init scripts...
    kafka-server: unrecognized service
    JMX enabled by default
    Using config: /etc/zookeeper/conf/zoo.cfg
    [QuickStart] Disabling CDH services on boot...
    error reading information on service kafka-server: No such file or directory
    [QuickStart] Starting Cloudera Manager server...
    [QuickStart] Waiting for Cloudera Manager API...
    [QuickStart] Starting Cloudera Manager agent...
    [QuickStart] Activating trial license for Enterprise...
    [QuickStart] Configuring deployment...
    Submitted jobs: 16
    [QuickStart] Deploying client configuration...
    Submitted jobs: 17
    [QuickStart] Starting Cloudera Management Service...
    Submitted jobs: 25
    [QuickStart] Enabling Cloudera Manager daemons on boot...
    ________________________________________________________________________________
    Success! You can now log into Cloudera Manager from the QuickStart VM's browser:
        http://quickstart.cloudera:7180
        Username: cloudera
        Password: cloudera

配置host映射

root@wang-pc:/home/wang# cat /etc/hosts
    127.0.0.1	localhost wang-pc quickstart.cloudera

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70]  
测试hdfs, yarn 是否正常

su hdfs
    cd ; echo "a,b,c">a.txt; hdfs dfs -put a.txt /
    hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount /a.txt /mapred-out-wc.txt

[https_www.cloudera.com_documentation_enterprise_5-13-x_topics_quickstart_docker_container.html_importing_quickstart_image]: https://www.cloudera.com/documentation/enterprise/5-13-x/topics/quickstart_docker_container.html#importing_quickstart_image
[https_hub.docker.com_r_cloudera_quickstart]: https://hub.docker.com/r/cloudera/quickstart
[http_quickstart.cloudera_7180]: http://quickstart.cloudera:7180
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70]: /images/20220403/1ce85ca9b6404c7288f1d7ca14920190.png