Flink 学习笔记 3. docker-compose 搭建Flink环境

素颜马尾好姑娘i 2022-12-30 03:37 175阅读 0赞

> docker-compose version 1.27.4, build 40524192
> 
> flink image tag: 1.11.3-scala\_2.12-java8

docker hub  查看可用的flink镜像，这里选用的是 **1.11.3-scala\_2.12-java8** 这个tag的镜像

![20201223202342463.png][]

1. 拉取Flink 1.11.3 镜像

docker pull flink:1.11.3-scala_2.12-java8

2. 本地创建一个docker-flink目录用于搭建flink环境，包括docker-compose.yaml文件和后面挂载目录，docker-compose.yaml文件内容如下：

version: "3.7"
    services:
      flink-jobmanager-01:
        image: flink:1.11.3-scala_2.12-java8
        container_name: flink-jobmanager-01
        hostname: flink-jobmanager-01
        expose:
          - "6123"
        ports:
          - "8081:8081"
        command: jobmanager
        environment:
          - JOB_MANAGER_RPC_ADDRESS=flink-jobmanager-01
    
      flink-taskmanager-01:
        image: flink:1.11.3-scala_2.12-java8
        container_name: flink-taskmanager-01
        hostname: flink-taskmanager-01
        expose:
          - "6121"
          - "6122"
        depends_on:
          - flink-jobmanager-01
        command: taskmanager
        links:
          - "flink-jobmanager-01:jobmanager"
        environment:
          - JOB_MANAGER_RPC_ADDRESS=flink-jobmanager-01

3. 启动docker-compose

mit@mit:~/my_project/big_data/flink/docker-flink$ docker-compose up -d
    Creating network "docker-flink_default" with the default driver
    Creating flink-jobmanager-01 ... done
    Creating flink-taskmanager-01 ... done
    mit@mit:~/my_project/big_data/flink/docker-flink$ docker ps
    CONTAINER ID   IMAGE                           COMMAND                  CREATED         STATUS         PORTS                              NAMES
    23d92bdefcf5   flink:1.11.3-scala_2.12-java8   "/docker-entrypoint.…"   6 seconds ago   Up 4 seconds   6121-6123/tcp, 8081/tcp            flink-taskmanager-01
    c6c038a378df   flink:1.11.3-scala_2.12-java8   "/docker-entrypoint.…"   6 seconds ago   Up 5 seconds   6123/tcp, 0.0.0.0:8081->8081/tcp   flink-jobmanager-01

4. 打开 http://127.0.0.1:8081 可以看到flink dashboard

我的机器为8核CPU，所以task\_manager默认有8个task slots.

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MDIzODYyNQ_size_16_color_FFFFFF_t_70][]

[20201223202342463.png]: /images/20221120/de1d2013f99c4e20a66608772301fd16.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MDIzODYyNQ_size_16_color_FFFFFF_t_70]: /images/20221120/572b2e500e5d4167aff1f5c2ea97149f.png