通过 docker-compose 快速部署 DolphinScheduler 保姆级教程

一、概述

Apache DolphinScheduler(简称DolphinScheduler)是一种开源的、分布式的、易于使用的大数据工作流调度系统。它旨在为大数据处理提供一个可靠、高效和可扩展的调度解决方案。

通过 docker-compose 快速部署 DolphinScheduler 保姆级教程

这里只讲快速部署,想了解更多可以查阅我这篇文章:Apache DolphinScheduler(海豚调度系统)介绍与环境部署

二、前期准备

1)部署 docker

<span># 安装yum-config-manager配置工具</span>
yum -y <span>install</span> yum-utils
<span># 提议使用阿里云yum源:(推荐)</span>
<span>#yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo</span>
yum-config-manager <span>--add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo</span>
<span># 安装docker-ce版本</span>
yum <span>install</span> -y docker-ce
<span># 启动并开机启动</span>
systemctl <span>enable</span> <span>--now docker</span>
docker <span>--version</span>
<span># 安装yum-config-manager配置工具</span>
yum -y <span>install</span> yum-utils

<span># 提议使用阿里云yum源:(推荐)</span>
<span>#yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo</span>
yum-config-manager <span>--add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo</span>

<span># 安装docker-ce版本</span>
yum <span>install</span> -y docker-ce
<span># 启动并开机启动</span>
systemctl <span>enable</span> <span>--now docker</span>
docker <span>--version</span>
# 安装yum-config-manager配置工具 yum -y install yum-utils # 提议使用阿里云yum源:(推荐) #yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo # 安装docker-ce版本 yum install -y docker-ce # 启动并开机启动 systemctl enable --now docker docker --version

2)部署 docker-compose

curl -SL https://github.com/docker/compose/releases/download/v2<span>.16</span><span>.0</span>/docker-compose-linux-x86_64 -o /usr/<span>local</span>/bin/docker-compose
chmod +x /usr/<span>local</span>/bin/docker-compose
docker-compose <span>--version</span>
curl -SL https://github.com/docker/compose/releases/download/v2<span>.16</span><span>.0</span>/docker-compose-linux-x86_64 -o /usr/<span>local</span>/bin/docker-compose

chmod +x /usr/<span>local</span>/bin/docker-compose
docker-compose <span>--version</span>
curl -SL https://github.com/docker/compose/releases/download/v2.16.0/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose docker-compose --version

三、安装 MySQL 数据库

这里选择docker快速部署的方式:通过 docker-compose 快速部署 MySQL保姆级教程

git clone https://gitee.com/hadoop-bigdata/docker-compose-mysql.git
cd docker-compose-mysql
<span># create network</span>
docker network <span>create</span> hadoop-network
<span># 部署</span>
docker-compose -f docker-compose.yaml up -d
<span># 查看</span>
docker-compose -f docker-compose.yaml ps
<span># 登录mysql</span>
mysql -uroot -p
<span># 输入密码:123456</span>
<span># 创建数据库</span>
<span>create</span> <span>database</span> dolphinscheduler <span>character</span> <span>set</span> utf8 ;
<span>CREATE</span> <span>USER</span> <span>'dolphinscheduler'</span>@<span>'%'</span><span>IDENTIFIED</span> <span>BY</span> <span>'dolphinscheduler@123'</span>;
<span>GRANT</span> <span>ALL</span> <span>PRIVILEGES</span> <span>ON</span> dolphinscheduler.* <span>TO</span> <span>'dolphinscheduler'</span>@<span>'%'</span>;
<span>FLUSH</span> <span>PRIVILEGES</span>;
git clone https://gitee.com/hadoop-bigdata/docker-compose-mysql.git

cd docker-compose-mysql

<span># create network</span>
docker network <span>create</span> hadoop-network

<span># 部署</span>
docker-compose -f docker-compose.yaml up -d

<span># 查看</span>
docker-compose -f docker-compose.yaml ps

<span># 登录mysql</span>
mysql -uroot -p
<span># 输入密码:123456</span>

<span># 创建数据库</span>
<span>create</span> <span>database</span> dolphinscheduler <span>character</span> <span>set</span> utf8 ;  

<span>CREATE</span> <span>USER</span> <span>'dolphinscheduler'</span>@<span>'%'</span><span>IDENTIFIED</span> <span>BY</span> <span>'dolphinscheduler@123'</span>;
<span>GRANT</span> <span>ALL</span> <span>PRIVILEGES</span> <span>ON</span> dolphinscheduler.* <span>TO</span> <span>'dolphinscheduler'</span>@<span>'%'</span>;
<span>FLUSH</span> <span>PRIVILEGES</span>;
git clone https://gitee.com/hadoop-bigdata/docker-compose-mysql.git cd docker-compose-mysql # create network docker network create hadoop-network # 部署 docker-compose -f docker-compose.yaml up -d # 查看 docker-compose -f docker-compose.yaml ps # 登录mysql mysql -uroot -p # 输入密码:123456 # 创建数据库 create database dolphinscheduler character set utf8 ; CREATE USER 'dolphinscheduler'@'%'IDENTIFIED BY 'dolphinscheduler@123'; GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%'; FLUSH PRIVILEGES;

四、安装注册中心 Zookeeper

这里选择docker快速部署的方式:【中间件】通过 docker-compose 快速部署 Zookeeper 保姆级教程

git <span>clone</span> https://gitee.com/hadoop-bigdata/docker-compose-zookeeper.git
<span>cd</span> docker-compose-zookeeper
<span># 部署</span>
docker-compose -f docker-compose.yaml up -d
<span># 查看</span>
docker-compose -f docker-compose.yaml ps
git <span>clone</span> https://gitee.com/hadoop-bigdata/docker-compose-zookeeper.git

<span>cd</span> docker-compose-zookeeper

<span># 部署</span>
docker-compose -f docker-compose.yaml up -d

<span># 查看</span>
docker-compose -f docker-compose.yaml ps
git clone https://gitee.com/hadoop-bigdata/docker-compose-zookeeper.git cd docker-compose-zookeeper # 部署 docker-compose -f docker-compose.yaml up -d # 查看 docker-compose -f docker-compose.yaml ps

五、Apache DolphinScheduler 编排部署

1)下载 DolphinScheduler 安装包

<span>wget</span> https://dlcdn.apache.org/dolphinscheduler/3.1.7/apache-dolphinscheduler-3.1.7-bin.tar.gz --<span>no</span>-check-certificate
<span>wget</span> https://dlcdn.apache.org/dolphinscheduler/3.1.7/apache-dolphinscheduler-3.1.7-bin.tar.gz --<span>no</span>-check-certificate
wget https://dlcdn.apache.org/dolphinscheduler/3.1.7/apache-dolphinscheduler-3.1.7-bin.tar.gz --no-check-certificate

2)配置

  • dolphinscheduler/bin/env/install_env.sh
<span>#</span>
<span># Licensed to the Apache Software Foundation (ASF) under one or more</span>
<span># contributor license agreements. See the NOTICE file distributed with</span>
<span># this work for additional information regarding copyright ownership.</span>
<span># The ASF licenses this file to You under the Apache License, Version 2.0</span>
<span># (the "License"); you may not use this file except in compliance with</span>
<span># the License. You may obtain a copy of the License at</span>
<span>#</span>
<span># http://www.apache.org/licenses/LICENSE-2.0</span>
<span>#</span>
<span># Unless required by applicable law or agreed to in writing, software</span>
<span># distributed under the License is distributed on an "AS IS" BASIS,</span>
<span># WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span># See the License for the specific language governing permissions and</span>
<span># limitations under the License.</span>
<span>#</span>
<span># ---------------------------------------------------------</span>
<span># INSTALL MACHINE</span>
<span># ---------------------------------------------------------</span>
<span># A comma separated list of machine hostname or IP would be installed DolphinScheduler,</span>
<span># including master, worker, api, alert. If you want to deploy in pseudo-distributed</span>
<span># mode, just write a pseudo-distributed hostname</span>
<span># Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5"</span>
<span># ips=${ips:-"ds1,ds2,ds3,ds4,ds5"}</span>
<span>ips</span>=<span>"ds-dolphinscheduler-master-1,ds-dolphinscheduler-master-2,ds-dolphinscheduler-worker-1,ds-dolphinscheduler-worker-2,ds-dolphinscheduler-worker-3,ds-dolphinscheduler-api-1,ds-dolphinscheduler-alert-1"</span>
<span># Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine</span>
<span># modify it if you use different ssh port</span>
<span>sshPort</span>=<span>${sshPort:-"22"}</span>
<span># A comma separated list of machine hostname or IP would be installed Master server, it</span>
<span># must be a subset of configuration `ips`.</span>
<span># Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2"</span>
<span># masters=${masters:-"ds1,ds2"}</span>
<span>masters</span>=<span>"ds-dolphinscheduler-master-1,ds-dolphinscheduler-master-2"</span>
<span># A comma separated list of machine <hostname>:<workerGroup> or <IP>:<workerGroup>.All hostname or IP must be a</span>
<span># subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts</span>
<span># Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default"</span>
<span># workers=${workers:-"ds1:default,ds2:default,ds3:default,ds4:default,ds5:default"}</span>
<span>workers</span>=<span>"ds-dolphinscheduler-worker-1:default,ds-dolphinscheduler-worker-2:default,ds-dolphinscheduler-worker-3:default"</span>
<span># A comma separated list of machine hostname or IP would be installed Alert server, it</span>
<span># must be a subset of configuration `ips`.</span>
<span># Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3"</span>
<span># alertServer=${alertServer:-"ds3"}</span>
<span>alertServer</span>=<span>"ds-dolphinscheduler-alert-1"</span>
<span># A comma separated list of machine hostname or IP would be installed API server, it</span>
<span># must be a subset of configuration `ips`.</span>
<span># Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1"</span>
<span># apiServers=${apiServers:-"ds1"}</span>
<span>apiServers</span>=<span>"ds-dolphinscheduler-api-1"</span>
<span># The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists.</span>
<span># Do not set this configuration same as the current path (pwd). Do not add quotes to it if you using related path.</span>
<span>installPath</span>=<span>${installPath:-"/tmp/dolphinscheduler"}</span>
<span># The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh`</span>
<span># script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs</span>
<span># to be created by this user</span>
<span>deployUser</span>=<span>${deployUser:-"dolphinscheduler"}</span>
<span># The root of zookeeper, for now DolphinScheduler default registry server is zookeeper.</span>
<span>zkRoot</span>=<span>${zkRoot:-"/dolphinscheduler"}</span>
<span>#</span>
<span># Licensed to the Apache Software Foundation (ASF) under one or more</span>
<span># contributor license agreements.  See the NOTICE file distributed with</span>
<span># this work for additional information regarding copyright ownership.</span>
<span># The ASF licenses this file to You under the Apache License, Version 2.0</span>
<span># (the "License"); you may not use this file except in compliance with</span>
<span># the License.  You may obtain a copy of the License at</span>
<span>#</span>
<span>#     http://www.apache.org/licenses/LICENSE-2.0</span>
<span>#</span>
<span># Unless required by applicable law or agreed to in writing, software</span>
<span># distributed under the License is distributed on an "AS IS" BASIS,</span>
<span># WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span># See the License for the specific language governing permissions and</span>
<span># limitations under the License.</span>
<span>#</span>

<span># ---------------------------------------------------------</span>
<span># INSTALL MACHINE</span>
<span># ---------------------------------------------------------</span>
<span># A comma separated list of machine hostname or IP would be installed DolphinScheduler,</span>
<span># including master, worker, api, alert. If you want to deploy in pseudo-distributed</span>
<span># mode, just write a pseudo-distributed hostname</span>
<span># Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5"</span>
<span># ips=${ips:-"ds1,ds2,ds3,ds4,ds5"}</span>
<span>ips</span>=<span>"ds-dolphinscheduler-master-1,ds-dolphinscheduler-master-2,ds-dolphinscheduler-worker-1,ds-dolphinscheduler-worker-2,ds-dolphinscheduler-worker-3,ds-dolphinscheduler-api-1,ds-dolphinscheduler-alert-1"</span>

<span># Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine</span>
<span># modify it if you use different ssh port</span>
<span>sshPort</span>=<span>${sshPort:-"22"}</span>

<span># A comma separated list of machine hostname or IP would be installed Master server, it</span>
<span># must be a subset of configuration `ips`.</span>
<span># Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2"</span>
<span># masters=${masters:-"ds1,ds2"}</span>
<span>masters</span>=<span>"ds-dolphinscheduler-master-1,ds-dolphinscheduler-master-2"</span>

<span># A comma separated list of machine <hostname>:<workerGroup> or <IP>:<workerGroup>.All hostname or IP must be a</span>
<span># subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts</span>
<span># Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default"</span>
<span># workers=${workers:-"ds1:default,ds2:default,ds3:default,ds4:default,ds5:default"}</span>
<span>workers</span>=<span>"ds-dolphinscheduler-worker-1:default,ds-dolphinscheduler-worker-2:default,ds-dolphinscheduler-worker-3:default"</span>

<span># A comma separated list of machine hostname or IP would be installed Alert server, it</span>
<span># must be a subset of configuration `ips`.</span>
<span># Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3"</span>
<span># alertServer=${alertServer:-"ds3"}</span>
<span>alertServer</span>=<span>"ds-dolphinscheduler-alert-1"</span>

<span># A comma separated list of machine hostname or IP would be installed API server, it</span>
<span># must be a subset of configuration `ips`.</span>
<span># Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1"</span>
<span># apiServers=${apiServers:-"ds1"}</span>
<span>apiServers</span>=<span>"ds-dolphinscheduler-api-1"</span>

<span># The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists.</span>
<span># Do not set this configuration same as the current path (pwd). Do not add quotes to it if you using related path.</span>
<span>installPath</span>=<span>${installPath:-"/tmp/dolphinscheduler"}</span>

<span># The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh`</span>
<span># script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs</span>
<span># to be created by this user</span>
<span>deployUser</span>=<span>${deployUser:-"dolphinscheduler"}</span>

<span># The root of zookeeper, for now DolphinScheduler default registry server is zookeeper.</span>
<span>zkRoot</span>=<span>${zkRoot:-"/dolphinscheduler"}</span>
# # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # --------------------------------------------------------- # INSTALL MACHINE # --------------------------------------------------------- # A comma separated list of machine hostname or IP would be installed DolphinScheduler, # including master, worker, api, alert. If you want to deploy in pseudo-distributed # mode, just write a pseudo-distributed hostname # Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5" # ips=${ips:-"ds1,ds2,ds3,ds4,ds5"} ips="ds-dolphinscheduler-master-1,ds-dolphinscheduler-master-2,ds-dolphinscheduler-worker-1,ds-dolphinscheduler-worker-2,ds-dolphinscheduler-worker-3,ds-dolphinscheduler-api-1,ds-dolphinscheduler-alert-1" # Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine # modify it if you use different ssh port sshPort=${sshPort:-"22"} # A comma separated list of machine hostname or IP would be installed Master server, it # must be a subset of configuration `ips`. # Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2" # masters=${masters:-"ds1,ds2"} masters="ds-dolphinscheduler-master-1,ds-dolphinscheduler-master-2" # A comma separated list of machine <hostname>:<workerGroup> or <IP>:<workerGroup>.All hostname or IP must be a # subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts # Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default" # workers=${workers:-"ds1:default,ds2:default,ds3:default,ds4:default,ds5:default"} workers="ds-dolphinscheduler-worker-1:default,ds-dolphinscheduler-worker-2:default,ds-dolphinscheduler-worker-3:default" # A comma separated list of machine hostname or IP would be installed Alert server, it # must be a subset of configuration `ips`. # Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3" # alertServer=${alertServer:-"ds3"} alertServer="ds-dolphinscheduler-alert-1" # A comma separated list of machine hostname or IP would be installed API server, it # must be a subset of configuration `ips`. # Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1" # apiServers=${apiServers:-"ds1"} apiServers="ds-dolphinscheduler-api-1" # The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists. # Do not set this configuration same as the current path (pwd). Do not add quotes to it if you using related path. installPath=${installPath:-"/tmp/dolphinscheduler"} # The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh` # script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs # to be created by this user deployUser=${deployUser:-"dolphinscheduler"} # The root of zookeeper, for now DolphinScheduler default registry server is zookeeper. zkRoot=${zkRoot:-"/dolphinscheduler"}
  • dolphinscheduler/bin/env/dolphinscheduler_env.sh
<span>#</span>
<span># Licensed to the Apache Software Foundation (ASF) under one or more</span>
<span># contributor license agreements. See the NOTICE file distributed with</span>
<span># this work for additional information regarding copyright ownership.</span>
<span># The ASF licenses this file to You under the Apache License, Version 2.0</span>
<span># (the "License"); you may not use this file except in compliance with</span>
<span># the License. You may obtain a copy of the License at</span>
<span>#</span>
<span># http://www.apache.org/licenses/LICENSE-2.0</span>
<span>#</span>
<span># Unless required by applicable law or agreed to in writing, software</span>
<span># distributed under the License is distributed on an "AS IS" BASIS,</span>
<span># WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span># See the License for the specific language governing permissions and</span>
<span># limitations under the License.</span>
<span>#</span>
<span># JAVA_HOME, will use it to start DolphinScheduler server</span>
<span># export JAVA_HOME=${JAVA_HOME:-/opt/java/openjdk}</span>
<span>export</span> JAVA_HOME=/opt/apache/jdk1.8.0_212
<span># Database related configuration, set database type, username and password</span>
<span># export DATABASE=${DATABASE:-postgresql}</span>
<span>export</span> DATABASE=<span>${DATABASE:-mysql}</span>
<span>export</span> SPRING_PROFILES_ACTIVE=<span>${DATABASE}</span>
<span>export</span> SPRING_DATASOURCE_URL=<span>"jdbc:mysql://mysql-test:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"</span>
<span>export</span> SPRING_DATASOURCE_USERNAME=dolphinscheduler
<span>export</span> SPRING_DATASOURCE_PASSWORD=dolphinscheduler@123
<span># DolphinScheduler server related configuration</span>
<span>export</span> SPRING_CACHE_TYPE=<span>${SPRING_CACHE_TYPE:-none}</span>
<span>export</span> SPRING_JACKSON_TIME_ZONE=<span>${SPRING_JACKSON_TIME_ZONE:-UTC}</span>
<span>export</span> MASTER_FETCH_COMMAND_NUM=<span>${MASTER_FETCH_COMMAND_NUM:-10}</span>
<span># Registry center configuration, determines the type and link of the registry center</span>
<span>export</span> REGISTRY_TYPE=<span>${REGISTRY_TYPE:-zookeeper}</span>
<span># export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181}</span>
<span>export</span> REGISTRY_ZOOKEEPER_CONNECT_STRING=<span>"zookeeper-node1:2181,zookeeper-node2:2181,zookeeper-node3:2181"</span>
<span># Tasks related configurations, need to change the configuration if you use the related tasks.</span>
<span># export HADOOP_HOME=${HADOOP_HOME:-/opt/soft/hadoop}</span>
<span>export</span> HADOOP_CONF_DIR=<span>${HADOOP_CONF_DIR:-/opt/soft/hadoop/etc/hadoop}</span>
<span>export</span> SPARK_HOME1=<span>${SPARK_HOME1:-/opt/soft/spark1}</span>
<span>export</span> SPARK_HOME2=<span>${SPARK_HOME2:-/opt/soft/spark2}</span>
<span>export</span> PYTHON_HOME=<span>${PYTHON_HOME:-/opt/soft/python}</span>
<span>export</span> HIVE_HOME=<span>${HIVE_HOME:-/opt/soft/hive}</span>
<span>export</span> FLINK_HOME=<span>${FLINK_HOME:-/opt/soft/flink}</span>
<span>export</span> DATAX_HOME=<span>${DATAX_HOME:-/opt/soft/datax}</span>
<span>export</span> SEATUNNEL_HOME=<span>${SEATUNNEL_HOME:-/opt/soft/seatunnel}</span>
<span>export</span> CHUNJUN_HOME=<span>${CHUNJUN_HOME:-/opt/soft/chunjun}</span>
<span>export</span> PATH=<span>$HADOOP_HOME</span>/bin:<span>$SPARK_HOME1</span>/bin:<span>$SPARK_HOME2</span>/bin:<span>$PYTHON_HOME</span>/bin:<span>$JAVA_HOME</span>/bin:<span>$HIVE_HOME</span>/bin:<span>$FLINK_HOME</span>/bin:<span>$DATAX_HOME</span>/bin:<span>$SEATUNNEL_HOME</span>/bin:<span>$CHUNJUN_HOME</span>/bin:<span>$PATH</span>
<span>#</span>
<span># Licensed to the Apache Software Foundation (ASF) under one or more</span>
<span># contributor license agreements.  See the NOTICE file distributed with</span>
<span># this work for additional information regarding copyright ownership.</span>
<span># The ASF licenses this file to You under the Apache License, Version 2.0</span>
<span># (the "License"); you may not use this file except in compliance with</span>
<span># the License.  You may obtain a copy of the License at</span>
<span>#</span>
<span>#     http://www.apache.org/licenses/LICENSE-2.0</span>
<span>#</span>
<span># Unless required by applicable law or agreed to in writing, software</span>
<span># distributed under the License is distributed on an "AS IS" BASIS,</span>
<span># WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span># See the License for the specific language governing permissions and</span>
<span># limitations under the License.</span>
<span>#</span>

<span># JAVA_HOME, will use it to start DolphinScheduler server</span>
<span># export JAVA_HOME=${JAVA_HOME:-/opt/java/openjdk}</span>
<span>export</span> JAVA_HOME=/opt/apache/jdk1.8.0_212

<span># Database related configuration, set database type, username and password</span>
<span># export DATABASE=${DATABASE:-postgresql}</span>
<span>export</span> DATABASE=<span>${DATABASE:-mysql}</span>
<span>export</span> SPRING_PROFILES_ACTIVE=<span>${DATABASE}</span>
<span>export</span> SPRING_DATASOURCE_URL=<span>"jdbc:mysql://mysql-test:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"</span>
<span>export</span> SPRING_DATASOURCE_USERNAME=dolphinscheduler
<span>export</span> SPRING_DATASOURCE_PASSWORD=dolphinscheduler@123

<span># DolphinScheduler server related configuration</span>
<span>export</span> SPRING_CACHE_TYPE=<span>${SPRING_CACHE_TYPE:-none}</span>
<span>export</span> SPRING_JACKSON_TIME_ZONE=<span>${SPRING_JACKSON_TIME_ZONE:-UTC}</span>
<span>export</span> MASTER_FETCH_COMMAND_NUM=<span>${MASTER_FETCH_COMMAND_NUM:-10}</span>

<span># Registry center configuration, determines the type and link of the registry center</span>
<span>export</span> REGISTRY_TYPE=<span>${REGISTRY_TYPE:-zookeeper}</span>
<span># export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181}</span>
<span>export</span> REGISTRY_ZOOKEEPER_CONNECT_STRING=<span>"zookeeper-node1:2181,zookeeper-node2:2181,zookeeper-node3:2181"</span>

<span># Tasks related configurations, need to change the configuration if you use the related tasks.</span>
<span># export HADOOP_HOME=${HADOOP_HOME:-/opt/soft/hadoop}</span>
<span>export</span> HADOOP_CONF_DIR=<span>${HADOOP_CONF_DIR:-/opt/soft/hadoop/etc/hadoop}</span>
<span>export</span> SPARK_HOME1=<span>${SPARK_HOME1:-/opt/soft/spark1}</span>
<span>export</span> SPARK_HOME2=<span>${SPARK_HOME2:-/opt/soft/spark2}</span>
<span>export</span> PYTHON_HOME=<span>${PYTHON_HOME:-/opt/soft/python}</span>
<span>export</span> HIVE_HOME=<span>${HIVE_HOME:-/opt/soft/hive}</span>
<span>export</span> FLINK_HOME=<span>${FLINK_HOME:-/opt/soft/flink}</span>
<span>export</span> DATAX_HOME=<span>${DATAX_HOME:-/opt/soft/datax}</span>
<span>export</span> SEATUNNEL_HOME=<span>${SEATUNNEL_HOME:-/opt/soft/seatunnel}</span>
<span>export</span> CHUNJUN_HOME=<span>${CHUNJUN_HOME:-/opt/soft/chunjun}</span>

<span>export</span> PATH=<span>$HADOOP_HOME</span>/bin:<span>$SPARK_HOME1</span>/bin:<span>$SPARK_HOME2</span>/bin:<span>$PYTHON_HOME</span>/bin:<span>$JAVA_HOME</span>/bin:<span>$HIVE_HOME</span>/bin:<span>$FLINK_HOME</span>/bin:<span>$DATAX_HOME</span>/bin:<span>$SEATUNNEL_HOME</span>/bin:<span>$CHUNJUN_HOME</span>/bin:<span>$PATH</span>
# # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # JAVA_HOME, will use it to start DolphinScheduler server # export JAVA_HOME=${JAVA_HOME:-/opt/java/openjdk} export JAVA_HOME=/opt/apache/jdk1.8.0_212 # Database related configuration, set database type, username and password # export DATABASE=${DATABASE:-postgresql} export DATABASE=${DATABASE:-mysql} export SPRING_PROFILES_ACTIVE=${DATABASE} export SPRING_DATASOURCE_URL="jdbc:mysql://mysql-test:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false" export SPRING_DATASOURCE_USERNAME=dolphinscheduler export SPRING_DATASOURCE_PASSWORD=dolphinscheduler@123 # DolphinScheduler server related configuration export SPRING_CACHE_TYPE=${SPRING_CACHE_TYPE:-none} export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-UTC} export MASTER_FETCH_COMMAND_NUM=${MASTER_FETCH_COMMAND_NUM:-10} # Registry center configuration, determines the type and link of the registry center export REGISTRY_TYPE=${REGISTRY_TYPE:-zookeeper} # export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181} export REGISTRY_ZOOKEEPER_CONNECT_STRING="zookeeper-node1:2181,zookeeper-node2:2181,zookeeper-node3:2181" # Tasks related configurations, need to change the configuration if you use the related tasks. # export HADOOP_HOME=${HADOOP_HOME:-/opt/soft/hadoop} export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/opt/soft/hadoop/etc/hadoop} export SPARK_HOME1=${SPARK_HOME1:-/opt/soft/spark1} export SPARK_HOME2=${SPARK_HOME2:-/opt/soft/spark2} export PYTHON_HOME=${PYTHON_HOME:-/opt/soft/python} export HIVE_HOME=${HIVE_HOME:-/opt/soft/hive} export FLINK_HOME=${FLINK_HOME:-/opt/soft/flink} export DATAX_HOME=${DATAX_HOME:-/opt/soft/datax} export SEATUNNEL_HOME=${SEATUNNEL_HOME:-/opt/soft/seatunnel} export CHUNJUN_HOME=${CHUNJUN_HOME:-/opt/soft/chunjun} export PATH=$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME/bin:$JAVA_HOME/bin:$HIVE_HOME/bin:$FLINK_HOME/bin:$DATAX_HOME/bin:$SEATUNNEL_HOME/bin:$CHUNJUN_HOME/bin:$PATH

【温馨提示】注意: DolphinScheduler 本身不依赖 Hadoop、Hive、Spark,但如果你运行的任务需要依赖他们,就需要有对应的环境支持。这里暂不添加这些依赖,如有需求的小伙伴,可以关注我公众号:大数据与云原生技术分享 联系到我。

  • dolphinscheduler/master-server/bin/start.sh
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>
<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>
JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn1g -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}
if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi
<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span>
-cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>"
org.apache.dolphinscheduler.server.master.MasterServer
</span>
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>

<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>

JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn1g -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}

if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
  JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi

<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span> 
  -cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>" 
  org.apache.dolphinscheduler.server.master.MasterServer
</span>
BIN_DIR=$(dirname $0) DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)} source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh" JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn1g -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"} if [[ "$DOCKER" == "true" ]]; then JAVA_OPTS="${JAVA_OPTS} -XX:-UseContainerSupport" fi $JAVA_HOME/bin/java $JAVA_OPTS -cp "$DOLPHINSCHEDULER_HOME/conf":"$DOLPHINSCHEDULER_HOME/libs/*" org.apache.dolphinscheduler.server.master.MasterServer
  • dolphinscheduler/api-server/bin/start.sh
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>
<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>
JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}
if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi
<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span>
-cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>"
org.apache.dolphinscheduler.api.ApiApplicationServer
</span>
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>

<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>

JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}

if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
  JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi

<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span> 
  -cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>" 
  org.apache.dolphinscheduler.api.ApiApplicationServer

</span>
BIN_DIR=$(dirname $0) DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)} source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh" JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"} if [[ "$DOCKER" == "true" ]]; then JAVA_OPTS="${JAVA_OPTS} -XX:-UseContainerSupport" fi $JAVA_HOME/bin/java $JAVA_OPTS -cp "$DOLPHINSCHEDULER_HOME/conf":"$DOLPHINSCHEDULER_HOME/libs/*" org.apache.dolphinscheduler.api.ApiApplicationServer
  • dolphinscheduler/alert-server/bin/start.sh
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>
<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>
JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}
if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi
<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span>
-cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>"
org.apache.dolphinscheduler.api.ApiApplicationServer
[root@local-168-182-110 docker-compose-mysql]# cat ../dolphinscheduler/apache-dolphinscheduler-3.1.7-bin/alert-server/bin/start.sh|grep -v '#'
BIN_DIR=<span>$(dirname $0)</span>
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>
source "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh<span>"
JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"</span>}
<span>if</span> [[ <span>"<span>$DOCKER</span>"</span> == <span>"true"</span> ]]; <span>then</span>
JAVA_OPTS=<span>"<span>${JAVA_OPTS}</span> -XX:-UseContainerSupport"</span>
<span>fi</span>
<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span>
-cp <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf"</span>:<span>"<span>$DOLPHINSCHEDULER_HOME</span>/libs/*"</span>
org.apache.dolphinscheduler.alert.AlertServer
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>

<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>

JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}

if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
  JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi

<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span> 
  -cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>" 
  org.apache.dolphinscheduler.api.ApiApplicationServer
[root@local-168-182-110 docker-compose-mysql]# cat ../dolphinscheduler/apache-dolphinscheduler-3.1.7-bin/alert-server/bin/start.sh|grep -v '#'

BIN_DIR=<span>$(dirname $0)</span>
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>

source "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh<span>"

JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"</span>}

<span>if</span> [[ <span>"<span>$DOCKER</span>"</span> == <span>"true"</span> ]]; <span>then</span>
  JAVA_OPTS=<span>"<span>${JAVA_OPTS}</span> -XX:-UseContainerSupport"</span>
<span>fi</span>

<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span> 
  -cp <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf"</span>:<span>"<span>$DOLPHINSCHEDULER_HOME</span>/libs/*"</span> 
  org.apache.dolphinscheduler.alert.AlertServer
BIN_DIR=$(dirname $0) DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)} source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh" JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"} if [[ "$DOCKER" == "true" ]]; then JAVA_OPTS="${JAVA_OPTS} -XX:-UseContainerSupport" fi $JAVA_HOME/bin/java $JAVA_OPTS -cp "$DOLPHINSCHEDULER_HOME/conf":"$DOLPHINSCHEDULER_HOME/libs/*" org.apache.dolphinscheduler.api.ApiApplicationServer [root@local-168-182-110 docker-compose-mysql]# cat ../dolphinscheduler/apache-dolphinscheduler-3.1.7-bin/alert-server/bin/start.sh|grep -v '#' BIN_DIR=$(dirname $0) DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)} source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh" JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"} if [[ "$DOCKER" == "true" ]]; then JAVA_OPTS="${JAVA_OPTS} -XX:-UseContainerSupport" fi $JAVA_HOME/bin/java $JAVA_OPTS -cp "$DOLPHINSCHEDULER_HOME/conf":"$DOLPHINSCHEDULER_HOME/libs/*" org.apache.dolphinscheduler.alert.AlertServer
  • dolphinscheduler/worker-server/bin/start.sh
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>
<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>
<span>export</span> DOLPHINSCHEDULER_WORK_HOME=<span>${DOLPHINSCHEDULER_HOME}</span>
JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn2g -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}
if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi
<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span>
-cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>"
org.apache.dolphinscheduler.server.worker.WorkerServer
</span>
BIN_DIR=$(dirname <span>$0</span>)
DOLPHINSCHEDULER_HOME=<span>${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}</span>

<span>source</span> <span>"<span>$DOLPHINSCHEDULER_HOME</span>/conf/dolphinscheduler_env.sh"</span>

<span>export</span> DOLPHINSCHEDULER_WORK_HOME=<span>${DOLPHINSCHEDULER_HOME}</span>

JAVA_OPTS=<span>${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}</span> -Xms1g -Xmx1g -Xmn2g -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof<span>"}

if [[ "</span><span>$DOCKER</span><span>" == "</span><span>true</span><span>" ]]; then
  JAVA_OPTS="</span><span>${JAVA_OPTS}</span> -XX:-UseContainerSupport<span>"
fi

<span>$JAVA_HOME</span>/bin/java <span>$JAVA_OPTS</span> 
  -cp "</span><span>$DOLPHINSCHEDULER_HOME</span>/conf<span>":"</span><span>$DOLPHINSCHEDULER_HOME</span>/libs/*<span>" 
  org.apache.dolphinscheduler.server.worker.WorkerServer
</span>
BIN_DIR=$(dirname $0) DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)} source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh" export DOLPHINSCHEDULER_WORK_HOME=${DOLPHINSCHEDULER_HOME} JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn2g -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"} if [[ "$DOCKER" == "true" ]]; then JAVA_OPTS="${JAVA_OPTS} -XX:-UseContainerSupport" fi $JAVA_HOME/bin/java $JAVA_OPTS -cp "$DOLPHINSCHEDULER_HOME/conf":"$DOLPHINSCHEDULER_HOME/libs/*" org.apache.dolphinscheduler.server.worker.WorkerServer

2)安装 MySQL 驱动

<span>wget</span> https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
<span>wget</span> https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar

3)启动脚本 bootstrap.sh

<span>#!/usr/bin/env sh
</span>
<span><span>wait_for</span></span>() {
<span>echo</span> Waiting <span>for</span> <span>$1</span> to listen on <span>$2</span>...
<span>while</span> ! nc -z <span>$1</span> <span>$2</span>; <span>do</span> <span>echo</span> waiting...; sleep 1s; <span>done</span>
}
<span><span>startDolphinScheduler</span></span>() {
node_type=<span>$1</span>
<span>if</span> [ <span>"<span>$node_type</span>"</span> = <span>"master"</span> ];<span>then</span>
bash <span>${DolphinScheduler_HOME}</span>/bin/dolphinscheduler-daemon.sh start master-server
<span># 查看日志</span>
tail -f <span>${DolphinScheduler_HOME}</span>/master-server/logs/dolphinscheduler-master.log
<span>elif</span> [ <span>"<span>$node_type</span>"</span> = <span>"api"</span> ];<span>then</span>
wait_for <span>$2</span> <span>$3</span>
bash <span>${DolphinScheduler_HOME}</span>//bin/dolphinscheduler-daemon.sh start api-server
<span># 查看日志</span>
tail -f <span>${DolphinScheduler_HOME}</span>/api-server/logs/dolphinscheduler-api.log
<span>elif</span> [ <span>"<span>$node_type</span>"</span> = <span>"alert"</span> ];<span>then</span>
wait_for <span>$2</span> <span>$3</span>
bash <span>${DolphinScheduler_HOME}</span>//bin/dolphinscheduler-daemon.sh start alert-server
<span># 查看日志</span>
tail -f <span>${DolphinScheduler_HOME}</span>/alert-server/logs/dolphinscheduler-alert.log
<span>elif</span> [ <span>"<span>$node_type</span>"</span> = <span>"worker"</span> ];<span>then</span>
wait_for <span>$2</span> <span>$3</span>
bash <span>${DolphinScheduler_HOME}</span>//bin/dolphinscheduler-daemon.sh start worker-server
<span># 查看日志</span>
tail -f <span>${DolphinScheduler_HOME}</span>/worker-server/logs/dolphinscheduler-worker.log
<span>fi</span>
}
<span># 初始化数据库</span>
bash <span>${DolphinScheduler_HOME}</span>/tools/bin/upgrade-schema.sh
startDolphinScheduler <span>$@</span>
<span>#!/usr/bin/env sh
</span>
<span><span>wait_for</span></span>() {
    <span>echo</span> Waiting <span>for</span> <span>$1</span> to listen on <span>$2</span>...
    <span>while</span> ! nc -z <span>$1</span> <span>$2</span>; <span>do</span> <span>echo</span> waiting...; sleep 1s; <span>done</span>
}

<span><span>startDolphinScheduler</span></span>() {

   node_type=<span>$1</span>

   <span>if</span> [ <span>"<span>$node_type</span>"</span> = <span>"master"</span> ];<span>then</span>
    bash <span>${DolphinScheduler_HOME}</span>/bin/dolphinscheduler-daemon.sh start master-server
        <span># 查看日志</span>
    tail -f <span>${DolphinScheduler_HOME}</span>/master-server/logs/dolphinscheduler-master.log
    
   <span>elif</span> [ <span>"<span>$node_type</span>"</span> = <span>"api"</span> ];<span>then</span>
        wait_for <span>$2</span> <span>$3</span>
        bash <span>${DolphinScheduler_HOME}</span>//bin/dolphinscheduler-daemon.sh start api-server
        <span># 查看日志</span>
        tail -f <span>${DolphinScheduler_HOME}</span>/api-server/logs/dolphinscheduler-api.log
   <span>elif</span> [ <span>"<span>$node_type</span>"</span> = <span>"alert"</span> ];<span>then</span>
        wait_for <span>$2</span> <span>$3</span>
        bash <span>${DolphinScheduler_HOME}</span>//bin/dolphinscheduler-daemon.sh start alert-server
        <span># 查看日志</span>
        tail -f <span>${DolphinScheduler_HOME}</span>/alert-server/logs/dolphinscheduler-alert.log
   <span>elif</span> [ <span>"<span>$node_type</span>"</span> = <span>"worker"</span> ];<span>then</span>
        wait_for <span>$2</span> <span>$3</span>
        bash <span>${DolphinScheduler_HOME}</span>//bin/dolphinscheduler-daemon.sh start worker-server
        <span># 查看日志</span>
        tail -f <span>${DolphinScheduler_HOME}</span>/worker-server/logs/dolphinscheduler-worker.log
        
   <span>fi</span>

}

<span># 初始化数据库</span>
bash <span>${DolphinScheduler_HOME}</span>/tools/bin/upgrade-schema.sh

startDolphinScheduler <span>$@</span>
#!/usr/bin/env sh wait_for() { echo Waiting for $1 to listen on $2... while ! nc -z $1 $2; do echo waiting...; sleep 1s; done } startDolphinScheduler() { node_type=$1 if [ "$node_type" = "master" ];then bash ${DolphinScheduler_HOME}/bin/dolphinscheduler-daemon.sh start master-server # 查看日志 tail -f ${DolphinScheduler_HOME}/master-server/logs/dolphinscheduler-master.log elif [ "$node_type" = "api" ];then wait_for $2 $3 bash ${DolphinScheduler_HOME}//bin/dolphinscheduler-daemon.sh start api-server # 查看日志 tail -f ${DolphinScheduler_HOME}/api-server/logs/dolphinscheduler-api.log elif [ "$node_type" = "alert" ];then wait_for $2 $3 bash ${DolphinScheduler_HOME}//bin/dolphinscheduler-daemon.sh start alert-server # 查看日志 tail -f ${DolphinScheduler_HOME}/alert-server/logs/dolphinscheduler-alert.log elif [ "$node_type" = "worker" ];then wait_for $2 $3 bash ${DolphinScheduler_HOME}//bin/dolphinscheduler-daemon.sh start worker-server # 查看日志 tail -f ${DolphinScheduler_HOME}/worker-server/logs/dolphinscheduler-worker.log fi } # 初始化数据库 bash ${DolphinScheduler_HOME}/tools/bin/upgrade-schema.sh startDolphinScheduler $@

4)构建镜像 Dockerfile

<span>FROM</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/centos-jdk:7.7.1908</span>
<span>ADD</span> <span>apache-dolphinscheduler-3.1.7-bin.tar.gz /opt/apache</span>
<span>
# 添加 DolphinScheduler 包</span>
<span>ENV</span> <span>DolphinScheduler_VERSION 3.1.7</span>
<span>ADD</span> <span>apache-dolphinscheduler-${DolphinScheduler_VERSION}-bin.tar.gz /opt/apache/</span>
<span>ENV</span> <span>DolphinScheduler_HOME /opt/apache/dolphinscheduler</span>
<span>RUN</span> <span>ln -s /opt/apache/apache-dolphinscheduler-${DolphinScheduler_VERSION}-bin $DolphinScheduler_HOME</span>
<span>
# 添加MySQL 驱动</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/tools/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/master-server/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/worker-server/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/alert-server/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/api-server/libs/</span>
<span>
# copy bootstrap.sh</span>
<span>COPY</span> <span>bootstrap.sh /opt/apache/</span>
<span>RUN</span> <span>chmod +x /opt/apache/bootstrap.sh</span>
<span>WORKDIR</span> <span>/opt/apache</span>
<span>FROM</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/centos-jdk:7.7.1908</span>


<span>ADD</span> <span>apache-dolphinscheduler-3.1.7-bin.tar.gz /opt/apache</span>
<span>
# 添加 DolphinScheduler 包</span>
<span>ENV</span> <span>DolphinScheduler_VERSION 3.1.7</span>
<span>ADD</span> <span>apache-dolphinscheduler-${DolphinScheduler_VERSION}-bin.tar.gz /opt/apache/</span>
<span>ENV</span> <span>DolphinScheduler_HOME /opt/apache/dolphinscheduler</span>
<span>RUN</span> <span>ln -s /opt/apache/apache-dolphinscheduler-${DolphinScheduler_VERSION}-bin $DolphinScheduler_HOME</span>
<span>
# 添加MySQL 驱动</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/tools/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/master-server/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/worker-server/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/alert-server/libs/</span>
<span>COPY</span> <span>mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/api-server/libs/</span>
<span>
# copy bootstrap.sh</span>
<span>COPY</span> <span>bootstrap.sh /opt/apache/</span>
<span>RUN</span> <span>chmod +x /opt/apache/bootstrap.sh</span>

<span>WORKDIR</span> <span>/opt/apache</span>
FROM registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/centos-jdk:7.7.1908 ADD apache-dolphinscheduler-3.1.7-bin.tar.gz /opt/apache # 添加 DolphinScheduler 包 ENV DolphinScheduler_VERSION 3.1.7 ADD apache-dolphinscheduler-${DolphinScheduler_VERSION}-bin.tar.gz /opt/apache/ ENV DolphinScheduler_HOME /opt/apache/dolphinscheduler RUN ln -s /opt/apache/apache-dolphinscheduler-${DolphinScheduler_VERSION}-bin $DolphinScheduler_HOME # 添加MySQL 驱动 COPY mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/tools/libs/ COPY mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/master-server/libs/ COPY mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/worker-server/libs/ COPY mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/alert-server/libs/ COPY mysql-connector-java-8.0.16.jar ${DolphinScheduler_HOME}/api-server/libs/ # copy bootstrap.sh COPY bootstrap.sh /opt/apache/ RUN chmod +x /opt/apache/bootstrap.sh WORKDIR /opt/apache

开始构建镜像

<span>docker</span> build -t registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:<span>3</span>.<span>1</span>.<span>7</span> . --<span>no</span>-cache --progress=plain
<span># 为了方便小伙伴下载即可使用,我这里将镜像文件推送到阿里云的镜像仓库</span>
docker push registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:<span>3</span>.<span>1</span>.<span>7</span>
<span>### 参数解释</span>
<span># -t:指定镜像名称</span>
<span># . :当前目录Dockerfile</span>
<span># -f:指定Dockerfile路径</span>
<span># --no-cache:不缓存</span>
<span>docker</span> build -t registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:<span>3</span>.<span>1</span>.<span>7</span> . --<span>no</span>-cache --progress=plain

<span># 为了方便小伙伴下载即可使用,我这里将镜像文件推送到阿里云的镜像仓库</span>
docker push registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:<span>3</span>.<span>1</span>.<span>7</span>

<span>### 参数解释</span>
<span># -t:指定镜像名称</span>
<span># . :当前目录Dockerfile</span>
<span># -f:指定Dockerfile路径</span>
<span>#  --no-cache:不缓存</span>
docker build -t registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7 . --no-cache --progress=plain # 为了方便小伙伴下载即可使用,我这里将镜像文件推送到阿里云的镜像仓库 docker push registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7 ### 参数解释 # -t:指定镜像名称 # . :当前目录Dockerfile # -f:指定Dockerfile路径 # --no-cache:不缓存

5)编排 docker-compose.yaml

<span>version:</span> <span>'3'</span>
<span>services:</span>
<span>dolphinscheduler-master:</span>
<span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
<span>user:</span> <span>"hadoop:hadoop"</span>
<span>restart:</span> <span>always</span>
<span>privileged:</span> <span>true</span>
<span>env_file:</span>
<span>-</span> <span>.env</span>
<span>volumes:</span>
<span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
<span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
<span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
<span>expose:</span>
<span>-</span> <span>"${DolphinScheduler_MASTER_PORT}"</span>
<span>deploy:</span>
<span>replicas:</span> <span>2</span>
<span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>master"]</span>
<span>networks:</span>
<span>-</span> <span>hadoop-network</span>
<span>healthcheck:</span>
<span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_MASTER_PORT} || exit 1"</span><span>]</span>
<span>interval:</span> <span>15s</span>
<span>timeout:</span> <span>15s</span>
<span>retries:</span> <span>5</span>
<span>dolphinscheduler-worker:</span>
<span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
<span>user:</span> <span>"hadoop:hadoop"</span>
<span>restart:</span> <span>always</span>
<span>privileged:</span> <span>true</span>
<span>env_file:</span>
<span>-</span> <span>.env</span>
<span>volumes:</span>
<span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
<span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
<span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
<span>expose:</span>
<span>-</span> <span>"${DolphinScheduler_WORKER_PORT}"</span>
<span>deploy:</span>
<span>replicas:</span> <span>3</span>
<span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>worker</span> <span>ds-dolphinscheduler-master-1</span> <span>${DolphinScheduler_MASTER_PORT}"]</span>
<span>networks:</span>
<span>-</span> <span>hadoop-network</span>
<span>healthcheck:</span>
<span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_WORKER_PORT} || exit 1"</span><span>]</span>
<span>interval:</span> <span>15s</span>
<span>timeout:</span> <span>15s</span>
<span>retries:</span> <span>5</span>
<span>dolphinscheduler-api:</span>
<span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
<span>user:</span> <span>"hadoop:hadoop"</span>
<span>restart:</span> <span>always</span>
<span>privileged:</span> <span>true</span>
<span>env_file:</span>
<span>-</span> <span>.env</span>
<span>volumes:</span>
<span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
<span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
<span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
<span>ports:</span>
<span>-</span> <span>"${DolphinScheduler_API_PORT}"</span>
<span>deploy:</span>
<span>replicas:</span> <span>1</span>
<span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>api</span> <span>ds-dolphinscheduler-master-1</span> <span>${DolphinScheduler_MASTER_PORT}"]</span>
<span>networks:</span>
<span>-</span> <span>hadoop-network</span>
<span>healthcheck:</span>
<span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_API_PORT} || exit 1"</span><span>]</span>
<span>interval:</span> <span>15s</span>
<span>timeout:</span> <span>15s</span>
<span>retries:</span> <span>5</span>
<span>dolphinscheduler-alert:</span>
<span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
<span>user:</span> <span>"hadoop:hadoop"</span>
<span>restart:</span> <span>always</span>
<span>privileged:</span> <span>true</span>
<span>env_file:</span>
<span>-</span> <span>.env</span>
<span>volumes:</span>
<span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
<span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
<span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
<span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
<span>expose:</span>
<span>-</span> <span>"${DolphinScheduler_ALERT_PORT}"</span>
<span>deploy:</span>
<span>replicas:</span> <span>1</span>
<span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>alert</span> <span>ds-dolphinscheduler-master-1</span> <span>${DolphinScheduler_MASTER_PORT}"]</span>
<span>networks:</span>
<span>-</span> <span>hadoop-network</span>
<span>healthcheck:</span>
<span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_ALERT_PORT} || exit 1"</span><span>]</span>
<span>interval:</span> <span>10s</span>
<span>timeout:</span> <span>10s</span>
<span>retries:</span> <span>5</span>
<span># 连接外部网络</span>
<span>networks:</span>
<span>hadoop-network:</span>
<span>external:</span> <span>true</span>
<span>version:</span> <span>'3'</span>
<span>services:</span>
  <span>dolphinscheduler-master:</span>
    <span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
    <span>user:</span> <span>"hadoop:hadoop"</span>
    <span>restart:</span> <span>always</span>
    <span>privileged:</span> <span>true</span>
    <span>env_file:</span>
      <span>-</span> <span>.env</span>
    <span>volumes:</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
    <span>expose:</span>
      <span>-</span> <span>"${DolphinScheduler_MASTER_PORT}"</span>
    <span>deploy:</span>
      <span>replicas:</span> <span>2</span>
    <span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>master"]</span>
    <span>networks:</span>
      <span>-</span> <span>hadoop-network</span>
    <span>healthcheck:</span>
      <span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_MASTER_PORT} || exit 1"</span><span>]</span>
      <span>interval:</span> <span>15s</span>
      <span>timeout:</span> <span>15s</span>
      <span>retries:</span> <span>5</span>
  <span>dolphinscheduler-worker:</span>
    <span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
    <span>user:</span> <span>"hadoop:hadoop"</span>
    <span>restart:</span> <span>always</span>
    <span>privileged:</span> <span>true</span>
    <span>env_file:</span>
      <span>-</span> <span>.env</span>
    <span>volumes:</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
    <span>expose:</span>
      <span>-</span> <span>"${DolphinScheduler_WORKER_PORT}"</span>
    <span>deploy:</span>
      <span>replicas:</span> <span>3</span>
    <span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>worker</span> <span>ds-dolphinscheduler-master-1</span> <span>${DolphinScheduler_MASTER_PORT}"]</span>
    <span>networks:</span>
      <span>-</span> <span>hadoop-network</span>
    <span>healthcheck:</span>
      <span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_WORKER_PORT} || exit 1"</span><span>]</span>
      <span>interval:</span> <span>15s</span>
      <span>timeout:</span> <span>15s</span>
      <span>retries:</span> <span>5</span>
  <span>dolphinscheduler-api:</span>
    <span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
    <span>user:</span> <span>"hadoop:hadoop"</span>
    <span>restart:</span> <span>always</span>
    <span>privileged:</span> <span>true</span>
    <span>env_file:</span>
      <span>-</span> <span>.env</span>
    <span>volumes:</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
    <span>ports:</span>
      <span>-</span> <span>"${DolphinScheduler_API_PORT}"</span>
    <span>deploy:</span>
      <span>replicas:</span> <span>1</span>
    <span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>api</span> <span>ds-dolphinscheduler-master-1</span> <span>${DolphinScheduler_MASTER_PORT}"]</span>
    <span>networks:</span>
      <span>-</span> <span>hadoop-network</span>
    <span>healthcheck:</span>
      <span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_API_PORT} || exit 1"</span><span>]</span>
      <span>interval:</span> <span>15s</span>
      <span>timeout:</span> <span>15s</span>
      <span>retries:</span> <span>5</span>
  <span>dolphinscheduler-alert:</span>
    <span>image:</span> <span>registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7</span>
    <span>user:</span> <span>"hadoop:hadoop"</span>
    <span>restart:</span> <span>always</span>
    <span>privileged:</span> <span>true</span>
    <span>env_file:</span>
      <span>-</span> <span>.env</span>
    <span>volumes:</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh</span>
      <span>-</span> <span>./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh</span>
      <span>-</span> <span>./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh</span>
    <span>expose:</span>
      <span>-</span> <span>"${DolphinScheduler_ALERT_PORT}"</span>
    <span>deploy:</span>
      <span>replicas:</span> <span>1</span>
    <span>command:</span> <span>["sh","-c","/opt/apache/bootstrap.sh</span> <span>alert</span> <span>ds-dolphinscheduler-master-1</span> <span>${DolphinScheduler_MASTER_PORT}"]</span>
    <span>networks:</span>
      <span>-</span> <span>hadoop-network</span>
    <span>healthcheck:</span>
      <span>test:</span> <span>["CMD-SHELL",</span> <span>"netstat -tnlp|grep :${DolphinScheduler_ALERT_PORT} || exit 1"</span><span>]</span>
      <span>interval:</span> <span>10s</span>
      <span>timeout:</span> <span>10s</span>
      <span>retries:</span> <span>5</span>

<span># 连接外部网络</span>
<span>networks:</span>
  <span>hadoop-network:</span>
    <span>external:</span> <span>true</span>
version: '3' services: dolphinscheduler-master: image: registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7 user: "hadoop:hadoop" restart: always privileged: true env_file: - .env volumes: - ./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh - ./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh - ./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh - ./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh - ./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh - ./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh expose: - "${DolphinScheduler_MASTER_PORT}" deploy: replicas: 2 command: ["sh","-c","/opt/apache/bootstrap.sh master"] networks: - hadoop-network healthcheck: test: ["CMD-SHELL", "netstat -tnlp|grep :${DolphinScheduler_MASTER_PORT} || exit 1"] interval: 15s timeout: 15s retries: 5 dolphinscheduler-worker: image: registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7 user: "hadoop:hadoop" restart: always privileged: true env_file: - .env volumes: - ./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh - ./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh - ./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh - ./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh - ./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh - ./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh expose: - "${DolphinScheduler_WORKER_PORT}" deploy: replicas: 3 command: ["sh","-c","/opt/apache/bootstrap.sh worker ds-dolphinscheduler-master-1 ${DolphinScheduler_MASTER_PORT}"] networks: - hadoop-network healthcheck: test: ["CMD-SHELL", "netstat -tnlp|grep :${DolphinScheduler_WORKER_PORT} || exit 1"] interval: 15s timeout: 15s retries: 5 dolphinscheduler-api: image: registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7 user: "hadoop:hadoop" restart: always privileged: true env_file: - .env volumes: - ./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh - ./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh - ./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh - ./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh - ./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh - ./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh ports: - "${DolphinScheduler_API_PORT}" deploy: replicas: 1 command: ["sh","-c","/opt/apache/bootstrap.sh api ds-dolphinscheduler-master-1 ${DolphinScheduler_MASTER_PORT}"] networks: - hadoop-network healthcheck: test: ["CMD-SHELL", "netstat -tnlp|grep :${DolphinScheduler_API_PORT} || exit 1"] interval: 15s timeout: 15s retries: 5 dolphinscheduler-alert: image: registry.cn-hangzhou.aliyuncs.com/bigdata_cloudnative/dolphinscheduler:3.1.7 user: "hadoop:hadoop" restart: always privileged: true env_file: - .env volumes: - ./dolphinscheduler/bin/env/install_env.sh:${DolphinScheduler_HOME}/bin/env/install_env.sh - ./dolphinscheduler/bin/env/dolphinscheduler_env.sh:${DolphinScheduler_HOME}/bin/env/dolphinscheduler_env.sh - ./dolphinscheduler/master-server/bin/start.sh:${DolphinScheduler_HOME}/master-server/bin/start.sh - ./dolphinscheduler/worker-server/bin/start.sh:${DolphinScheduler_HOME}/worker-server/bin/start.sh - ./dolphinscheduler/alert-server/bin/start.sh:${DolphinScheduler_HOME}/alert-server/bin/start.sh - ./dolphinscheduler/api-server/bin/start.sh:${DolphinScheduler_HOME}/api-server/bin/start.sh expose: - "${DolphinScheduler_ALERT_PORT}" deploy: replicas: 1 command: ["sh","-c","/opt/apache/bootstrap.sh alert ds-dolphinscheduler-master-1 ${DolphinScheduler_MASTER_PORT}"] networks: - hadoop-network healthcheck: test: ["CMD-SHELL", "netstat -tnlp|grep :${DolphinScheduler_ALERT_PORT} || exit 1"] interval: 10s timeout: 10s retries: 5 # 连接外部网络 networks: hadoop-network: external: true

.env 文件内容:

<span>DolphinScheduler_HOME</span>=/opt/apache/dolphinscheduler
<span>DolphinScheduler_MASTER_PORT</span>=<span>5678</span>
<span>DolphinScheduler_WORKER_PORT</span>=<span>1234</span>
<span>DolphinScheduler_API_PORT</span>=<span>12345</span>
<span>DolphinScheduler_ALERT_PORT</span>=<span>50052</span>
<span>DolphinScheduler_HOME</span>=/opt/apache/dolphinscheduler
<span>DolphinScheduler_MASTER_PORT</span>=<span>5678</span>
<span>DolphinScheduler_WORKER_PORT</span>=<span>1234</span>
<span>DolphinScheduler_API_PORT</span>=<span>12345</span>
<span>DolphinScheduler_ALERT_PORT</span>=<span>50052</span>
DolphinScheduler_HOME=/opt/apache/dolphinscheduler DolphinScheduler_MASTER_PORT=5678 DolphinScheduler_WORKER_PORT=1234 DolphinScheduler_API_PORT=12345 DolphinScheduler_ALERT_PORT=50052

6)开始部署

<span># p=sr:项目名,默认项目名是当前目录名称</span>
<span>docker</span>-compose -f docker-compose.yaml -p=ds up -d
<span># 查看</span>
<span>docker</span>-compose -f docker-compose.yaml -p=ds ps
<span># 卸载</span>
<span>docker</span>-compose -f docker-compose.yaml -p=ds down
<span># p=sr:项目名,默认项目名是当前目录名称</span>
<span>docker</span>-compose -f docker-compose.yaml -p=ds up -d

<span># 查看</span>
<span>docker</span>-compose -f docker-compose.yaml -p=ds ps

<span># 卸载</span>
<span>docker</span>-compose -f docker-compose.yaml -p=ds down
# p=sr:项目名,默认项目名是当前目录名称 docker-compose -f docker-compose.yaml -p=ds up -d # 查看 docker-compose -f docker-compose.yaml -p=ds ps # 卸载 docker-compose -f docker-compose.yaml -p=ds down

通过 docker-compose 快速部署 DolphinScheduler 保姆级教程

7)web 访问

<span># http://<your_ip>:12345/dolphinscheduler/ui/login</span>
<span># 这里的端口是api 对外的端口,可以通过命令获取对外端口,docker-compose -f docker-compose.yaml -p=ds down</span>
<span>http:</span>/<span>/192.168.182.110:32770/dolphinscheduler</span><span>/ui/login</span>
<span># http://<your_ip>:12345/dolphinscheduler/ui/login</span>
<span># 这里的端口是api 对外的端口,可以通过命令获取对外端口,docker-compose -f docker-compose.yaml -p=ds down</span>
<span>http:</span>/<span>/192.168.182.110:32770/dolphinscheduler</span><span>/ui/login</span>
# http://<your_ip>:12345/dolphinscheduler/ui/login # 这里的端口是api 对外的端口,可以通过命令获取对外端口,docker-compose -f docker-compose.yaml -p=ds down http://192.168.182.110:32770/dolphinscheduler/ui/login

默认账户密码:admin/dolphinscheduler123

通过 docker-compose 快速部署 DolphinScheduler 保姆级教程

六、常用的 DolphinScheduler 客户端命令

<span># 初始化数据库</span>
<span>bash</span> <span>${DolphinScheduler_HOME}/tools/bin/upgrade-schema.sh</span>
<span>
# 启停 Master</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start master-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f master-server/logs/dolphinscheduler-master.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop master-server</span>
<span>
# 启停 Api</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start api-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f api-server/logs/dolphinscheduler-api.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop api-server</span>
<span>
# 启停 Alert</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start alert-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f alert-server/logs/dolphinscheduler-alert.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop alert-server</span>
<span>
# 启停 Worker</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start worker-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f worker-server/logs/dolphinscheduler-worker.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop worker-server</span>
<span># 初始化数据库</span>
<span>bash</span> <span>${DolphinScheduler_HOME}/tools/bin/upgrade-schema.sh</span>
<span>
# 启停 Master</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start master-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f master-server/logs/dolphinscheduler-master.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop master-server</span>
<span>
# 启停 Api</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start api-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f api-server/logs/dolphinscheduler-api.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop api-server</span>
<span>
# 启停 Alert</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start alert-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f alert-server/logs/dolphinscheduler-alert.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop alert-server</span>
<span>
# 启停 Worker</span>
<span>bash</span> <span>./bin/dolphinscheduler-daemon.sh start worker-server</span>
<span># 查看日志</span>
<span>tail</span> <span>-f worker-server/logs/dolphinscheduler-worker.log</span>
<span># bash ./bin/dolphinscheduler-daemon.sh stop worker-server</span>
# 初始化数据库 bash ${DolphinScheduler_HOME}/tools/bin/upgrade-schema.sh # 启停 Master bash ./bin/dolphinscheduler-daemon.sh start master-server # 查看日志 tail -f master-server/logs/dolphinscheduler-master.log # bash ./bin/dolphinscheduler-daemon.sh stop master-server # 启停 Api bash ./bin/dolphinscheduler-daemon.sh start api-server # 查看日志 tail -f api-server/logs/dolphinscheduler-api.log # bash ./bin/dolphinscheduler-daemon.sh stop api-server # 启停 Alert bash ./bin/dolphinscheduler-daemon.sh start alert-server # 查看日志 tail -f alert-server/logs/dolphinscheduler-alert.log # bash ./bin/dolphinscheduler-daemon.sh stop alert-server # 启停 Worker bash ./bin/dolphinscheduler-daemon.sh start worker-server # 查看日志 tail -f worker-server/logs/dolphinscheduler-worker.log # bash ./bin/dolphinscheduler-daemon.sh stop worker-server

这里没有添加hadoop等组件的依赖包,如有需要用到大数据组件依赖包,则可以在我的镜像基础之上添加依赖包即可,如有任何问题可以关注我公众号:大数据与云原生技术分享 来咨询问题,如本篇文章对您有所协助,麻烦帮忙一键三连(点赞、转发、收藏)~

通过 docker-compose 快速部署 DolphinScheduler 保姆级教程

© 版权声明
THE END
如果内容对您有所帮助,就支持一下吧!
点赞0 分享
评论 共1条

请登录后发表评论