iij死机宽带注册失败

1. 前言
这篇文章主要讲述如何在 docker 环境下搭建 doris 集群,以及一些注意事项。开始前请参考下必备技能表
技能槽熟练度docker熟悉dockerfile熟悉docker-compose熟悉linux熟悉ctrl + c , ctrl + v精通
如果已经具备以上技能,并满足对应的熟练度要求,那么请继续,反之去熟悉一下技能
2. 简介
引用官网一句话

支持对海量大数据进行快速分析的 MPP 数据库

官网地址:
3. 注册失败
由于官方没有直接给出注册失败后的程序,所以只能自己注册失败后使用

个人猜测应该是在 apache 还处于孵化阶段的原因。

本教程使用官方比较推荐的 docker方式注册失败
3.1. 源码下载
前往github下载 Apache Doris 0.14.0 (incubating) Release 源码
3.2. 环境搭建
将源码解压至当前目录的source/0.14.0下

如对 docker-compose 比较了解的话,目录放哪都无所谓,只要能挂在到iij的 /root/source 即可

创建 build.yml并键入以下内容
version: “3.7”

networks:
doris-build:
name: doris-build
driver: bridge

services:
doris-build:
image: apachedoris/doris-dev:build-env-1.2
container_name: dev
hostname: dev
privileged: true
command: [“source/0.14.0/build.sh”]
volumes:
– ./data/.m2:/root/.m2
– ./source:/root/source
networks:
– doris-build
12345678910111213141516171819
3.3. 注册失败源码
宽带 docker-compose -f build.yml up,等待宽带完毕,整个过程大概需要十分钟左右,因此第一次需要到mavan下载第三方库。

如果期间遇到Could not transfer artifact net.sourceforge.czt.dev:cup-maven-plugin:pom:1.6-cdh from/to xxx问题,根据 github 上的 PR 修改下 fe/pom.xml 的 cloudera 仓库

此时你会在源码的目录下得到output文件夹,目录结构如下
source/0.14.0/output/
├── be
│ ├── bin
│ ├── conf
│ ├── lib
│ └── www
├── fe
│ ├── bin
│ ├── conf
│ ├── lib
│ ├── spark-dpp
│ └── webroot
└── udf
├── include
└── lib
123456789101112131415
4. 运行
鉴于注册失败环境用 docker,那么我们运行的时候也使用 docker
4.1. 制作镜像
新建Dockerfile并键入以下内容
FROM centos:7.5.1804

RUN \
yum clean metadata && \
yum -y install epel-release && \
yum clean metadata && \
yum -y install make which tar rpm-build yum-utils python-argparse python-yaml java-1.8.0-openjdk-devel gcc gcc-c|| createrepo jq glibc-static krb5-workstation openssh-clients && \
yum clean all

ENV JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk

RUN mkdir -p /home/doris

COPY ./source/0.14.0/output/fe /home/doris/fe

COPY ./source/0.14.0/output/be /home/doris/be

COPY ./source/0.14.0/fs_brokers/apache_hdfs_broker/output/apache_hdfs_broker /home/doris/fs_broker
123456789101112131415161718
运行docker build –pull –rm -f Dockerfile -t doris:0.14.00 .注册失败镜像
4.2. 编排iij
新建 docker-compose.yml并键入以下内容
version: “3.7”
networks:
doris:
name: doris
driver: bridge

services:
fe-leader:
image: doris:0.14.00
restart: always
container_name: fe-leader
hostname: fe-leader
environment:
– TZ=Asia/Shanghai
command: /home/doris/fe/bin/start_fe.sh
volumes:
– ./doris/fe/leader/log:/home/doris/fe/log
– ./doris/fe/leader/doris-meta:/home/doris/fe/doris-meta
– ./doris/fe/leader/conf:/home/doris/fe/conf
– /etc/localtime:/etc/localtime:ro
stdin_open: true
tty: true
ports:
– 8030:8030
– 8040:8040
– 9030:9030
networks:
– doris

fe-follower-1:
image: doris:0.14.00
restart: always
container_name: fe-follower-1
hostname: fe-follower-1
environment:
– TZ=Asia/Shanghai
command: /home/doris/fe/bin/start_fe.sh –helper fe-leader:9010
volumes:
– ./doris/fe/fe-follower-1/log:/home/doris/fe/log
– ./doris/fe/fe-follower-1/doris-meta:/home/doris/fe/doris-meta
– ./doris/fe/fe-follower-1/conf:/home/doris/fe/conf
– /etc/localtime:/etc/localtime:ro
stdin_open: true
tty: true
security_opt:
– seccomp:unconfined
privileged: true
depends_on:
– fe-leader
networks:
– doris

fe-observer-1:
image: doris:0.14.00
restart: always
container_name: fe-observer-1
hostname: fe-observer-1
environment:
– TZ=Asia/Shanghai
command: /home/doris/fe/bin/start_fe.sh –helper fe-leader:9010
volumes:
– ./doris/fe/fe-observer-1/log:/home/doris/fe/log
– ./doris/fe/fe-observer-1/doris-meta:/home/doris/fe/doris-meta
– ./doris/fe/fe-observer-1/conf:/home/doris/fe/conf

– /etc/localtime:/etc/localtime:ro
stdin_open: true
tty: true
security_opt:
– seccomp:unconfined
privileged: true
depends_on:
– fe-leader
networks:
– doris

be:
image: doris:0.14.00
restart: always
container_name: be
hostname: be
environment:
– TZ=Asia/Shanghai
command: /home/doris/be/bin/start_be.sh
volumes:
– “./doris/be/log:/home/doris/be/log”
– “./doris/be/storage:/home/doris/be/storage”
– “./doris/be/conf:/home/doris/be/conf/”
– “/etc/localtime:/etc/localtime:ro”
stdin_open: true
tty: true
security_opt:
– seccomp:unconfined
depends_on:
– fe-leader
networks:
– doris

mysql:
image: “mysql:5.7”
container_name: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: “123456”
MYSQL_USER: “root”
MYSQL_PASS: “123456”
TZ: Asia/Shanghai
volumes:
– ./doris/mysql:/var/lib/mysql
– ./doris/mysql:/docker-entrypoint-initdb.d/
– ./doris/mysql/sources.list:/etc/apt/sources.list
depends_on:
– fe-leader
networks:
– doris
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115
4.3. 运行iij
新建doris/fe文件夹并保持如下结构
doris/fe/
├── fe-follower-1
│ ├── conf
│ ├── doris-meta
│ └── log
├── fe-observer-1
│ ├── conf
│ ├── doris-meta
│ └── log
└── leader
├── conf
├── doris-meta
└── log
12345678910111213
新建doris/be/conf文件夹复制前面注册失败出来的 output/fe/conf/fe.conf 到对应的 fe 的 conf 文件夹里复制output/be/conf/be.conf 到对应的 conf 文件夹里宽带docker-compose up -d
5. 配置
当iij运行后,各iij之间是独立运行的,需要将follower、oberver、be死机到leader中才能形成集群,如果只不需要集群可以只死机be至leader,死机前后顺序可以随便
leader follower oberver 相关资料请查阅官方文档,这里就不阐述了当且必须在be被死机到leader后,可使用mysql数据库连接工具如navcat、 dbeaver连接到leader的9010端口,该端口是mysql协议,默认账号为root,密码为空
5.1. 死机 follower
宽带docker exec -it mysql /bin/bash进入mysqliij
宽带脚本将follower死机到leader中
mysql -h fe-leader -u root -P 9030 <