在本地部署洞态IAST时，dtctl脚本自动从docker hub官方仓库拉取镜像，由于国内访问限制导致总是拉取失败的解决办法。

先修改docker的镜像源

cat /etc/docker/daemon.json

{
  "registry-mirrors" : ["https://docker.registry.cyou",
"https://docker-cf.registry.cyou",
"https://dockercf.jsdelivr.fyi",
"https://docker.jsdelivr.fyi",
"https://dockertest.jsdelivr.fyi",
"https://mirror.aliyuncs.com",
"https://dockerproxy.com",
"https://mirror.baidubce.com",
"https://docker.m.daocloud.io",
"https://docker.nju.edu.cn",
"https://docker.mirrors.sjtug.sjtu.edu.cn",
"https://docker.mirrors.ustc.edu.cn",
"https://mirror.iscas.ac.cn",
"https://docker.rainbond.cc",
"https://do.nark.eu.org",
"https://dc.j8.work",
"https://dockerproxy.com",
"https://gst6rzl9.mirror.aliyuncs.com",
"https://registry.docker-cn.com",
"http://hub-mirror.c.163.com",
"http://mirrors.ustc.edu.cn/",
"https://mirrors.tuna.tsinghua.edu.cn/",
"http://mirrors.sohu.com/" 
],
 "insecure-registries" : [
    "registry.docker-cn.com",
    "docker.mirrors.ustc.edu.cn"
    ],
"debug": true,
"experimental": false
}

拉取镜像

拉取以下镜像 docker pull dongtai/dongtai-web:latest

dongtai/dongtai-web:latest
dongtai/dongtai-server:latest
dongtai/dongtai-mysql:latest
dongtai/dongtai-redis:latest
dongtai/dongtai-openapi:latest
dongtai/dongtai-logrotate:latest
dongtai/dongtai-logstash:latest
dongtai/dongtai-engine:latest
dongtai/dongtai-engine-task:latest
dongtai-engine-task:latest

如果你选择自己的mysql、redis可以忽略拉取dongtai的镜像。

image-20250807150429811

修改 dtctl安装脚本的验证函数

image-20250807150520469

把 get_latest_image_tag_from_dockerhub 函数直接改成

get_latest_image_tag_from_dockerhub() {
   echo "latest"
}

注意 docker compose 的版本要求version 2.1.0及以上

image-20250807150533426

#先删除旧版
sudo apt remove docker-compose -y

# 创建插件目录
sudo mkdir -p /usr/local/lib/docker/cli-plugins

# 下载最新版本
sudo curl -SL https://github.com/docker/compose/releases/latest/download/docker-compose-linux-x86_64 \
  -o /usr/local/lib/docker/cli-plugins/docker-compose

# 赋予执行权限
sudo chmod +x /usr/local/lib/docker/cli-plugins/docker-compose

#验证版本
docker compose version

Docker-compose v2检测问题

新版本的docker compose没有连接符-

image-20250807150548164

但是dtctl安装脚本调用的全是 docker-compose 命令

image-20250807150604984

通过创建软连接的方式一次性解决问题

ln -sf /usr/local/lib/docker/cli-plugins/docker-compose /usr/bin/docker-compose

image-20250807150613127

使用国内过期的阿里云镜像仓库导致访问失败问题

image-20250807150625763

在 dtctl 脚本中找到 aliyuncs.com/huoxian_pub

image-20250807150639932

改成 dongtai，另外可以直接自己使用 docker pull 拉取镜像，不要用脚本去调用，直接删除pull操作

image-20250807150652408

#docker-compose -p $PROJECT_NAME -f <(echo "$docker_compose_file") pull

不过由于我们已经本地下载了latest tag的镜像，所以可以在安装时进行指定即可，这样也能直接跳过docker的pull操作，避免产生网络访问问题。

./dtctl install -v latest

本地自己要pull的镜像如下

# 拉取洞态所需全部镜像（对应最新版，如需指定版本把 :latest 换掉）
images=(
  dongtai/dongtai-web:latest
  dongtai/dongtai-server:latest
  dongtai/dongtai-mysql:latest
  dongtai/dongtai-redis:latest
  dongtai/dongtai-openapi:latest
  dongtai/dongtai-logrotate:latest
  dongtai/dongtai-logstash:latest
  dongtai/dongtai-engine:latest
  dongtai/dongtai-engine-task:latest
  dongtai-engine-task:latest
)

for img in "${images[@]}"; do
  docker pull "$img"
done

后台接口报错404问题

按照上述步骤操作完成后，发现后台的一些接口报错404，通过看日志发现是dongtai-server没有启动。继续回去查看安装脚本 dtctl，当我们使用latest，没有指定具体的tag时，脚本会默认生成下面这个docker-compose.yaml，而这个配置文件只启动了web、web-api、mysql、redis、openapi、engine

没有启动server、logstash、logrotate、以及依赖server分角色启动的一些worker服务，导致后台接口产生问题。

image-20250807150707641

查看上面其他tag版本的docker-compose.yaml，将缺失的配置复制过来，*注意（挂载使用了iast-vol的因为在生成配置时，脚本会自动拼接/dev/fd/log，导致服务启动报错，所以需要把iast-vol改成 $PWD/log/）*

image-20250807150720357

version: "2"
networks:
  dongtainetwork:
    name: dongtainetwork
    ipam:
      config:
        - subnet: 172.31.200.0/24
services:
  $MYSQL_STR
  $REDIS_STR
  dongtai-webapi:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "webapi"]
    ports:
      - "8000:8000"
    restart: always
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    depends_on:
      - dongtai-mysql
      - dongtai-redis
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-web:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-web:$CHANGE_THIS_VERSION"
    restart: always
    ports:
      - "$WEB_SERVICE_PORT:80"
    volumes:
      - "$PWD/nginx.conf-legacy:/etc/nginx/nginx.conf"
    depends_on:
      - dongtai-webapi
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-server:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    restart: always
    sysctls:
      net.core.somaxconn: 4096
    volumes:
      - "$PWD/log/:/tmp/logstash/"
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-openapi:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-openapi:$CHANGE_THIS_VERSION"
    restart: always
    volumes:
       - "$PWD/config-tutorial.ini-legacy:/opt/dongtai/openapi/conf/config.ini"
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
    depends_on:
      - dongtai-mysql
  dongtai-engine:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-engine:$CHANGE_THIS_VERSION"
    restart: always
    volumes:
      - "$PWD/config-tutorial.ini-legacy:/opt/dongtai/engine/conf/config.ini"
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
    depends_on:
      - dongtai-mysql
  dongtai-engine-task:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-engine:$CHANGE_THIS_VERSION"
    restart: always
    command: ["/opt/dongtai/engine/docker/entrypoint.sh", "task"]
    volumes:
      - "$PWD/config-tutorial.ini-legacy:/opt/dongtai/engine/conf/config.ini"
    depends_on:
      - dongtai-engine
      - dongtai-mysql
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-worker-task:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    restart: always
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "beat"]
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-worker-beat:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    restart: always
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "worker-beat"]
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-worker-other:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    restart: always
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "worker-other"]
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-worker-high-freq:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    restart: always
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "worker-high-freq"]
    environment:
      - DONGTAI_CONCURRENCY=-P gevent --concurrency=128
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-worker-sca:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    restart: always
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "worker-sca"]
    environment:
      - DONGTAI_CONCURRENCY=-P gevent --concurrency=10
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-worker-es:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    restart: always
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "worker-es"]
    environment:
      - DONGTAI_CONCURRENCY=-P gevent --concurrency=64
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-logrotate:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-logrotate:$DONGTAI_SERVER_VER"
    restart: always
    user: root
    volumes:
      - "$PWD/log/:/tmp/logstash/"
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
  dongtai-logstash:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-logstash:$DONGTAI_SERVER_VER"
    restart: always
    user: root
    environment:
      - DATABASE=${MYSQL_IP}/${MYSQL_DATABASES}
      - USERNAME=${MYSQL_USERNAME}
      - PASSWORD=${MYSQL_PASSWORD}
    volumes:
      - "$PWD/log/:/tmp/logstash/"
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
volumes:
  mysql-vol:

使用脚本启动后发现，后台接口还是报404错误。直接把日志拿去问ai，ai提示dongtai-webapi镜像太旧了，而前端镜像目前更新到了最新版本，官方已经没有维护webapi了，在新版本的dongtai-server中自带了dongtai-webapi服务。所以还有改一下配置。

image-20250807150738569

dongtai-webapi的配置修改如下

dongtai-webapi:
    networks:
      - dongtainetwork
    image: "$registry/dongtai-server:$DONGTAI_SERVER_VER"
    entrypoint: ["/opt/dongtai/deploy/docker/entrypoint.sh", "webapi"]
    ports:
      - "8000:8000"
    restart: always
    volumes:
      - $PWD/config-tutorial.ini:/opt/dongtai/dongtai_conf/conf/config.ini
    depends_on:
      - dongtai-mysql
      - dongtai-redis
    logging:
      driver: "json-file"
      options:
        max-size: "10m"

Server连接mysql报错缺失字段问题

前端响应502，查看server的日志发现如下报错

(1054, "Unknown column 'iast_sensitive_info_rule.system_type' in 'where clause'")

只要 dongtai-server/dongtai-webapi 启动时就抛这个错，Django 会返回 ***500***** → Nginx 收到 500，再转给前端就成 ****502******。

补充数据库字段

# 1. 进入 MySQL
docker exec -it dongtai-dongtai-mysql-1 \
  mysql -uroot -pdongtai-iast dongtai_webapi

# 2. 手工加缺失列
ALTER TABLE iast_sensitive_info_rule ADD COLUMN system_type TINYINT(1) NOT NULL DEFAULT 1;

之前我还遇到一个报错，也是缺失一个数据库字段

(1054, "Unknown column 'iast_hook_strategy.modified' in 'where clause'")
#先删除之前的 modified
ALTER TABLE iast_hook_strategy DROP COLUMN modified;

#添加缺失的列
ALTER TABLE iast_hook_strategy ADD COLUMN modified TINYINT(1) NOT NULL DEFAULT 0;

如果还有别的列缺失，可让 dongtai-server 跑一次 migrate，发现有缺失字段按上面配置添加进数据库

docker exec -it dongtai-dongtai-server-1 \
  python manage.py migrate --noinput

直到执行完迁移全部提示OK

image-20250807150752239

启动命令如下

./dtctl install -v latest

image-20250807150804405

等待一段时间加载好，在用docker ps -a 查看容器进程

OK，现在都是正常的，没有异常启动的进程

image-20250807150823453

访问一下后台

/api/v2/*相关的接口也全部响应200正常。到此所有报错解决了，我部署的这个版本是 1.16.0

image-20250807150843732