Seatunnel在Dolphinscheduler调度中报错

Seatunnel在Dolphinscheduler调度中报错

[INFO] 2022-07-06 13:12:16.714 - [taskAppId=TASK-393-81727-908774]:[238] - process has exited, execute path:/data/dolphinscheduler/exec/process/9/393/81727/908774, processId:25215 ,exitStatusCode:1 ,processWaitForStatus:true ,processExitValue:1[INFO] 2022-07-06 13:12:17.201 - [taskAppId=TASK-393-81727-908774]:[138] - -> tools size = 2813343 4574389 5445379 4238956 4253580 /data/dolphinscheduler/exec/process/9/393/81727/908774/393_81727_908774_node.sh: line 7: [: too many arguments dbos-bigdata-clickhouse001 8000 0 2 0 dbos-bigdata-clickhouse003 8000 0 1 0 dbos-bigdata-clickhouse002 8000 0 0 0 SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See for further details. Exception in thread "main" java.nio.file.AccessDeniedException: plugins.tar.gz at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at org.apache.seatunnel.core.base.utils.CompressionUtils.tarGzip(CompressionUtils.java:62) at org.apache.seatunnel.core.spark.SparkStarter$ClusterModeSparkStarter.buildCommands(SparkStarter.java:390) at org.apache.seatunnel.core.spark.SparkStarter.main(SparkStarter.java:103)

方案：

查看脚本中的源码，需要代码中cd /opt/apache-seatunnel-incubating-2.1.2/

#!/bin/bashhive_table_name='dim_dbos_mac_userid_relation_his'hdfs_size=`hdfs dfs -du oss://dbbigdatapro/group/user/tools/meta/hive-temp-table/${hive_table_name}/ | awk '{print $1}'`echo `whoami`echo "size = ${hdfs_size}" if [ $hdfs_size -eq 0 ]then echo "没有数据不需要导出"else clickhouse-client -h 172.18.2.111 --port 8000 -u default -d default -m -q "TRUNCATE TABLE dmp.dim_dbos_mac_userid_relation_his on CLUSTER cluster_clickhouse" cd /opt/apache-seatunnel-incubating-2.1.2 /opt/apache-seatunnel-incubating-2.1.2/bin/start-seatunnel-spark.sh --master yarn --deploy-mode cluster --config /home/tools/bigsource/etl/DMP/new/clickhouse_ddl/${hive_table_name}.conffi

报错解除：

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。