錯誤調用BashOperator：bash命令失敗

Question

這裏是我的DAG文件和BashOperator任務：

my_dag = { 
dag_id = 'my_dag', 
start_date = datetime(year=2017, month=3, day=28), 
schedule_interval='01***', 
} 

my_bash_task = BashOperator(
task_id="my_bash_task", 
bash_command=bash_command, 
dag=my_dag) 

bash_command = "/home/jak/my_projects/workflow_env/repo_workflow/db_backup_bash.sh ""

在此之後answer我甚至給了bash的文件後空間，以避免TemplateNotFound錯誤。但是，當運行此任務給了我這個錯誤：airflow.exceptions.AirflowException：Bash命令失敗。

bash_command文件內容爲：

#!/bin/bash 
DATABASE=db_name 
FILE=$DATABASE-`date +%F-%H%M%S`.backup 
export PGPASSWORD=password 
pg_dump -h localhost -p 5432 -U developer -F c -b -v -f ~/Dropbox/database_backup/location/$FILE db_name 
unset PGPASSWORD 

但是代替bash_command指向bash的文件寫入多行字符串命令的工作原理：

bash_command = """ 
DATABASE=db_name 
FILE=$DATABASE-`date +%F-%H%M%S`.backup 
export PGPASSWORD=password 
pg_dump -h localhost -p 5432 -U developer -F c -b -v -f ~/Dropbox/database_backup/location/$FILE db_name 
unset PGPASSWORD 
""" 

正因爲如此，我假設錯誤不是因爲bash命令。 我甚至嘗試用＃！/ bin/sh替換bash文件中的＃！/ bin/bash，但這也不起作用。

我跑了終止符sh db_back_up_bash.sh，它工作正常。

更新的實際代碼：

bash_file_location_to_backup_db = '{{"/home/jak/my_projects/workflow_env/repo_workflow/db_backup_bash.sh"}}' 
# bash_file_location_to_backup_db = "/home/jak/my_projects/workflow_env/repo_workflow/db_backup_bash.sh " 
bash_command = """ 
DATABASE=ksaprice_scraping 
FILE=$DATABASE-`date +%F-%H%M%S`.backup 
export PGPASSWORD=password 
pg_dump -h localhost -p 5432 -U developer -F c -b -v -f ~/Dropbox/database_backup/ksaprice/$FILE ksaprice_scraping 
unset PGPASSWORD 
""" 

backup_scraped_db_in_dropbox_task = BashOperator(
    task_id="backup_scraped_db_in_dropbox_task", 
    # bash_command=bash_command,# this works fine 
    bash_command=bash_file_location_to_backup_db,#this give error :airflow.exceptions.AirflowException: Bash command failed 
    dag=dag_crawl 
) 

錯誤跟蹤：

[2017-04-11 20:02:14,905] {bash_operator.py:90} INFO - Output: 
2017-04-11 20:02:14,905 | INFO| root : Output: 
[2017-04-11 20:02:14,906] {bash_operator.py:94} INFO - /tmp/airflowtmp7FffJ2/backup_scraped_db_in_dropbox_taskQ6IVxm: line 1: /home/jak/my_projects/workflow_env/repo_workflow/db_backup_bash.sh: Permission denied 
2017-04-11 20:02:14,906 | INFO| root : /tmp/airflowtmp7FffJ2/backup_scraped_db_in_dropbox_taskQ6IVxm: line 1: /home/jak/my_projects/workflow_env/repo_workflow/db_backup_bash.sh: Permission denied 
[2017-04-11 20:02:14,906] {bash_operator.py:97} INFO - Command exited with return code 126 
2017-04-11 20:02:14,906 | INFO| root : Command exited with return code 126 
[2017-04-11 20:02:14,906] {models.py:1417} ERROR - Bash command failed 
Traceback (most recent call last): 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run 
    result = task_copy.execute(context=context) 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute 
    raise AirflowException("Bash command failed") 
AirflowException: Bash command failed 
2017-04-11 20:02:14,906 | ERROR| root : Bash command failed 
Traceback (most recent call last): 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run 
    result = task_copy.execute(context=context) 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute 
    raise AirflowException("Bash command failed") 
AirflowException: Bash command failed 
[2017-04-11 20:02:14,907] {models.py:1441} INFO - Marking task as FAILED. 
2017-04-11 20:02:14,907 | INFO| root : Marking task as FAILED. 
[2017-04-11 20:02:14,947] {models.py:1462} ERROR - Bash command failed 
2017-04-11 20:02:14,947 | ERROR| root : Bash command failed 
Traceback (most recent call last): 
    File "/home/jak/my_projects/workflow_env/bin/airflow", line 28, in <module> 
    args.func(args) 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 585, in test 
    ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True) 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper 
    result = func(*args, **kwargs) 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run 
    result = task_copy.execute(context=context) 
    File "/home/jak/my_projects/workflow_env/local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute 
    raise AirflowException("Bash command failed") 
airflow.exceptions.AirflowException: Bash command faile 

Answer 1

我認爲這是在氣流中的錯誤，神社不應該指望.SH文件包含模板信息在BashOperator中。

我周圍有通過將命令轉換爲格式神社將正確解釋：

bash_command = '{{"/home/jak/my_projects/workflow_env/repo_workflow/db_backup_bash.sh"}}'