如何設置Airflow的電子郵件配置以發送錯誤電子郵件？

Question

我試圖通過傳遞一個不起作用的Bash行（thisshouldnotrun）來故意排除故障並排除錯誤。氣流正在輸出以下內容：

[2017-06-15 17:44:17,869] {bash_operator.py:94} INFO - /tmp/airflowtmpLFTMX7/run_bashm2MEsS: line 7: thisshouldnotrun: command not found 
[2017-06-15 17:44:17,869] {bash_operator.py:97} INFO - Command exited with return code 127 
[2017-06-15 17:44:17,869] {models.py:1417} ERROR - Bash command failed 
Traceback (most recent call last): 
    File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run 
    result = task_copy.execute(context=context) 
    File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute 
    raise AirflowException("Bash command failed") 
AirflowException: Bash command failed 
[2017-06-15 17:44:17,871] {models.py:1433} INFO - Marking task as UP_FOR_RETRY 
[2017-06-15 17:44:17,878] {models.py:1462} ERROR - Bash command failed 
Traceback (most recent call last): 
    File "/home/ubuntu/.local/bin/airflow", line 28, in <module> 
    args.func(args) 
    File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/bin/cli.py", line 585, in test 
    ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True) 
    File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper 
    result = func(*args, **kwargs) 
    File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run 
    result = task_copy.execute(context=context) 
    File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute 
    raise AirflowException("Bash command failed") 
airflow.exceptions.AirflowException: Bash command failed 

Airflow會發送電子郵件來處理這些類型的錯誤嗎？如果不是，發送這些錯誤的電子郵件的最佳方式是什麼？

我甚至不確定airflow.cfg是否設置正確...由於最終目標是測試電子郵件警報通知，因此我想確保airflow.cfg安裝正確。這裏的設置：

[email] 
email_backend = airflow.utils.email.send_email_smtp 


[smtp] 
# If you want airflow to send emails on retries, failure, and you want to use 
# the airflow.utils.email.send_email_smtp function, you have to configure an 
# smtp server here 
smtp_host = emailsmtpserver.region.amazonaws.com 
smtp_starttls = True 
smtp_ssl = False 
# Uncomment and set the user/pass settings if you want to use SMTP AUTH 
# smtp_user = airflow_data_user 
# smtp_password = password 
smtp_port = 587 
smtp_mail_from = airflow_data_user@domain.com 

什麼是smtp_starttls？我無法在文檔或在線中找到任何信息。如果我們有查看電子郵件所需的雙因素身份驗證，那麼這對於Airflow來說會是一個問題嗎？

這裏是我的bash命令：

task1_bash_command = """ 
export PATH=/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin 
export rundate=`TZ='America/Los_Angeles' date +%F -d "yesterday"` 
export AWS_CONFIG_FILE="/home/ubuntu/.aws/config" 

/home/ubuntu/bin/snowsql -f //home/ubuntu/sql/script.sql 1> /home/ubuntu/logs/"$rundate"_dev.log 2> /home/ubuntu/logs/"$rundate"_error_dev.log 

if [ -e /home/ubuntu/logs/"$rundate"_error_dev.log ] 
then 
    exit 64 
fi 

而我的任務：

task1 = BashOperator(
    task_id = 'run_bash', 
    bash_command = task1_bash_command, 
    dag = dag, 
    retries = 2, 
    email_on_failure = True, 
    email = 'username@domain.com') 

Answer 1

smtp_starttls基本上意味着使用TLS

將其設置爲False，並設置smtp_ssl到True如果你想使用改爲SSL。您可能需要smtp_user和smtp_password。

氣流不會處理2步驗證。但是，您是否使用AWS，因爲您的SMTP（SES）證書與您的AWS證書不同，您可能不需要它。

請參閱here。

編輯： 爲了讓氣流在發生故障時發送電子郵件，有幾件事情需要在您的任務上設置，email_on_failure和email。

在這裏看到，例如：

def throw_error(**context): 
    raise ValueError('Intentionally throwing an error to send an email.') 



t1 = PythonOperator(task_id='throw_error_and_email', 
        python_callable=throw_error, 
        provide_context=True, 
        email_on_failure=True, 
        email='your.email@whatever.com', 
        dag=dag)