Untitled

Tools

• boundary-layer : Declarative Airflow Workflows

  https://codeascraft.com/2018/11/14/boundary-layer -declarative-airflow-workflows/

Task

• Airflow: Call 3208 crashed but not updated the statuses, although logged them:

  [2018-10-29 09:59:21,443] {logging_mixin.py:84} INFO - Setting ca_status = [-112], lta_status = [-109], status = [-12]

  *http://10.0.13.207:8080/admin/airflow/log?task_id=call_analysis&dag_id=voice_analysis_params_2&execution_date=2018-10-29T09:58:09*

• Check stuck call #3272, patient 348 stuck on 7

• Airflow: send emails from **[email protected], not Arik to Shira and Itay**

• Important: Set status when a call “caught” by sensor to prevent multiple DAGs on it

• Send better exceptions

• Analytics: result class

Airflow 1.10

• airflow.cfg (1.10)
• executor = LocalExecutor sql_alchemy_conn - use MySQL DB, e.g. *mysql+[mysqlconnector://root:toor6@localhost:3306/airflow](mysqlconnector://root:toor6@localhost:3306/airflow)* dag_concurrency = 4 max_active_runs_per_dag = 4 load_examples = False enable_xcom_pickling = False base_url = ??? server URL for links dag_orientation = TB navbar_color = #2CABE2 [smtp] settings fernet_key - set with the result of: $ python>>> from cryptography.fernet import Fernet

k=Fernet.generate_key() print(k) ...

• Mysql
• Increase com value from BLOB to MEDIUMBLOB Create /etc/my.cnf [mysqld] explicit_defaults_for_timestamp = 1

airflow (1.9)

• pip install --upgrade pip

Machine