Airflow Data Pipeline

This example walks through how to set up an ETL pipeline using Airflow. I set up at DAG to query the openweathermap.org API everyday, process the json data and store it in a PostgreSQL database. For explanation of how this code works see this blog post

Requirements

Airflow

Python 2.7

PostgreSQL

psycopg

SQLAlchemy

SQLAlchemy-Utils

To install the requirements (except for Python and postgres) type:

pip install -r requirements.t