Proyectos
Ver en GitHub
MartyC-137/Data-Engineering
MartyC-137/Data-Engineering has 32 GitHub stars and 5 forks. GitHot measured 0 stars in 24 hours, 0 in 7 days, and 0 in 15 days. Last checked 2026-11-21 19:49 UTC. Last push 2023-09-05.
Crecimiento de estrellas medido
- in 24h
- 0
- in 7 days
- 0
- in 15 days
- 0
- in 1 month
- Recopilando historial
Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.
Acerca de
A project portfolio to accompany my resume
DatosREADME.md
OriginalData Engineering Portfolio
Introduction
This repository contains numerous work examples of code I use in my day to day work as a data engineer, all of which has been modified as minimum reproducible examples. My favourite tools are Snowflake, Python, and dbt, and I also have an interest in DevOps as it pertains to data engineering.
Table of Contents
- Python Examples
- Snowpark example - backload data from SQL Server
- Snowpark example - backload data from API
- Automated SQL insert statements from a CSV file
- Extract data from SQL Server, transform, and load to Snowflake
- Batch load JSON files to Snowflake
- SQL Server data Pull - 100 Records from every view in a database
- SQL Examples
- Only grant permissions on tables with > 0 rows of data - Snowflake
- Auto Ingest Snowpipe from Azure Blob to Snowflake
- Shorten large union queries using Snowflake
- Basic Snowflake CDC Pipeline using Streams and Tasks
- Find missing dates in a date field - Snowflake
- Snowflake data pipeline from internal stage
- Snowflake CI/CD using Azure Pipelines - SQLFluff testing, build and deploy using SnowSQL
- SQLFluff and yamllint pipelines for a dbt project
Usage
# Clone the repository
$ git clone https://github.com/MartyC-137/Data-Engineering.git
# Connect to the repository
$ cd Data-Engineering
Explorar proyectos relacionados
Cargando…