Programming Study日本語

Disclosure: We earn a commission from Udemy when you buy a course through links on this page.

The Complete GCP Data Engineering Project - Retailer Domain

The Complete GCP Data Engineering Project - Retailer Domain

Udemy · English

About this course

Language
English
Category
Provider
Udemy
Data updated

This project focuses on building a data lake in Google Cloud Platform (GCP) for Retailer DomainThe goal is to centralize, clean, and transform data from multiple sources, enabling Retailers providers and insurance companies to streamline billing, claims processing, and revenue tracking.GCP Services Used: Google Cloud Storage (GCS): Stores raw and processed data files. BigQuery: Serves as the analytical engine for storing and querying structured data. Dataproc: Used for large-scale data processing with Apache Spark. Cloud Composer (Apache Airflow): Automates ETL pipelines and workflow orchestration. Cloud SQL (MySQL): Stores transactional Electronic Medical Records (EMR) data. GitHub & Cloud Build: Enables version control and CI/CD implementation.CICD (Continuous Integration & Continuous Deployment): Automates deployment pipelines for data processing and ETL workflows. Techniques involved : Metadata Driven ApproachSCD type 2 implementationCDM(Common Data Model)Medallion Architecture Logging and MonitoringError HandlingOptimizationsCICD implementationmany more best practicesData SourcesMySQL Retailer DatabaseMySQL Supplier DatabaseAPI Reviews (api-reviews)Expected OutcomesEfficient Data Pipeline: Automating the ingestion and transformation of RCM data. Structured Data Warehouse: gold tables in BigQuery for an

Courses in the same category