Synchronize Postgres to Elasticsearch

CDC noticeboard scraper

This project shows how to capture changes from postgres database and stream them into kafka

streaming data pipeline platform

A bot for students of ComfortDelgro Driving Centre (CDC) to monitor and reserve earlier sessions of lessons.

Built a real-time streaming pipeline to extract stock data, using Apache Nifi, Debezium, Kafka, and Spark Streaming. Loaded the transformed data into Glue database and created real-time dashboards using Power BI and Tableau with Athena. The pipeline is orchestrated using Airflow.

An acquisition and processing toolkit for open access phenology data.

Epidemiological weeks calculation based on the US CDC (MMWR) and ISO week numbering systems

This project aims IIT KGP students who are sitting for placement and internships to get the alerts on time

Python Kafka consumer - Change Data Capture from MongoDB to MySQL

A Python package for the National Syndromic Surveillance Program (NSSP) and its Community of Practice. A collection of classes and methods to advance the practice of Syndromic Surveillance.

Code to set up CDC applications Enable change data capture on RDS for MySQL applications that are using XA transactions blog post

Keep in sync RDB table with Hive structured store. Added Kafka as a buffer between those two tables.

Lightweight CDC patterns for SQLite

This is a tryout I prepared to demonstrate CDC (change data capture) using MySQL, Maxwell and Kafka.

Showcases real-time data replication from RDS (MariaDB) to Kinesis using AWS DMS on LocalStack. Implements both full-load and Change Data Capture (CDC) tasks to stream database changes for analytics.

An ensemble of BERTs for classifying injury narratives

How to build a complete Data Platform -> Here

疾病管制署部分開放資料集鏡像

the binlog-repository of mysql CDC(Change Data Capture)