ETL Pipeline Migration Engineer

When a database platform changes, the ETL pipelines feeding it rarely survive unmodified. Source queries need rewriting for the new dialect. Transformation logic tied to vendor-specific functions must be reimplemented. Staging table patterns, bulk load mechanisms, and incremental extraction strategies all depend on platform-specific features that may not exist or behave differently on the target. This AI assistant specializes in the intersection of database migration and data pipeline engineering, helping you redesign and migrate ETL workflows alongside the database itself.

The assistant helps you inventory your existing pipeline landscape — whether built in traditional tools like Informatica, DataStage, or SSIS, modern orchestrators like Apache Airflow, Prefect, or dbt, or custom SQL and scripting solutions. It analyzes each pipeline's source queries, transformation steps, and load mechanisms, then identifies which components need rewriting and which can be adapted with minor changes.

For SQL-heavy pipelines, the assistant rewrites extraction queries for the new source or target dialect, adapting window functions, date arithmetic, string manipulation, and aggregation syntax as needed. For loading steps, it recommends the most efficient bulk load approach on the target platform — COPY commands, BCP, external tables, or streaming inserts — and produces the configuration and surrounding script for each.

The assistant also addresses incremental extraction strategies during and after the migration. It helps you redesign high-water mark patterns, CDC-based extraction, and partition-based parallel loading to work with the new platform's capabilities and limitations.

Ideal for data engineers and analytics teams migrating a data warehouse or operational database while keeping downstream pipelines functional. Also valuable for teams modernizing from legacy ETL tools to code-first orchestration frameworks as part of a broader platform modernization initiative.