New: Cookbooks and AI ExplanationsStep-by-Step recipes to solve problems connected to Roadmaps and Cheat Sheets. Need more details? Use AI buttons for structured and simple explanations with concrete examples throughout the whole platform.Take a look
Leadership just announced the on-prem Hadoop cluster is being decommissioned in six months, and half the team's pipelines still run on it.
What you'll have at the end
A migration plan phased by pipeline priority, with a rollback point at each phase and the first pipeline cut over
You need
Read access to the Hadoop cluster's scheduler, HDFS paths, and Hive metastore for the pipelines in scope, plus a working account on the cloud platform the team has already picked as the destination.
Not covered
Physically decommissioning the Hadoop hardware or closing its infrastructure contracts once every wave is done; that starts only after the last wave in this plan clears, and it usually belongs to an infrastructure team rather than the pipeline owner.
An inherited pipeline, mapped stage by stage
for building the per-pipeline map itself when nobody left behind documentation on what a given pipeline actually does
Turn a hard-to-reverse platform choice into a written ADR
for choosing and justifying the destination platform itself, a decision this recipe assumes is already made
Checked 26 Aug 2026
Part of the Data Engineering Core cookbook