How one big retail chain moved its data warehouse to a Hadoop environment.
Name of Organization: Impetus Technologies
Industry: Retail
Opportunity or Challenge Encountered: Impetus was contacted by a U.S.-based multinational retail chain to batch-migrate its data warehouse from Teradata to a Hadoop/Hive environment. Batch migration includes data storage, the creation of batch jobs, moving migrated data to target systems, performing data validation, conversion of Teradata specific operations to new systems and training assistance.
See also: Case Study: Building the Refinery of the Future with IoT
How this Opportunity or Challenge Was Met: Impetus developed a two key migration solution for the retail chain. One-time migration of historical data from Teradata, scripts and queries used in batch and ad-hoc to access Teradata and analytical post-processed data was all moved onto the new Hadoop environment. Incremental migration of daily, weekly, and monthly increments of data from source to Hadoop. Impetus’ solution features an automated utility that converts BTEQ and SQL scripts into SparkQL/HiveQL equivalent, reducing the time it takes to migrate drastically. Impetus used its Workload Migration and IBM DataStage for incremental data migration to Hadoop.
Benefits of This Initiative: The migration to Hadoop reduces the overall cost of running the data warehouse, moves to a modern technology that is more easily accessible to basic users and adds business risk mitigation. Teradata’s high licensing cost and high barrier to entry are two reasons we see a surge in migration interest to Apache’s platforms, which are open source and updated more frequently.
