Zero-downtime Hadoop / HBase Cross-datacenter Migration

摘要：

SPN team in Trend Micro had been running Hadoop over 8 years, we had dig out many valuable insights to provide Threat Expert Services to our internal/external users, from our own data assets and Big Data platform by relying on our Trend Micro Hadoop ecosystem. After many years goes by, our Big Data platform hit physical bottlenecks in our old IDC, regarding to networks, rack design and spaces, etc. Which would impact the reliability and growth of Big Data platform. So then we decided to migrate our Big Data platform to another modern IDC for eliminating the bottlenecks previous mentioned. There are multiple on PROD services running on Big Data platform, therefore, the availability and reliability of Big Data platform is crucial for these services during we doing cross IDC migration.
 
In this sharing, we would like to talk about 1) how and what strategies we used, to migrate our Big Data platform to another IDC with zero-downtime requirements, especially focus on Hadoop and HBase. and 2) we also upgrade to the new version of Hadoop ecosystem released recently from Apache communities.

Trend Micro SPN team已經使用Hadoop有8年的經驗。我們基於內部的Big Data平台儲存並運算我們從世界收集回來的大數據，已經發掘出很多具有商業價值的洞見，並對我們內外部使用者提供專業的Threat Expert服務；而此Big Data平台就是建構在Hadoop ecosystem之上。從Big Data平台啟用到現今已經過不少年，它已經到達了我們資料中心的物理瓶頸，例如內部網路的頻寬，機櫃的空間設計和不足等。這些種種瓶頸會間接地影響我們Big Data平台的成長和穩定性。所以我們決定遷移Big Data平台到另一個新的資料中心來解決上述提到的瓶頸。現行已有不少PROD服務在Big Data平台上運行，所以如何在遷移Big Data平台的時候，仍讓這些服務平穩地運行，是這次遷移工作的主要目標之一。
 
在這個分享裡，我們想分享 1) 我們如何在遷移Big Data平台到另一個資料中心的同時，也能滿足PROD服務不中斷的需求 (Zero-downtime)，分享會聚焦在Hadoop和HBase上。 2) 我們除了做Big Data平台的遷移，也會把Hadoop ecosystem同時升級到目前Apache社群所釋出的最新版本。

講者簡介：

Scott Miao is an infra. team member. He is responsible of maintaining the TrendMicro SPN Big Data platform. He mainly focuses on Hadoop ecosystem. He as 4 years experiences on maintaining Hadoop ecosystem components, investigate new related tech., patch critical issues, and help to make the roadmap for our Big Data platform, in on-premise and on the cloud as well.

• ← How Apache Bigtop leveraged Docker for build automation and one click Hadoop provisioning
• Multi-Cluster Live Synchronization with Kerberos Federated Hadoop →