Multi-Cluster Live Synchronization with Kerberos Federated Hadoop

講者:張雅芳 Mammi Chang / 趨勢科技 Trend Micro / 資深工程師
地點:3F – 第二會議室
講題:Multi-Cluster Live Synchronization with Kerberos Federated Hadoop


In enterprise on-premises data center, we may have multiple Secured Hadoop clusters for different purpose. Sometimes, these Hadoop clusters might have different Hadoop distribution, Hadoop version, or even locate in different Data Center. To fulfill business requirement, data synchronize between these clusters could be an important mechanism. However, the story will be more complicated within the real world secured multi-cluster, compare to distcp between two same version and non-secured Hadoop clusters.
We would like to go through our experience on enable live data synchronization for multiple kerberos enabled Hadoop clusters. Which include the functionality verification, multi-cluster configurations and automation setup process, etc. After that, we would share the use cases among those kerberos federated Hadoop clusters. Finally, provide our common practice on multi-cluster data synchronization.


Mammi Chang (張雅芳) works as a DevOps engineer in Trend Micro. Her job duty is to build the modern and stable Big Data infrastructure. She is well-experienced in operation, deployment, automation of Hadoop Ecosystem and AWS. Also, Mammi was invited as a speaker to share the Trend Micro Hadoop Overview in HadoopCon 2014.