Skill: Job description:
• Set up and maintain data transfer tools (possibly sqoop).
• Set up and optimize Hcatalog.
• Create a Hive metadata on a separate machine (MySql) Setup version control to S3. Build/optimize an AWS cluster.
• Create a framework to manage Hive external meta-store concurrency.
• Set up and maintain a scheduling tool.
• Set up and maintain monitoring and alerts (possibly DataDog). Tools to manage S3 (cleanup, etc)