Job TITLE: Senior Data Engineer

Location: San Jose, CA

Term: Full Time

Skill: Coding in Python and Shell Scripting using UNIX platform for automation. Provide cleansed data for business users to take mission critical decisions quickly. Create and publish rich and interactive dashboards using latest data visualization technologies. Setup data pipelines using big data technologies like Kafka and Spark streaming for ingesting huge amounts of data, test and deploy ETL for processing the data acquired from various data sources in Hadoop using the technologies like Hive, MapReduce, Spark SQL and create data marts. Create data models by working with the senior data architects. Load processed data into Teradata for downstream application and reporting needs. Load data for user analytics. Create and publish Tableau dashboards daily using data in Teradata, Hive and Spark. Fine tune the processes by writing shell scripts and python scripts wherever necessary. Provide support to business needs by doing lot of ad-hoc analyses on data by writing complex SQL queries, spark code etc. Work in Agile environment and follow the strict release guidelines

Experience: Requires 1 year of experience in the job offered, Software Engineer, Consultant, Programmer Analyst, Systems Analyst, Developer or related

Education: Working on large data sets using distributed computing methodologies. Setup highly scalable data pipelines for continuous data ingestion from various sources like databases, servers, Kafka clusters etc. Participate in developing and documenting user stories, including development estimates and QA. Coding, technical design, data modeling, root cause analysis, investigation, debugging, testing the ETL for data mart creation processes. Load data into multiple destinations like Teradata, Hive for various use cases. Collaboration with the business partners, product managers other engineering teams. Scripting in Hadoop to process petabytes of data efficiently. Working on Teradata physical design and implementation, Teradata SQL performance optimization . Coding in Python and Shell Scripting using UNIX platform for automation. Provide cleansed data for business users to take mission critical decisions quickly. Create and publish rich and interactive dashboards using latest data visualization technologies. Setup data pipelines using big data technologies like Kafka and Spark streaming for ingesting huge amounts of data, test and deploy ETL for processing the data acquired from various data sources in Hadoop using the technologies like Hive, MapReduce, Spark SQL and create data marts. Create data models by working with the senior data architects. Load processed data into Teradata for downstream application and reporting needs. Load data for user analytics. Create and publish Tableau dashboards daily using data in Teradata, Hive and Spark. Fine tune the processes by writing shell scripts and python scripts wherever necessary. Provide support to business needs by doing lot of ad-hoc analyses on data by writing complex SQL queries, spark code etc. Work in Agile environment and follow the strict release guidelines. Discuss with Business Users to understand the requirements, scope it and work with product managers in defining each goal and SLAs for each deliverable. Automate job scheduling and also setup proper alerting mechanisms for taking on-time actions to meet SLAs. Responsible for daily effort tracking and documentation. Requirements: Required Masters or foreign equivalent in Computer Science, Computer Applications, CIS, MIS, Engineering (Any), or any related field. . Must be able to travel/relocate to various client sites throughout the U.S. To apply please send resumes to HR Manager, Flexton Inc., 2590 North First Street, Suite 101, San Jose, CA – 95131 or email hr@flextoninc.com