Job summary:

Job description:

***Long Term Contract***100% Remote******Note: interview will be in depth technical - so please make sure you are comfortable before you apply ***

Job Desription:

• Develop Python scripts for automation of the data flow process from Snap Logic application to Apache spark.
• Implement ETL transformations using tools and services like Snap Logic, Apache Spark.
• Develop complex SQL queries to perform data analysis to identify any need for data transformation.
• Integrate Snap Logic pipelines to make the data flow from on-prem to cloud Apache spark.
• Schedule all the pipelines to make it automated to minimize manual intervention.
• Implement Data profiling, Data quality and Data standardization for all the workflows based on EDM standards (Enterprise Data Management).
• Build framework to felicitate cloud data analytics using Apache Spark.
• Migrate tables from traditional RDBMS into Data Lake using Snap Logic.
• Develop complex pipelines to implement data migration and data analytics using Spark snap from Snap Logic tool to implement data streaming to Spark resilient distributed dataset (RDD).
• Design and implement data models in Apache spark cloud cluster.
• Prepare Technical Specification documents that captures Snap Logic pipeline information and Python scripting along with Apache spark data analytics. 
• Prepare data warehouse models for migrating from existing Snap Logic application data to Apache spark analytics.
• Create Technical design documents after reverse engineering to use for Python scripting and Apache Spark data analysis.
• Analyze databases to design extract strategies Apache spark and data warehouse model.
• Participate in business requirement meetings, perform requirement, and impact analysis with Business Analysts.
• Coordinate with business users to understand the data sources/entities.
• Gather Snap Logic pipeline information for reverse engineering.
• Perform Unit and regression testing on the developed data migration with Snap Logic and Apache spark, prepare unit test case documents for comparing the results with business requirements.
• Perform Unit testing on newly developed Python script for the data analytic and documenting the same.
• Work on various data needs for Unit testing to identify the possibility of bugs in the development.
• Help QA in preparing a comprehensive test plan to ensure proper functioning of the application.
• Assist both QA and UAT team with the test data needs.
• Fix the code issues on timely manner reported from QA and UAT testing and tracking the same with proper documentation.
• Deploy the recent version of script/code/pipelines to Dev, QA, UAT & Production environment within given timeline provided by business user.
• Implement the DI and DQ check on data on the migrated data post deployment to Dev, QA, UAT & Production environment.
• Assess the quality of the data and provide recommendations to improve the data quality at the source.
• Perform root cause analysis for the production issues and resolving the issue.
• Monitor scripts & pipeline execution to make sure to failure happens along with performance tuning 

Contact the recruiter working on this position:

The recruiter working on this position is Suhasini Thota(Ravi Team)
His/her contact number is +(1) (703) 3494245
His/her contact email is suhasini@msysinc.com

Our recruiters will be more than happy to help you to get this contract.