Data Pipeline Developer

Company: Connvertex Technologies Inc.
Location: Raleigh
Posted on: October 17, 2020

Job Description:

Connvertex is seeking a Data Pipeline Developer with our direct client. (BIDP) solution. The position has accountability around providing technical expertise to ensure the quality and accuracy of that data, then process, design and present it in ways to help those leveraging the BIDP. The ETL Developer is responsible for Working with Data Architects, Analysts, and Scientist to aid in BIDP data pipeline efforts Configure ingestion and format validation of the new workstreams Verify initial and incremental data uploads and maintain the workstreams Implement data pipeline-based technologies within and associated with the BIDP environment Preform data pipeline testing Conducting data pipeline optimization, troubleshooting and debugging, report regularly on health and performance of jobs Maintain ownership of release activities interacting with pipeline projects Support and improve data pipeline automation Assist in discovering, evaluating, and qualifying new technologies around BIDP data pipeline functions Partner with BIDP stakeholders to establish and maintain policies, procedures, operational standards around the data pipeline functions Help business in leveraging the BIDP through the data pipeline process 15 Preform data pipeline process management and lifecycle support by outlining the process and setting the boundaries for data consumption and processing within the BIDP. Provide architecture and data flows for the data pipelines. Document the requirements of the data pipeline process, and tools, and manage its development. Work with stakeholder in the development and maintenance of all data pipeline-based documentation. Take part in the development and implementation of data consumption tools within the BIDP environment. Ensure that data pipelines comply with all regulatory standards. Prepare and report on activities and utilization around data pipelines. 65 Develop data pipelines, processes and designs for data being consumed within the BIDP. Identify trends and opportunities for growth through analysis of complex data sets. Evaluate methods and provide source-to-target mappings and information-model specification documents for data sets. Work directly with stakeholders to gather requirements for data pipelines for the BIDP. Work closely with business understand and maintain focus on their analytical needs, including identifying critical metrics and KPIs, and deliver actionable insights to relevant stakeholders. Define and implement data acquisition and integration logic, selecting appropriate combination of methods and tools within defined technology stack to ensure optimal scalability and performance of the solution 20 Conduct testing of tools and data pipelines. Perform root cause analysis on all processes and resolve all production issues and validate all data and perform routine tests on data sets and provide support to all data pipeline connections. Document all test procedures for BIDP data pipeline tools and processes, and coordinate with stakeholders and exchange partners to resolve issues and maintain quality. Competencies, Knowledge, Skills and Abilities Required in this Position Strong knowledge and hands-on experience with data pipeline tools Experience in data modeling Knowledge of compliance frameworks PCI, SOX, SOC 2, ISO 27001 and the ability to apply their requirements and concepts to a complex environment Experience in supporting a data warehouse in identifying and revising reporting requirements Knowledge and hands-on expertise with processing confidential data and information according to guidelines Strong interpersonal communication and ability to solve complex problems Hands-on experiences with databases e.g., SQL, MYSQL, Oracle, Redshift (preferred), etc. Experience in model design, segmentation techniques, and ETL Frameworks Knowledge and understanding of micro-services architecture Strong analytic skills, including mining, evaluation, analysis, and visualization AWS, Azure or Google Cloud Platform background (preferably all three) Experience with enterprise level cloud-based development, deployment, and auditing, including PAAS, IAAS, and SAAS. (preferred) Proficiency in scripting languages e.g., XML, JavaScript, etc. Education and Experience Required Bachelor's degree in Computer Science, Computer Information Systems, Computer Engineering, Or other related technical degree from an appropriately accredited institution and 3 years of progressive expertise as a data analystscientist Or bachelor's degree from an appropriately accredited institution and eight years progressive experience in the field of information technology or an equivalent combination of education and experience. Connvertex is a trusted System Integration Partner for providing innovative mobility, system integration and modernization solutions to global clients in Health Human Services, Child Welfare, Healthcare Workforce Services. Connvertex has a decade-long history in IT Professional Service industry servicing several clients across U.S. We bridge the IT talent demand-supply gap through an innovative, best-in-class recruiting engine, backed by a dedicated client-centric service focus. Thanks and Regards, Piyush Kharpate

Keywords: Connvertex Technologies Inc., Raleigh , Data Pipeline Developer, IT / Software / Systems , Raleigh, North Carolina