company-img2

Data Engineer I Big Data Ecosystems | A healthcare tech startup | 3 6 years

  • 0 yrs
  • Not Disclosed

Job Description

Key goals for the position:
• Develop technical solutions for a data platform based on existing and emerging patterns and technologies
• Enable advanced analytics including AI ML and data driven decision making by building pipelines that are event driven, can transform cleanse manipulate data and facilitate visualization of data integrated from enterprise systems, internet, and many other disparate sources
• Enable data exchange amongst enterprise systems via real time event streaming platform
• Champion conceptual and technical aspects of real time data platform development and support, using various design patterns, dynamically scalable technologies, and Devops practices
Team Member Minimum Requirements
Preferred Formal Education Qualifications:
• Bachelor of Computer Science
• Other qualifications in IT or related discipline Proven Experience:
• Developing, Testing and Supporting data lakes, data warehouses and large scale data processing systems.
• Working effectively in data platform scrum teams for technical design, development and deployment of solutions.
• Implementing and supporting data platform technologies both on premise and cloud based.
• Deep understanding and adoption of Agile delivery techniques, including Continuous Integration & Continuous Delivery (CI CD).
• Developing solutions using a variety of technologies and tools to marry on premise and cloud based systems together.
• Participate in technology tools framework evaluation to recommend influence adoption
• Follow best practices to ensure high standards of data availability, reliability, completeness, efficiency and quality. Skills Knowledge Abilities Technology Used:
• Advanced SQL working knowledge and experience working with a variety of relational databases, SQL query authoring.
• Expert understanding and hands on working knowledge of message queueing, real time event streaming architecture for data platforms including Kafka, Kinesis, SNS, SQS etc.
• Deep understanding and ability to build automations for data transformation, processing of data structures, metadata, dependencies and workload management for very high volume, velocity and variety of data.
• Deep working knowledge of AWS technologies such as S3, EC2, EMR or Bigdata Hadoop, RDS, Lambda, Elasticsearch, Redshift, Cloudformation, Terraform, Cloudwatch etc.
• Sound to advanced level working knowledge of Git and CI CD pipeline technologies such as Jenkins, Chef, Kubernetes & Docker containers
• Deep experience with object oriented object function scripting languages: Python, Scala, Java, R, C++, Golang etc.
• Working knowledge of Data warehouses, NoSQL databases and ETL technologies like Informatica, Talend etc
• Understanding of modern SaaS based monitoring and logging tools like New Relic Sumologic or equivalent
• Must have experience with Linux, shell scripting
• Familiarity with BI tools a plus
Area of Accountability Key Responsibilities & Deliverables Performance Measures & Targets
Development and Operations:
• Collaborate with team members to design and implement data solutions in alignment with the project schedule.
• Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.
• Peer and customer feedback
• Line manager observations
• Line manager observations
• Create data flow diagrams for business systems.
• Implement security as part and parcel of all development.
• Builds automation tools to provide a self service data platform and enable CICD
• Creates and maintains a data catalogue.
• Develops standards and processes.
• Champions change management
• Take Care Safety • Accountable for a safe site for everyone, every day by implementing and evaluating safe work practices, improving safety performance and
celebrating safety achievements
• Follows a Devops mindset and practice
Communication:
• Conducts product demonstrations, showcases, briefings on trending technologies, processes and solutions relating to the data platform.
• Line manager observations
• Line manager observations
• Communicates effectively with stakeholders, partners, vendors.
• Peer and customer feedback

Other Areas of Accountability Key Responsibilities Major Activities:
Values and Behaviour
• Live Integrity
• Think Customer
• Grow Together
• Reach Higher