About the job
Data Engineer – Junior (Up to 3 years), Intermediate (6 Years) and Senior (6+ Years)
- Technical role responsible for operating and ingesting data into COMPANY data repositories (e.g., Digital Data Platform, Data warehouse, etc.) to ensure the availability of data for data visualization, data analytics and digital use cases by designing, building and reviewing data integration pipelines.
- Maintain close alignment with the data governance, data architecture, data security and overall IS strategy.
- Continuous collaboration with the various COMPANY business entities to ensure the data governance practices are applied on the data repositories (e.g., Data Quality).
- Job dimensions
- Data Management covers (but not limited to) the following main business entities: Reservoir and Geosciences, Field Operations, Drilling and Wells, Logistics, Engineering & Constructions and Major Projects, HR and HSE-Q.
- Data is one of the 5 pillars of IS strategy to enable the value of data analytics and digitalization
- Al Shaheen field produces petroleum data from different COMPANY entities with large data sets with complex data streams (RGS) and interfaces (drilling and production real-time data)
- Technical leadership and strategic direction for the technologies, standards, processes and architectures for data across the enterprise.
- Data and Information management along with its associated concepts, policies and standards in COMPANY
- Handles all types of data in its structured and unstructured data
– Approximately 70 Data Sources
– Approximately 120 Server Applications
Scope of responsibilities
- Operate the Digital Data Platform (DDP) to support data analytics, data integration data visualization and digital use cases
- Build DDP components and technology to support COMPANY activities in alignment with the data architect and Digital Project group
- Implement data ingestion requirements following Data Governance, Data Architecture and Data Security principles
- Direct involvement in the Data management lifecycle, from producing data integration mapping specifications, design and validation through to development and production deployments
- Design, Develop & Monitor Data integration pipeline
- Develop, construct, test and maintain data integration architectures including migrating on premises Integration pipelines to Cloud based.
- Align architecture with business requirements
- Identify ways to improve data reliability, performance, efficiency and quality
- Deploy complex analytical algorithms, machine learning and statistical methods using programming languages and tools
- Support data analytics through preparing and developing descriptive, diagnostic, predictive and prescriptive modeling
- Use data to discover patterns and tasks that can be automated
- Develop data set processes and data acquisitions
- Contribute to the design, development, implementation and maintenance of complex data systems and solutions.
- Drive initiatives in collaboration with IS teams especially data architect, data analysts and the various business entities to deliver solutions that maximizes the value of data.
- Evaluate and recommend innovative solutions to mature COMPANY’s data operations and overall data management
- Build and maintain effective working relationships with key entities and stakeholders in COMPANY and other parties as needed.
- Support COMPANY to define new requirements, participating in End user meetings to derive and define the business requirement, propose cost effective solutions for data analytics and familiarize the team with the requirements, specifications, design targets & techniques to support task performance and delivery.
- Review & Propose industry best tools & technology for ever changing business rules and data set from data engineering point of view. Conduct Proof of Concepts (POC) with new tools & technologies to derive convincing benchmarks and deliver value-based solution in Quick time.
- Bachelor’s degree from an accredited university in Information Management or Computer Science or an engineering/business related equivalent.
- Deep experience in petroleum operations data and good understanding of the related data sources
- Full understanding of Data Discovery, Data Visualization, Data Warehousing, Data Governance (e.g., datalogging, profiling, ownership/stewardship, Data Quality, etc.)
- Solid track record implementing and maintaining data lakes, databases &data warehousing solutions
- Strong experience in database technologies and techniques (e.g., SQL, NoSQL, Oracle, etc.)
- Strong hands-on experience designing, developing, deploying, testing and maintaining in Informatica suit (IICS, Lake house management, CDQ MDM, EDC, IDQ, Axon and MDM)
- Technical experience in Informatica Intelligent Cloud Services. (IICS) including Cloud Mass Ingestion (CDMI) and Cloud Data Integration (CDI)
- Strong hands-on knowledge on Informatica administration activities which includes creating connections, upgrades, secure agent installation.
- Ability to understand and work with Microsoft Common Data Model (CDM) & Rest API.
- Identify and understand architectural pieces, provide direction and technical expertise in design, development and systems
- Able to make quick decisions and solve technical problems to provide an efficient environment for project implementation. Able to make quick decisions and solve technical problems to provide an efficient environment for project implementation.
- Strong hands-on experience on Azure data services (ADLS, Azure Synapse, Azure SQL, Azure Databricks, etc.)
- Awareness of data integration patterns, data modeling and data architecture
- Strong experience in programming Languages (e.g., Python, SQL, etc.)
- Strong knowledge in ETLs, APIs, algorithms, data structures and business intelligence tools (Power BI)
- Ability to regularly learn and adopt new technology
- Expert in SQL and data analysis and shell scripting, ability to debug and fix data issue
- Ability to communicate across all levels of the organization and work with diverse project teams
- 5 years of experience in Data and Information Management for Oil and Gas Upstream operations is an advantage.
- 5+ years of experience in Data Integration Projects
- Good Knowledge of Applications Architecture and Information Security Principles
- Fluent English both spoken and written