Overview:
**Responsibilities**:
- Constructing secure data pipelines, with a focus on automation through CI/CD techniques.
- Identifying and resolving data quality issues in collaboration with the data team.
- Preparing data for business users, data analysts, and data scientists.
- Communicating directly with clients and product owners.
- Supporting system design.
- Assisting with issue triage, diagnosis, and resolution.
- Collaborating on code reviews and solutions with other developers.
- Contributing to system development methodologies and staying informed about evolving technologies, techniques, architectures, and hardware relevant to data management and platforms.
- Playing a pivotal role in the introduction of new technologies and tools.
- Ensuring robust data security through encryption, access controls, and authentication mechanisms to maintain data integrity and confidentiality across its lifecycle.
Qualifications:
- 3-5 years of Data Engineering experience.
- Expert in SQL
- In-depth experience with at least one programming language (e.g., Java, Python, Bash, JavaScript).
**Required Technical Expertise**:
- Proficiency in mainstream databases (e.g., SQL Server, Oracle, MySQL, DB2).
- Must-have: experience in Microsoft Fabric ideally, or Azure stack as a compromise.
- Deep knowledge of large-scale analytic database technologies (e.g., Snowflake, Redshift, Azure Synapse).
- Experience constructing effective data pipelines.
- Data Modeling
- Understanding of diverse Data & Analytics hybrid architectures, including Data Lake, Data Warehouse, Data Mesh, Enterprise Data Hub, Data Fabric, and Delta Lake with real-time, batch, and micro-batch workloads
- Extensive Python experience.
- Expertise in API programming interfaces such as JDBC, ODBC, REST API, and SOAP API.
- Logging, auditing, and tracing knowledge.
- Proficiency in SQL.
- Intermediate Shell scripting skills (Bash, PowerShell, etc.).
- Data security and privacy expertise.
- Automation of data pipelines.
- Infrastructure-as-code implementation (e.g., Liquibase, Flyway).
- Hands-on experience with various data types (structured, semi-structured, unstructured, JSON, AVRO, ORC, Parquet, Iceberg, Arrow).
- Data Quality Assurance mastery.
- Knowledge of Big Data technologies (e.g., Hadoop, Hive, Spark, Kafka) and data engineering.
- Expertise in Data Warehousing and Cloud Platforms (e.g., Snowflake, Redshift, Azure Synapse).
- Proficiency in source code control using Git.
- Understanding of Data Science and Machine Learning tools and algorithms.
**Desirable Skills**:
- Experience with major cloud technologies (AWS, GCP, Azure).
- Data Application Development proficiency.
- Data Modeling expertise.
- Experience in large-scale data migrations.
- Extensive experience with Spark and Databricks.
- Proficiency in Scala, Java, and JavaScript.
- Familiarity with AWS EMR and Presto.
- Expertise in ETL/ELT tools (e.g., Informatica, Talend, DBT).
- Knowledge of Data Replication (e.g., HVR, Golden Gate, Fivetran).
- Understanding of Data Virtualization (e.g., Denodo).
- Hands-on experience with machine learning.
- Familiarity with Data Governance practices.
**Consultative Skills**:
- Dedication to adding value to projects and teams.
- Flexibility and customer-centric approach.
- Embracing feedback and showing growth.
- Enthusiastic about continuous learning and professional development.
- Identifying and addressing project challenges with Nortal leadership.
- Understanding the high-level business value of ongoing projects.
- Participation in mid-level technical meetings with clients.
- Strong written and verbal English communication.
- Self-directed work style.
- Mentorship potential and orientation for Junior Engineers.
- Initiative-driven and results-focused.
- Openness to new experiences and knowledge.
- Exceptional communication skills.
NortalCareers
**#LI-Remote