Data Engineer Jr- Mid Responsibilities: Actively participates in requirements gathering, understanding sessions, and research to estimate coding and/or pipeline automation times considering the data engineering lifecycle and "go to production" requests for data solutions. Contributes to all phases of the data engineering lifecycle using Agile methodology and adhering to rituals to track and meet the committed delivery dates on time and in a structured manner. Understands and applies Data Governance concepts throughout the data solution development process, ensuring their quality. Performs data ingestion and/or processing of structured data files, using relational databases (SQL); for data processing: batch, micro-batch. Ensures the continuity of digital data solutions, insights, dashboards, etc., to build and consolidate a Data-Driven culture. Documents processes or diagrams related to the processes involved in the development of the data solution under their responsibility, including everything necessary for "go to production," to guarantee continuity and efficient execution in a productive environment. Ensures the application of data privacy design concepts and/or strategy. Alerts their leader of anomalies and/or risks found in data solution developments or under their responsibility, so they can be addressed without affecting the business. Communicates the status of their solution development activities to meet committed deadlines in a timely and structured manner. Required Knowledge: 1 year of experience as a Data Engineer. Understanding of data engineering concepts and principles, including ETL (Extract, Transform, Load) processes, data pipelines, and data warehousing. Strong knowledge of Python fundamentals, including data structures, functions, error handling, and modules. Familiarity with Python libraries and frameworks commonly used in data engineering, such as Pandas, NumPy, and PySpark. Intermediate knowledge of different types of databases (SQL and NoSQL), including relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra). Strong understanding of data modeling concepts and techniques, including normalization, denormalization, and schema design. Familiarity with various data processing methods, such as batch processing and stream processing. Understanding of file processing concepts, including reading and writing files in different formats (e.g., CSV, JSON, Parquet). Basic understanding of data governance principles, including data quality, data lineage, and data privacy. Familiarity with the end-to-end data engineering lifecycle, from data ingestion and processing to storage and retrieval. Basic understanding of various types of data architectures, including data lakes, data warehouses, and data marts. Knowledge of version control systems (e.g., Git) and experience using platforms like GitHub or GitLab. Basic understanding of data visualization tools (e.g., Tableau, Power BI) and reporting techniques. Basic familiarity with cloud computing (GCP and AWS) stack. Participation in projects with Objectives and Key Results (OKRs), ability to identify risks, and contribute to delivering business value. Intermediate ability to transparently communicate project status.
#J-18808-Ljbffr
