.Develops end-to-end data engineering solution projects.
Contributes by estimating tasks and using an agile methodology.
Ensures code quality and performance by adopting best engineering principles, following the defined data engineering lifecycle.
Collaborates closely with multidisciplinary teams, contributing their expertise and learning from their peers to solve technical problems and actively participates in continuous improvement.Main Responsabilities:Contributes to identify inputs and data origins, as well as the feasibility of requested requirements, identifying risk situations and communicating them to relevant participants during these sessions.Contributes to all phases of the data engineering lifecycle.Estimates the time it will take to code data solutions, considering the stages of the data engineering lifecycle, documentation, "go to production," and go-live.Performs ingestion or processing of structured and semi-structured data files, using relational and non-SQL databasesEnsures the continuity of digital data solutions, insights, dashboards, etc.Documents processes or diagrams related to data architecture / processes / technical memory, to ensure continuity and efficient execution in a productive environment.Generates code versions in the repository, artifacts, and components of data solutions and/or data products throughout the data engineering lifecycle, project closure, and/or post-mortemRequired Knowledge and Experience:2-4 years of experience as a Data EngineerIn-depth understanding of core data engineering concepts and principles, including complex ETL (Extract, Transform, Load) processes, scalable data pipelines, and advanced data warehousing techniques.Advanced proficiency in Python, including writing efficient and optimized code, using advanced features like decorators, generators, and context managers.Extensive experience with Python libraries and frameworks commonly used in data engineering, such as Pandas, NumPy, PySpark, and Dask.Strong knowledge of both SQL and NoSQL databases, including advanced querying, indexing, and optimization techniques.Experience with database design, normalization, and performance tuning.Advanced understanding of data modeling concepts and techniques, including star schema, snowflake schema, and dimensional modeling.Experience with data modeling tools and best practices.Proficient in various data processing methods, including batch processing, stream processing, and real-time data processing.Extensive experience with data processing frameworks like Apache Spark, Apache Kafka, and Apache Flink.Advanced knowledge of file processing concepts, including handling large datasets and working with different file formats (e.G., CSV, JSON, Parquet, Avro).Strong understanding of data governance principles, including data quality management, data lineage, data security, and data privacy