Industry: E-commerce (Fashion & Home)
Location: Portugal
Work Model: 100% Remote
Start Date: ASAP
Project Language: English
Project Overview:Join a leading e-commerce company in fashion and home décor, as they undertake critical data engineering projects to support their evolving business needs. This is an exciting opportunity to contribute to high-impact data solutions for an innovative, customer-focused brand, leveraging the latest in data engineering best practices.
Responsibilities:Project Understanding and Communication:Analyze business and technical challenges from a user perspective.Collaborate with Data Architects and Project Managers to ensure solutions align with the client's data architecture.Data Pipeline Development:Design, build, and deploy efficient data pipelines according to project requirements.Apply best practices for performance, scalability, and maintainability.Use Terraform to deploy and manage infrastructure efficiently.Testing and Deployment:Define test cases and conduct testing in collaboration with the Project Manager.Present completed developments to Data Architects and Lead DataOps, ensuring smooth deployment and active monitoring post-deployment.Documentation and Peer Review:Document processes, tests, and results thoroughly.Conduct peer reviews and participate in code reviews for quality assurance.Requirements:Hard Skills:Proficiency in PySpark and Spark SQL for data processing.Experience with Databricks and Delta Live Tables for ETL and workflow orchestration.Familiarity with Azure Data Lake Storage for data storage and management.At least 1 year of experience with Terraform and GitOps practices for infrastructure deployment.Strong understanding of ETL/ELT processes, data warehousing, data lakes, and data modeling.Knowledge of orchestration tools (e.g., Apache Airflow) for pipeline scheduling and management.Experience with data partitioning and lifecycle management in cloud storage.Optional:Experience with Databricks Asset Bundles, Kubernetes, Apache Kafka, and Vault is a plus.
Soft Skills:English Fluency: Strong written and verbal English skills in a working environment.Communication: Ability to convey technical concepts effectively and understand user needs.Organizational Skills: Detail-oriented with the ability to maintain structured documentation.Problem-Solving: Proactive approach to understanding and addressing data challenges.
#J-18808-Ljbffr