We are seeking a highly skilled Data Engineer to spearhead the collection, storage, processing, and analysis of extensive datasets. As a pivotal member of our team, you will play a crucial role in identifying and implementing optimal solutions for these tasks while ensuring their seamless integration into our client architectures. Your responsibilities will encompass the full lifecycle of data management, from initial design and implementation to ongoing maintenance and monitoring. This role offers an exciting opportunity to leverage your expertise in data engineering to drive innovation and deliver actionable insights for our clients.
About Data Masters
Data Masters is a company specialized in the field of Data Science. We are a dedicated and passionate team, extremely motivated to address the challenges of any company and help them become Masters of their Data.
Our team members possess knowledge and expertise in Data Warehousing, Data Engineering, Machine Learning, and Analytical Product Development. We support clients from all over the world, and we would love to welcome new team members that share our passion, mission, and vision.
Job description
Responsibilities
- Spearhead the selection and integration of cutting-edge Big Data tools and frameworks to unlock innovative capabilities and drive actionable insights.
Design and implement robust ETL processes to ensure seamless data flow and transformation, empowering our teams with accurate and timely information.
Proactively monitor system performance metrics, providing strategic recommendations for infrastructure enhancements to optimize efficiency and scalability.
Shape and enforce data retention policies, ensuring compliance with regulations while maximizing the value and usability of our data assets.
Collaborate closely with cross-functional teams to understand evolving business needs and translate them into scalable data solutions that deliver tangible impact.
Stay abreast of industry trends and emerging technologies, continuously enhancing our data engineering practices to maintain our competitive edge.
Foster a culture of collaboration, innovation, and continuous improvement within the data engineering team, inspiring excellence and driving collective success.
Join our team and be at the forefront of transforming raw data into actionable insights that drive meaningful business outcomes. If you’re passionate about leveraging cutting-edge technologies and shaping the future of data engineering, we want to hear from you!
Skills and Qualifications
- BSc/BA in Computer Science, Engineering, or a relevant field
- At least 3 years in the business intelligence or data engineering role
- Experience with integration of data from multiple data sources
- Knowledge of various ETL techniques and frameworks
- At least 3 years of python experience
- Experience with PySpark or Scala
- Knowledge of stream processing frameworks such as Apache Kafka or Apache Flink
- Experience in developing ETL pipelines in at least one of the following cloud providers(AWS, Azure, GCP)
- Knowledge of data visualization tools such as Power BI
- Familiarity with DevOps principles and practices, including CI/CD pipelines and automated testing
- Familiarity with containerization technologies such as Docker and container orchestration tools like Kubernetes
