- Design, develop, and maintain robust data pipelines using Java within AWS infrastructure.
- Implement scalable solutions for data analysis and transformation using Apache Spark and PySpark.
- Utilise Airflow for efficient workflow orchestration in complex data processing tasks.
- Ensure fast and interactive querying capabilities through the use of Presto.
- Containerise applications using Docker for streamlined deployment and scaling.
- Orchestrate and manage containers effectively with Kubernetes in production environments.
- Implement infrastructure as code using Terraform for provisioning and managing AWS resources.
- Collaborate with cross-functional teams to understand data requirements and architect scalable solutions aligned with business goals.
- Ensure data quality and reliability through robust testing methodologies and monitoring solutions.
- Stay updated with emerging technologies and industry trends to continuously enhance the data engineering ecosystem.
- Bachelor's degree in Computer Science, Engineering, or related field.
- Minimum 4 years of hands-on experience in Java / Scala / Python development, emphasising object-oriented principles.
- Proficient in Apache Spark or PySpark for large-scale data processing.
- Experience with Airflow for workflow orchestration in production environments.
- Familiarity with Docker for containerisation and Kubernetes for container orchestration.
- Knowledge of Terraform for infrastructure as code implementation in AWS environments.
- Experience managing AWS services such as S3, EMR, Glue, Athena, and Redshift.
- Strong background in SQL and relational databases, with proficiency in technologies like Postgres.
- Preference for experience with streaming platforms such as Kafka for real-time data processing.
- Excellent English language communication skills, both verbal and written.
- Ability to collaborate effectively with technical and non-technical stakeholders.
-
Data Engineer
há 3 dias
Dellent Consulting Porto, PortugalSe és um profissional apaixonado pelo universo automóvel e pela inovação na mobilidade, e queres enfrentar novos desafios, estás no sitio certo. Procuramos pessoas que queiram fazer parte de um projeto líder no setor, contribuindo para a transformação e excelência num ambiente di ...
-
Data Engineer
há 1 dia
Dellent Porto, Portugal Tempo integralA Dellent é uma empresa de consultoria especializada em Sistemas de Informação e Telecomunicações. O nosso objetivo é conseguirmos ajudar os nossos consultores e candidatos a integrarem projetos que vão ao encontro dos seus interesses, motivações e expectativas. · Se és um profis ...
-
Data Engineer
há 3 dias
Mind Source Porto, PortugalLOCAL: Porto (Híbrido) DESCRITIVO DA FUNÇÃO: A Mind Source está a reforçar a sua Equipa de Business Insights através de um Data Engineer.PERFIL:- Licenciatura em Engenharia Informática, Informática de Gestão ou similar;- Pelo menos 3 anos de experiência com modelagem de dados e P ...
-
Data Engineer
há 3 dias
Pkf-Vmb Belgium Porto, PortugalQuantas vezes te aborreceste à procura de uma oportunidade de emprego que fizesse sentido para ti? Apostamos que perdeste a conta... Se ainda não ouviste falar de nós, temos uma história para ti. Se já ouviste, acreditamos que o que te vamos contar vai fazer a diferença · A agap2 ...
-
Data Engineer
há 1 semana
Mind Source Porto, Portugal Tempo integralLOCAL: Porto (Híbrido) · DESCRITIVO DA FUNÇÃO: · A Mind Source está a reforçar a sua Equipa de Business Insights através de um Data Engineer. · PERFIL: · Licenciatura em Engenharia Informática, Informática de Gestão ou similar; · Pelo menos 3 anos de experiência com modelagem de ...
-
Data Platform Engineer
há 3 dias
Quickops Porto, PortugalRef. AO2089Estamos à procura de um Data Platform Engineer para se juntar a empresa líder na área de retalho.Se te consideras uma pessoa flexível e proativa e desejas enfrentar novos desafios profissionais, envia-nos a tua candidatura Estamos ansiosos por fazer parte do teu cresci ...
-
Data Platform Engineer
há 2 semanas
Quickops Porto, PortugalRef. AO2089 · Estamos à procura de um Data Platform Engineer para se juntar a empresa líder na área de retalho. · Se te consideras uma pessoa flexível e proativa e desejas enfrentar novos desafios profissionais, envia-nos a tua candidatura Estamos ansiosos por fazer parte do teu ...
-
Full-Stack Data Engineer
há 2 dias
Employment Pro Limited Porto, PortugalBuscar vagas de emprego e oportunidades de carreira · Encontramos 735 vagas para Vagas de emprego full-stack-data-engineer .Receba alerta de vagas em seu email. Registre-se agoraFunção: Full Stack EngineerEmpresa: DevoteamTipo: Full-timeContrato: n/dLocalidade: Aveiro, Lisboa, Po ...
-
Data Engineering Intern
há 1 semana
Adidas Porto, PortugalData Engineering is all about making sure that the organizational data, coming from both internal and external sources, is stored, processed and analysed as effectively and efficiently as possible. Being a Data Engineer means having a deep understanding of techniques and technolo ...
-
Data Engineer
há 5 dias
AskBlue Porto, PortugalWe are currently looking for a Data Engineer to integrate one of your project teams in Lisbon or Porto. Main Tasks: Develop and design data pipelines/ETL in Python; Industrialize processes for full automation; Assess client needs, understand the business and deliver a proper solu ...
-
Data Engineer
há 5 dias
Closer Consulting Porto, PortugalCloser is a tech company created in . Carrying innovation in our DNA, we are a 100% Portuguese company with offices in Portugal, Brazil, the United Kingdom, and Hong Kong. Being proud of our steady growth and impact in the IT market, we are conscious of the importance of the role ...
-
Senior Data Engineer/ Data Engineer
há 1 semana
Maersk Porto, Portugal Tempo integralMaersk is going through times of unprecedented change. From the farm to your refrigerator, the factory to your wardrobe, we are developing solutions that meet customer needs from one end of the supply chain to the other. Digitization and IT are taking center stage in enabling our ...
-
Data Engineer
há 6 dias
Dbservices Portugal Porto, PortugalDBServices helps to design, develop and support digital platforms for large and mid-sized companies in ITC, Banking, Retail, Government, Health, and other industries in Europe and Americas. · TasksCollaborate closely with business users to understand their requirements and transl ...
-
Data Engineer
há 3 dias
We Are Meta Porto, PortugalData Engineer (French Speaker)PortoPython,Pyspark,Hadoop,GCP,machine learning · Are you a Data Engineer? Become a Rhino join the #crash We are META, more than a company a TEAM that will shake your idea about tech recruitment. Based in Évora but acting mainly in a remote way, we ...
-
Data Engineer
há 3 dias
Multipessoal Porto, PortugalWe are recruiting a Data Engineer for a multinational company on the Energy Sector. · - Design and develop robust data pipelines and architectures to facilitate efficient data processing, transformation, and loading; · - Implement scalable and reliable ETL processes to extract, t ...
-
Data Engineer
há 1 semana
DBServices Portugal Porto, Portugal CDIDBServices helps to design, develop and support digital platforms for large and mid-sized companies in ITC, Banking, Retail, Government, Health, and other industries in Europe and Americas. · Tasks · Collaborate closely with business users to understand their requirements and tra ...
-
Data Engineer
há 5 dias
WE ARE META Porto, Portugal Data EngineerSeeking a Data Engineer: Rhino, are you there? · At WE ARE META, we're focused on finding the perfect · match between our Rhinos and our clients. · Why join us? · Expand your network: As a consultant at We Are Meta you'll get to have access to a network of national an ...
-
Data Engineer
há 5 dias
Keyrus Porto, PortugalEmpowered by our motto, " Making Data Matter ", Keyrus stands at the forefront of global consultancy in Data & Analytics, AI & Machine Learning, and Digital Transformation. With a presence in 27 countries across 4 continents, we're on a mission to elevate the impact of data. · We ...
-
Data Engineer
há 2 dias
Dellent Consulting Porto, PortugalWould you like to make an impact in different industries with your work? Then this project is for you. Here you will have the opportunity to be part of a multicultural and inclusive team with global projection that contacts with areas such as consultancy, law, strategy, insurance ...
-
Data Engineer
há 6 dias
Db Services Portugal Porto, PortugalDBServices helps to design, develop and support digital platforms for large and mid-sized companies in ITC, Banking, Retail, Government, Health, and other industries in Europe and Americas. · TasksCollaborate closely with business users to understand their requirements and transl ...
Middle/Senior Data Engineer - Porto, Portugal - Devexperts
Descrição
Company Description
Devexperts has been working for nearly two decades consulting and developing for the financial industry. We solve complex technological challenges facing the most well-respected financial institutions worldwide.
By becoming a part of Devexperts, you'll become a part of a company that fosters self-improvement and actively seeks out-of-the-box ideas. Our teams work together to create the next generation of financial software solutions. We welcome all candidates who believe, as we do, that innovation is grounded in education.
Job Description
We are looking for a Middle/Senior Data Engineer with a Java / Scala / Python background to join the project for a Top-5 US retail broker (by the number of users). The project is devoted to trading experience, finance reports, and risk management.
You will join a cross-functional team that excels in getting features done from zero to production.
Key responsibilities:
1. Data Pipeline Development:
2. Infrastructure Management:
3. Collaboration and Communication:
Qualifications
1. Education and Experience:
2. Technical Proficiency:
3. Communication Skills: