We are looking for talented profiles to help build and maintain the distributed data collection system that is at the heart of our business.
We are a data-driven company which collects and processes more than 500GB of raw data daily. We leverage big data technologies such as Serverless, Spark on AWS EMR to crunch these volumes of data and make it queryable.
In this role you will ensure that our data collection engine, which consists of distributed web crawlers, is state of the art and ahead of our competition. You will ensure that we can scrape any webshop, no matter the ban-detection that has been put in place. Next to that it will be important that the proper monitoring tools are in place. We are currently scraping 60 sites and your goal is to at least triple that without losing completeness and quality.
As a distributed systems engineer you will be responsible for the following topics:
On top of all this you’ll make sure that Daltix stays competitive in terms of data collection by using the latest & most suitable technologies throughout our stack.
About the stack
What can Daltix offer you?
Daltix’ offers a competitive wage (including various benefits etc) and a young, dynamic and international (we have offices in Belgium and Portugal) atmosphere to work in.
You will also receive the possibility to work from home if you prefer (even if you live in Lisbon).
When you start working at Daltix, you will get a deep dive experience. You learn all you need to know about us, our journey, your future colleagues, the tools we work with, etc.
Going beyond, is coded in our company DNA. As soon as you start working, we expect a hands-on approach, with an entrepreneurial mentality.
You will also be able to participate in relevant trainings to stay at the top of this field.
Besides developing your technical skills you will also have the opportunity to grow into the following skill sets:
Technical/architectural lead. SW project management. Team leading & coaching.