Big Data Engineer

Job Description

MGID was founded in 2008 and is one of the leading companies in native advertising. We enable our media partners to monetize their audience and help brands to promote their services and goods effectively.

MGID offers a range of integrated solutions covering the promotion process every step of the way; we offer services ranging from planning out the marketing strategy to its thoughtful implementation and optimization. Our clients include major international brands like Renault, Domino’s, Airbnb, PizzaHut, Qatar Airlines, and many others, including media organizations and web agencies.

MGID is:

• A proprietary Highload service that delivers 185 billion advertisements to 850 million unique users in more than 70 languages;
• The winner of multiple AdTech awards for innovation and product quality;
• A workforce of 700+ employees operating from offices in the US, Europe, and Asia.

We are looking for a Big Data Engineer to efficiently work with large datasets and support our Data Science team in developing, improving, and delivering our ML and AI solutions and algorithms.
If you’re passionate about building scalable data solutions and staying up-to-date with the latest industry trends and technologies, we want to hear from you!

Requirements

Who you are:

— Proven experience in developing and optimizing PySpark applications.
— Strong knowledge of distributed computing principles and concepts.
— Practical experience working with large datasets using technologies such as Hadoop, Spark, ClickHouse.
— Proficiency in programming languages such as Python, SQL.
— Experience with Linux/Unix command-line interface.
— Familiarity with data visualization and dashboarding tools.
— Strong communication skills and ability to work effectively in a remote team environment.
— Excellent problem-solving skills and attention to detail.

Will be a plus:
— Bachelor’s or Master’s degree in Computer Science or a related field.
— Practical experience with ClickHouse.
— Practical experience with stream processing and messaging systems such as Kafka.
— Practical experience with NoSQL databases (for example MongoDB), especially Aerospike.
— Knowledge of AdTech domain — understanding of online advertising, RTB.
— Familiarity with containerization technologies such as Docker and Kubernetes, cloud computing platforms.
— Familiarity with data governance and security best practices.
— Knowledge of machine learning concepts and frameworks.

What You Will Do:
— Collaborate with Data Scientists, Data Analysts, and other stakeholders to understand data needs and develop solutions.
— Design, develop, and optimize PySpark applications for processing and analyzing large sets of structured and unstructured data.
— Monitor and evaluate data to ensure accuracy and integrity, troubleshoot and debug PySpark code.
— Build and maintain data pipelines for ingesting, processing, and storing data, optimizing for performance and scalability.
— Develop and maintain data visualization dashboards and reports to enable insights and decision-making.
— Create and maintain tools and libraries for efficient data processing.
— Stay up-to-date with industry trends and new technologies to continuously improve data processing capabilities.

Join MGID, a company known for its results-driven culture and drive for innovation in AdTech. As part of our team, you will feel supported, connected, and have the flexibility you need to thrive in both your personal and professional life. We value your background, ideas, enthusiasm, and desire to improve every day.

MGID is an equal opportunity employer. We value our colleagues for who they are, no matter what they look like, where they come from, or what language they speak.