Senior Data Engineer in Bucuresti

Hadoop (advanced) Spark (advanced) Big Data (advanced) Data (advanced) Svitla Systems Inc. is looking for a Senior Data Engineer for a full-time position (40 hours per week) in Poland. Our client is one of the world’s largest retailers and the largest private employer in the United Kingdom and serves millions of customers weekly in the stores and online. They operate in Hungary, Poland, the Czech Republic, and Slovakia in Europe, offering customers daily value. You will join a multi-functional and specialist team that drives operational excellence of services, improves scale for the systems and processes globally, and creates business-leading capabilities. The project’s team builds products that power the engine of the Supply Chain that makes sure customers get what products they need when they walk into the client’s store or shop through other channels. They influence product availability through advanced forecasting and proper ordering. It's challenging since it needs to be done while keeping waste reduction and optimal stock holding. It helps keep the operational costs in control and helps the bottom line. The more they save, the more they can pass on to the customers w.r.t the right price. It’s fun since the client is building better capabilities on newer platforms. It involves a lot of learning since we are transforming forecasting intelligence into algorithms on big data platforms. They crunch a large history volume, react to recent trends and promotional events, and turn demands into the correct orders for suppliers and stores. The nature of the products lends itself to real-time data inputs, machine learning, and applied statistics, safely allowing them to say it’s a great problem space to work in and help solve. The team takes responsibility for the software's entire lifecycle, practices continuous integration and delivery, and supports their code to production and beyond. It is a mixed work mode role with some office presence in Kraków expected, probably 1-3 times per month for teamwork and meetings, to help build teamwork and team spirit (can be negotiated). Requirements:6+ years of commercial experience in software development.4+ years of experience in a big data platform.Expertise in Hadoop.Expertise in Spark and Spark SQL.Knowledge of PySpark.Knowledge of Hive (HiveLLAP).Background in HBase, Phoenix, and other SQL and NoSQL (Druid, Kylin would be added advantage).Hands-on experience in data ingestion and transformation (Sqoop/NiFi /Flume/Kafka, consuming APIs).Understanding of how to efficiently structure data on HDFS (columnar stores, compression, partitions)Good experience in productionizing algorithms.Understanding design/development batch and streaming framework and when to use which.Ability to give design streaming applications.Ability to turn a business problem into an algorithmic formulation (incl. breaking it down into multiple parts if necessary).Understanding ML Techniques.Ability to build data platform capabilities: Backtesting, Disaster Recovery, query engine for analysts, etc.Problem-solving, analysis, and computational skills. You are comfortable designing systems and reasoning about them. When tackling problems, you take a thorough and logical approach to solving them and retain ownership through resolution.Experience in designing and developing systems for high availability and scalability.Experience working with data and algorithms and setting up solid data platform capabilities.Depth of professional software engineering experience and best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations.Experience in running and supporting software in production. Responsibilities:Consistently deliver against the product and engineering backlogs.Support your and your team’s code to production.Lead and evangelize the use of Monitoring & Alerting tooling (Splunk) to “do production right.”Adhere to the processes, designs, and systems your teams have agreed to.Contribute to the definition of features for services owned by your team.Continuously improve yourself and your team by suggesting and helping implement new engineering processes and practices.Keep a customer focus. You find the right balance between outcome delivery and technical excellence.Be a technical authority on one or more specialized areas of your team, someone, the rest of the team turns to for engineering advice or guidance.Lead in setting coding standards, testing standards, and operational best practices.Help less experienced developers.Effectively communicate your and your team’s progress to your team members, manager, and the project’s colleagues. We Offer: Salary of 5 500 - 7 000 USD/per month on a B2B contract.US and EU projects based on advanced technologies.Competitive compensation based on skills and experience.Annual performance appraisals.Flexibility in workspace, either remote or in our welcoming office. Comprehensive medical insurance after one month.MultiSport card with access to 2500 sports facilities all over PolandBonuses for recommendations of new employees.Bonuses for article writing, public talks, other activities.15 vacation days, 10 national holidays, sick leaves, family days off.Educational activities reimbursement on the monthly basis.Free webinars, meetups and conferences organized by Svitla.Gifts for birthdays, New Year and other holidays.Fun corporate celebrations and activities.Awesome team, friendly and supportive community! About Svitla:Svitla Systems is a global trusted IT solutions company headquartered in California, with business and development offices throughout the US, Latin America, Europe, and Asia. Svitla is an outspoken advocate of workplace flexibility, best known for its well-established remote culture, individual approach to our teammate’s professional and personal growth, and family-like environment. Since 2003, Svitla has served a wide range of clients, from innovative start-ups in California to mega-large corporations such as Ingenico, Amplience, InvoiceASAP and Global Citizen. At Svitla, developers work with clients’ teams directly, building lasting and successful partnerships, as a result of seamless integration with on-site processes. Svitla Systems’ global mission is to build a business that contributes to the well-being of our partners, personnel and their families, improves our communities, and makes a lasting difference in the world. Join us! If you are interested in our vacancy, please send your CV. We will be happy to see you in our friendly team