Amberdata
Senior Big Data Engineer
RemoteFull-timeGlobal
📊 Mid🏠Remote
RemoteRemote work position availableActivePosted within the last 30 days
Job Description
[AI-summarized by JobStash]
YOU will design and implement scalable data pipelines that process petabytes of data and power analytics. YOU will write performant, clean, and well tested code and contribute to a high performance distributed system. YOU will work with modern data processing frameworks and DevOps practices to deliver reliable data products in a fast paced environment. YOU will collaborate with cross functional teams to ensure data quality and reliability.
Requirements
- â—Ź8+ years of hands on coding and software design experience
- â—ŹFluent in at least two programming languages and deep expertise with either Java Scala Go or Python
- â—ŹExperience in implementing scalable and reliable services using big data processing frameworks and tools like Hadoop EMR Pig Hive Databricks and equivalents
- â—ŹExperience with designing and managing data workflows using orchestration tools such as Airflow Luigi Dagster and similar technologies
- â—ŹExperience working in stream processing frameworks such as Flink or Storm
- â—ŹExperience in establishing and promoting high standards in pipeline monitoring data validation testing
- â—ŹKnowledge of creating efficient data models for analytical operational use
- â—ŹPassionate about big data problems distributed systems and an appetite for learning and growing
- â—ŹExperience with the DevOps mindset infrastructure as code Continuous Delivery and TDD
- â—ŹHands on experience with AWS cloud native architectures Kubernetes Docker etc
- ●Bachelor’s master’s or doctorate degree in a related field or an intriguing reason for not having one
- â—ŹExcellent communication skills
Responsibilities
- â—ŹWorking on interesting and complex technical challenges
- â—ŹDesigning end to end solutions that process and store petabytes of data
- â—ŹWriting performant clean and well tested code
- â—ŹUsing Continuous Delivery practices and DevOps tools to deliver software
- â—ŹDefining big data best practices and guidelines
- â—ŹWorking in a collaborative and fast paced environment
- â—ŹEnjoying a high degree of autonomy contributing to ideas and shaping architecture
- â—ŹWorking on a high performance distributed system
Benefits & Perks
- â—ŹCompetitive Salary including very attractive stock options package
- â—ŹEmployee benefits including healthcare (medical dental vision) and 401k
- â—ŹUnparalleled opportunities to help build a category defining company
- â—Ź100% remote
Tech Stack
datadatamodelingDatabricksarchitectureEMRPythonworkflowDevOpsDagsterScala