Keyrock
Senior Data Engineer
Brussels, Abu Dhabi, Amsterdam, ...Full-timeGlobal
š Midš Remote
Job Description
[AI-summarized by JobStash]
You will build and operate streaming and batch data pipelines that ingest, normalise, and distribute market, trading, and portfolio data. You will design the lakehouse and time-series layers around consumer query patterns, own data contracts and schema evolution, and implement data quality, lineage, and self-healing. You will provide self-serve tooling, instrument observability, treat infrastructure as code, and work openly with architecture, infrastructure, platform, and product stakeholders. You will produce derived analytics such as cross-exchange spreads, VWAP, order book microstructure, and portfolio/performance views.
Requirements
- ā8+ years of building production data systems
- āStrong proficiency in Python
- āStrong proficiency in SQL and reasoning about query engines
- āStrong understanding of data modelling for streaming and analytical workloads
- āExperience designing and operating streaming systems (Kafka, Redpanda, MSK, or Kinesis)
- āExperience with time-series stores in production (ClickHouse, TimescaleDB, QuestDB, or similar)
- āExperience with lakehouse architectures and table layout, partitioning, and compaction decisions
- āExperience building for idempotency and self-healing with safe reprocessing
- āExperience with Docker, Terraform, and CI/CD
- āExperience instrumenting logs, metrics, and traces for observability
- āExperience designing data quality, governance, contracts, validation, lineage, and ownership
- āUnderstanding of financial market data (order books, trades, reference data, portfolios, exposures)
- āAbility to design, ship, operate, and improve end-to-end data systems
- āNice to have: Lakehouse experience with Apache Iceberg or Delta Lake
- āNice to have: Familiarity with DataHub or similar metadata/lineage platforms
- āNice to have: Rust familiarity
Responsibilities
- āBuild streaming and batch pipelines that ingest, normalise, and distribute market, trading, and portfolio data resilient to feed and exchange failures
- āBuild self-serve tooling (SDKs, patterns, templates, AI agents) for publishing and consuming data products
- āOwn data contracts and manage schema evolution
- āDesign the lakehouse and time-series layer around consumer query patterns
- āBuild and evolve data governance and data quality frameworks including stale-feed detection, schema validation, range checks, idempotent writes, lineage, and ownership
- āBuild derived analytics such as cross-exchange spreads, VWAP at depth, order book microstructure, portfolio views, exposure, and performance
- āMake observability, cost, and performance first-class
- āTreat infrastructure as code (Docker, Terraform, CI/CD)
- āWrite documentation and partner closely with Architecture, Infrastructure, Platform, and other teams
Benefits & Perks
- āFlexible hours
- āRemote-first
- āBusiness-hours on-call shared across the team
- āRegular online get-togethers
- āYearly onsite
- āAutonomy on how you work
- āStrong cross-functional partners
Tech Stack
DockerPythonSQLstreamingClickHouseRustmetricsportfolioVictoriaMetricstime series