Senior Software Engineer - Husky Storage

Husky is what we call the distributed, petabyte-scale columnar event store at the heart of our Event Platform, which powers dozens of Datadog’s most popular products – Logs, RUM, APM, Cloud Network Monitoring, Netflow, and many more. Husky was built from the ground up at Datadog to store and query massive volumes of...

As a Senior Software Engineer on this team, you'll work hands-on with a distributed storage system at petabyte scale, and help us evolve our platform to support the next level of scale that will power the AI era. We will move from a custom storage format towards open standards (Parquet, Arrow, DataFusion) enabling o...

Datadog has a culture that is increasingly rare in mature technology companies. We are an engineering driven, bottom up, high agency/high responsibility culture–where everyone is empowered and expected to drive the technological and strategic direction of the business over time. We are a blameless culture where we b...

Drive Husky to the next level of adoption by migrating our storage format to open source Parquet and re-imagine compaction on a format-agnostic, DataFusion-based substrate
Drive concrete cost-reduction work: block-level inverted index compression, combining the database with external objects at the row level, and storage tiering/caching
Scale our solution to enable customers to manage event versioning at scale, using temporal-database techniques.
Play an influential role in operating and deciding how to best operate our service going forward.
Work across the write path (with Ingestion) and toward a cleaner Iceberg-based handoff to ADP/Lakehouse for batch/analytical workloads

Enthusiastic and experienced with large-scale distributed systems and/or storage engines
Comfortable with improving and operating large distributed systems that customers rely on for critical business functions
Strong fundamentals in columnar/file storage formats (Parquet, ORC, Arrow) or comparable systems
Bonus: experience with Go, Java, or Rust; with compaction or compression/encoding schemes (RLE, dictionary, delta, gorilla); with object storage (S3/GCS/Azure Blob) at scale, including caching and cost optimization; with Kafka and stream processing; or with Kubernetes

 
Datadog values people from all walks of life. We understand not everyone will meet all the above qualifications on day one. That's okay. If you’re passionate about technology and want to grow your skills, we encourage you to apply.