Data Engineer - LLM Pipeline & Data Infrastructure

We're building an AI-powered conversational system for drive-thru automation. As our Data Engineer, you'll design and implement the infrastructure that powers our multi-stage LLM pipeline, from data capture to processing, model training, and deployment.

• Build scalable real-time data pipelines for audio processing, LLM interactions, and model training
• Design comprehensive data storage solutions across object storage, NoSQL, and analytical databases
• Implement data quality management with filtering, normalization, and enrichment capabilities
• Create automated processes for data preparation, model evaluation, and continuous improvement
• Develop observability systems with monitoring, alerting, and performance dashboards
• Establish data security and compliance protocols, including privacy protection measures
• Build resilient data systems with error recovery, backup, and integrity verification

What You'll Need

• Experience designing data pipelines for AI/ML applications
• Expertise with Apache Airflow for workflow orchestration
• Strong knowledge of Apache Spark for large-scale data processing
• Experience with Apache Kafka for real-time event streaming
• Proficiency with object storage systems (S3/MinIO) and database technologies (Cassandra/ScyllaDB, ClickHouse)
• Understanding of monitoring tools (OpenTelemetry) and observability platforms
• Experience implementing data security and compliance measures
• Advanced Python programming skills

Preferred Experience

• Audio data processing and conversational AI systems
• LLM training and fine-tuning pipelines
• Data quality frameworks (Great Expectations) and versioning tools (LakeFS, DVC)
• Kubernetes for container orchestration
• Multi-region deployment and distributed systems

• Build cutting-edge conversational AI systems with real-world impact
• Work with modern, open-source technology stack
• Help shape the future of automated customer service
• Competitive compensation and flexible work arrangements

If you're passionate about building robust data systems for AI applications and excited by complex real-time data challenges, we'd love to talk.