Job Overview
Design and implement enterprise data architecture solutions to ensure seamless data flow across systems. Focus on modern architectures, including AI/ML data pipelines, real-time streaming, and infrastructure for autonomous agent systems. Align data assets with business goals, security, and optimization for analytics and AI workloads.
Responsibilities
- Design and implement secure, scalable data architectures on cloud platforms using data lake, warehouse, and Lakehouse patterns.
- Model datasets for financial reporting, analytics, and AI/ML workloads.
- Implement batch and real-time streaming data pipelines.
- Build infrastructure supporting AI/ML model training, inference, and autonomous agent data access.
- Gain experience with vector databases and embedding storage for AI applications.
- Support data mesh and data product initiatives for self-service data consumption.
- Implement knowledge graph concepts and semantic layers for AI reasoning.
- Create data integration processes for enterprise systems.
- Work with analysts, data scientists, and stakeholders to translate needs into data models.
- Implement data governance, quality, cataloging, lineage tracking, and metadata management.
- Apply responsible AI practices including bias detection and explainability.
- Optimize architectures for performance, cost, and operational excellence.
- Collaborate with IT security and data teams on best practices.
- Communicate project status and requirements with stakeholders.
- Stay updated on cloud data services, AI/ML requirements, and emerging technologies.
- Research infrastructure for Agentic Architecture and evaluate vector databases, knowledge graphs.
- Foster innovative team culture and contribute to data governance.
Qualifications
- Demonstrated experience with data modeling techniques and warehousing principles, including schema design.
- Experience with cloud data platforms such as Azure Data Lake, Synapse, Databricks, Snowflake, AWS Redshift, BigQuery.
- Proficiency in SQL and programming languages like Python, Scala, or Spark for data manipulation and transformation.
- Experience with real-time data streaming platforms such as Kafka, Kinesis, Event Hubs, Pub/Sub.
- Knowledge of data integration patterns.
