Data Engineering AI Architect

Responsibilities

Key Responsibilities

Data Architecture for AI

• Architect AI data foundations including ingestion, transformation, enrichment, and serving layers
• Design data architectures supporting RAG, embeddings, feature stores, and training data pipelines
• Define standards for data quality, lineage, versioning, and governance for AI workloads
• Ensure data platforms support scalability, performance, and low latency AI use cases

Data Quality & Assurance

• Architect data validation and testing frameworks for AI and analytics systems
• Enable automated validation for data correctness, drift, bias, and completeness
• Define test strategies for data migration, data transformation, and AI readiness
• Collaborate with QE teams to embed data assurance into pipelines and platforms

Platform & Integration

• Integrate data platforms with AI services and analytics tools
• Define secure access patterns for data used in training, inference, and evaluation
• Enable observability for data pipelines and AI data consumption
• Guide teams on best practices for AI enabled BI and data driven systems

Core Platforms, Frameworks & Tooling

• LLM and foundation model platforms (e.g., AWS Bedrock, Azure OpenAI, Vertex AI)
• Agentic AI and orchestration frameworks (LangChain, LangGraph, CrewAI, AutoGen, Google ADK or equivalent)
• CI/CD and MLOps tooling for AI pipelines (GitHub Actions, Azure DevOps, Jenkins)
• Data ingestion and processing platforms (Spark, Kafka, cloud native ETL/ELT frameworks)
• Data quality and validation frameworks (Great Expectations, Amazon Deequ, custom reconciliation frameworks)
• Feature stores and embedding pipelines (Feast, embedding generation pipelines, vector databases)
• Data drift, bias, and consistency monitoring tools (Evidently, statistical data quality monitors)
• Metadata, lineage, and governance platforms (DataHub, Apache Atlas, cloud data catalogs)
• AI enabled analytics and Generative BI platforms (Power BI with Copilot, semantic layers, NLQ enabled BI)
• Cloud native data platforms and storage (object storage, distributed query engines, data lakehouses)

Client Orientation & Leadership

• Partner with product and engineering teams to identify Data for AI opportunities and shape roadmaps
• Support client workshops, RFPs, and solution presentations
• Mentor engineers on AI/ML/Gen AI best practices and emerging technologies
• Translate complex AI concepts into business-friendly narratives

Technical requirements

Must Have Qualifications

• 13+ years of experience in software engineering with 3+ years in AI with strong architecture ownership
• Strong expertise in data engineering, data quality, and data governance
• Experience supporting AI use cases such as RAG, feature engineering, and model training
• Proficiency with data platforms, cloud services, and distributed data systems
• Solid understanding of QE practices related to data validation and testing

Good to Have Skills

• Experience with Generative BI or AI assisted analytics
• Knowledge of metadata management, lineage tools, and data observability
• Exposure to AI ethics and bias in data sets
• Cloud data certifications

Preferred skills

• Technology › AI-AI Engineering › AI/ML Solution Architecture and Design
• Technology › AI-AI Engineering › Databricks AI Engineering Services
• Technology › AI-AI Engineering › LLMOps
• Technology › AI-Data science › Databricks Machine Learning
• Technology › Architecture › Architecture - ALL
• Technology › Data Engineering › Databricks
• Technology › Data Management › Data Architecture › Data Architecture - Metadata Management
• Technology › Enterprise Architecture › Digital Architecture
• Technology › ETL & Data Quality › IBM Infosphere Datastage › IBM Infosphere Datastage - Datastage

About the role

• Role: Senior Technology Architect
• Experience: 8 - 15 years
• Education: Bachelor of Engineering