Contact Us

Hybrid Principal Data Engineer – AI Platform

at TASC
Visit TASC websiteVisit TASC LinkedIn

Posted 5 hours ago

Apply now

About the role

  • Drive modernization from legacy and on-prem systems to modern, cloud-native, and hybrid data platforms.
  • Architect and lead the development of a Multi-Agent ETL Platform for batch and event streaming, integrating AI agents to autonomously manage ETL tasks such as data discovery, schema mapping, and error resolution.
  • Define and implement data ingestion, transformation, and delivery pipelines using scalable frameworks (e.g., Apache Airflow, Nifi, dbt, Spark, Kafka, or Dagster).
  • Leverage LLMs, and agent frameworks (e.g., LangChain, CrewAI, AutoGen) to automate pipeline management and monitoring.
  • Ensure robust data governance, cataloging, versioning, and lineage tracking across the ETL platform.
  • Define project roadmaps, KPIs, and performance metrics for platform efficiency and data reliability.
  • Establish and enforce best practices in data quality, CI/CD for data pipelines, and observability.
  • Collaborate closely with cross-functional teams (Data Science, Analytics, and Application Development) to understand requirements and deliver efficient data ingestion and processing workflows.
  • Establish and enforce best practices, automation standards, and monitoring frameworks to ensure the platform’s reliability, scalability, and security.
  • Build relationships and communicate effectively with internal and external stakeholders, including senior executives, to influence data-driven strategies and decisions.
  • Continuously engage and improve teams’ performance by conducting recurring meetings, knowing your people, managing career development, and understanding who is at risk.
  • Oversee deployment, monitoring, and scaling of ETL and agent workloads across multi cloud environments.
  • Continuously improve platform performance, cost efficiency, and automation maturity.

Requirements

  • Hands-on experience in data engineering, data platform strategy, or a related technical domain.
  • Proven experience leading global data engineering or platform engineering teams.
  • Proven experience in building and modernizing distributed data platforms using technologies such as Apache Spark, Kafka, Flink, NiFi, and Cloudera/Hadoop.
  • Strong experience with one or more of data pipeline tools (Nifi, Airflow, dbt, Spark, Kafka, Dagster, etc.) and distributed data processing at scale.
  • Experience building and managing AI-augmented or agent-driven systems will be a plus.
  • Proficiency in Python, SQL, and data ecosystems (Oracle, AWS Glue, Azure Data Factory, BigQuery, Snowflake, etc.).
  • Deep understanding of data modeling, metadata management, and data governance principles.
  • Proven success in leading technical teams and managing complex, cross-functional projects.
  • Passion for staying current in a fast-paced field with proven ability to lead innovation in a scaled organization.
  • Excellent communication skills, with the ability to tailor technical concepts to executive, operational, and technical audiences.
  • Expertise and ability to lead technical decision-making considering scalability, cost efficiency, stakeholder priorities, and time to market.
  • Proven track leading high-performing teams with experience leading and coaching director level reports and experienced individual contributors.
  • Advanced degree in Data Science, Computer Science, Information Technology, Business Administration, or a related field. Equivalent experience will also be considered.

Benefits

  • insurance (including medical, prescription drug, dental, vision, disability, life insurance)
  • flexible spending account and health savings account
  • paid leaves (including 16 weeks of new parent leave and up to 20 days of bereavement leave)
  • 80 hours of Paid Sick and Safe Time, 25 days of vacation time and 5 personal days, pro-rated based on date of hire
  • 10 annual paid U.S. observed holidays
  • 401k with a best-in-class company match
  • deferred compensation for eligible roles
  • fitness reimbursement or on-site fitness facilities
  • eligibility for tuition reimbursement

Job title

Principal Data Engineer – AI Platform

Job type

Full Time

Experience level

Lead

Salary

$170,000 - $284,000 per year

Degree requirement

Postgraduate Degree

Tech skills

AirflowApacheAWSAzureBigQueryCloudETLHadoopKafkaOraclePythonSparkSQL

Location requirements

HybridArlingtonUnited States

Report this job

See something inaccurate? Let us know and we'll update the listing.

Report job

Similar roles

Browse all Data Engineer jobs
FindasenseSenior Data Engineer – Intelligence

Data Engineer Senior responsible for building data architecture and optimizing pipelines for Business Intelligence. Collaborating with analysts to develop insights using Power BI and Azure technologies.

Hybrid Role
BogotáColombiaData Engineer
TASCPrincipal Data Engineer – AI Platform

Principal Data Engineer driving modernization from legacy systems to cloud - native platforms at Mastercard. Architecting and developing ETL platforms with AI integration and establishing data - driven strategies.

Onsite Role
VancouverCanadaData Engineer
CA$125,000 - CA$206,000 per year
Sabiá AdministraçãoSenior Data Engineer

Data Engineer designing, implementing, and maintaining data pipelines at Sabiá Gaming. Focused on high - quality data access and integration for enhanced decision - making.

Hybrid Role
PompéuBrasilData Engineer
MassMutualQuantitative Data Engineer

Quantitative Data Engineer developing data solutions and automations for MassMutual's investment management. Working with data orchestration tools within a collaborative team environment.

Hybrid Role
SpringfieldUnited StatesData Engineer
$104,900 - $137,700 per year
NinjaTraderSenior Data Engineer

Data Engineer developing architecture and pipelines for data analytics at NinjaTrader. Empowering analysts and improving business workflows through data - driven solutions.

Hybrid Role
ChicagoUnited StatesData Engineer
$140,000 - $160,000 per year
SpecsaversData Engineer

Data Engineer designing and developing data solutions at Specsavers. Collaborating on data pipelines and migrations with analysts and stakeholders.

Onsite Role
FarehamUnited KingdomData Engineer
AlterricData Engineer

Data Engineer joining Alterric to collaborate on data platform projects and analytics solutions. Working with Azure Cloud technologies to ensure data quality and integrity for informed decision - making.

Hybrid Role
GermanyData Engineer
KyndrylELK Data Engineer

Data Engineer at Kyndryl transforming raw data into actionable insights using ELK Stack. Responsible for developing, implementing, and maintaining data pipelines and processing workflows.

Hybrid Role
BangaloreIndiaData Engineer