Cloud Systems Engineer managing cloud infrastructure projects for federal clients. Supporting stability and reliability of cloud - based systems and networks with a focus on innovative solutions.
About the role
- Senior Cloud Systems Engineer providing Databricks administration for government/financial services. Responsible for platform management, security, and automation in a hybrid setup.
Responsibilities
- Administer Databricks accounts and workspaces across SDLC environments.
- Standardize configuration, naming conventions, and operational practices.
- Configure and maintain clusters, compute policies, SQL warehouses, runtime versions, libraries, jobs, repositories, and workspace settings.
- Monitor platform health through operational dashboards, alerts, and monitoring tools.
- Maintain operational documentation, runbooks, and platform procedures.
- Implement and enforce least-privilege access controls across platform resources.
- Manage identity integrations including SSO, SCIM provisioning, and role-based access control.
- Administer service principals and group-based access permissions.
- Enable audit logging and support security monitoring and compliance reviews.
- Implement secure secrets management and connectivity patterns.
- Administer Unity Catalog including metastores, catalogs, schemas, and tables.
- Manage data ownership, permission grants, and governance policies.
- Configure and maintain external locations and storage credentials.
- Support data classification, tagging, and lineage integrations with governance teams.
- Coordinate with cloud and network teams to establish secure connectivity patterns.
- Implement storage access controls and secure object storage integrations.
- Support cloud logging, monitoring, and security integration with enterprise platforms.
- Automate platform configuration and administration using APIs, CLI tools, and Infrastructure-as-Code frameworks.
- Implement CI/CD pipelines for deploying jobs, notebooks, and configurations across environments.
- Implement Databricks Asset Bundles (DABs) for standardized deployment workflows.
- Reduce configuration drift through automated deployment processes.
- Implement cost control policies such as cluster policies and auto-termination rules.
- Analyze usage metrics and provide recommendations to improve cost efficiency.
- Monitor and optimize SQL warehouse performance and cluster autoscaling.
- Implement Delta Lake optimization strategies including OPTIMIZE, VACUUM, and Z-ordering.
- Administer Delta Live Tables pipelines and support data engineering teams.
- Monitor pipeline health and address job failures or performance issues.
- Support integrations with business intelligence tools and metadata catalog systems.
- Assist with troubleshooting data pipeline and query performance issues.
- Maintain platform configuration documentation and governance standards.
- Develop onboarding materials and self-service guides for platform user.
- Support user onboarding and workspace access provisioning.
- Provide guidance to platform users and development teams on best practices.
- Conduct capacity planning and forecast resource usage based on platform growth.
- Monitor concurrent workloads and resource allocation.
- Recommend scaling strategies to support increased platform usage.
- Ensure platform stability during peak usage periods.
Requirements
- Bachelor’s Degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience.
- 7+ years of experience in cloud infrastructure, data platform administration, or enterprise platform operations.
- 3+ years of hands-on experience administering Databricks environments.
- Hands-on experience managing Databricks workspaces, clusters, compute policies, SQL warehouses, runtime versions, jobs, and repositories.
- Experience administering Unity Catalog including metastores, catalogs, schemas, permissions, service principals, and storage access.
- Strong knowledge of identity and access management including SSO, SCIM provisioning, and role-based access control.
- Experience implementing platform security including secrets management, audit logging, and secure connectivity.
- Experience with automation tools such as Terraform, REST APIs, or CLI-based configuration management.
- Experience implementing CI/CD pipelines for notebooks, jobs, and configuration promotion across environments.
- Understanding of lakehouse architecture concepts including Delta Lake and compute-storage separation.
Benefits
- Flexible work arrangements
- Professional development
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior Systems Engineer supporting DIAs National Digital Exploitation and OSINT Center for a tech company serving public sector clients. Manage systems engineering tasks and lead a scrum software development team.
Principal Engineer defining and architecting distributed AI systems across heterogeneous compute platforms at Intel. Focusing on dynamic execution and optimization of large - scale AI computation graphs.
Lead Supply Management for Micron's semiconductor materials and delivery in Penang, Malaysia. Collaborate cross - functionally for global supply fulfillment and manage supplier relationships effectively.
System Engineer for network technology in telecommunications company offering tailored internet services. Responsible for firewall administration, network components, and strategic project management.
Principal Platform Systems Engineer leading test automation for maritime AI solutions. Focused on building infrastructure for cloud, edge compute, and embedded systems.
System Engineer focusing on Citrix and Azure Virtual Desktop at DATAGROUP in Kaunas. Develop technical solutions, implement them independently, and ensure smooth operations.
Principal Software Systems Engineer supporting Northrop Grumman's Sentinel program based in Colorado Springs or Huntsville. Involves collaboration in software development and best practices.
Systems Engineer specializing in automated warehouse systems for key warehouse automation projects at Dematic. Overseeing technical integration and collaborating with engineering teams to deliver innovative solutions.
Senior Business Systems Analyst focusing on Oracle Revenue Management Cloud Service. Ensure reliability and scalability of critical systems while collaborating with technical teams.