Hybrid Data Platform Engineer

at Nexus Cognitive

• Job title: Data Platform Engineer
• Job type: Full Time
• Experience level: Mid level, Senior
• Salary: Not specified
• Degree requirement: Bachelor's Degree
• Tech skills: Airflow, AWS, Azure, Cloud, Google Cloud Platform, Hadoop, Kafka, Linux, Open Source, Spark
• Location requirements: Hybrid, Atlanta, United States

About the role

• Hands-on Data Platform Engineer managing NexusOne products using cloud and on-premise technologies. Collaborate with data engineers and architects to design scalable data solutions.

Responsibilities

• Manage daily operations of NexusOne products on on-premise and cloud based technologies using open-source Linux-based data technologies.
• Install, configure, and support CDP clusters in both cloud (AWS, Azure, GCP) and on-premises environments, ensuring seamless integration and functionality.
• Maintain, patch, and upgrade existing CDP setups, ensuring minimal downtime and adherence to Cloudera’s best practices.
• Evaluate and optimize complex distributed production deployments, identifying bottlenecks and recommending performance enhancements.
• Configure and manage security using tools like Ranger and Kerberos, ensuring robust data protection and compliance.
• Configure Datadog and develop integrations to monitor and alert operations teams
• Develop and maintain technical documentation, including administration runbooks and knowledge base articles, to support operational excellence and knowledge sharing.
• Work in an agile team and Participate in an on-call rotation, providing expert-level support and troubleshooting for critical issues.
• Lead and mentor junior team members, fostering a culture of continuous learning and improvement. Collaborate with cross-functional teams, including data engineers, solution architects, and DevOps, to design and implement scalable data solutions.
• Stay current with emerging technologies and industry trends, applying this knowledge to improve the CDP environment and drive innovation.

Requirements

• US Citizenship Required
• In-depth understanding of both on-premise and cloud network architectures, ensuring seamless integration and efficient data flow.
• Minimum of 5 years of experience in installing and administering the Cloudera Data Platform (CDP), with a proven track record of managing large-scale deployments.
• Hands-on experience with open-source Linux-based data technologies, including Iceberg, Spark, Nifi, Jupyter Notebooks, Cloudera, Databricks, Kubeflow, MLFlow, and Kafka.
• Proven ability to build and support solutions across major cloud platforms (AWS, Azure, GCP), leveraging cloud-native tools and open source for optimal performance.
• Strong Cloudera expertise, with a deep understanding of Spark and Airflow for orchestrating complex data workflows.
• Experience integrating Azure Active Directory with FreeIPA and other directory services such as LDAP, enhancing security and user management.
• Up-to-date knowledge of the Hadoop Big Data ecosystem, staying current with the latest technologies and best practices.
• Excellent troubleshooting skills, with a thorough understanding of CDP capacity planning, identifying bottlenecks, and optimizing memory utilization, CPU usage, OS performance, storage, and network configurations.
• Strong analytical and problem-solving abilities, capable of diagnosing and resolving complex technical issues efficiently.
• Nice to have: Cloudera Certified Administrator certifications is a plus. Prior experience as a support or development engineer with a focus on Data, Analytics, and AI.

Benefits

• A collaborative team culture built on curiosity and respect
• Challenging work where your contributions clearly matter
• A leadership team that invests in learning and development
• The opportunity to work at the intersection of cloud, data, and AI innovation