Hybrid Principal Splunk Engineer

at EEOC

• Job title: Principal Splunk Engineer
• Job type: Full Time
• Experience level: Lead
• Salary: $184,000 - $230,000 per year
• Degree requirement: Bachelor's Degree
• Tech skills: Cloud, Linux, Splunk
• Location requirements: Hybrid, Scottsdale, United States

About the role

• Principal Splunk Engineer managing and optimizing logging architecture for Early Warning's services. Leading decisions in technology direction and mentoring the engineering team.

Responsibilities

• Leads the Splunk Engineering team in technical and risk based decision making, focusing on data quality and business needs
• Architects Splunk Enterprise infrastructure and leads tuning Splunk for optimal onboarding of data, performance, and capacity management; identifies gaps and areas of duplication; provides recommendations for optimization
• Leads operating and maintaining efforts for a complex multi-site hybrid environment and ensures the infrastructure remains available and scalable.
• Significantly influences the overall technology direction of Early Warning from the perspective of log management
• Works collaboratively with business customers to intake and define new logging architectures and solutions for various business units at Early Warning
• Leads and approves changes for logging infrastructure from a security perspective
• Ensures our logging environment provides for effective threat detection and response in direct partnership with information security teams
• Participates in incident management & incident response during an outage or security investigation when needed
• Creates documentation for any addition or change to our environment. Reviews and updates on a regular basis to ensure accuracy.
• Ensures the tools are supporting all compliance efforts in collaboration with auditors
• Provides metrics for platform performance, capacity, and user management
• Leads root cause analysis efforts pertaining to log engineering issues
• On call rotation with other Splunk engineers to cover 24x7 response
• Sets and supports best practices for end users and company standards. Stays current on the latest industry technologies, trends, and strategies
• Support the company’s commitment to protect the integrity and confidentiality of systems and data
• Advises as the subject matter expert and contributes to the development of Early Warning security policy and procedures
• Mentors new team members

Requirements

• Education and experience typically obtained through completion of a Bachelor's degree in Computer Science, Engineering, Math or Physical Science
• Typically has 15 years of progressive Splunk administration, Splunk architect and/or logging experience in a multisite environment is necessary
• Certification: Splunk Certified Administrator required
• Subject matter expert: Splunk Enterprise and Cloud
• Splunk SPL query language
• Common ingestion strategies such as UFs, HFs, HEC, TAs/Add-ons and syslog
• Demonstrated proficiency with the full Splunk lifecycle, including all major components for enterprise deployments.
• Must have solid foundation in Linux and possess a competence to troubleshoot various aspects of the integration including operating system, application, and networking components as they relate to both Splunk and syslog implementations
• Experience with supporting technologies such as Cribl
• Ability to handle large projects as well as take care of day-to-day operations
• Strong sense of responsibility, ownership, and pride in delivering quality results. In addition, troubleshooting and organizational skills with a can-do attitude and the ability to adjust to changing requirements are essential.
• Demonstrated ability to communicate across all levels of the organization is necessary; must be able to clearly articulate technical ideas to a non-technical audience both verbally and in writing.

Benefits

• Healthcare Coverage – Competitive medical (PPO/HDHP), dental, and vision plans as well as company contributions to your Health Savings Account (HSA) or pre-tax savings through flexible spending accounts (FSA) for commuting, health & dependent care expenses.
• 401(k) Retirement Plan – Featuring a 100% Company Safe Harbor Match on your first 6% deferral immediately upon eligibility.
• Paid Time Off – Flexible Time Off for Exempt (salaried) employees, as well as generous PTO for Non-Exempt (hourly) employees, plus 11 paid company holidays and a paid volunteer day.
• 12 weeks of Paid Parental Leave
• Maven Family Planning – provides support through your Parenting journey including egg freezing, fertility, adoption, surrogacy, pregnancy, postpartum, early pediatrics, and returning to work. And SO much more!