Senior hpc administrator, secure hpc platform - high performance computational and data ecosystem - technology specialist iii - hybrid/onsite

The Scientific Computing and Data group at the Icahn School of Medicine at Mount Sinai partners with scientists to accelerate scientific discovery. To achieve these aims, we support a cutting-edge high-performance computing and data ecosystem along with MD/PhD-level support for researchers. The group is composed of a high-performance computing team, a clinical data warehouse team and a data services team.

RESPONSIBILITIES

The Senior HPC Administrator, High Performance Computational and Data Ecosystem is responsible for the operation, security, and governance of a secure HPC environment supporting multiple regulated research and clinical workloads. These include CMS (Centers for Medicare & Medicaid Services) controlled data subject to NIST SP 800‑53 (Data Management Self-Attestation Questionnaire - DMP SAQ) and NIH‑controlled genomic datasets subject to NIST SP 800‑171, and equivalent regulatory frameworks. This role ensures continuous compliance with federal, institutional, and sponsor cybersecurity and data governance requirements while delivering high‑performance, reliable, and auditable research computing services.

To support Sinai’s scientific and clinical mission, the Senior HPC Administrator brings deep technical expertise in computational systems, data platforms, and research software environments, with a strong emphasis on cybersecurity, compliance, and researcher‑focused service delivery. The role requires independent leadership, advanced troubleshooting skills, and the ability to drive complex initiatives to completion with minimal supervision.

Scientific Computing operates a large‑scale computational and data ecosystem, including a 30,000+ core HPC cluster and 50+ PB of usable storage, as well as a clinical data warehouse and secure software development environments. The HPC system is ranked among the fastest academic biomedical computing platforms globally (Top500).

This position reports to the Senior Director for Computational & Data Ecosystem in Scientific Computing. Specific responsibilities are listed below.

HPC Administration, and HPC Cybersecurity

1. Provide full lifecycle administration of secure HPC infrastructure, including compute, storage, networking, and supporting services. Manage system configuration, authentication and authorization, monitoring, logging, performance tuning and analytics, usage reporting, and capacity planning.
2. Administer and support core HPC services such as job scheduling platforms (e.g., LSF), Open OnDemand user access environments, and PostgreSQL databases.
3. Design, maintain, and enforce secure network architectures, including segmentation, firewall rules, and controlled ingress/egress.
4. Lead the troubleshooting, isolation and resolution of all technical issues including system, hardware, software, and network.
5. Implement and maintain NIST‑aligned security controls, including encryption of data at rest and in transit, secure enclaves, access control, centralized logging, audit trails, and monitoring. Lead system security hardening, patching, vulnerability management, and technical risk remediation activities.
6. Ensure continuous compliance with CMS DMP SAQ requirements, NIST SP 800‑171/800‑53 controls, NIH data governance policies, and Mount Sinai cybersecurity standards.
7. Develop, maintain, and regularly update security policies, operational procedures, and technical documentation supporting regulatory compliance.
8. Serve as the primary technical and process liaison with DTP Cybersecurity, GCO, and compliance stakeholders. Coordinate incident response activities and security investigations in partnership with DTP Cybersecurity.
9. Organize and manage annual external audits and recertification efforts, ensuring timely remediation and sustained audit readiness.
10. Stay current with emerging HPC and cybersecurity technologies to keep the organization’s infrastructure up-to-date.
11. Work effectively and productively with other team members within the group and across Mount Sinai.
12. Support researchers and users with computational, data, and analytics needs, including data transfers, software installation, batch jobs and troubleshooting etc. Answers and resolves user tickets. Create and update user-facing technical documentation for the CMS cluster.
13. Provide after-hours support as needed.
14. Perform other duties as assigned or requested.

QUALIFICATIONS

• Bachelors degree in a technical discipline in computer science, engineering or related scientific field. Master's or PhD preferred
• 8 years minimum experience (higher preferred), 12 years of related experience, including a demonstrated ability in technology area. In-depth knowledge of associated technology areas that could impact area of responsibility; healthcare technology experience preferred.

Specific experience for this position:

• 8+ years of progressive HPC systems administration experience in large‑scale Red Hat/CentOS Linux batch environments.
• Hands-on experience with HPC schedulers, parallel file systems, configuration management (e.g., xCAT, Puppet, Ansible), databases, web services, globus and high‑performance networking (InfiniBand, Ethernet).
• Must be an expert troubleshooter; Must be a team player and customer focused
• Strong background in cybersecurity practices within regulated environments (e.g., healthcare, government), including firewalls, data protection, access controls, monitoring, and incident response.
• In‑depth knowledge of NIST SP 800‑171 and NIST SP 800‑53, with hands‑on experience implementing, and operating compliant HPC systems.
• Script and programming experience
• Ability to manage multiple priorities effectively in a dynamic environment
• Excellent analytical ability, strong judgment and management skills, and the ability to work effectively as a liaison between both research and technology teams.
• Excellent written, verbal, and interpersonal communication skills.

Preferred Experience

• Experience with IBM LSF and GPFS Spectrum Scale.
• Experience in an academic or research environment.
• Experience with confidential computing and privacy-preserving linking of PHI.
• Experience with public cloud platforms (Azure, AWS, or GCP).

Compensation Statement

The Mount Sinai Health System (MSHS) provides salary ranges that comply with the New York City Law on Salary Transparency in Job Advertisements. The salary range for the role is $113,011.58 - $202,155.00 Annually. Actual salaries depend on a variety of factors, including experience, education, and operational need. The salary range or contractual rate listed does not include bonuses/incentive, differential pay or other forms of compensation or benefits.

Non-Bargaining Unit, 297 - DTP Infrastructure Planning - MSH, Mount Sinai Hospital

Strength through Unity and Inclusion

The Mount Sinai Health System is committed to fostering an environment where everyone can contribute to excellence. We share a common dedication to delivering outstanding patient care. When you join us, you become part of Mount Sinai’s unparalleled legacy of achievement, education, and innovation as we work together to transform healthcare. We encourage all team members to actively participate in creating a culture that ensures fair access to opportunities, promotes inclusive practices, and supports the success of every individual.

At Mount Sinai, our leaders are committed to fostering a workplace where all employees feel valued, respected, and empowered to grow. We strive to create an environment where collaboration, fairness, and continuous learning drive positive change, improving the well-being of our staff, patients, and organization. Our leaders are expected to challenge outdated practices, promote a culture of respect, and work toward meaningful improvements that enhance patient care and workplace experiences. We are dedicated to building a supportive and welcoming environment where everyone has the opportunity to thrive and advance professionally. Explore this opportunity and be part of the next chapter in our history.

About the Mount Sinai Health System:

Mount Sinai Health System is one of the largest academic medical systems in the New York metro area, with more than 48,000 employees working across eight hospitals, more than 400 outpatient practices, more than 300 labs, a school of nursing, and a leading school of medicine and graduate education. Mount Sinai advances health for all people, everywhere, by taking on the most complex health care challenges of our time — discovering and applying new scientific learning and knowledge; developing safer, more effective treatments; educating the next generation of medical leaders and innovators; and supporting local communities by delivering high-quality care to all who need it. Through the integration of its hospitals, labs, and schools, Mount Sinai offers comprehensive health care solutions from birth through geriatrics, leveraging innovative approaches such as artificial intelligence and informatics while keeping patients’ medical and emotional needs at the center of all treatment. The Health System includes more than 9,000 primary and specialty care physicians; 13 joint-venture outpatient surgery centers throughout the five boroughs of New York City, Westchester, Long Island, and Florida; and more than 30 affiliated community health centers. We are consistently ranked by U.S. News & World Report's Best Hospitals, receiving high "Honor Roll" status, and are highly ranked: No. 1 in Geriatrics, top 5 in Cardiology/Heart Surgery, and top 20 in Diabetes/Endocrinology, Gastroenterology/GI Surgery, Neurology/Neurosurgery, Orthopedics, Pulmonology/Lung Surgery, Rehabilitation, and Urology. New York Eye and Ear Infirmary of Mount Sinai is ranked No. 12 in Ophthalmology. U.S. News & World Report’s “Best Children’s Hospitals” ranks Mount Sinai Kravis Children's Hospital among the country’s best in several pediatric specialties. The Icahn School of Medicine at Mount Sinai is ranked No. 11 nationwide in National Institutes of Health funding and in the 99th percentile in research dollars per investigator according to the Association of American Medical Colleges. Newsweek’s “The World’s Best Smart Hospitals” ranks The Mount Sinai Hospital as No. 1 in New York and in the top five globally, and Mount Sinai Morningside in the top 20 globally.

Equal Opportunity Employer

The Mount Sinai Health System is an equal opportunity employer, complying with all applicable federal civil rights laws. We do not discriminate, exclude, or treat individuals differently based on race, color, national origin, age, religion, disability, sex, sexual orientation, gender, veteran status, or any other characteristic protected by law. We are deeply committed to fostering an environment where all faculty, staff, students, trainees, patients, visitors, and the communities we serve feel respected and supported. Our goal is to create a healthcare and learning institution that actively works to remove barriers, address challenges, and promote fairness in all aspects of our organization.