Updates on HPSS at IN2P3 Computing Center
- Infrastructure and HPSS upgrade
- ARM architecture support
Last year at NERSC we retired our long-standing nagios-based HPSS monitoring deployment in favor of VictoriaMetrics, Loki and Alertmanager. We would like to share our experience and lessons learned on the way.
- Motivation for making this transition
- Limitations of Nagios-style monitoring
- How does VictoriaMetrics address these?
- General overview of our monitoring...
As part of our efforts to upgrade our site to HPSS 10.3, Indiana University recently began development of a virtualized HPSS environment that we can use to quickly iterate on testing and development initiatives without tying up limited bare-metal hardware resources. This virtual-machine environment is patterned on the implementation created by IBM's HPSS Support team for use at the recent...
Starting from version 10.3, HPSS has an S3 interface. We at MPCDF have installed it on our test system to try it out in several usage scenarios including cloud sync - using Ceph Cloud Sync module as well as rclone, generating presigned URLs, and just using different S3 clients. Among our test actions, we are trying out put, get, remove S3 objects as well as getting S3 objects from tape. This...
NERSC Site Report HUF24
Karlsruhe Institute of Technology Campus Nord from 09-12 September 2024
Abstract
Topics to discuss
- NERSC Stats - PBs, etc
- Upgrade HPSS 7.4.3 to 9.3
New RHEL8 Core Servers.
New FS7300 metadata arrays.
Update existing movers to RHEL8.
Updated PAM auth module to work with NERSC Auth. - Install 4th TS4500...
2024 HUF Presentations by IBM
!!duration 1h
2024 HUF Presentations by IBM
!! duration: 1,5h
Imagine you turn on your work laptop or arrive at the office and find this message from the customer service team: "we have a user that is trying to retrieve many files from the HPSS archive. At the current retrieval rate, we estimate it will take 6 months for the user to retrieve all the files in the dataset. Can you help?"
What do you do? How do you proceed? What features does HPSS...
Abstract: Data lifecycle management poses significant challenges, particularly in academic and research environments where data accumulation is rapid and perpetual. This presentation delves into the complexities surrounding data retention and abandonment, highlighting the prevalent issues of data hoarding and the lack of structured deletion policies. Specifically, it addresses the dilemma...
At LLNL we have been using commodity hardware more and more to serve our parallel filesystems and archival storage clusters. We wanted to explore how to use this same hardware for our HPSS Core Server systems. In order to make the system as reliable as possible, ZFS emerged as the underlying filesystem we wanted to utilize for its reliability and other advanced features. How would traditional...
2024 HUF Presentations by IBM
!! duration 1h
We will present our activities with HPSS since the last HUF, including our upgrade to HPSS 10.3.
2024 HUF Presentations by IBM
!!duration 1h
SSC Site Report will be a recap of previous HUF presentations and focus on :
- Solution overview
- Review of components throughout upgrades
- HPNLS (High Performance Nearline Storage) architecture
- HPSS and RHEL Upgrade
- HPSS monitoring
- User tools and environment
The recent operation status of JAXA HPSS "J-SPACE", the plans and issues for its replacement in 2025, and monitoring functionality will be reported.
2024 HUF Presentations by IBM
!!duration 1h
