Sep 9 – 12, 2024
SCC
Europe/Berlin timezone

Staging ~2 Million Files from Tape for a User

Sep 11, 2024, 11:30 AM
20m
Aula (SCC)

Aula

SCC

Hermann von Helmholtz Platz 1 76344 Eggenstein-Leopoldshafen
public to everybody attending HUF

Speaker

Mr Geoff Cleary (LLNL)

Description

Imagine you turn on your work laptop or arrive at the office and find this message from the customer service team: "we have a user that is trying to retrieve many files from the HPSS archive. At the current retrieval rate, we estimate it will take 6 months for the user to retrieve all the files in the dataset. Can you help?"

What do you do? How do you proceed? What features does HPSS offer to help with this situation?

I'll answer those questions and more as we examine LLNL's approach to retrieving nearly 2 million files across 10's of tape volumes with tools like quaid, SQLite, and RabbitMQ (along with a bit of custom Python code).

Timeslot 15 min

Primary author

Mr Geoff Cleary (LLNL)

Presentation materials

There are no materials yet.