Welcome to 3rd CORSIA 8 focus week
Summary of Monday discussions:
General motto and philosophy:
Physics first. Code clean. Optimize and profile next. Watch out for potential "dead end", but go ahead. We cannot avoid to eventually go back and perform major code revisions for further optimizations. Data structures are at least as important as code!
General impression:
We are going in the right direction. A lot of progress. CORSIKA 8 prototype can already do many things.
- Brief review of status, MR, issues, milestones
- Have code dependencies in source tree
- complicated/binary dependencies in containers
- E.g. for PYTHIA, have helper script to remove all bloat -> create minimal package
-> Need maintenance project with tools. I suggest in corsika-docker
- check "git large file system", but license issues. Alternative: git binaries
- discuss paper plans and general strategy for the next time on Friday
- list of conferences, who is going where?
Anatoli presented a talk about parallel running and scheduling
EAS -> branching tree.
CPU time is almost linear function of sub-shower energy
Need master to distribute work, and organize
-> scheduling. sorting according to Wall time -> dispatch work tasks
-> need also sequential mode (at initialization)
-> advantages: load balancing, monitoring, log on scratch space transfer only if needed
About 3 parameters needed for scheduler to optimize. Can be easily optimized on any computing cluster.
- data serialization is critical. Large number of workers -> may need "data concentrator" nodes, or/and concurrent writes to files
- MPI is important on large machines, but annoying on personal computers and small clusters. Need with and without option.
- OpenMP corresponds to "thread model"
- Data transfer between nodes is very minimal
--> Define abstraction to allow different backend implementations