GridKa School 2016 - Data Science on Modern Architectures (29 August 2016

GridKa School 2016 - Data Science on Modern Architectures

from Monday, 29 August 2016 (08:00) to Friday, 2 September 2016 (12:00)

Monday, 29 August 2016

12:00

12:00 - 14:00
Room: Foyer
14:00
14:00 - 18:00
Room: Aula
Contributions

14:00 Welcome to KIT - Hannes Hartenstein (KIT)

14:30 Welcome to GridKa School 2016 - Manuel Giffels (KIT)

14:50 Data Science - Past, Present and Future - John Wood (RDA Council Co-Chair)

16:00 ROOT as a Service - Jupyter Notebook - Lorenzo Moneta (CERN)

16:40 Data Challenges in Photon Science - Manuela Kuhn (DESY)

17:20 Data Management in NoSQL Databases - Lena Wiese (Georg-August-Universität Göttingen)
Tuesday, 30 August 2016
09:00 Plenary Talks
Plenary Talks
09:00 - 12:00
Contributions

09:00 Technologies to build hybrid clouds on public-private infrastructures - Jurry De la Mar (T-Systems)

09:40 Docker and Linux Containers - Elvin Sindrilaru (CERN)

10:40 Hitachi Data Systems - Digital Transformation within existing Big Data Environments - Krebs Jürgen (Hitachi)

11:20 Openstack @ CERN - Jose Castro Leon (CERN)
12:00 Lunch
Lunch
12:00 - 13:00
13:00 Docker Tutorial - Elvin Sindrilaru (CERN)
Docker Tutorial
- Elvin Sindrilaru (CERN)
13:00 - 18:00
Room: Room 157 The Docker tutorial will walk you through the basic steps of setting up a Docker environment on your machine. There will be a series of exercises that will detail the various concepts presented during the plenary talk which are critical that you understand for the later part of the tutorial. The final goal of the tutorial is to build and deploy a couple of containers that replicate the usual analysis workflow in High Enery Physics: you will have a container running a XRootD server providing the storage for the data and a different container that runs the ROOT framework where you will do your analysis. The tutorial will discuss into depth the concepts of port forwarding, volumes and resource management in the context of containers with a focus on understanding the advantages of containers over traditional virtual machines. In order to benefit to the maximum of the Docker tutorial part, there are some pre-requisites one needs to take into consideration. First of all, you should be comfortable working with the Linux terminal, installing packages over the command line, using the ssh client to connect to a remote machine and last but not least, editing files using one of the common editors in Linux: emacs, vi, nano etc.
Introduction to OpenStack - Tobias Kurze (KIT) Peter Krauss (KIT)
Introduction to OpenStack
- Tobias Kurze (KIT)
- Peter Krauss (KIT)
13:00 - 18:00
Room: Room 156
Jupyter Notebooks for Science (ROOT as a Service) - Lorenzo Moneta (CERN)
Jupyter Notebooks for Science (ROOT as a Service)
- Lorenzo Moneta (CERN)
13:00 - 18:00
Room: Room 163 The objective of the exercises is to give to show the participants how to be even more productive in they everyday tasks thanks to Jupyter notebooks. During this hands on session the concepts treated during the front lesson are seen in action. At the beginning the opportunities given by the notebooks and the Jupyter web interface are reviewed. Elements of the several keyword shortcuts and web user interface management are tried out. The usage of markdown cells, code cells, the way to manage the output and notebooks' conversion to various formats such as HTML or pdf is illustrated. Once the basics are addressed, we focus on Python and C++ notebooks, starting from selected examples coming from High Energy Physics such as CMS Opendata or Large Hadron Collider machine studies. Libraries like ROOT, Matplotlib and Pandas are used. Particular attention is dedicated to machine learning activities. Finally the interplay of Python and C++ made possible by the ROOT Python bindings, PyROOT, is illustrated by examples.
Puppet I - Dimitri Nilsen (KIT) Pavel Weber (KIT) Sven Sternberger (DESY)
Puppet I
- Dimitri Nilsen (KIT)
- Pavel Weber (KIT)
- Sven Sternberger (DESY)
13:00 - 18:00
Room: Room 155 Puppet is today the leading configuration management system. It allows to represent and to control the complete infrastructure of complex IT environments. In this course we will learn what Puppet is and how to use it. We will start to write simple examples and understand how the Puppet workflow could look like. We will also have a look in the surrounding ecosystem git, foreman and hiera. And will show how continous integration could help us to develop our sites.
Using R to improve data analyses Python workflows - Eileen Kühn (KIT)
Using R to improve data analyses Python workflows
- Eileen Kühn (KIT)
13:00 - 18:00
Room: Aula The choice between R and Python is often considered when selecting the best tool to use for data science workflows. Many people often prefer Python over R, since it offers a lot of flexibility and is commonly used in every-day programming. In contrast, R has a steep learning curve. However it offers a huge amount of specialised statistics libraries that are not available in Python so far. Furthermore it offers leading-edge features for visualisation. Combining R and Python in your workflows allows you to benefit from both. Throughout the tutorial, you will learn how to benefit from both tools. To get you running, we introduce several possibilities how to utilise R from Python. The tutorial itself is hands-on, applying concepts from every-day workflows. The tutorial targets participants who are familiar with using Python for their workflows. Experience with R is not required, though helpful.
18:30

18:30 - 22:00
Wednesday, 31 August 2016
09:00 Plenary Talks
Plenary Talks
09:00 - 12:00
Room: Aula
Contributions

09:00 Building HPC Clusters as Code in the [Almost] Infinite Cloud - Brendan Bouffler (Amazon.com, Inc.)

09:40 Introduction to HTCondor - Andrew Lahiff (Rutherford Appleton Laboratory)

10:40 HEP Software Foundation - Benedikt Hegner (CERN)

11:20 Scientific Computing in Climate Science - Peter Braesicke (KIT)
12:00 Lunch Break
Lunch Break
12:00 - 13:00
13:00 CEPH Tutorial - Diana Gudu (KIT)
CEPH Tutorial
- Diana Gudu (KIT)
13:00 - 18:00
Room: Room 163 Ceph is an open-source, software-defined distributed storage system that strives to achieve scalability and reliability through an innovative decentralised design. Distributed file systems nowadays face multiple challenges: scaling to peta-byte capacity and providing high performance, while protecting against failures. Moreover, file systems should be able to adapt to dynamic distributed workloads to provide the best performance. Ceph tries to tackle these challenges with a completely decentralised architecture that has no single point of failure. Reliability is achieved through distributed data placement and replication. Ceph's dynamic metadata partitioning feature helps deal with dynamic workloads. This session is an introduction to Ceph and it will cover theoretical background on Ceph's architecture, as well as hands-on exercises, such as installation and configuration of a Ceph cluster, basic usage and monitoring, and a dive into Ceph's secret to extreme scalability: the CRUSH algorithm. After completing this session, you should be able to understand and discuss Ceph concepts, and deploy and manage a Ceph Storage Cluster. Basic knowledge of Linux and storage concepts is required.
Concurrent Programming in C++ - Graeme Stewart (University of Glasgow)
Concurrent Programming in C++
- Graeme Stewart (University of Glasgow)
13:00 - 18:00
Room: Room 157 In this course we will introduce how to program for concurrency in C++, taking advantage of modern CPUs ability to run multi-threaded programs on different CPU cores. Firstly, we will explore the new concurrency features of C++14 itself, which will also serve as a general introduction to multi-threaded programming. Students will learn the basics of asynchronous execution, thread spawning, management and synchronisation. Some elementary considerations about deadlocks and data races will be introduced, which will illustrate the common problems that can arise when programming with multiple threads. After this the Threaded Building Block template library will be introduced. We shall see how the features of this library allow programers to exploit multi-threading at a higher level, not needing to worry about so many of the details of thread management. Students should be familiar with C++ and the standard template library. Some familiarity with makefiles and/or CMake would be useful.
Performance, profiling and debugging - Nathalie Rauschmayr (CERN)
Performance, profiling and debugging
- Nathalie Rauschmayr (CERN)
13:00 - 18:00
Room: Room 156 Nowadays, multi- and manycore CPUs provide many different performance dimensions. In order to profit from these, it needs to be understood how an application interacts with the hardware. This is done through profiling, which helps to determine resource requirements and to identify performance bottlenecks. The session will discuss the different perfomance dimensions of modern CPUs, how to perform benchmarking and high-/low-level application profiling in Linux. Commonly used profiling and debugging utilities will be presented. The session requires a basic understanding of Linux and CPU architecture.
Puppet II - Sven Sternberger (DESY) Pavel Weber (KIT) Dimitri Nilsen (KIT)
Puppet II
- Sven Sternberger (DESY)
- Pavel Weber (KIT)
- Dimitri Nilsen (KIT)
13:00 - 18:00
Room: Room 155 Puppet is today the leading configuration management system. It allows to represent and to control the complete infrastructure of complex IT environments. In this course we will learn what Puppet is and how to use it. We will start to write simple examples and understand how the Puppet workflow could look like. We will also have a look in the surrounding ecosystem git, foreman and hiera. And will show how continous integration could help us to develop our sites.
Scientific Computing on Amazon Web Services - Brendan Bouffler (Amazon.com, Inc) Christian Petters (Amazon Web Services GmbH)
Scientific Computing on Amazon Web Services
- Brendan Bouffler (Amazon.com, Inc)
- Christian Petters (Amazon Web Services GmbH)
13:00 - 18:00
Room: Aula A survey will be send around to all participants shortly before the GridKa School to asked for a specific sub-topic to cover. The following topics are available: <ol> <li>Introduction to AWS (Basics for all services)</li> <li>Compute:</li> <ul> <li>Amazon Elastic Compute (EC2)</li> <li>EC2 Container Services</li> <li>Lambda - Run Code in Response to Events</li> </ul> <li>Storage</li> <ul> <li>Scalable Storage in the Cloud (S3)</li> <li>CloudFront</li> <li>Glacier </li> <li>Elastic File System</li> </ul> <li>Database</li> <ul> <li>RDS (Managed Relational Database service)</li> <li>DynamoDB (Managed NoSQL database services)</li> <li>Redshift (Fast Simple Cost-Effiective Data)</li> </ul> <li>Management Tools</li> <ul> <li>CloudWatch (Monitor Resources)</li> <li>CloudFormation (Create and manage Resources)</li> <li>CloudTrail (track user activities)</li> <li>Config (Track resource inventory)</li> <li>OpsWorks (Automatic operations)</li> <li>Service Catalog (Create and use Standardized services)</li> </ul> <li>Security & Identity</li> <ul> <li>Identity and Access Management</li> <li>Certificate Manager</li> </ul> <li>Analytics</li> <ul> <li>Elastic Map Reduce (EMR)</li> <li>Data Pipeline (Data driven workflows)</li> <li>Elastic search searvice</li> <li>Kinesis (Work with streaming data)</li> <li>Machine learning</li> </ul> <li>Internet of Things - Connect Devices to the cloud</li> <li>Mobile Services</li> <ul> <li>Device Farm : Test Android, iOS and Web Apps</li> <li>Mobile analytics (Collect, View and Export App Analytics)</li> <li>SNS Simple notification service</li> </ul> <li>Application services</li> <ul> <li>API Gateway (Build deploy and manage gateways)</li> <li>AppStream (low latency app streaming)</li> <li>SQS (Message Queue Service)</li> <li>SWF (Workflow Service for Coordinating Application components)</li> </ul> </ol>
18:30 Evening Lecture: "Automated and Connected Driving from a User-Centric Viewpoint" - Frank Köster (Deutsches Zentrum für Luft- und Raumfahrt)
Evening Lecture: "Automated and Connected Driving from a User-Centric Viewpoint"
- Frank Köster (Deutsches Zentrum für Luft- und Raumfahrt)
18:30 - 20:00
Room: Aula Automated and connected driving can be seen as a paradigm shift in the automotive domain which will change the today’s role of the human driver substantially. More and more parts of the driving task are to be taken over by technical components for different scenarios. The application areas reach from fully automated parking up to automated driving on a highway with different involvements of the driver in monitoring tasks (SAE level 2 to 5). Thus, it is essential to understand how this role change will influence the human driver and how we can support him by designing an easy to understand, safe and comfortable interaction with the automated vehicle. From Human Factors research we know that it is crucial to ensure that the driver has a correct mental model about the automation levels and its current status to avoid mode confusion and resulting errors in operation. In addition, transitions of control between the driver and the automated vehicle need a careful design that is adjusted to the current traffic situation and the driver status. For example, mobile devices that the driver uses during the automated driving can be integrated smoothly in the overall interaction design concept to support transitions. The talk will give an overview on selected research topics in the area of user-centric development of automated and connected vehicles at DLR. It will underline why it is important to put the human into focus when designing a new, exciting technology such as automated and connected vehicles.
Thursday, 1 September 2016
09:00 Plenary Talks
Plenary Talks
09:00 - 12:00
Room: Aula
Contributions

09:00 Big Data Analytics in Biology: Biomolecular Structure Prediction and Beyond by Tracing Residue Co-Evolution - Alexander Schug (KIT)

09:40 GPU Computing: Platform, Programming, and Pitfalls - Andreas Herten (FZ Jülich)

10:40 The computer that could be smarter than us - Cognitive Computing - Ingolf Wittmann (IBM Deutschland)

11:20 HDF5 as a standard file format for synchrotron radiation experiments - Eugen Wintersberger (DESY)
12:00 Lunch Break
Lunch Break
12:00 - 13:00
13:00 Advanced Python Programming for Data Science - Max Fischer (KIT)
Advanced Python Programming for Data Science
- Max Fischer (KIT)
13:00 - 18:00
Room: Aula In recent years, Python has been adopted in many fields, be it scientific, commercial and other. For most new users, Python is easy to learn and allows to quickly write small but powerful scripts. At the same time, it is also feasible to create web services, distributed processing systems, and other complex applications. Beginners have an easy start with Python: little code is required for many tasks as the language has "batteries included". Yet, data driven science often relies on custom workflows and algorithms. Writing clean, efficient, and comprehensive code is vital; perhaps not for the next deadline, but the one after. Luckily, Python provides many features to even manage complex tasks with ease. The course focuses on advanced concepts of Python programming, useful for data science. This covers features of the language, how to use them, and also best practices. We present this as an interactive hands on tutorial, which gives you a feeling for the capabilities of Python. The course targets people familiar with Python or similar languages. You should feel comfortable writing small scripts for solving problems.
Apache Spark in Scientific Applications - Mirko Kämpf (Cloudera)
Apache Spark in Scientific Applications
- Mirko Kämpf (Cloudera)
13:00 - 18:00
Room: Room 155 The workshop Spark in Scientific Applications covers fundamentale development and data analysis techniques using Apache Hadoop and Apache Spark. Beside an introduction into the theoretical background about Map-Reduce- and Bulk-Synchronous-Parallel processing, also the machine learning library MLlib and the graph processing framework GraphX are used. We work on sample data sets from Wikipedia, financial market data, and from a generic data generator. During the tutorial sessions we illustrate the Data Science Workflow and present the right tools for the right task. All practical exercises are well prepared in a pre-configured virtual machine. Participants get access to required data sets on a „one node pseudo-distributed“ cluster with all tools inside. This VM is also a starting point for further experiments after the workshop.
Application development with relational and non-relational databases - Mario Lassnig (CERN)
Application development with relational and non-relational databases
- Mario Lassnig (CERN)
13:00 - 18:00
Room: Room 163 In this workshop, the students will learn how to use relational and non-relational databases to build multi-threaded applications. The focus of the workshop is to teach efficient, safe, and fault-tolerant principles when dealing with high-volume and high-throughput database scenarios. This includes, but is not limited to, systems such as PostgreSQL, Redis or Elasticsearch. A basic understanding of the following things is required: - A programming language (preferably Python or any C-like) - Basic SQL (CREATE, DROP, SELECT, UPDATE, DELETE) - Linux shell scripting (bash or zsh) The course will cover the following three topics: - When to use relational databases, and when not * Relational primer * Non-relational primer * How to design the data model - Using SQL for fun and profit * Query plans and performance analysis * Transactional safety in multi-threaded environments * How to deal with large amounts of sparse metadata * Competetive locking and selection strategies - Building a fault-tolerant database application * Distributed transactions across relational and non-relational databases * SQL injection and forceful breakage * Application-level mitigation for unexpected database issues
GPU - Felice Pantaleo (CERN)
GPU
- Felice Pantaleo (CERN)
13:00 - 18:00
Room: Room 156 While the computing community is racing to build tools and libraries to ease the use of heterogeneous parallel computing systems, effective and confident use of these systems will always require knowledge about the low-level programming interfaces in these systems. This workshop is designed to introduce the CUDA programming language, through examples and hands-on exercises so as to enable the user to recognize CUDA friendly algorithms and completely exploit the computing potential of a heterogeneous parallel system.
Introduction into Go - Sebastien Binet (Laboratoire de Physique Corpusculaire de Clermont-Ferrand (LPC))
Introduction into Go
- Sebastien Binet (Laboratoire de Physique Corpusculaire de Clermont-Ferrand (LPC))
13:00 - 18:00
Room: Room 164 # Introduction to Go ## Introduction In this workshop, we will introduce the basics of programming in Go and then work our way up to concurrency programming with this relatively new language. We'll start with the usual "Hello World" program, introduce functions, variables, packages and then interfaces. Then, we will tackle the two main tools at the disposal of the Go programmer (colloquially known as a gopher): the channels and the goroutines. This will be done by implementing a small peer to peer application transmitting text messages over the network. The workshop wraps up with a whirlwind tour of scientific and non-scientific libraries readily available, and prospects/news about the next Go version. ## References - https://golang.org - https://tour.golang.org - https://talks.golang.org People will have to install the Go compiler on their laptop. The instructions to do so for their favorite operating system are detailed at: https://golang.org/doc/install To get a taste of what Go looks like and wet their feet, people can also follow the interactive, browser-based, installation-free tour from: https://tour.golang.org
Modern & Idiomatic C++14 (with a perspective on C++17) - Daniel Hofmann (Mapbox)
Modern & Idiomatic C++14 (with a perspective on C++17)
- Daniel Hofmann (Mapbox)
13:00 - 18:00
Room: Room 157 In this workshop we explain not only modern and idiomatic C++14 features such as ownership semantics, Rule of Zero and Expression SFINAE from the ground up. We also show modern software-engineering techniques for safely wrapping C-libraries, how to provide FFI access to your C++ library (e.g. to provide Python, Node.js or Haskell bindings), and how to provide library ABI stability guarantees. The workshop wraps up with an outlook on the C++17 standardization and proposals (such as Concepts, Variant, Optional), and a discussion on how to emulate some of those features in C++14.
20:00

20:00 - 23:00
Friday, 2 September 2016
09:00 Plenary Talks
Plenary Talks
09:00 - 12:00
Room: Aula
Contributions

09:00 Mesoscale brain modelling in the Human Brain Project - Liesbeth Vanherpe (École polytechnique fédérale de Lausanne)

09:40 Trends in HPC System Architectures: Towards 'The Machine' and Beyond - Frank Baetke (Hewlett Packard Enterprise)

10:40 Radiomics - Klaus Maier-Hein (Deutsches Krebsforschungszentrum Heidelberg)

11:20 Conclusions - Manuel Giffels (KIT)