Enhancements to Multiprocessing in ROOT
| Main Authors: | Guiraud, Enrico, Ganis, Gerardo |
|---|---|
| Format: | Report Journal |
| Terbitan: |
, 2015
|
| Subjects: | |
| Online Access: |
https://zenodo.org/record/31854 |
Daftar Isi:
- Project Specification ROOT is a C++ library developed by CERN for data analysis and plotting. PROOF is a multi-process framework to run ROOT data analytics in parallel on distributed resources. It implements a 3-tier client-master-workers architecture, with the master in charge of dynamic work distribution and collection of the results. PROOF-Lite is a version of PROOF optimized for multi-core machines. It has found much success in the HEP community because of its direct and simple way of exploit multi-cores. However, it suffers from some difficulties coming from the fact that it inherits the setup technology from PROOF. In particular, setting up the environment of the workers requires the manual replication of the client configuration, a fragile process open to inconsistencies and failures. In this project the student will prototype the possibility to create the workers from the client using process forking just before the execution of the query, so eliminating the need of any additional configuration of the workers and taking advantage of copy-on-write system feature. Abstract In this report we outline the results and the products of our investigation of ROOT multi-processing capabilities: not only we showed that it is possible to fork a ROOT session running in Linux environment, but we also built a framework that allows to easily exploit this capability to build parallel applications based on a client-worker architecture. Moreover, using this framework as a foundation, we built a new ROOT feature, the Map function, inspired by python’s pool.map: this function allows to execute the same task many times in parallel on different arguments, giving users an easy and lightweight access to multi-processing. Section 1 is dedicated to a brief explanation of the current approach to multi-processing in ROOT, namely the PROOF and PROOF-Lite facilities. Section 2 describes our new approach, its advantages and issues and some implementation details. Section 3 describes the Map application, a few usage examples, some implementation details and the analysis of its performance. Section 4 outlines some of the possible future developments of the work done so far.