Working together on large datasets
Monday, 9 December 2019 -
13:15
Monday, 9 December 2019
13:15
Coffee
Coffee
13:15 - 13:30
Room: room MAX III, 4th floor
13:30
MAX IV DataSTaMP project and MAX IV involvement in ExPaNDS
-
Magnus Klingberg
Zdenek Matej
Darren Spruce
(MAX IV Laboratory)
MAX IV DataSTaMP project and MAX IV involvement in ExPaNDS
Magnus Klingberg
Zdenek Matej
Darren Spruce
(MAX IV Laboratory)
13:30 - 13:40
Room: room MAX III, 4th floor
MAX IV is developing storage and data services for MAX IV users within the DataSTaMP project thanks to the funding from KAW. MAX IV is also one of 11 partner institutions in ExPaNDS which aims for developing Photon and Neutron Data Services within the European Open Science Cloud (EOSC). A very brief introduction will be given about these projects. [DataSTaMP] https://www.maxiv.lu.se/accelerators-beamlines/technology/kits-projects/datastamp/ [ ExPaNDS] https://expands.eu/
13:40
Towards end-to-end data-management for large scale x-ray facilities
-
Brian Vinter
(Copenhagen University)
Towards end-to-end data-management for large scale x-ray facilities
Brian Vinter
(Copenhagen University)
13:40 - 14:10
Room: room MAX III, 4th floor
Large scale scientific facilities, including x-ray facilities, face an extreme growth in data from instruments. With x-ray instruments data grow exponentially with the increased size of detectors, another exponential factor from the frequency one many sample with and finally x-ray sources are not robust enough that a large set of experiments can be automated, bringing a large increase in the number of experiments an instrument can perform in a session. Thus storing the data alone is a challenge. The challenges are furthered from the fact that detector-size has grown to a resolution where samples, at least tomograms, cannot fit in the memory of a PC for data-analysis, and thus must be moved onto server-class computers with sufficient memory to hold a raw-data sample and a processed version as well. The increase in data-rate and number of experiments also mean that running through all samples manually easily becomes unfeasible and some means of batch processing must be introduced. A final challenge is that users of x-ray facilities is widening and many of the new users are not comfortable with data-analysis and need to work with others in that part. This means that large communities, typically geographically distributed, need to collaborate on these, very large, datasets. The talk will presents our ideas for an integrated solution to the above problems, include status and plans, and also introduce – for discussion – an idea for such an integrated system to help fight scientific fraud. The work is supported in parts by H2020 European training network MUMMERING.
14:10
ICOS Carbon Portal
-
Maggie Hellström
(Lund University (Carbon Portal, Sweden))
Karolina Pantazatou
(ICOS ERIC (Carbon Portal, Sweden))
ICOS Carbon Portal
Maggie Hellström
(Lund University (Carbon Portal, Sweden))
Karolina Pantazatou
(ICOS ERIC (Carbon Portal, Sweden))
14:10 - 14:40
Room: room MAX III, 4th floor
ICOS - Integrated Carbon Observation System, a European Research Infrastructure. ICOS is the European measurement system for high quality and high precision greenhouse gas observations. The ICOS Carbon Portal provides free and open access to all ICOS data.
14:40
Discussion and coffee
Discussion and coffee
14:40 - 15:00
Room: room MAX III, 4th floor