DeepESDL ml4xcube - Machine Learning Toolkits for Data Cubes#
AI is becoming increasingly important in Earth observations as most parts of the Earth system are continuously monitored by sensors and AI is able to cope with both the volume of data and the heterogeneous data characteristics. For instance, satellites monitor the atmosphere, land, and ocean with unprecedented accuracy. In course of DeepESDL, the Earth System Data Lab (ESDL) capabilities have been extended to support the application of machine learning (ML) methods on Earth System Data Cubes (ESDC).
Various Python-based Jupyter Notebooks/Scripts illustrate a use cases to
demonstrate the capabilities of ml4xcube, utilizing state-of-the-art machine
learning libraries on ESDCs in the DeepESDL Hub environment. Each Jupyter
Notebook involves a self-contained workflow, markdown cells,
comments and plots for user-friendly application and guidance and is based
on one of the three well established open source ML libraries respectively:
- scikit-learn
For classical machine learning such as support vector machines, decision trees, regressions or clustering, scikit-learn provides a broad set of features that fulfils many basic requirements. - PyTorch
For larger neural networks and support for Deep Learning additional ML toolchains are necessary, for example the python-based ML stack PyTorch. With PyTorch, experienced users are supported. It provides low-level API and allows for flexibility to develop and customize deep learning models. It allows for GPU computation and supports transfer learning, domain adaptation, or diverse methods for fine tuning of models. - TensorFlow/Keras
Keras provides a high-level API that can be run on the popular execution backend TensorFlow. Due to its simplicity, it fits well to the requirements of those Earth system scientist that do not require to newly develop neuronal network architectures. As PyTorch, TensorFlow is python-based, allows for GPU computation and it supports Deep Learning applications including transfer learning or domain adaptation.