DeepESDL ml4xcube - Machine Learning Toolkits for Data Cubes#

AI is becoming increasingly important in Earth observations as most parts of the Earth system are continuously monitored by sensors and AI is able to cope with both the volume of data and the heterogeneous data characteristics. For instance, satellites monitor the atmosphere, land, and ocean with unprecedented accuracy. In course of DeepESDL, the Earth System Data Lab (ESDL) capabilities have been extended to support the application of machine learning (ML) methods on Earth System Data Cubes (ESDC).

Various Python-based Jupyter Notebooks/Scripts illustrate a use cases to demonstrate the capabilities of ml4xcube, utilizing state-of-the-art machine learning libraries on ESDCs in the DeepESDL Hub environment. Each Jupyter Notebook involves a self-contained workflow, markdown cells, comments and plots for user-friendly application and guidance and is based on one of the three well established open source ML libraries respectively:

  1. scikit-learn
    For classical machine learning such as support vector machines, decision trees, regressions or clustering, scikit-learn provides a broad set of features that fulfils many basic requirements.
  2. PyTorch
    For larger neural networks and support for Deep Learning additional ML toolchains are necessary, for example the python-based ML stack PyTorch. With PyTorch, experienced users are supported. It provides low-level API and allows for flexibility to develop and customize deep learning models. It allows for GPU computation and supports transfer learning, domain adaptation, or diverse methods for fine tuning of models.
  3. TensorFlow/Keras
    Keras provides a high-level API that can be run on the popular execution backend TensorFlow. Due to its simplicity, it fits well to the requirements of those Earth system scientist that do not require to newly develop neuronal network architectures. As PyTorch, TensorFlow is python-based, allows for GPU computation and it supports Deep Learning applications including transfer learning or domain adaptation.

Overview#

  1. Getting Started
  2. Example Use Cases and Juypter Notebooks
  3. ml4xcube API