Simplifying geospatial AI with TerraTorch 1.0
An affiliate project of the AI Alliance, TT 1.0 makes it easier to develop and benchmark geospatial models.
We are excited to announce the release of TerraTorch (TT) 1.0, an important milestone in geospatial AI. TerraTorch was developed to meet the needs of Geospatial AI researchers seeking to fine-tune foundation models and apply them to their data. It simplifies this process by integrating and standardizing foundation models as reusable backbones while enabling seamless combinations with state-of-the-art decoders and heads.
Different tasks (including regression, classification, segmentation, object detection, et cetera) require different heads, and decoders exhibit varying performance on different dataset, backbone, and head combinations, making them part of the optimization objective. TerraTorch has rapidly evolved into a powerful framework for foundation model fine-tuning and inference, specifically designed for geospatial, weather, climate, and Earth observation applications.
Recently, TerraTorch introduced the concept of "necks" — intermediate layers that adapt between incompatible model components, thereby enhancing flexibility in architectural design. Since all neural network components are modularized and accessible via both the Python API and a TerraTorch execution configuration YAML file, the framework not only improves reproducibility and integration but also reduces user error by minimizing the need for excessive glue code.
Finally, this configuration-based approach enables hyperparameter optimization (HPO) and neural network architecture search (NAS) through seamless integration with the TT Iterate plugin. This tool assists users in optimizing the selection of backbones, decoders, and heads, as well as in fine-tuning hyperparameters for improved model performance. Additionally, TerraTorch plays a key role in benchmarking foundation models according to the GeoBench standard, ensuring rigorous evaluation and comparison of models in geospatial AI.
"TerraTorch makes it very easy to adapt geospatial foundation models to EO applications. The built-in metrics help accelerate model validation and compare models with each other," said Rahul Ramachandran, senior research scientist at NASA’s Marshall Space Flight Center. "Our researchers have used TerraTorch to fine tune Prithvi-EO-2.0 for several applications, including flood, wildfire, and burn intensity mapping, landslide detection, and crop segmentation, and the extended community is showing great uptake."
TT 1.0 provides fingertip access to a wide range of foundation models, with opinionated performance optimizations (default values based on best practices) that are still configurable, to help users get started quickly while maintaining flexibility for fine-tuning. This allows for rapid experimentation and deployment, making it easier to explore and apply state-of-the-art geospatial AI models.
A key highlight of this release is TerraTorch’s acceptance as an Affiliate Project of the AI Alliance. This collaboration brings TerraTorch into a larger community of AI researchers and practitioners, ensuring alignment with best practices in open AI development and fostering innovation in geospatial machine learning.
A key part of TT 1.0 is the Iterate plugin for HPO and NAS. Iterate simplifies model selection and optimization by automating the search for the best architectures and hyperparameters running on top of Optuna and optionally using MLFlow and Ray Tune. Additionally, Iterate supports benchmarking foundation models according to the GeoBench standard, ensuring robust evaluation of geospatial AI models against real-world datasets. Here is a link to a hello world example.
TT 1.0 comes with a vast collection of foundation models and reusable data modules, making it easier than ever to use geospatial AI models. Users can leverage TT 1.0 for pre-training, fine-tuning, and inference, all configured either via a flexible YAML configuration file or executed using PyTorch Lightning-compatible code.
To expand its capabilities, TT 1.0 seamlessly integrates with TorchGeo, providing access to all models and data modules supported by the TorchGeo ecosystem. Moreover, many open-source geospatial foundation models are now natively supported, including the Prithvi family, the Granite family, Clay, SatMAE, Satlas, DeCur and DOFA, reinforcing TerraTorch’s role as a premier geospatial AI framework.
Through flexible heads, TT 1.0 supports semantic segmentation, multivariate multiple regression, and image classification.
It also supports parameter efficient fine-tuning (PEFT) using LoRA and ViT-Adapter. VPT will be available with TT 1.1.
In addition, any model from timm or smp can also be used in TT 1.0, giving the user full flexibility on the type of model to use for their specific task.
TerraTorch is the powerhouse behind IBM's Geospatial Studio, driving scalable AI inference and fine-tuning for geospatial applications. As the backbone of the platform, it enables seamless data preprocessing, model training, and real-time analysis of satellite imagery — all within a unified environment.
Since Geospatial Studio is built on an open-source foundation, TerraTorch minimizes vendor lock-in, promotes transparency, and accelerates innovation by leveraging community-driven advancements. In this way, Geospatial Studio automatically benefits from all of the state-of-the-art developments in foundation models and AI best practices, without being limited to IBM’s stack.
vLLM is an optimized open-source library for LLM inference, featuring advanced memory management, dynamic batching, and Paged Attention for efficient streaming with lower latency.
We're expanding vLLM beyond text to multi-modal AI, leveraging TerraTorch to enable consumption and production of geospatial data. A key milestone has been onboarding IBM Research's Prithvi-EO-2.0 — the first non-text input/non-text output model — in vLLM. Current work looks to accelerate this shift by enabling non-text post-processing stages in vLLM, which in the case of geospatial models can leverage TerraTorch for computer vision tasks. This is laying the foundations for making vLLM truly multimodal.
The launch of TT 1.0 is just the beginning. We are committed to continuous improvements, expanding support for new models, datasets, tasks (including object detection), and benchmarking standards to ensure TerraTorch remains at the forefront of geospatial AI innovation.
We invite the AI and geospatial communities to explore TT 1.0, contribute to its development, and collaborate in shaping the future of geospatial AI. Try it out today and be part of this exciting journey!
For more information, visit our GitHub repository and join the discussion on Slack.
Lead image source: European Center for Medium-Range Weather Forecasts.