v1.2.1

What's Changed

• v1.2.1 by @Tschoun in #99

New Features

Enhanced VHI Calculation

Snow Coverage Masking

• Added functionality to mask snow-covered areas and low illumination angles in VHI calculations.
• Introduced a new function calculateSnowMask to compute the Normalized Difference Snow Index (NDSI).

Illumination Angle Filtering

• Implemented masking for areas with NDSI values of 0.43 or higher, effectively filtering out snow-covered regions.
• Added masking for areas with illumination angles of 65° or higher to improve accuracy in poorly lit terrain.

Technical Details

• The new calculateSnowMask function utilizes NDSI for snow detection.
• Threshold values:
  • NDSI: ≥ 0.43 for snow coverage
  • Illumination angle: ≥ 65° for low light conditions

Full Changelog: v1.2.0...v1.2.1

v1.2.0

Release Notes for Version 1.2.0

• Merging dev-20240917 to main by @Tschoun in #98

General Updates

• Fixed the wrong collection name for LST_reference_data.

Bug Fixes and Security

• Fix for Issue #95.
  • Solved with sorting the unique_filename_day list.
• Small changes to util_moveasset.py.
• "Fix" for bug #97:
  • Updated the terrainShadowMask band to now contain values from 0-90° representing the illumination angle based on the solar position and the SRTM 30m DEM. Values above 90° have been clamped to 90.
  • Adapted the functions addTerrainShadow and addTerrainShadow_predefined to add the mask (calculated from SwissALTI3D or SwissSURFACE3D) as the value 100 to the terrainShadowMask band, which now contains the illumination angle.
  • Adapted the function addMaskedPixelCount to set the threshold for the terrainShadowMask to greater than 99.

Configuration Updates

• Updated prod_config.py.

Notes

This release includes important fixes related to the terrainShadowMask and the handling of illumination angles. Users should be aware of the changes made to the DEM sources and the adjustments in the associated functions.

Full Changelog: v1.1.0...v1.2.0

v1.1.0

Release Notes for Version 1.1.0

• based on Dev 20240717 to v.1.1.0 by @davidoesch in #94

General updates

• Product Vegetation Health Index VHI is operational
• Co-registration is done with the help of AEROSICS and GRI

New Features and Enhancements

• Utility Scripts

  • Created util_upload_dxdy.py for uploading DX and DY files
  • Added util_deleteassets.py to remove assets from an image collection
  • Implemented util_checkasset for asset verification

• MSG / MFG Adaptations

  • Updated MSG and MFG processing for operational CLIMA delivery
  • Modified LST climatology processing
  • Updated step0_processor_msg_lst_clima.py to use MSG instead of MFG

• S2 SR Processing

  • Implemented precalculated coregistration usage switch[6]
  • Added check for DX DY availability in S2_SR processing

• VHI Processing

  • Updated final VHI calculation to use Meteosat data instead of VIIRS
  • Implemented daily VHI calculation regardless of S2_SR availability
  • Added checks for existing VHI assets and empty asset lists

• Google Cloud Storage

  • Added Google Cloud Storage export options

Improvements and Optimizations

• Error Handling and Performance

  • Implemented fix for Google Drive timeout issues
  • Added handling for STAC timeouts[8]
  • Reduced printing of STAC commands and GDAL output

• Documentation and Diagrams

  • Updated satromo_processor.drawio and added SVG version

• Configuration Updates

  • Added OED configuration
  • Updated dev_config.py with new settings and asset sizes

Bug Fixes and Security

• Fixed vulnerabilities in dependencies
• Resolved Fiona dependency issue in GeoPandas

Other Changes

• Removed backup on S3 INT option with rclone
• Added asset size check function to publish.py
• Updated documentation in MSG Processor

Notes

• This release includes significant changes to VHI processing, LST data handling, and utility scripts.
• Users should be aware of the switch from VIIRS to Meteosat data for LST in VHI calculations.
• The new DX DY upload utility and precalculated coregistration switch may affect S2 SR processing workflows.

Full Changelog: v.1.0.1...v1.1.0

v.1.0.1

Release Notes for Version 1.0.1

• based on Dev 20240517 to main by @davidoesch in #89

General Updates

• Prepared for deployment to production.

Features

• Added satromo_processor.drawio.svg
• Updated satromo_processor.drawio
• Updated README.md
  • Added description for single scene processing via command line.

Enhancements

• Added MSG LST step0

  • Introduced a new script (step0_processor_msg_lst.py) to access and upload LandSurface (LST) data over Switzerland to Google Earth Engine (GEE).
  • Utilized CMS SAF data provided by MeteoSwiss via s3 on data.geo.admin.ch to store as SATROMO assets and to calculate VCI, TCI, and VHI.
  • Added functionality to retrieve and process NetCDF files, extracting relevant attributes and dimensions.
  • Implemented the export of NetCDF bands to GeoTIFF format and scaling operations using Rasterio.
  • Added functions to generate LST mosaics for specific dates and handle the upload process to Google Cloud Storage and GEE.
  • Ensured proper error handling and cleanup of temporary files and datasets.
  • Added in dev_config a new PRODUCT_MSG which configures the LST.
  • Moved util_addproperties to main_functions.

• Updated requirements.txt for MSG LST

  • Updated dependencies in requirements.txt for LST METEOSWISS processor.
  • Removed unnecessary mount point to Google Cloud.

• Added MSG LST CLIMA

  • MSG LST and MSG LST CLIMA can be calculated with the step0 function as the other products. They won't have a step1, easing reprocessing.
  • Added PRODUCT_MSG_CLIMA and PRODUCT_MSG exit points in satromo processor.
  • Added PRODUCT_MSG_CLIMA and PRODUCT_MSG in dev_config.
  • New scripts: step0_processor_msg_lst.py and step0_processor_msg_lst_clima.py.

• Added command line option for date

  • Enabled reprocessing of a specific date via command line: python satromo_processor.py prod_config.py 2024-06-12. The config file is mandatory when used with the date option.

• First steps to get rid of the black part in the visualization

  • Outside of the Swiss perimeter is nan. Inside the Swiss perimeter but outside the flight path is zero.
  • Addressed issues caused by line 605 in step0_processor_s2_sr.py.

Bug Fixes

• Removes black border
  • Fix for #56.

Asset Management

• Moved ch_buffer_5000m to assets
• Moved ch_buffer_5000m.shp to assets path

Thumbnail and Script Updates

• Thumbnail

  • Fixed aspect ratio for VHI and S2 SR.
  • Updated background to accommodate new S2 SR tiles cut to the effective path.
  • Changed no data values accordingly.

• util_listassets.py

  • Added functionality to list all assets of a collection.

• util_moveassets_terrainshadow.py

  • Script to initialize Google Earth Engine (GEE) and Google Drive authentication.
  • Converts Day-Of-Year (DOY) to datetime strings.
  • Manages the copying and updating of assets within Google Earth Engine.

• satromo_publish.py

  • Added detection for VHI cases to use the Swiss buffer.

• step0_processor_s2_sr.py

  • Replaced swisswalti with swisssurface.
  • Set terrainShadowDetectionPrecalculated to True to use pre-calculated shadows.

• step1_processor_vhi.py

  • Updated to use T235959 time format instead of 240000 as per definitions with BAFU and MeteoSwiss.

Updates to Requirements

• Update requirements.txt

  • Set certifi to latest.

• Catch gdrive errors, remove backup on S3 INT

  • Fix for #86.
  • Removed the backup on S3 INT option with rclone.

• Functional util_addproperties.py

  • Script to add properties to existing GEE assets.
  • #79.

Full Changelog: v1.0.0...v.1.0.1

v1.0.0

Release Notes for Version 1.0.0

General Updates

• Set up the step1_processor_vhi:

  • Made changes to the dev_config file regarding additional variables for the V1 (VHI) product.
  • Renamed from step1_processor_pv1 to step1_processor_vhi, adjusted in the satromo_processor.
  • Deleted step1_processor_pv1 for better clarity.
  • Completed the setup of step1_processor_vhi.

• Created util_castassetbands.py:

  • Addressed issue #73.

• Color Table Update:

  • Changed color table for VHI thumbnail.
  • Added white background for no data in the thumbnail.

• Filename and Timestamp Changes:

  • Renamed export drive from _10m to -10m.
  • Updated VHI timestamp to 240000 format, addressing changes in issue #76.

• Added Metadata:

  • Included 'pixel_size_meter' to S2_SR and VHI, as noted in issue #79.

• Processing Logic Updates:

  • Stopped processing if data is not yet complete, addressing issues #61 and #80.
  • Fixed conditions for incomplete tile coverage, ensuring accurate number of tiles for orbit 108 and correcting conditional logic.

• Date Handling Improvements:

  • Fixed datetime error in publishing process, ensuring correct ISO8601 format handling.
  • Updated publish script to check for "240000" format, accommodating the latest ISO8601 standard.

• Preview and Thumbnail Enhancements:

  • Updated preview link to map.geo, fixing issue #77.
  • Enhanced VHI thumbnail to include Swiss borders, rivers, and lakes.

• Warning Regions Updates:

  • Fixed extraction issues for warnregions, addressing issue #72.
  • Updated requirements.txt for the main_extract_warnregions function to include pandas and pyarrow.
  • Added official versions of warning regions (Version 2.5.2024) and BAFU_Grossregionen_v3.
  • Adjusted main_extract_warnregions.py to handle empty values.

• Asset and Configuration Adjustments:

  • Deleted obsolete assets and scripts.
  • Updated prod_config for more intuitive naming conventions and added no_data property.

Functionality Updates

• Operationalization of swissEO S2-SR:
  • Finalized and operationalized the VHI product, swissEO S2-SR.

Bug Fixes

• Fixed Incomplete Processing Conditions:

  • Ensured processing stops if the required number of tiles or cloud probability collections are incomplete.

• Resolved Datetime and ISO8601 Issues:

  • Corrected issues with the datetime format for publish and reverted end-of-day format to 23:59:59.

Full Changelog: v0.0.2...v1.0.0

v0.0.2

Release Notes for Version 0.0.2

this is bugfix release

Bug Fixes

• HotFix cs+ s2_sr cast to int8 for cloudprobabillity export to asset to fix S2 SR for #73

Full Changelog: v0.0.2...v0.0.2

v0.0.1

Release Notes for Version 0.0.1

General Updates

• Operational Product as of now from 28.3.2024 onward: swissEO S2-SR: Optical satellite data (Sentinel-2)
• Preparatory Work: Preparation for the release of version 0.0.1 has been completed.
• Switched Asset Resources: Asset resources have been switched to production environment.
• Added Priority Column: A new "priority" column has been added.
• Fix DATA_SOURCE: Issue with DATA_SOURCE has been fixed.
• Updated Requirements: Requirements for earthengine-api have been updated.
• Fixed SWISSTOPO_RELEASEVERSION: Issue with SWISSTOPO_RELEASEVERSION has been resolved.
• Changed Run Type Naming: Naming convention for run types has been adjusted.
• Added Properties to Asset Export: Additional properties have been included in asset export.
• Refined CS+ Cloud Shadow Masking: Improvements have been made to CS+ cloud shadow masking.
• Replaced join with .linkCollection: join function has been replaced with .linkCollection.
• Fix for #57: Issue #57 has been resolved.
• Added CloudScore+ as Default: CloudScore+ has been set as the default cloud detection algorithm.

Functionality Updates

• Added clip_outermost_rows Function: New function clip_outermost_rows has been added to S2_TOA.
• Added S3 TOA: Initial support for S3 TOA has been added.
• Added Date Acquired to S2 TOA: Date acquired information has been added to S2 TOA.
• Added Initial S3 TOA: Initial support for S3 TOA processing has been added.
• Added l89 toa and sr: Support for l89 TOA and SR has been included.
• Moved Functions to Main Utils: Functions have been relocated from processor to main_utils.
• Added l57 toa and draft step1: Initial support for l57 TOA and draft step1 has been added.
• Removed get_github_info Function: get_github_info() function has been removed from step0 to main_utils.
• Initial Step1 for l57_ar: Initial implementation for step1 of l57_ar has been added.
• Added PROD: PROD environment has been incorporated into the system.
• Fix Missing Thumbnail Conditions: Conditions for missing thumbnails have been corrected.

Bug Fixes

• Fixed Data Source: Issue with data source has been rectified.
• Updated Gray Value for VHI Thumbnail: Gray value for VHI thumbnail has been updated.
• Changed Gray in No Info No Data Pixels: Gray value in no info and no data pixels has been adjusted.
• Fix Prod for Collection Issue: Issue with production for collection has been fixed.
• Set Back to Operation Mode Regarding Date: System has been reverted to operation mode regarding date.

These release notes summarize the changes made in version 0.0.1 of the project. For more details, refer to the commit history on GitHub.