Skip to contents

Changelog

Source: NEWS.md

Rwapor 1.0.6

Production readiness & scientific review fixes

  • Full Tiled Indicator Parity & Loud Validation (R-01 & R-02): wapor_normalize_analysis_indicators() now expands user-facing indicator aliases ("peff", "peff_green_blue", "variability", "biomass", "yield", "cv", "theil") and fails loudly with an explicit error on unsupported indicator names instead of silently dropping them. wapor_run_seasonal_analysis() and wapor_run_seasonal_analysis_tiled() now compute, export, and record provenance for all 24 built-in indicator steps across memory, stream, and tiled modes (including standalone et_rain, seasonal_npp, cwp_raster, bwp_raster, cwp_summary, cv_aeti, theil_aeti, and out-of-core global class adequacy_p95), and include indicators in the tiled manifest.json resume config_hash.
  • Explicit L3 Selection & mosaic_all Coverage Provenance (R-04): wapor_run_seasonal_analysis() now enforces wapor_resolve_l3_selection() for remote L3 variables, mosaicking all intersecting L3 regions per dekad via wapor_write_l3_mosaic() when l3_mode = "mosaic_all" and recording per-region coverage_fraction in results$l3_coverage and provenance.json. The Shiny download and seasonal analysis modules (inst/shiny/mod_download.R, inst/shiny/mod_analysis.R) now wire l3_region / l3_mode = "mosaic_all" and require explicit disambiguation when an AOI intersects multiple L3 schemes.
  • Canonical Indicator Specification (R-06) & Cross-Implementation Golden Harness (R-03): Added inst/spec/indicators.yaml and docs/architecture.md as the single machine-readable indicator specification synchronized with wapor_list_indicator_steps() (tests/testthat/test-indicator-spec.R), alongside a 6×6 6-dekad cross-implementation golden fixture (tests/testthat/fixtures/crossval/) verified across memory, stream, and tiled modes (tests/testthat/test-crossval-waporbox.R).
  • Exported GeoTIFF Unit Stamping & Offline Seasonal Pipeline Test (R-07): wapor_export_analysis_outputs() now stamps terra::units() on every exported GeoTIFF from results$provenance$outputs[[key]]$units and exports seasonal_npp, cwp, bwp, cwp_by_class, and cv_aeti/theil_aeti summary metrics (tests/testthat/test-integration-seasonal-offline.R).
  • Climatic Kc Adjustment (FAO-56 Eq. 62 & 65): Added wapor_adjust_kc_climate(), wapor_extract_agera5_climate(), and wapor_adjust_crop_params_agera5() to calibrate tabulated Kc,midK_{c,\text{mid}} and Kc,endK_{c,\text{end}} to local climatic conditions (wind speed u2u_2, minimum relative humidity RHminRH_{\min}, and plant height hh). Automatically derives stage-specific mean climate directly from AgERA5 reanalysis via wapor_ts().
  • Net Biomass Water Productivity (NBWP): New wapor_calc_nbwp() and wapor_math_nbwp() calculate Net Biomass Water Productivity (NBWP=Biomass/(Transpiration×10)NBWP = \text{Biomass} / (\text{Transpiration} \times 10) in kg/m3\text{kg/m}^3), isolating beneficial water consumption through transpiration from non-beneficial soil evaporation and canopy interception. Fully integrated into wapor_run_seasonal_analysis(), analysis_registry, wapor_export_analysis_outputs(), and Shiny analysis tab.
  • ETLook Energy Balance Mass Balance Auditor: New wapor_audit_etlook_mass_balance() audits mass conservation (AETI=E+T+IAETI = E + T + I) across rasters or numeric totals with customizable tolerance.
  • Beneficial Fraction Clamping: wapor_calc_beneficial_fraction() and wapor_math_beneficial_fraction() now clamp ratios strictly to [0,1][0, 1] across both raster and numeric execution paths.
  • Crop Parameter Validation: Added strict validation in wapor_calc_yield_npp() to ensure moisture content MC∈[0,1)MC \in [0, 1), harvest index HI∈(0,1]HI \in (0, 1], and positive correction factors (fc,AOTf_c, AOT).
  • Phenology Proportional Scaling in Processing Kernel: Harmonized .wapor_profile_kc and step_etc with wapor_build_kc_by_class() so that short seasons (TD<Lini+Lmid+LlateTD < L_{ini} + L_{mid} + L_{late}) proportionally scale stage lengths across chunked, tiled, and in-memory execution paths.
  • Net Irrigation Requirement (NIR) in Seasonal Analysis: Integrated "nir" into wapor_run_seasonal_analysis(), calculating monthly NIR=max⁡(0,ETc−Peff)NIR = \max(0, ET_c - P_{\text{eff}}) and seasonal totals, exporting monthly and seasonal rasters, summaries, and provenance sidecars.
  • Transpiration Summary Column Fix: Fixed column naming in seasonal_t$by_class in analysis_engine.R from mean_seasonal_aeti to mean_seasonal_t.
  • Literature Citations & Derivations: Added formal mathematical equations, parameter derivations (e.g. carbon fraction 10/0.45=22.22210 / 0.45 = 22.222 for TBP), and authoritative peer-reviewed literature citations across 11 core indicators in R/analysis_indicators.R and R/performance_indicators.R.
  • Short-Season Crop Phenology: In wapor_build_kc_by_class(), when season duration is shorter than the sum of standard growth stages (TD<Lini+Lmid+LlateTD < L_{ini} + L_{mid} + L_{late}), stage lengths are now proportionally scaled to fit the observed satellite phenology rather than aborting with numeric(0).
  • Generalized Temperature Conversion: is_temperature_variable() and wapor_convert_temperature() now recognize all AgERA5 temperature products (TMIN, TMAX, TAVG, TDEW, and TEMP), automatically converting Kelvin to Celsius during download and caching.
  • Farm Monitoring Aggregation: wapor_generate_seasonal_raster() and wapor_apply_seasonal_mask_recalc() apply dekadal day length multipliers (N∈[8,11]N \in [8, 11] days) before seasonal summation of flux variables, eliminating the ~10x seasonal accumulation underestimation.
  • DuckDB Windowed Streaming: Monitoring raster retrieval prioritizes COG file paths with windowed reads (terra::rast(path, win = f_ext)) over full raster BLOB deserialization, dramatically reducing memory overhead.
  • Vignette Harmonization: Updated vignettes/wheat-water-productivity.Rmd to use canonical indicator peff_green_blue and cwp_summary output structures.
  • Cross-Platform & Packaging: Added SystemRequirements: GDAL (>= 3.0.0), GEOS (>= 3.8.0), PROJ (>= 6.0.0) in DESCRIPTION. Added macos-latest to live API CI/CD matrix. Excluded large unversioned archives from package builds via .Rbuildignore.

Irrigation performance indicators

Download once, work offline

  • New wapor_download() saves one GeoTIFF per time step for an area, in the layout the seasonal analysis reads with data_source = "local". Files already saved are skipped, so the call can be repeated to complete or extend a period; without a connection it returns the saved files and warns that it could not check the period. Values keep the source unit with the WaPOR scale factor applied once. Files are written under a temporary name and renamed when complete, and a folder is tied to the area of its first download.
  • wapor_run_seasonal_analysis() accepts config$cache_dir: with data_source = "api" the sources a run needs are saved there first (only the missing files) and then read locally, so later runs for the same area work without internet. Results equal a local run on the same files. Not available with l3_mode = "mosaic_all".
  • Streaming is now about as fast as local files (see “Faster remote reads”), so these are for offline work and repeated analyses, not a requirement for speed.

Faster remote reads

  • The package no longer sets the GDAL HTTP chunk size (CPL_VSIL_CURL_CHUNK_SIZE) to 10 MB on load. GDAL reads remote files in whole chunks, so every remote file open downloaded far more than the window it needed. Measured with wapor_ts() on 150 polygons and 36 dekads: Level 2 took 343 s and requested 1,091 MB before, 21 s and 17 MB now; Level 3 took 66 s and 232 MB before, 27 s and 14 MB now. Results are identical. GDAL fixes this value at the first remote read of a session, so the per-job chunk size in wapor_plan_processing() never applied and has been removed (gdal_chunk_bytes is no longer part of the plan). wapor_configure_gdal(chunk_size = ) now defaults to NULL (leave GDAL’s default) and accepts 1024 to 10485760; to use your own value, set it before the first remote read.
  • While Rwapor reads remote rasters it limits /vsicurl/ to .tif files, which stops two failing side-file requests per raster. The limit is removed after each read; switch it off with options(Rwapor.remote_extension_filter = FALSE).
  • The PROJ fix on package load no longer depends on RWAPOR_AUTO_CONFIG; it has its own switch, options(Rwapor.fix_proj = FALSE). Without the fix, a foreign PROJ database on the path (for example PostGIS) costs about 0.35 s per raster open.
  • wapor_ts() and wapor_map() print one note when a request reads more than about 2 GB of cells, naming the coarser level as an option.
  • New inst/bench/remote_io_benchmark.R counts HTTP requests and requested megabytes for a standard polygon time series and fails when its limits are exceeded.

Zonal statistics and masks

  • New wapor_zonal_stats(): area-weighted statistics for any polygons at one or several nested levels (for example scheme and farm), on a raster or directly on the result of wapor_run_seasonal_analysis(). It reports crop share (mask_fraction) and data coverage separately, weighted mean, median and quantiles (equal to quantile(type = 7) for equal weights), population or sample standard deviation, CV, Christiansen uniformity, low-quarter distribution uniformity, Gini, Theil, volumes in m3 and million m3 (from depths, or from rates with days), and class shares with every class listed. Output is a long table with the id columns; format = "wide" or "sf" and wapor_zonal_wide() give one row per zone and layer.
  • New wapor_rasterize_mask() (polygons to a mask or a covered fraction on any grid, with an area check) and wapor_harmonize_mask() (a classified map at any resolution to a mask and a class fraction). The fractions are the weights input of wapor_zonal_stats().
  • Fixed before release, found while completing this batch: polygons sharing an id were not merged, so a parent level with several polygons was reported once per polygon; on lon/lat rasters cell areas came from a sphere (0.3% too large at 32 degrees north) and now use the WGS84 ellipsoid; fractions of adjacent polygons in one cell now add up; cells without data in a classified map count as “not the class”.
  • wapor_zonal_stats() returns the same table about eight to ten times faster (400 zones and 12 layers: 56 s before, 6.5 s now; 2,025 zones: 237 s before, 27 s now). Only the requested statistics are computed, and layers are read in groups sized to the memory budget (options(Rwapor.memory_budget_mb = )), so memory no longer grows with the number of layers. inst/bench/zonal_benchmark.R times the standard cases.
  • wapor_zonal_stats(format = "sf") now gives every row the geometry of its own zone. With more than one statistic, rows carried the geometry of another zone or an empty geometry.
  • wapor_zonal_stats() works on lon/lat rasters (Level 1 and Level 2) without the lwgeom package. Zone areas are measured in an equal-area projection centred on the zones; the session’s s2 setting is no longer changed.
  • wapor_classify() treats NA and NaN as missing (class NA) instead of stopping, as its error message already said; only infinite values are refused. This also makes wapor_zonal_stats(stats = "class_share", breaks = ...) work for zones with NoData cells.

Classification

  • Added configurable fixed and type-7 quantile classification with literature-backed adequacy, equity, irrigation-method uniformity, and productivity-spot schemes. Results retain thresholds and source metadata, including through GeoTIFF metadata.

Reliability

  • Known-answer tests pin every output of wapor_run_seasonal_analysis() (236 values, memory, stream and tiled modes) to real WaPOR data from the 2026 training case studies (citrus year from 1 March, wheat season from 1 November), independently checked against plain formulas. A change in any number now fails the test suite.
  • wapor_save_raster_blobs() (monitoring) applies the WaPOR scale factor exactly once, independent of the terra version; reports layers it could not save with a warning and returns list(variable, saved, existing, failed); never stores a layer under today’s date when its date cannot be read.
  • options(Rwapor.verbose = FALSE) silences progress messages (warnings still show). All package options are listed in ?Rwapor.
  • The live-API release gate also checks seasonal ETc and adequacy against an independent daily Kc x RET calculation.

Rwapor 1.0.5

  • wapor_run_seasonal_analysis() computes ETc with the Kc curve aligned to the season again. Since 1.0.4 (94b1533) the processing kernel rebased the season days to the period year for the weights but aggregated Kc with the unshifted dekad table, so any season not starting on 1 January of the period year used a shifted Kc curve: ETc (and adequacy_etc, monthly_etc) came out too low, e.g. about 9% for a citrus year starting 1 March and 0 mm (with adequacy NaN) for a wheat season starting 1 November. Re-run ETc and adequacy results made with 1.0.4. A regression test checks kernel ETc against the daily Kc x RET sum for a 1 March season.
  • wapor_map() applies the source file scale (0.1 for most WaPOR layers) exactly once. It clears the stored scale when it opens the files and applies it after the crop, so the result no longer depends on the terra version. The 1.0.4 fix multiplied values that were already physical by the catalogue scale again, giving outputs 10 times too low (e.g. 2.5 instead of 25 mm/dekad) with terra 1.7 and 1.9. Re-download maps made with 1.0.4.
  • wapor_map() again returns the documented character vector of output paths (a named list of them for several variables). Since 1.0.1 it returned a status list, so terra::rast(wapor_map(...)) from the getting-started vignette failed, and the dashboard reported “Download failed … expected files were not found” after every successful non-seasonal download. Run details are in attr(x, "wapor_status"); code that used x$status or x$output_paths from 1.0.1 to 1.0.4 must switch to the attribute or to x.
  • wapor_write_cog() no longer truncates fractional values. Its integer datatype probe read only the first rows of the first layer, so a band of zeros (e.g. a masked edge) wrote a float raster as INT4U. It now checks every value (up to 5 million) or a regular sample across all layers.
  • wapor_ts() polygon statistics no longer fail with “Could not find expected columns for layer 1” when a batch holds a single layer (for example a one-dekad period): exactextractr names single-layer columns without the layer suffix. Found by the new live-API release gate.
  • Local analysis (data_source = "local") reads <VAR>_seasonal aggregates only when the per-time-step <VAR> folder has no files, so a seasonal map saved next to dekadal files is no longer counted as an extra layer.

Disk use and speed of seasonal analysis

  • Seasonal analysis writes about 45 % less to disk and runs 10 to 40 % faster (1 M cells, 18 dekads, 5 variables: 1,288 MB to 715 MB; 90 s to 70 to 80 s with defaults, 110 s to 65 to 77 s with keep_intermediates = TRUE), with unchanged results:
    • keep_intermediates is FALSE unless set. Behaviour change: results$dekadal_stacks and results$season_weights are no longer returned in memory mode by default; set keep_intermediates = TRUE.
    • wapor_export_analysis_outputs(include_dekadal = FALSE) is the new default. Behaviour change: set include_dekadal = TRUE for dekadal_stacks/.
    • File-backed results, tiles and exports are compressed Float32 (relative difference below 1e-7).
    • The run logs an estimate of the disk space it needs and warns when the output or temporary folder has less free space (options(Rwapor.disk_check = FALSE) turns this off).
  • inst/bench/disk_footprint_benchmark.R reproduces these measurements.

New and improved functions

  • New wapor_unstack_map() splits a multi-band wapor_map() file into one file per date, the layout local analysis reads. wapor_local_rasters() now warns about such files instead of skipping them silently.
  • wapor_fetch_l3_regions() gains timeout and retry.
  • wapor_configure_gdal() gains overwrite. When the package loads, GDAL variables that are already set (by the user, .Renviron or an institutional setup) are kept, and RWAPOR_AUTO_CONFIG=false or options(Rwapor.configure_gdal = FALSE) skips the configuration.
  • wapor_write_cog() no longer prints GDAL “does not support creation option” notes on every write.

Dashboard

  • The Visualisation tab no longer needs the retired raster package (terra rasters go straight to leaflet >= 2.1.2).
  • Start-up no longer blocks about 30 s on API retries when the L3 region list cannot be fetched.
  • The Analysis tab defaults to dekadal RET and precipitation (L1-RET-D, L1-PCP-D) instead of annual AgERA5 products.
  • The active tab label is readable.

Installation and CI

  • README Step 3.2 installs knitr and rmarkdown, which install_github(build_vignettes = TRUE) needs, and documents the install without vignettes. raster and pkgdown are no longer in Suggests.
  • CI runs on every version-* branch and every pull request; a new install smoke test installs Rwapor as the README describes on Windows, macOS and Linux and starts the dashboard. The pkgdown site deploys from the default branch.
  • Fixed a Windows-only race in a test fixture that kept CI red on 1.0.4; a network test now skips when the WaPOR API is unreachable.

Rwapor 1.0.4

Installation and analysis safety fixes

  • GitHub CI now runs for the default version-1.0.4 branch.
  • wapor_export_analysis_outputs() accepts a single-season result directly; it no longer mistakes its first raster for a multi-season result.
  • Seasonal analysis accepts season rasters made with the historic ref_year = 1970 default by rebasing their internal profile keys.
  • A supplied crop mask is used by default. An explicit use_crop_mask = FALSE now warns because it analyses the whole AOI as crop class 1.
  • Non-seasonal wapor_map(separate_files = TRUE) writes physical WaPOR values when a remote source has lost its scale metadata, so the output is safe to reuse with data_source = "local". (Superseded in 1.0.5: this fix scaled values twice.)

New features

  • New wapor_plot_polygon_seasons() draws selected polygon units over several raster layers: one row per unit (for example a farm), one column per layer (for example one seasonal AETI map per season). Every panel of a row shows the unit at the same size, with its polygon outlines, and all panels share one colour scale computed from all layers. scale = "percentile_stretch" keeps one continuous gradient but anchors its colours at percentiles of the data (ticks at the anchor values), so differences stand out more. Row and column titles can be set, for example the seasonal ETc under each season.

  • New wapor_plot_unit_series() draws one time series per unit (seasonal or monthly, character, Date or numeric time), coloured by a group column, with an optional reference series such as ETc as a black dashed line, to see which units stay above or below the reference.

Bug fixes

  • wapor_plot_polygon_grid() drew the continuous colour bar about five times longer than intended (ggplot2 draws a colour bar 5 times legend.key.height), so it was taller than the panel grid. The bar is now legend_size times the default length, and a single row of panels (or a panel with its own scale) gets a bar as tall as its panels.

  • Seasonal analysis on local files (data_source = "local") no longer counts the days of a dekad twice. wapor_map() saves dekadal mm/day products as mm/dekad by default (unit_conversion = NULL), but the analysis multiplied every local layer by the days in its dekad again, so seasonal AETI, RET, ETc, precipitation and the indicators built on them were about 10 times too high. The analysis now reads each file’s units and divides out the saved conversion (mm/dekad, mm/month or mm/year); files in mm/day, runs on the WaPOR API, and folders mixing both give the same totals. The WaPOR scale factors (0.1, 0.001) were always applied correctly on read and are unchanged: verified against the raw COG values for L1 AETI, PCP, RET, NPP and L3 AETI, in all processing modes and in wapor_ts().

Rwapor 1.0.3

New features

  • New wapor_plot_polygon_grid() draws a raster as a grid of small maps, one panel per polygon unit (for example one panel per farm, with all of its parcels outlined). Panels are cropped to each unit plus a buffer (metres), square or in the unit’s own shape, and laid out per_page to a page; each page can be saved as a PNG. The colour scale is shared by all panels by default and computed from the whole raster: a continuous gradient (2nd to 98th percentile, full range or your limits), percentile classes, or your own class breaks. common_scale = FALSE gives every panel its own scale. Needs ggplot2; own scales and true shapes also need patchwork.

Bug fixes

  • The registered peff_green_blue indicator step (wapor_get_indicator_step("peff_green_blue")) now sums the monthly green/blue splits, like wapor_run_seasonal_analysis() since 1.0.2. It still split the seasonal totals, so a direct call could give more green and less blue water than the engine.

Rwapor 1.0.2

Bug fixes

  • Seasonal green and blue water (green_water, blue_water from wapor_run_seasonal_analysis()) are now the sum of the monthly splits: green = sum over months of min(AETI, Peff), blue = sum over months of max(0, AETI - Peff). Before, the split was applied to the seasonal totals, which let surplus rain in a wet month offset irrigation in a dry month, so green water was overestimated and blue water underestimated whenever wet and dry months fell in the same season. Green + blue still equals seasonal AETI. Seasons that span a single month are unchanged. The monthly layers (monthly_green_water, monthly_blue_water) were already correct.

Rwapor 1.0.1

Rwapor now chooses how to process a job from its size, and seasonal totals are computed without altering the values the WaPOR server returns. To reproduce results from before this release, install the tag v1.0.0-final (remotes::install_github("almutaz9000/Rwapor@v1.0.0-final")).

Size-aware processing

  • New wapor_plan_processing() estimates the memory a job needs and chooses a mode: "memory" (whole area at once), "stream" (whole area, dekads read in batches) or "tiled" (square tiles assembled with a VRT). The plan reports its estimate, budget and reasons. The budget is half the free RAM divided by the number of future workers; override it with options(Rwapor.memory_budget_mb = ...) and the mode thresholds with options(Rwapor.plan_thresholds = ...).
  • wapor_run_seasonal_analysis() (via config$processing), wapor_map() and wapor_ts() gain processing = c("auto", "memory", "stream", "tiled"), defaulting to "auto". batch_size now defaults to NULL (chosen by the planner); an explicit number still wins.
  • All modes run the same window kernel, so results are identical whichever mode runs (tested to 1e-6 relative). A small farm runs in memory without tiling; a large 20 m scheme is tiled automatically. Tiles run in parallel under the active future::plan(), with terra and GDAL caches divided per worker.
  • wapor_run_seasonal_analysis_tiled() is now a thin wrapper that forces tiled mode. It keeps the run manifest, resume, checksums and COG output.
  • The dashboard’s “Optimize Memory” checkbox is replaced by a processing-mode selector and a badge showing the planned mode and memory estimate.
  • config$keep_intermediates (default TRUE in memory mode, FALSE otherwise) controls whether dekadal_stacks and season_weights are returned. Registered indicator steps that read ctx$stacks or ctx$season_weights still get them; they are built on first access.

Accuracy (results can differ from 1.0.0)

  • Seasonal and monthly totals are summed at each source’s native resolution and resampled once onto the analysis grid, instead of resampling every dekad.
  • Continuous variables now use nearest-neighbour resampling by default (was bilinear), so every output value is a raw server value or a sum of raw values. AgERA5-derived results (RET, ETc, precipitation, Peff, adequacy, green and blue water) change for most pixels; AETI, NPP and T on their native grid do not. Use config$resampling_method = list(ret = "bilinear") to interpolate.
  • New config$min_coverage (default 1): a pixel is NA unless every dekad in its season has data. Previously missing dekads were silently counted as 0 mm. The per-variable share of season days with data is returned in results$coverage.
  • When aoi_region is not supplied, the analysis area now defaults to the crop mask (or season raster) extent, with a log message. Previously it used the full extent of the first source file, which for WaPOR L1 is the whole globe.
  • With bilinear resampling, 1.0.0 cropped each source to the area before resampling, which biased the outer one to two rows and columns. Sources are now read with a halo of neighbouring cells.
  • ETc for cross-year seasons (end day of year before start) is now computed; 1.0.0 dropped those profiles.
  • Per-pixel area uses terra::cellSize() (exact ellipsoidal area) instead of a latitude approximation, which slightly changes area-weighted CWP/BWP means.

Bug fixes

  • The tiled engine cropped source rasters with the crop mask’s row/column indices, so any source on a different grid (every remote WaPOR file) produced all-NA tiles without an error (ISS-20260923-001).
  • WaPOR dekad file labels (2023-01-D1, D2, D3) are now matched to their dekads. In 1.0.0, wapor_run_seasonal_analysis() with data_source = "api" stopped with “Missing data for some dekads in the analysis period” for every request (verified against the live API); it now runs.
  • The GDAL capability check at package load always warned that /vsicurl/ was missing (it searched a driver column that does not exist), and options(Rwapor.remote_fallback = "download") therefore always downloaded whole files. curl support is now detected from GDAL’s HTTP driver.

Memory and speed

  • Exact P95 (wapor_calc_p95_aeti()) and Theil index (wapor_calc_theil()) are computed block-wise with bounded memory; results are unchanged.
  • linear_trend() uses closed-form layer arithmetic instead of an R function per pixel; results are unchanged.
  • wapor_masked_sum(), wapor_map() seasonal mode and the crop-mask checks no longer build whole-stack temporaries or read full rasters into R.
  • wapor_map() writes tiled, LZW-compressed GeoTIFFs (BigTIFF when needed) in a single pass; wapor_ts() no longer crops before polygon extraction and computes seasonal weighted means in one extraction pass.
  • The GDAL HTTP chunk size is matched to each job’s window size (256 KB to 10 MB) instead of a fixed 10 MB.
  • wapor_suggest_tile_size() now assumes 8-byte values (terra’s in-memory type) and gains n_vars and overhead arguments.

Rwapor 1.0.0 (development)

Core geospatial processing

  • Tiled seasonal engine now uses deterministic square tiles, a versioned run_manifest.json, per-tile GeoTIFF/COG assets, checksums, and VRT assembly. Completed tiles can be resumed; mismatched manifests are refused.
  • Source rasters are windowed onto each tile before analysis so the engine does not load the full AOI stack. Tile-local block reducers compute AETI, RET, PCP, Peff, ETc, adequacy, biomass, and related indicators one layer at a time. Remote-COG fixtures and a tile-vs-AOI memory benchmark cover /vsicurl/ windowing.
  • Incomplete seasonal coverage now fails unless partial = TRUE.
  • mosaic_all supports multiple variables and periods, and wapor_ts() extracts every intersecting L3 source. Shiny no longer preselects the first of several L3 codes and offers mosaic-all.
  • Closing a dashboard browser tab no longer calls stopApp().
  • wapor_write_cog() publishes atomically through a .partial.tif file and applies LZW compression, datatype predictors, overviews, and a BigTIFF policy.

Monitoring database (DuckDB)

  • wapor_save_raster_blobs() rewritten: pending dates are resolved from URL metadata before any network open (avoids redundant vsicurl calls for already-saved dates), and remaining layers for one variable are opened as a single batched /vsicurl/ multi-band stack instead of one dataset handle per layer.
  • Each saved raster now writes both the existing in-memory compressed GeoTIFF blob (backward-compatible read path) and a file-backed COG under a new sibling <db>_raster_store/ directory, recorded in monitoring_rasters.raster_path together with gdal_version, terra_version, and band_count provenance columns.
  • monitoring_metadata now carries a schema_version with an automatic migration path (.wapor_monitoring_migrate()), and a raster_grid_registry table rejects mixed-resolution writes for the same variable.
  • farm_timeseries gained a season_id column (part of the primary key) and now upserts via INSERT OR REPLACE instead of failing on overlapping reruns.

Shiny dashboard

  • The Download tab sidebar is now a 5-step numbered accordion wizard (Project, Variables, Period, L3 Region, Run) instead of a flat two-panel layout.
  • New Dual Compare tab (mod_dual_map.R): two independent Leaflet maps kept in pan/zoom sync via leaflet.extras2::addLeafletsync(), for side-by-side raster comparison distinct from the existing single-map overlay “Dual Compare” mode inside the Visualisation tab.
  • app.R now guards all Shiny/dashboard package dependencies with a clear install message instead of a bare library() failure, and .onLoad configures GDAL unconditionally rather than only when an environment variable is set.
  • Dashboard: grouped variable choices, save/load analysis configuration to JSON, and user-adjustable stress thresholds for the monitoring module.

Testing & documentation

  • All network-dependent test files are guarded with skip_if_wapor_offline() so a machine without internet access gets a clean skip instead of a false failure.
  • Fixed a stale live-API test (wapor_map no longer accepts multiple variables in seasonal mode by design; the test now asserts the documented error and exercises the multi-variable workflow via one wapor_map() call per variable).
  • Removed a redundant intermediate warning in download_seasonal_rasters() that duplicated information already in the final aggregated “INCOMPLETE” warning/error.
  • Regenerated stale man/ pages that had drifted from code (wapor_ts, wapor_map, wapor_init_monitoring_db, wapor_save_raster_blobs, wapor_suggest_tile_size).
  • README installation section restructured to the install -> verify -> quick start convention, with a fixed CI badge branch reference and citation version bump to 1.0.0.

Rwapor 0.9.9

Download Robustness

  • Replaced the disk URL-cache key (.wapor_url_hash()) with a proper SHA-256 digest (digest::digest()). The previous byte-sum-based checksum could collide for different queries whose request strings were character permutations of each other, silently serving cached URLs for the wrong date range within the 24h cache TTL.
  • Added retry-with-backoff to the seasonal raster loader (download_seasonal_rasters()), matching the retry behavior already used by the non-seasonal wapor_ts() path, so a single transient /vsicurl/ failure no longer drops an entire resolution group.
  • download_seasonal_rasters() now reconciles the download plan against what was actually loaded: any code group with no URLs, any plan row that fails to match a URL, or any raster stack that fails to load after retries is tracked and surfaced as a single clear warning (with the exact missing period IDs and day-coverage shortfall) instead of being silently dropped. The missing periods are also exposed via attr(result, "missing_periods") on wapor_ts(seasonal = TRUE) output.
  • API request retries (collect_responses(), metadata pagination) now use exponential backoff, honor a numeric Retry-After response header, and treat HTTP 429/5xx as transient, instead of a flat 2-second delay.

Fixes and Documentation

  • Fixed broken code-fence in README Example 2 that caused Example 3 to render incorrectly on GitHub.
  • Corrected README Example 3 to use wapor_analysis_pipeline() — the function wapor_run_seasonal_analysis() referenced previously does not exist.
  • Removed two dead internal links in README (.github/L3-AUTO-DETECTION.md and docs/REPOSITORY_STRUCTURE.md).
  • Removed stale importFrom directives for stats::coef, stats::lm, utils::capture.output, and utils::getFromNamespace that produced R CMD CHECK notes.
  • Fixed wapor_analysis_pipeline() to respect config$ref_year instead of silently deriving it from the start date.
  • Added on.exit() guard for temp-file cleanup in wapor_map() to prevent orphaned files when an error occurs mid-processing.
  • Fixed duplicate ID / identifier column in wapor_ts() output.
  • Added .Deprecated() call for the download_locally parameter in wapor_ts().
  • Resolved @export + @keywords internal contradiction on internal helpers.

Rwapor 0.9.8

Download and Unit Conversion

  • Hardened unit-conversion behavior for wapor_map() and wapor_ts() to correctly handle dekadal, monthly, and annual products without double-scaling.

Rwapor 0.9.7

Major Synchronization and Optimization

This release synchronizes the local development features with the latest remote optimizations. It includes optimized zonal statistics, enhanced monitoring module, and improved unit conversion logic.

Breaking Changes

  • All public functions now use the wapor_* prefix consistently. The old rwapor_* aliases (e.g. rwapor_harmonize_crop_mask, rwapor_build_daily_kc) have been removed. See the migration guide below.

Function Renames (from pre-1.0 development versions)

Old name New name
rwapor_harmonize_crop_mask() wapor_harmonize_raster()
rwapor_harmonize_to_template() wapor_harmonize_raster()
rwapor_build_season_weights_dekad() wapor_build_season_weights()
rwapor_calc_seasonal_aeti_masked() wapor_calc_seasonal_aeti()
rwapor_calc_seasonal_ret_masked() wapor_calc_seasonal_ret()
rwapor_get_crop_defaults() wapor_crop_defaults()
rwapor_build_daily_kc() wapor_build_kc()
rwapor_calc_seasonal_etc_incremental() wapor_calc_seasonal_etc()
rwapor_apply_masked_sum() wapor_masked_sum()

Repository Cleanup

  • Removed development-only test scripts from tests/ root (ad hoc benchmarks, integration probes, fix verifications). Formal test suite lives in tests/testthat/.
  • Removed archived_reports/ development notes.
  • Removed memories/ agent session files.
  • Removed root-level debug scripts (check_db_summary.R) and stray data files (Pivots_savola2.geojson).

Documentation

  • Updated README with correct function names and a complete step-by-step seasonal analysis tutorial.
  • Updated installation instructions (removed duplicate section).
  • Added tidyverse lifecycle badge update from experimental to stable.

Rwapor 0.9.5

Changes

Shiny Dashboard

  • REMOVED: Split-screen slider (swipe) display mode from Dual Raster Comparison. The “Dual Raster Display” panel now offers “Overlay” and “Intersection Only” modes.
  • FIXED: Debug cat() statements removed from Visualization module server-side renderUI outputs.
  • FIXED: Added missing viridisLite and RColorBrewer packages to Suggests in DESCRIPTION.
  • FIXED: Removed stale raster_left/raster_right layer-group clearing calls that were only used by the removed swipe mode.

Rwapor 0.9.3

Major Enhancements

Shiny Dashboard Improvements

  • NEW: L3 Region Auto-Detection System
    • Automatically detects which L3 regions overlap with user-defined AOI
    • Filters L3 region dropdown to show only relevant regions
    • Auto-selects when exactly one region overlaps
    • Provides real-time feedback messages about detection status
    • Supports vector files, raster files, and drawn geometries as AOI
    • Leverages wapor_guess_region() for spatial intersection analysis
  • REORGANIZED: Download workflow UI
    • AOI selection moved immediately after Project Folder
    • More intuitive linear workflow: Folder → AOI → Variables → Time Period
    • Better guidance for users selecting L3 variables
  • NEW: Comprehensive Shiny dashboard documentation

Analysis Pipeline Refactoring

  • NEW: wapor_analysis_pipeline() - Modularized seasonal crop water productivity analysis pipeline
    • Orchestrates data loading, harmonization, seasonal aggregation, and indicator computation
    • Can be run from Shiny or standalone scripts
    • Supports both API streaming and local file modes
    • Enhanced error logging with detailed diagnostics
    • Progress callback support for integration with UI
    • Automatic output saving (rasters + CSV summaries)

Validation Framework

Spatial Anomaly Detection

  • NEW: wapor_detect_aeti_anomalies() - Identify water stress anomalies
    • Flags pixels with AETI significantly below class median
    • Configurable threshold (default 50% of median)
    • Per-class anomaly statistics
    • Generates anomaly maps for visualization
  • NEW: wapor_detect_compound_anomalies() - Multi-indicator anomaly detection
    • Combines AETI, adequacy, and yield indicators
    • Identifies areas with compound stress
  • NEW: wapor_detect_zscore_anomalies() - Statistical anomaly detection using z-scores
  • NEW: wapor_detect_spatial_hotspots() - Spatial cluster analysis for stress areas

Multi-Season Comparison

  • NEW: wapor_compare_seasons() - Compare multiple season analysis results
    • Side-by-side comparison of indicators across seasons
    • Supports both overall and per-class comparisons
    • Automatic change percentage calculation for 2-season comparisons
  • NEW: wapor_trend_analysis() - Linear trend analysis for 3+ seasons
    • Computes slopes, R-squared, and p-values
    • Supports both overall and per-class trends
  • NEW: wapor_export_comparison_report() - Export formatted comparison reports

Performance Improvements

  • Parallel path resolution for multiple variables
  • Enhanced harmonization caching with intelligent invalidation
  • Memory-efficient incremental aggregation option
  • Optimized ETc computation with unique profile deduplication

User Experience Enhancements

  • Detailed progress feedback during analysis
  • Enhanced error logging with stack traces and configuration snapshots
  • Improved validation messages with specific recommendations
  • Better handling of missing data with auto-download suggestions

Shiny Dashboard Improvements

  • Modularized analysis module for easier maintenance
  • Missing data detection with one-click download
  • Pre-run validation with collapsible diagnostic panel
  • Enhanced error recovery with log file generation
  • Improved progress notifications with granular updates

Bug Fixes

  • Fixed raster harmonization cache invalidation on file re-upload
  • Improved handling of cross-year seasons in Julian day calculations
  • Better error messages for failed API requests
  • Fixed memory leaks in long-running analysis sessions

Documentation

  • Added comprehensive examples for new validation functions
  • Updated analysis workflow documentation
  • Added anomaly detection vignette examples
  • Improved function documentation with use case examples

Rwapor 0.9.2

Previous Release

See previous release notes for 0.9.2 and earlier versions.

Rwapor 0.1.0

Initial Release

This is the first release of Rwapor, an R package for downloading and processing WaPOR and AgERA5 data from the FAO GIS Manager API. ### New Features

  • wapor_map(): Download and save raster maps for specified regions and time periods. Now supports optional batching and parallel chunk processing.
  • wapor_ts(): Extract time series with zonal statistics for polygons. Now supports parallel batching and custom polygon identifiers.
  • run_wapor(): Launch an interactive Shiny application for data selection and visualization.
  • wapor_fix_proj(): Automatically resolve PROJ_LIB environment variable conflicts on Windows.
  • wapor_generate_urls(): Generate download URLs for WaPOR/AgERA5 resources (memoized).
  • parse_region(): Parse region inputs (bounding box, vector file, or L3 code).
  • get_date_info(): Extract date information from WaPOR URL filenames.
  • df_unit_convertor(): Convert DataFrame values between temporal units.
  • raster_unit_convertor(): Convert raster values between temporal units.
  • get_variable_metadata(): Retrieve variable metadata with memoization.

Data Support

  • WaPOR Level 1 (L1) variables: AETI, E, I, NPP, PCP, GBWP, NBWP
  • WaPOR Level 2 (L2) variables: AETI, E, I, NPP, T, GBWP, NBWP
  • WaPOR Level 3 (L3) regional data support
  • AgERA5 climate variables: ET0, TMIN, TMAX, SRF, WS, PF

Temporal Resolutions

  • Daily (E)
  • Dekadal (D) - 10-day periods
  • Monthly (M)
  • Annual (A)

Infrastructure

  • Parallel download support via furrr package
  • Progress reporting via progressr package
  • API response caching via memoise package
  • Unit conversion between day/dekad/month/year