rsyncrosim: introduction to spatial data

Source: vignettes/a04_rsyncrosim_vignette_spatial.Rmd

This vignette will cover incorporating spatial data into SyncroSim models using the rsyncrosim package within the SyncroSim software framework. For an overview of SyncroSim and rsyncrosim, as well as a basic usage tutorial for rsyncrosim, see the Introduction to rsyncrosim vignette. To learn how to use iterations in the rsyncrosim interface, see the rsyncrosim: introduction to uncertainty vignette. To learn how to link models using pipelines in the rsyncrosim interface, see the rsyncrosim: introduction to pipelines vignette.

SyncroSim Package: helloworldSpatial

To demonstrate how to use spatial data in the rsyncrosim interface, we will be using the helloworldSpatial SyncroSim package. helloworldSpatial was designed to be a simple package to show off some key functionalities of SyncroSim, including the ability to use both spatial and non-spatial data.

The package takes 3 inputs, mMean, mSD, and a spatial raster file of intercept (b) values. For each iteration, a value m, representing the slope, is sampled from a normal distribution with mean of mMean and standard deviation of mSD. These values are run through 2 models to produce both spatial and non-spatial outputs.

Infographic of helloworldSpatial package
Infographic of helloworldSpatial package

For more details on the different features of the helloworldSpatial SyncroSim package, consult the SyncroSim Enhancing a Package: Integrating Spatial Data tutorial.

Setup

Install SyncroSim

Before using rsyncrosim you will first need to download and install the SyncroSim software. Versions of SyncroSim exist for both Windows and Linux.

Installing and loading R packages

You will need to install the rsyncrosim R package, either using CRAN or from the rsyncrosim GitHub repository. Versions of rsyncrosim are available for both Windows and Linux. You may need to install the raster and rgdal packages from CRAN as well.

In a new R script, load the necessary packages. This includes the rsyncrosim and terra R packages.

# Load R packages
library(rsyncrosim)  # package for working with SyncroSim
## Warning: package 'rsyncrosim' was built under R version 4.3.3
library(terra)       # package for working with spatial data
## terra 1.7.55
## 
## Attaching package: 'terra'
## The following object is masked from 'package:rsyncrosim':
## 
##     project

Connecting R to SyncroSim using session()

Finish setting up the R environment for the rsyncrosim workflow by creating a SyncroSim Session object. Use the session() function to connect R to your installed copy of the SyncroSim software.

mySession <- session("path/to/install_folder")      # Create a Session based SyncroSim install folder
mySession <- session()                              # Using default install folder (Windows only)
mySession                                           # Displays the Session object
## class               : Session
## filepath [character]: C:/Program Files/SyncroSim
## silent [logical]    : TRUE
## printCmd [logical]  : FALSE
## condaFilepath [NULL]:

Use the version() function to ensure you are using the latest version of SyncroSim.

version(mySession)
## [1] "2.5.11"

Installing SyncroSim packages using addPackage()

Install helloworldSpatial using the rynscrosim function addPackage(). This function takes a package name as input and then queries the SyncroSim package server for the specified package.

# Install helloworldSpatial
addPackage("helloworldSpatial")
## Package <helloworldSpatial> installed

helloworldSpatial should now be included in the package list returned by the package() function in rsyncrosim:

# Get list of installed packages
package()
##                name                                   description version
## 1 helloworldSpatial Example demonstrating how to use spatial data   1.1.0
##                                                   location status
## 1 C:\\Users\\sarah\\SyncroSim\\Packages\\helloworldSpatial     OK

Create a modeling workflow

When creating a new modeling workflow from scratch, we need to create objects of the following scopes:

For more information on these scopes, see the Introduction to rsyncrosim vignette.

Set up Library, Project, and Scenario 

# Create a new Library
myLibrary <- ssimLibrary(name = "helloworldLibrary.ssim",
                         session = mySession,
                         package = "helloworldSpatial",
                         overwrite = TRUE)

# Open the default Project
myProject = rsyncrosim::project(ssimObject = myLibrary, project = "Definitions")

# Create a new Scenario (associated with the default Project)
myScenario = scenario(ssimObject = myProject, scenario = "My spatial scenario")

View model inputs using datasheet()

View the Datasheets associated with your new Scenario using the datasheet() function from rsyncrosim.

# View all Datasheets associated with a Library, Project, or Scenario
datasheet(myScenario)
##       scope                                    name                displayName
## 1   library                             core_Backup                     Backup
## 2   library                        core_CondaConfig        Conda Configuration
## 3   library                           core_JlConfig        Julia Configuration
## 4   library                         core_LNGPackage   Last Known Good Packages
## 5   library                    core_Multiprocessing            Multiprocessing
## 6   library                            core_Options                    Options
## 7   library               core_ProcessorGroupOption    Processor Group Options
## 8   library                core_ProcessorGroupValue     Processor Group Values
## 9   library                           core_PyConfig       Python Configuration
## 10  library                            core_RConfig            R Configuration
## 11  library                           core_Settings                   Settings
## 12  library                          core_SysFolder                    Folders
## 13  library                       corestime_Options            Spatial Options
## 14  project                         core_AutoGenTag       Auto Generation Tags
## 15  project                 core_RunSchedulerOption      Run Scheduler Options
## 16  project               core_RunSchedulerScenario    Run Scheduler Scenarios
## 17  project                          core_StageName               Stage Groups
## 18  project                         core_StageValue            Stages by Group
## 19  project                        core_Transformer                     Stages
## 20  project                        corestime_Charts                     Charts
## 21  project              corestime_DistributionType              Distributions
## 22  project          corestime_ExternalVariableType         External Variables
## 23  project                      corestime_MapFacet               Map Faceting
## 24  project                          corestime_Maps                       Maps
## 25 scenario                    core_AutoGenTagValue Auto Generation Tag Values
## 26 scenario                           core_Pipeline                   Pipeline
## 27 scenario             corestime_DistributionValue              Distributions
## 28 scenario                      corestime_External                   External
## 29 scenario         corestime_ExternalVariableValue         External Variables
## 30 scenario               corestime_Multiprocessing    Spatial Multiprocessing
## 31 scenario        helloworldSpatial_InputDatasheet             InputDatasheet
## 32 scenario helloworldSpatial_IntermediateDatasheet      IntermediateDatasheet
## 33 scenario       helloworldSpatial_OutputDatasheet            OutputDatasheet
## 34 scenario            helloworldSpatial_RunControl                Run Control

From the list of Datasheets above, we can see that there are four Datasheets specific to the helloworldSpatial package, including an Input Datasheet, an Intermediate Datasheet, an Output Datasheet, and a Run Control Datasheet.

Configure model inputs using datasheet() and addRow()

Currently our input Scenario Datasheets are empty! We need to add some values to our input Datasheet (InputDatasheet) and run control Datasheet (RunControl) so we can run our model. Since this package also uses pipelines, we also need to add some information to the core Pipeline Datasheet to specify which models are run in which order. For more information on using pipelines, see the rsyncrosim: introduction to pipelines vignette and the SyncroSim Enhancing a Package: Linking Models tutorial.

Input Datasheet

First, assign the contents of the input Datasheet to a new data frame variable using datasheet(), then check the columns that need input values.

# Load input Datasheet to a new R data frame
myInputDataframe <- datasheet(myScenario,
                              name = "helloworldSpatial_InputDatasheet")

# Check the columns of the input data frame
str(myInputDataframe)
## 'data.frame':    0 obs. of  3 variables:
##  $ mMean              : num 
##  $ mSD                : num 
##  $ InterceptRasterFile: chr

The input Datasheet requires three values:

  • mMean : the mean of a normal distribution that will determine the slope of the linear equation.
  • mSD : the standard deviation of a normal distribution that will determine the slope of the linear equation.
  • InterceptRasterFile : the file path to a raster image, in which each cell of the image will be an intercept in the linear equation.

In this example, the external file we are using for the InterceptRasterFile is a simple 5x5 raster TIF file generated using the raster package in R. The file used in this vignette can be found here.

InterceptRasterFile input image
InterceptRasterFile input image

Add these values to a new data frame, then use the addRow() function from rsyncrosim to update the input data frame

# Create input data and add it to the input data frame
myInputRow <- data.frame(mMean = 0, mSD = 4,
                         InterceptRasterFile = "path/to/input-raster.tif")
myInputDataframe <- addRow(myInputDataframe, myInputRow)

# Check values
myInputDataframe
##   mMean mSD      InterceptRasterFile
## 1     0   4 path/to/input-raster.tif

Finally, save the updated R data frame to a SyncroSim Datasheet using saveDatasheet().

# Save input R data frame as a SyncroSim Datasheet
saveDatasheet(ssimObject = myScenario, data = myInputDataframe,
              name = "helloworldSpatial_InputDatasheet")
## Datasheet <helloworldSpatial_InputDatasheet> saved

RunControl Datasheet

The RunControl Datasheet sets the number of iterations and the minimum and maximum time steps for our model. We’ll assign the contents of this Datasheet to a new data frame variable as well and then add then update the information in the data frame using addRow(). We need to specify data for the following four columns:

  • MaximumIteration : total number of iterations to run the model for.
  • MinimumTimestep : the starting time point of the simulation.
  • MaximumTimestep : the end time point of the simulation.

Note: A fourth hidden column, MinimumIteration, also exists in the RunControl Datasheet (default=1).

# Load RunControl Datasheet to an R data frame
runSettings <- datasheet(myScenario, name = "helloworldSpatial_RunControl")

# Check the columns of the run control data frame
str(runSettings)
## 'data.frame':    0 obs. of  3 variables:
##  $ MaximumIteration: num 
##  $ MinimumTimestep : num 
##  $ MaximumTimestep : num
# Create RunControl data and add it to the RunControl data frame
runSettingsRow <- data.frame(MaximumIteration = 5,
                             MinimumTimestep = 1,
                             MaximumTimestep = 10)
runSettings <- addRow(runSettings, runSettingsRow)

# Check values
runSettings
##   MaximumIteration MinimumTimestep MaximumTimestep
## 1                5               1              10
# Save run control R data frame to a SyncroSim Datasheet
saveDatasheet(ssimObject = myScenario, data = runSettings,
              name = "helloworldSpatial_RunControl")
## Datasheet <helloworldSpatial_RunControl> saved

Pipeline Datasheet

The helloworldSpatial package also makes use of pipelines to link the output of one model to the input of a second model. To learn more about pipelines, see the rsyncrosim: introduction to pipelines vignette and the SyncroSim Enhancing a Package: Linking Models tutorial.

To implement pipelines in our package, we need to specify the order in which to run the Transformers (i.e. models) in our pipeline by editing the Pipeline Datasheet. The Pipeline Datasheet is part of the built-in SyncroSim core, so we access it using the “core_” prefix with the datasheet() function. From viewing the structure of the Pipeline Datasheet we know that the StageNameID is a factor with two levels: “First Model” and “Second Model”. We will set the data for this Datasheet such that “First Model” is run first, then “Second Model”. This way, the output from “First Model” is used as the input for “Second Model”.

# Load Pipeline Datasheet to an R data frame
myPipelineDataframe <- datasheet(myScenario, name = "core_Pipeline")

# Check the columns of the Pipeline data frame
str(myPipelineDataframe)
## 'data.frame':    0 obs. of  2 variables:
##  $ StageNameID: Factor w/ 2 levels "First Model",..: 
##  $ RunOrder   : num
# Create Pipeline data and add it to the Pipeline data frame
myPipelineRow <- data.frame(StageNameID = c("First Model", "Second Model"),
                            RunOrder = c(1, 2))

myPipelineDataframe <- addRow(myPipelineDataframe, myPipelineRow)

# Check values
myPipelineDataframe
##    StageNameID RunOrder
## 1  First Model        1
## 2 Second Model        2
# Save Pipeline R data frame to a SyncroSim Datasheet
saveDatasheet(ssimObject = myScenario, data = myPipelineDataframe,
              name = "core_Pipeline")
## Datasheet <core_Pipeline> saved

Run Scenarios

Setting run parameters with run()

We will now run our Scenario using the run() function in rsyncrosim. If we have a large modeling workflow and we want to parallelize the run using multiprocessing, we can set the jobs argument to be a value greater than one.

# Run the first Scenario we created
myResultScenario <- run(myScenario, jobs = 5)
## [1] "Running scenario [1] My spatial scenario"

After the Scenario has been run, a Results Scenario is created that contains results in the output Datasheets.

View results

The next step is to view the output Datasheets added to the Result Scenario when it was run.

Viewing non-spatial results with datasheet()

First, we will view the non-spatial results within the Results Scenarios. For each step in the pipeline, We can load the result tables using the datasheet() function.

# Load results of first Transformer in Pipeline
resultsSummary <- datasheet(myResultScenario,
                            name = "helloworldSpatial_IntermediateDatasheet")

# View results table of first Transformer in Pipeline
head(resultsSummary)
##   Iteration Timestep         y        OutputRasterFile
## 1         1        1 -130.7026 rasterMap_iter1_ts1.tif
## 2         1        2 -257.0712 rasterMap_iter1_ts2.tif
## 3         1        3 -383.4399 rasterMap_iter1_ts3.tif
## 4         1        4 -509.8085 rasterMap_iter1_ts4.tif
## 5         1        5 -636.1771 rasterMap_iter1_ts5.tif
## 6         1        6 -762.5457 rasterMap_iter1_ts6.tif
# Load results of second Transformer in Pipeline
resultsSummary2 <- datasheet(myResultScenario,
                             name = "helloworldSpatial_OutputDatasheet")

# View results table of second Transformer in Pipeline
head(resultsSummary2)
##   Iteration Timestep       yCum
## 1         1        1  -130.7026
## 2         1        2  -387.7738
## 3         1        3  -771.2137
## 4         1        4 -1281.0222
## 5         1        5 -1917.1993
## 6         1        6 -2679.7450

From viewing these Datasheets, we can see that the spatial output is contained within the IntermediateDatasheet, in the column called OutputRasterFile.

Viewing spatial results with datasheetSpatRaster()

For spatial results, we want to load the results as raster images. To do this, we will use the datasheetSpatRaster() function from rsyncrosim. The first argument is the result Scenario object. Next, we specify the name of the Datasheet containing raster images using the datasheet argument, and the column pertaining to the raster images using the column argument. The results contain many raster images, since we have a raster for each combination of iteration and time step. We can use the iteration and timestep arguments to specify a single raster image or a subset of raster images we want to view.

# Load raster files for first result Scenario with time step and iteration
rasterMaps <- datasheetSpatRaster(
  myResultScenario,
  datasheet = "helloworldSpatial_IntermediateDatasheet",
  column = "OutputRasterFile",
  iteration = 1,
  timestep = 5
  )

# View results
rasterMaps
## class       : SpatRaster 
## dimensions  : 5, 5, 1  (nrow, ncol, nlyr)
## resolution  : 0.4, 0.4  (x, y)
## extent      : -1, 1, -1, 1  (xmin, xmax, ymin, ymax)
## coord. ref. : lon/lat WGS 84 
## source      : rasterMap_iter1_ts5.tif 
## name        : rasterMap_iter1_ts5 
## min value   :           -27.89954 
## max value   :           -23.15529
plot(rasterMaps[[1]])

Viewing spatial results in the SyncroSim Windows User Interface

To create maps using the Results Scenario we just generated, open the current Library in the User Interface and sync the updates from rsyncrosim using the “refresh” button in the upper toolbar. All the updates made in rsyncrosim should appear in the User Interface. We can now add the Results Scenario to the Results Viewer and create our maps. For more information on generating map in the User Interface, see the SyncroSim tutorials on creating and customizing maps

Using rsyncrosim with the SyncroSim Windows User Interface to map spatial data
Using rsyncrosim with the SyncroSim Windows User Interface to map spatial data