Speed up moving Data via iShare Studio Tasks

Owned by Gavin Kelly
Last updated: 12 Apr, 2022
1 min read

Overview

Some jobs which are moving data to or from your Spatial Data Warehouse (SDW) may be taking a lot longer than necessary. The following has been identified as a reason for this as well as proving a solution that can reduce time taken in some example from over an hour to under one minute.

Performance issues in OGR 1.8

We have found that ogr2ogr in OGR 1.8 performs badly over a network. This can be vastly improved by applying the following flags to the Additional parameters for the Workflow Task.

-gt 20000 --config PG_USE_COPY YES

However note that this may fail if you try to load a CSV file with missing values in the data, even if it is correctly formatted - PG_USE_COPY doesn't seem to handle this.

Step-by-step guide

  1. Open iShare Studio and locate the Task that’s taking too long.

  2. Tick the Show expert mode and then add the following into the Additional parameters --config PG_USE_COPY YES -gt 20000

  3. Run the Task again.