/
Speed up moving Data via iShare Studio Tasks

Speed up moving Data via iShare Studio Tasks

Overview

Some jobs which are moving data to or from your Spatial Data Warehouse (SDW) may be taking a lot longer than necessary. The following has been identified as a reason for this as well as proving a solution that can reduce time taken in some example from over an hour to under one minute.

Performance issues in OGR 1.8

We have found that ogr2ogr in OGR 1.8 performs badly over a network. This can be vastly improved by applying the following flags to the Additional parameters for the Workflow Task.

-gt 20000 --config PG_USE_COPY YES

However note that this may fail if you try to load a CSV file with missing values in the data, even if it is correctly formatted - PG_USE_COPY doesn't seem to handle this.

Step-by-step guide

  1. Open iShare Studio and locate the Task that’s taking too long.

  2. Tick the Show expert mode and then add the following into the Additional parameters --config PG_USE_COPY YES -gt 20000

  3. Run the Task again.

 

Related content

Set My Maps Panels as Open by Default
Set My Maps Panels as Open by Default
Read with this
Configure a Spatial Data Transformation Task
Configure a Spatial Data Transformation Task
More like this
Reduce Shapefile Geometries Precision to a level PostgreSQL supports
Reduce Shapefile Geometries Precision to a level PostgreSQL supports
Read with this
Load Polygon & Multi Polygon data using Workflow
Load Polygon & Multi Polygon data using Workflow
More like this
How-To Articles
How-To Articles
Read with this
Load large CSV dataset into PostgreSQL
Load large CSV dataset into PostgreSQL
More like this