Overview

This workflow describes the GetINSPIREd Cloud Service Client process. This workflow will allow the client to publish their data through data.gov.uk and have it validated through the INSPIRE programme.

Here is a brief summary of the workflow tasks, also see the diagram below:

1. Client provides some data and a spreadsheet to a known location on their server.

2. Astun provides a GeoNetwork-based solution to the Client.

3. Client registers with data.gov.uk.

4. Client validates metadata using GeoNetwork.

5. Astun requests data.gov.uk harvesting.

6. Client validates metadata.

7. Success - data.gov.uk acceptance - compliance complete.

Image Removed

Documents

You should receive the first two Client specific documents, the remaining documentation is provided on-line and should cover everything else you need for configuring and using the GetINSPIREd Cloud Service:

1. The Client Configuration document - provides details that are specific to you.
2. The Client Template for Inspire spreadsheet - you'll have to add basic information to this document.
3. This document (Workflow Documentation Client Steps) - provides a description of the Workflow: i.e. what you should be doing and when.
4. GeoNetwork User Guidance - training material on how to use GeoNetwork and validate Metadata.
5. INSPIRE - Validation - common validations errors and how to fix them.
6. Harvesting and Validation on data.gov.uk - guidance on how to get your metadata accepted by data.gov.uk.

Please check that you have all the documentation to hand.

Reference

There is a lot of guidance available on the process outlined below, most of which can be found at this link:

http://data.gov.uk/location/guidance_and_tools

...

You should place your data in the location on your network specified by the Client Configuration document. If you are running iShare, this will generally this will be on a server already running the iShare Web Service application and the PostgreSQL service - otherwise Astun will provide an FTP location.

Some rules about the data;

• The geographic data should be provided as .shp or .tab files. 
• There should not be any sub-folders.
• File names should be lower case, contain only alphanumeric or underscore characters. Each file name should start with a letter and not end with an underscore. 
• The data should be provided in OSGB 1936/ British National Grid (ESPG:27700).
• There should be no geometry errors in the data.

You should be in receipt of a spreadsheet called the Client Template for INSPIRE. This document details what are the metadata elements are required under INSPIRE. In this document there is a worksheet called 'Basic Requirement for Client' - this is the section that you should fill in before this task is complete. This section holds generic information for the service such as name of the client organisation, and email address of the contact person. It will also contain the WFS & WMS online resources, which you can take from the Client Configuration document.

Additionally to the service information, you should provide an abstract and keyword list for each of the datasets. The record_name for a data set will match the file name, e.g. listed_buildings.shp will have a record_name of 'listed_buildings'.

Once you have added all the necessary information export the 'Basic Requirement for Client' worksheet as a csv file and place it in the same folder as the data.

Once done you can notify Astun that this task has been completed.

...

Astun staff will now perform the following tasks;

1. Import the data into a database on the server.
2. Run a migration check on the data that includes null geometries, number of geometries, geometry validity and extent of geometries, and pass back to the client if there are errors.
3. If the data / WMS / WFS is to be hosted by the client Astun will create an iShare INSPIRE overlay mapsource. If you are running iShare, this can be hidden if you wish so it is not visible in MyMaps. If Astun are to provide the hosting the data will be transferred to the Astun servers and the INSPIRE mapsource will created there.
4. Check the external WMS / WFS endpoints for the mapsource.
5. Create a group (for the authority) and user account(s) (for individuals) for you in GeoNetwork. 
6. Create metadata by harvesting from the WMS endpoint into GeoNetwork.

...

• Got existing metadata? If you have existing metadata to transfer to the Metadata Catalog, discuss this with your Astun Consultant who will advise on the best approach for transferring and converting the metadata to Gemini 2.3
  
• Starting from scratch? If you don't have any existing metadata, discuss with your Astun Consultant the approaches that are available to you for creating new metadata.

In both cases, your Astun Consultant will create a Metadata Portal for you, provide details on how to access it, and how to create a user account. This will either be pre-populated for you with your existing metadata, or you can create new records from scratch using these instructions.

All records, whether newly created or transferred from elsewhere, must be validated in GeoNetwork before they can be published to data.gov.uk. See Create a UK Gemini 2.3 compliant Metadata Record in GeoNetwork#6.Validatearecord for details.

Registering with data.gov.uk

You will need to register your authority as a publisher with data.gov.uk. Once you're registered, let Astun know, and we will then request to become an editor for your authority at data.gov.uk. This will give us the ability to organise harvesting at data.gov.uk on your behalf. This process can take some time but fortunately it is independent of validating the metadata in GeoNetwork, so should be started as soon as possible.

...

Your metadata has now been loaded into GeoNetwork, and where possible mandatory information has been generated. It's your task to now validate your metadata against the INSPIRE metadata standard, and make any corrections and additions to the metadata which are required.

You should now log on to geonetwork.astuntechnology.com and start the process of validating your metadata.

Use the Getting started with GeoNetwork document. This document explains the process of logging into GeoNetwork and how to complete and validate your metadata. Your user name and password will be in the Client Configuration document. It is recommended that you change your password after the first login.

When you start to validate your metadata you will undoubtedly hit a few validation errors. Common validation errors and the appropriate fixes have been captured in the INSPIRE - Validation document.

Once validation for all datasets and the service record is complete please notify Astun.

...

Astun will now prepare your metadata for harvesting on data.gov.uk. To do this, Astun exports the metadata for each dataset and service as an xml file. These will be held in a Web Accessible Folder (WAF), and the URL  for this can be found in the Client Configuration document. For reference the process of creating the Web Accessible Folder can be found in the Harvesting and Validation on data.gov.uk document.

Astun will apply to be an administrator for your organisation (through their helpdesk) and then organise the harvest from data.gov.uk and will notify you that this task has been completed.

...

Be prepared for validation failures as data.gov.uk's interpretation of the INSPIRE standard differs slightly from GeoNetwork's. The number of errors should be relatively small though. The Harvesting and Validation on data.gov.uk document contains the common errors that have been observed so far. Each error validation failure needs to fixed by editing the metadata in GeoNetwork. After you've corrected the errors, request Astun to re-run the data.gov.uk harvesting process.

...

Data.gov.uk harvesting

Astun will configure a data.gov.uk harvesting node that points at the csw-inspire endpoint for your metadata catalog and will run through the harvest for you to ensure there are no additional errors. If there are, Astun will work with you to resolve them and redo the harvest. Once records are being harvested correctly, Astun can either set up the harvesting job to run as a scheduled task, or it can be run manually. In either case, you should then be able to look after the process of harvesting yourselves.