In this section:

Introduction

Only validated and verified records enter the Butterflies for the New Millennium (BNM) and National Moth Recording Scheme (NMRS) databases. This ensures data quality to inform our conservation, research, and advocacy objectives. County Recorders and their Verification Assistants are responsible for performing validation on the data. This includes checking the date format of the records, grid references, location names and species names for accuracy. Butterfly Conservation recommends the use of the Butterfly and Moth Spreadsheet Validators for the validation process of checking records.

In this section we outline the essential stages of validation and recommended tools.

How to validate records

  1. Are the dates in the correct format – UK not US?
    All records entered to iRecord and Butterfly Conservation’s iRecord-linked surveys use the UK date format dd/mm/yyyy e.g. 01/06/2023.

    More care is needed when checking record dates in Excel, as under certain conditions Excel will transpose UK format dates to US format dates (e.g. 01/06/2023 to 06/01/2023.) Please check the locale of the date format in any spreadsheets you receive from your recorders. If your recorders are using the recommended spreadsheet template or equivalent the date format will be set correctly and the US date issue should not arise.

    You can use the Butterfly or Moth Validation Spreadsheet, created by one of our dedicated volunteers, in the resources section below, to check incoming data sheets’ date formats.
     
  2. Are the grid references within the vice-county / recording area?
    In iRecord, as registered verifier, you are provided access to records within the boundary of your vice-county or recording area, so grid references will be within the correct area. One exception may be when the record falls on the boundary of two recording areas. In this case, the record may appear in the verification grid of a neighbouring area(s). Please liaise with the neighbouring County Recorder(s) regarding who is ‘claiming’ the record.

    We recommend you use NBN Record Cleaner to check for spatial outliers in the data, see resources section below.
     
  3. Do the grid references and location names match up?
    Grid references should match up with a location name used on an Ordnance Survey map. “My garden”, “my house”, “my field” type locations and postal addresses should not be used. Any records with these types of locations need to be referred back to the recorder for editing of location name.
     
  4. Are the correct species names being used?
    iRecord uses the current taxon checklist/dictionary for Lepidoptera, this is based on the Checklist of the Lepidoptera of the British Isles by David J. L. Agassiz, Stella D. Beavan and Robert J. Heckford, and subsequent updates.

    The Checklist of the Lepidoptera of the British Isles contains the current scientific names & codes of butterflies and moths that are resident and transitory resident species along with species that are believed to be extinct and immigrants. It also includes adventive species- species that have been imported and found in the wild. The book is available to buy from booksellers and there is also a downloadable Excel spreadsheet which is regularly updated.

    In iRecord-linked surveys, the species names are auto-filled with Checklist standard names during data entry, thus minimising spelling mistakes you receive. Spreadsheets can be more problematic for species name spelling mistakes so additional caution is required. Supplying a spreadsheet template to your local recorders can help minimise this issue, see resources below.

    You can use the Butterfly or Moth Validation spreadsheet, in the resources section below, to check incoming data sheets’ species names.

Excel tools for cleaning incoming data

You will receive some records from recorders in spreadsheet format. Some inconsistencies can be present that you want to correct before you import records to your database. Excel has some useful inbuilt tools.

This Microsoft Excel support page will show you how to format dates in Excel. There can be redundant spaces at the end of cell entries in spreadsheets, which can cause database problems and use up unnecessary data space.

Incompatible characters and unwanted spaces can creep into spreadsheets. This Excel sheet contains a formula that will shorten grid references to six-figure and will clean and trim hidden characters. You can also refer to the following guide, which will show you how to trim extra spaces in Excel. And the following guide will help you Clean non-printable characters in Excel.

Another useful shortcut when working with spreadsheet data is the Lookup in Excel.

This spatial converter Excel add-in will convert GB and Irish Grid references to Eastings and Northings and Lat & Long. Please download the add-in and refer to these instructions.

It can also convert GB Grid references to 1km, 2km,10km and convert Eastings and Northings to various GB Grid reference resolutions.

 

Summary

Validation checks are vital for ensuring that the data are accurate prior to verification. Correct dates and grid references are important for the verification checks. The Butterfly and Moth Spreadsheet Validators and NBN Record Cleaner are great tools to use for validation checks.

If any of the grid reference, location name or species name is incorrect or missing, please contact the recorder and request the additional information required. Details on how to do this are in section 8.