Search and rescue helicopter statistics: Background Quality ... - Gov.uk
Statistical Document
6th December 2017
Search and Rescue Helicopter Statistics Background Quality Report
This background quality report relates to the publication of quarterly and annual data from civilian search and rescue helicopter (SARH) operations in the UK. The purpose of this document is to provide users of the statistics with information about the quality of the outputs, measured against different dimensions of statistical quality. As a result, this document helps to demonstrate how the Department complies with the Code of Practice for Official Statistics principle on quality. Section 1: Background to the statistics: Page 2 Section 2: Quality assessment •
Relevance Page 4
•
Accuracy and reliability Page 5
•
Timeliness and punctuality Page 6
•
Accessibility and clarity Page 6
•
Coherence and comparability Page 7
•
Trade-offs between output quality components Page 8
•
Assessment of user needs and perceptions Page 8
•
Performance, cost and respondent burden Page 9
•
Confidentiality, transparency and security Page 9
Section 3: Summary and conclusions: Page 10
RESPONSIBLE STATISTICIAN:
Matthew Tranter
Davita Patel
FURTHER INFORMATION:
Media: 020 7944 4312
Email: [email protected]
Section 1: Background to the statistics Search and rescue helicopters are on standby 24 hours a day to assist those in difficulty, both at sea and over land, as part of a wider search and rescue provision which includes maritime and land based operations. These statistics are collected to provide information on the number and type of SARH taskings. From April 2015, the mix of military and civilian search and rescue capacity, managed by the Ministry of Defence (MOD), began to transition to a single civilian contract. This is now managed by the Maritime and Coastguard Agency (MCA), with statistics on search and rescue helicopters (SARH) reported by the Department for Transport (DfT). A completely separate statistical series has now been established by DfT, covering the civilian bases, from the point at which they opened. As part of the transition to the civilian contract, two new SARH bases were opened in April 2015; two in July 2015; one in October 2015 and two more in January 2016. Since July 2017, all SARH bases have been fully operational across the United Kingdom. The Portland base ceased operation after June 2017. Data is available for this base from its initial operation in April 2015 to its last operation in June 2017. Map of UK SARH bases
Sumburgh Apr 2017
Stornoway July 2017
Inverness Apr 2015
Civilian base now closed Civilian base now open
Prestwick Jan 2016
Humberside Apr 2015
Caernarfon Jul 2015
St Athan Oct 2015
Lydd
Aug 2015
Newquay Jan 2016
Portland
Lee-on-Solent Apr 2017
Closed: June 2017
Search and Rescue Helicopter Statistics - Background Quality Report Page 2
Methodology and Production Between April 2015 and June 2017, data used to produce these statistics was derived from an administrative system used by the helicopter operators (known as iSAR) and provided to DfT by the Maritime and Coastguard Agency. From June 2017, the statistics are based on information recorded by the Aeronautical Rescue Co-ordination Centre (ARCC) based in Fareham. ARCC are responsible for the co-ordination of the search and rescue helicopters, from answering the initial request through to the conclusion of the tasking. These statistics are based on administrative data, with the process broadly as follows: • Following a tasking, the ARCC records incident data into a single spreadsheet. • This spreadsheet is sent to the MCA and DfT on a monthly basis, with MCA checking the incidents captured are correct for contractual purposes. • DfT validates the data to verify that the data items are accurate and consistent. This includes cross checking the details against those entered in the helicopter operator system (iSAR), checking for missing data and any anomalies, and checking that the location type and type of tasking matches with the tasking narrative. These checks are done using Excel and ArcGIS mapping software. • Any anomalies and amendments are sent back to the MCA for verification. DfT then signs off the monthly data for use in production of the statistics, on a quarterly basis. • Once the data is finalised, it is entered into a SQL database, where data tables are exported to Excel and the tables and statistical release are produced for publication. • Each statistical release produced is independently checked by a statistician where any discrepancies are resolved before being approved by the senior statistician. Currently the statistics are published each quarter, with an annual release published just after the end of each financial year (in June). DfT SARH process example
ARCC Data MCA DfT QA of tasking count Making sure all ARCC IDs for each tasking are unique and match with the old data source.
All data fields are included, no blank entries from each variable.
QA of variables (Excel) Tasking category Persons rescued Duration of tasking.
QA of variables (ArcGIS) Color code by base and check for any unusual taskings. E.g. a Sumburgh tasking taking place in South of England. Colour code type of tasking on ArcGIS. Check for any pre-arranged transfers in maritime.
Check that beach/cliff taskings occur near coast, vessel taskings mostly in maritime or rivers etc.
Search and Rescue Helicopter Statistics - Background Quality Report Page 3
Section 2: Quality assessment In this section, the quality of the statistics is considered in relation to the different dimensions of quality as stated in the European Statistical System (ESS) quality framework.
1. Relevance Relevance is the degree to which a statistical product meets user needs in terms of content and coverage.
The statistical outputs presented within SARH statistics include: • A statistical release containing key findings, trends over time and signposts to further information. • Open Document Spreadsheet (ODS) data tables containing information on the number of SARH taskings, location, individual level information etc. • An interactive data map highlighting each tasking in greater detail for the annual statistics. • A document outlining the changes in data sources after April 2017. Known users and uses of the statistics: Within the DfT and MCA, the statistics are used: • • • •
For ministerial briefing and to answer public enquiries; As background to policy development; For monitoring trends in search and rescue activity; and, By analysts in modelling overall search and rescue operations.
Outside of DfT, known users include: • • •
Search and rescue partners, national parks, tourist boards, mountaineering/ walking groups and others monitoring the use of search and rescue services by tourists/ members; Air ambulance charities monitoring SAR activity in their areas; and, Maritime organisations monitoring SAR activity related to their businesses (e.g. near ports).
How well the statistics meet user needs: DfT published the first quarterly SARH statistical release in November 2015 and alongside this, launched an online survey asking users for feedback on the statistical release, which ran until January 2016. A summary of the user response was published and provides information on the extent to which user needs are being met, and where user feedback has been addressed such as incorporating new analysis within the statistical reports. Since the introduction of the first SARH statistics publication, DfT have made a number of changes to better meet user needs. These include: • Changes in data source: using data from the ARCC has enabled DfT to produce new analysis such as the number of people rescued in a helicopter that were not previously available. • An interactive map containing individual level taskings to cater to users wanting specific information about a certain base, National Park, region etc. • Additional analyses including taskings by day of week, region, and a breakdown of location type. DfT will continue to further develop the statistical outputs in response to user feedback. Search and Rescue Helicopter Statistics - Background Quality Report Page 4
2. Accuracy and Reliability Accuracy refers to how close the estimated value in the output is to the true result.
These statistics aim to measure the true number of SARH taskings in the UK. As part of the transition to using data from ARCC, a comparison was made between the two data sources which suggested a consistent number of tasking was being captured by each. A summary was published to inform users about the change of data source. The following summarises potential sources of risks or errors which may arise throughout the process of compiling the SARH statistics. Potential source of error
Risk or error mitigation
Taskings missed, or As all SARH emergency calls are received by the ARCC and recorded into a wrongly included by spreadsheet, missing records are unlikely. In addition, taskings recorded are ARCC validated by MCA for contractual purposes and the data is compared with the iSAR system and checked by MCA. This ensures that any out of scope taskings are also identified. Missing or Inconsistent data or missing variables information are queried by DfT and MCA. inconsistent Where discrepancies are identified in relation to the classification of a tasking, the data items in the description of the tasking entered in the iSAR system is used to resolve them. spreadsheet In the financial year 2016/17 year (when the data was sourced from the iSAR system) approximately 10% of cases in the data supplied to DfT required querying with the MCA. These related mainly to the latitude/longitude coordinates, which are validated by the statistics team using a mapping software (ArcGIS). Production Tabulations and basic checks are also performed on the SARH data to make sure of statistical they are in line with expectation (e.g. we would expect SARH taskings to peak in the publication and data summer months in a given year due to greater leisure activity). The data are also tables checked with previous years, looking for any unexpected changes for each variable. Where uncertainties exist in data outputs, these are highlighted in the statistical releases.
DfT have a close working relationship with the MCA and the ARCC to ensure the quality of the data is maintained. The team at the ARCC are trained in recording incidents and guidance is supplied to those who are responsible for inputting data in order to ensure that this is as consistent as possible. Because of the thorough data validation process undertaken by DfT every month, it is rare for the SARH statistics to be revised.
Search and Rescue Helicopter Statistics - Background Quality Report Page 5
3. Timeliness and Punctuality
Timeliness describes the time between the date of publication and the date to which the data refers, and punctuality describes the time between the actual publication and the planned publication of a statistic.
SARH statistics are currently published quarterly. The production of these reports commences after the final month’s data for a given quarter is received. Publication dates for these statistical releases are agreed at least 2 months in advance and dates are published on the GOV.UK release calendar. We aim to publish the statistical releases around 2 months after the end of the period to which the statistics relate. For example statistics covering the period April to June 2017 were published in August 2017. To date, all SARH statistical releases have been published to the scheduled pre-announced date. Example of timescales for a SARH quarterly release Task Description
Timeline
DfT receive final months ARCC data, from MCA, after initial validation. Validations carried out using Excel and ArcGIS; queries raised to MCA and final data agreed.
Week 2 Week 2 - Week 4
Data amended based on verifications for final months data, and the entire quarter’s data loaded into SQL to produce final tables.
Week 5
Statistical release is produced, including putting together charts, maps and
Week 6
infographics. All tables, charts and figures are quality assured.
Week 7
Sign off required by senior statisticians and the MCA. Documents and tables are finalised and sent to web team ready for publication.
Week 8 Week 9
4. Accessibility and Clarity Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.
Accessibility: The outputs are published on the GOV.UK DfT statistics page in accessible formats: • Statistical releases and are available as PDFs, which are made accessible for those who use assistive technologies. • Data tables are available in ODS file format which can be accessed by using freely available software. The published quarterly tables include information on the number of taskings, type of tasking, location, number of people rescued and duration time. The annual tables, in addition contain information on region, time of day and day of week of taskings. • Individual level raw data tables and interactive data visualisations are also available annually for users interested in record level data on taskings. Search and Rescue Helicopter Statistics - Background Quality Report Page 6
Clarity: The statistical releases use plain language, in which technical terms, acronyms and definitions are defined where appropriate. The main findings are presented using a series of text, charts and maps, with maps used to show the distribution of SARH tasking locations. Full details of the strengths and weaknesses of the data are provided at the end of each statistical report for users who are interested in this. In addition to the statistical releases, further relevant supporting documentation includes a note outlining the changes in data sources from April 2017.
5. Coherence and Comparability Comparability is the degree to which data can be compared over time and domain. Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar.
Comparability: Over time: The statistical releases cover trends since April 2015, the first period for which data are published by DfT. Over this period, the number of civilian bases grew from April 2015 onwards. The final two bases (Prestwick and Newquay) began to fully operate from January 2016 onwards. Trends over time, across all bases, can only be made in a comparable way from this point. In addition, the transition of data sources from iSAR to ARCC took place in June 2017. DfT compared data from the two sources where this was possible (for the first 3 months of 2017) which showed a close correspondence between the two sources; consequently the change of source is not considered to have impacted the trends shown in the statistics. By base: The Portland base only operated a 12 hour service compared to the other bases that operates 24 hours a day. Therefore, this should be taken into account when interpreting the results. Previous UK SARH statistics: Historically, the MOD produced monthly, quarterly and annual statistics releases covering search and rescue helicopters. The statistics covering the military operations were badged as National Statistics and provided in-depth coverage, whereas the statistics for the civilian operations were much more limited and were not badged as National Statistics (they covered a brief summary table at the end of the publication). The DfT statistics series is currently in the ‘experimental’ stage, with the aim being to badge it as a National Statistic once a stable series is in place. During the transition period, the MOD continued to produce quarterly and annual statistical publications as usual, for those bases that remained open, until they ceased operations. The last MOD base closed on 31 December 2015, and the final statistics were published in February 2016. Although similar, as the two sets of figures were collected in from different sources, comparisons between them are not made.
Search and Rescue Helicopter Statistics - Background Quality Report Page 7
Coherence: The statistical publication reports on all SARH activity in the United Kingdom, including the Channel Islands, Isle of Wight and Isles of Scilly. The Irish Coastguard is responsible for all search and rescue helicopter activity in Ireland, but do not publish official statistics in a comparable format; some information on the number of Irish helicopter taskings is available as part of the Coastguard end of year statement. Due to the various different methods used for reporting search and rescue activity in different countries, it is not possible to directly compare the UK SARH statistics to others internationally. DfT intend to explore data from other search and rescue organisations such as lifeboats and coastal rescue teams, and whether it is possible to expand the coverage of these statistics to a cover wider UK Search and Rescue statistics in the future. The following four sections cover additional principles which Eurostat asks official statistics producers to comment on when reporting on quality.
6. Trade-offs between Output Quality Components Trade offs between output quality components describes the extent to which different aspects of quality are balanced against each other.
The main trade-off is accuracy against timeliness and respondent burden. DfT have a documented process to follow from receiving the data from suppliers, cross checking results with iSAR to internal checks and queries. Since the transition to the new data source took place, around 15% of the data was queried to the MCA and ARCC in the first three months of data received, with additional queries resolved internally within the DfT statistics team (for example, using the iSAR system to fill in missing data, such as incident co-ordinates). The impact of this trade-off is reduced timeliness, in return for ensuring accuracy. If none of the data validation queries were undertaken, the statistics could potentially be published around a month earlier, but would be likely to contain less accurate data.
7. Assessment of User Needs and Perceptions Assessment of user needs and perception covers the processes for finding out about users and uses, and their views on the statistical products.
DfT regularly engages with users by social media, email and face to face methods when possible. This includes requesting feedback on the quarterly and annual statistics series, with contact details provided in every statistical release. Each publication is promoted via Twitter and often the MCA Facebook page. DfT is also in regular contact with the MCA who are responsible for SARH operations, to ensure that their needs continue to be met by the published statistics. More in-depth user consultation was carried out alongside the initial publication of the statistics, between November 2015 and January 2016. This provided DfT with a good understanding of user interests which led to the development of new dissemination techniques (including an interactive map) and publication of additional information.
Search and Rescue Helicopter Statistics - Background Quality Report Page 8
DfT also regularly analyses web page usage, ad-hoc requests and social media analytics to monitor activity overtime. In line with the Code of Practice for Official Statistics, users will be informed about any changes or revisions to the data series.
8. Performance, Cost and Respondent Burden Performance, cost and respondent burden describes the effectiveness, efficiency and economy of the statistical output.
The outputs are produced from the administrative data, managed by the MCA. The ARCC data is reviewed by both the MCA and DfT on a monthly basis where validations and checks are in place to ensure the correct number of taskings are recorded. Previous discussions between ARCC, DfT and MCA have resulted in more consistent recording and a set of trusted statistics that are produced to inform policy and decision making. There is considered to be a minimal burden associated with the data supplied by the ARCC and validated by MCA, since the data is also collected for their internal purposes.
9. Confidentiality, Transparency and Security Confidentiality, transparency and security refers to the procedures and policy used to ensure sound confidentiality, security and transparent practices.
All data is stored, accessed and analysed using MCA and DfT secure IT systems. Data protection regulations are adhered to throughout the SARH statistics production process, and any information provided to DfT by MCA and ARCC will be kept securely where access to data is controlled in accordance with departmental policy. The information used to compile these statistics provides details on individual events (for example a brief description of the circumstances of an incident, including details such as the age and gender of those involved), though sensitive personal or identifiable personal data is not sent to DfT. The published statistics do not therefore reveal any private information about any individual or organisation. No statistical disclosure control methods are applied to the outputs, as the variables included in the statistics are not considered to be sensitive. DfT aims to publish as much data as is possible whilst ensuring that confidentiality is maintained. In line with this, a spreadsheet of record level data containing high level variables for each tasking was published alongside the annual statistics in June 2017 (Table SARH0112). DfT adheres to the principles and protocols laid out in the Code of Practice for Official Statistics and comply with pre-release access arrangements. The pre-release access lists are available on the DfT website.
Search and Rescue Helicopter Statistics - Background Quality Report Page 9
Section 3: Summary and conclusions This Background Quality Report presents information for users of the search and rescue helicopter statistics covering different aspects of their quality. The Department concludes, on the basis of the assessment outline above, that the statistics are of a quality which is considered fit for the purposes for which the statistics are being used. Comments and feedback on this report, or any other aspect of these statistics are welcome, and can be provided by email to [email protected] Further information Search and rescue helicopter statistics: • Alongside this publication a series of data tables have been published, where users can find the key statistics: https://www.gov.uk/government/statistical-data-sets/search-and-rescue-helicopter. • In June 2017, DfT produced an interactive map of SARH taskings based on 2016/17 data: http://maps.dft.gov.uk/sarh-statistics/. • The document outlining the changes in data sources from the helicopter operating system to ARCC data can after April 2017 can be found here: https://www.gov.uk/government/uploads/ system/uploads/attachment data/file/618846/sarh-changes-to-data-series.pdf • A summary of the user feedback document, published in March 2016 can be found here: https:// www.gov.uk/government/uploads/system/uploads/attachment data/file/505975/user-feedbackrecommendations.pdf Statistical quality: • The European Statistical System (ESS) handbook for quality reports provides comprehensive guidance on measuring the quality of statistical processes. • The Code of Practice sets out common standards that should be followed by all UK organisations that produce official Code of Practice statistics: https://www.statisticsauthority.gov. uk/osr/code-of-practice/.
Search and Rescue Helicopter Statistics - Background Quality Report Page 10
6th December 2017
Search and Rescue Helicopter Statistics Background Quality Report
This background quality report relates to the publication of quarterly and annual data from civilian search and rescue helicopter (SARH) operations in the UK. The purpose of this document is to provide users of the statistics with information about the quality of the outputs, measured against different dimensions of statistical quality. As a result, this document helps to demonstrate how the Department complies with the Code of Practice for Official Statistics principle on quality. Section 1: Background to the statistics: Page 2 Section 2: Quality assessment •
Relevance Page 4
•
Accuracy and reliability Page 5
•
Timeliness and punctuality Page 6
•
Accessibility and clarity Page 6
•
Coherence and comparability Page 7
•
Trade-offs between output quality components Page 8
•
Assessment of user needs and perceptions Page 8
•
Performance, cost and respondent burden Page 9
•
Confidentiality, transparency and security Page 9
Section 3: Summary and conclusions: Page 10
RESPONSIBLE STATISTICIAN:
Matthew Tranter
Davita Patel
FURTHER INFORMATION:
Media: 020 7944 4312
Email: [email protected]
Section 1: Background to the statistics Search and rescue helicopters are on standby 24 hours a day to assist those in difficulty, both at sea and over land, as part of a wider search and rescue provision which includes maritime and land based operations. These statistics are collected to provide information on the number and type of SARH taskings. From April 2015, the mix of military and civilian search and rescue capacity, managed by the Ministry of Defence (MOD), began to transition to a single civilian contract. This is now managed by the Maritime and Coastguard Agency (MCA), with statistics on search and rescue helicopters (SARH) reported by the Department for Transport (DfT). A completely separate statistical series has now been established by DfT, covering the civilian bases, from the point at which they opened. As part of the transition to the civilian contract, two new SARH bases were opened in April 2015; two in July 2015; one in October 2015 and two more in January 2016. Since July 2017, all SARH bases have been fully operational across the United Kingdom. The Portland base ceased operation after June 2017. Data is available for this base from its initial operation in April 2015 to its last operation in June 2017. Map of UK SARH bases
Sumburgh Apr 2017
Stornoway July 2017
Inverness Apr 2015
Civilian base now closed Civilian base now open
Prestwick Jan 2016
Humberside Apr 2015
Caernarfon Jul 2015
St Athan Oct 2015
Lydd
Aug 2015
Newquay Jan 2016
Portland
Lee-on-Solent Apr 2017
Closed: June 2017
Search and Rescue Helicopter Statistics - Background Quality Report Page 2
Methodology and Production Between April 2015 and June 2017, data used to produce these statistics was derived from an administrative system used by the helicopter operators (known as iSAR) and provided to DfT by the Maritime and Coastguard Agency. From June 2017, the statistics are based on information recorded by the Aeronautical Rescue Co-ordination Centre (ARCC) based in Fareham. ARCC are responsible for the co-ordination of the search and rescue helicopters, from answering the initial request through to the conclusion of the tasking. These statistics are based on administrative data, with the process broadly as follows: • Following a tasking, the ARCC records incident data into a single spreadsheet. • This spreadsheet is sent to the MCA and DfT on a monthly basis, with MCA checking the incidents captured are correct for contractual purposes. • DfT validates the data to verify that the data items are accurate and consistent. This includes cross checking the details against those entered in the helicopter operator system (iSAR), checking for missing data and any anomalies, and checking that the location type and type of tasking matches with the tasking narrative. These checks are done using Excel and ArcGIS mapping software. • Any anomalies and amendments are sent back to the MCA for verification. DfT then signs off the monthly data for use in production of the statistics, on a quarterly basis. • Once the data is finalised, it is entered into a SQL database, where data tables are exported to Excel and the tables and statistical release are produced for publication. • Each statistical release produced is independently checked by a statistician where any discrepancies are resolved before being approved by the senior statistician. Currently the statistics are published each quarter, with an annual release published just after the end of each financial year (in June). DfT SARH process example
ARCC Data MCA DfT QA of tasking count Making sure all ARCC IDs for each tasking are unique and match with the old data source.
All data fields are included, no blank entries from each variable.
QA of variables (Excel) Tasking category Persons rescued Duration of tasking.
QA of variables (ArcGIS) Color code by base and check for any unusual taskings. E.g. a Sumburgh tasking taking place in South of England. Colour code type of tasking on ArcGIS. Check for any pre-arranged transfers in maritime.
Check that beach/cliff taskings occur near coast, vessel taskings mostly in maritime or rivers etc.
Search and Rescue Helicopter Statistics - Background Quality Report Page 3
Section 2: Quality assessment In this section, the quality of the statistics is considered in relation to the different dimensions of quality as stated in the European Statistical System (ESS) quality framework.
1. Relevance Relevance is the degree to which a statistical product meets user needs in terms of content and coverage.
The statistical outputs presented within SARH statistics include: • A statistical release containing key findings, trends over time and signposts to further information. • Open Document Spreadsheet (ODS) data tables containing information on the number of SARH taskings, location, individual level information etc. • An interactive data map highlighting each tasking in greater detail for the annual statistics. • A document outlining the changes in data sources after April 2017. Known users and uses of the statistics: Within the DfT and MCA, the statistics are used: • • • •
For ministerial briefing and to answer public enquiries; As background to policy development; For monitoring trends in search and rescue activity; and, By analysts in modelling overall search and rescue operations.
Outside of DfT, known users include: • • •
Search and rescue partners, national parks, tourist boards, mountaineering/ walking groups and others monitoring the use of search and rescue services by tourists/ members; Air ambulance charities monitoring SAR activity in their areas; and, Maritime organisations monitoring SAR activity related to their businesses (e.g. near ports).
How well the statistics meet user needs: DfT published the first quarterly SARH statistical release in November 2015 and alongside this, launched an online survey asking users for feedback on the statistical release, which ran until January 2016. A summary of the user response was published and provides information on the extent to which user needs are being met, and where user feedback has been addressed such as incorporating new analysis within the statistical reports. Since the introduction of the first SARH statistics publication, DfT have made a number of changes to better meet user needs. These include: • Changes in data source: using data from the ARCC has enabled DfT to produce new analysis such as the number of people rescued in a helicopter that were not previously available. • An interactive map containing individual level taskings to cater to users wanting specific information about a certain base, National Park, region etc. • Additional analyses including taskings by day of week, region, and a breakdown of location type. DfT will continue to further develop the statistical outputs in response to user feedback. Search and Rescue Helicopter Statistics - Background Quality Report Page 4
2. Accuracy and Reliability Accuracy refers to how close the estimated value in the output is to the true result.
These statistics aim to measure the true number of SARH taskings in the UK. As part of the transition to using data from ARCC, a comparison was made between the two data sources which suggested a consistent number of tasking was being captured by each. A summary was published to inform users about the change of data source. The following summarises potential sources of risks or errors which may arise throughout the process of compiling the SARH statistics. Potential source of error
Risk or error mitigation
Taskings missed, or As all SARH emergency calls are received by the ARCC and recorded into a wrongly included by spreadsheet, missing records are unlikely. In addition, taskings recorded are ARCC validated by MCA for contractual purposes and the data is compared with the iSAR system and checked by MCA. This ensures that any out of scope taskings are also identified. Missing or Inconsistent data or missing variables information are queried by DfT and MCA. inconsistent Where discrepancies are identified in relation to the classification of a tasking, the data items in the description of the tasking entered in the iSAR system is used to resolve them. spreadsheet In the financial year 2016/17 year (when the data was sourced from the iSAR system) approximately 10% of cases in the data supplied to DfT required querying with the MCA. These related mainly to the latitude/longitude coordinates, which are validated by the statistics team using a mapping software (ArcGIS). Production Tabulations and basic checks are also performed on the SARH data to make sure of statistical they are in line with expectation (e.g. we would expect SARH taskings to peak in the publication and data summer months in a given year due to greater leisure activity). The data are also tables checked with previous years, looking for any unexpected changes for each variable. Where uncertainties exist in data outputs, these are highlighted in the statistical releases.
DfT have a close working relationship with the MCA and the ARCC to ensure the quality of the data is maintained. The team at the ARCC are trained in recording incidents and guidance is supplied to those who are responsible for inputting data in order to ensure that this is as consistent as possible. Because of the thorough data validation process undertaken by DfT every month, it is rare for the SARH statistics to be revised.
Search and Rescue Helicopter Statistics - Background Quality Report Page 5
3. Timeliness and Punctuality
Timeliness describes the time between the date of publication and the date to which the data refers, and punctuality describes the time between the actual publication and the planned publication of a statistic.
SARH statistics are currently published quarterly. The production of these reports commences after the final month’s data for a given quarter is received. Publication dates for these statistical releases are agreed at least 2 months in advance and dates are published on the GOV.UK release calendar. We aim to publish the statistical releases around 2 months after the end of the period to which the statistics relate. For example statistics covering the period April to June 2017 were published in August 2017. To date, all SARH statistical releases have been published to the scheduled pre-announced date. Example of timescales for a SARH quarterly release Task Description
Timeline
DfT receive final months ARCC data, from MCA, after initial validation. Validations carried out using Excel and ArcGIS; queries raised to MCA and final data agreed.
Week 2 Week 2 - Week 4
Data amended based on verifications for final months data, and the entire quarter’s data loaded into SQL to produce final tables.
Week 5
Statistical release is produced, including putting together charts, maps and
Week 6
infographics. All tables, charts and figures are quality assured.
Week 7
Sign off required by senior statisticians and the MCA. Documents and tables are finalised and sent to web team ready for publication.
Week 8 Week 9
4. Accessibility and Clarity Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.
Accessibility: The outputs are published on the GOV.UK DfT statistics page in accessible formats: • Statistical releases and are available as PDFs, which are made accessible for those who use assistive technologies. • Data tables are available in ODS file format which can be accessed by using freely available software. The published quarterly tables include information on the number of taskings, type of tasking, location, number of people rescued and duration time. The annual tables, in addition contain information on region, time of day and day of week of taskings. • Individual level raw data tables and interactive data visualisations are also available annually for users interested in record level data on taskings. Search and Rescue Helicopter Statistics - Background Quality Report Page 6
Clarity: The statistical releases use plain language, in which technical terms, acronyms and definitions are defined where appropriate. The main findings are presented using a series of text, charts and maps, with maps used to show the distribution of SARH tasking locations. Full details of the strengths and weaknesses of the data are provided at the end of each statistical report for users who are interested in this. In addition to the statistical releases, further relevant supporting documentation includes a note outlining the changes in data sources from April 2017.
5. Coherence and Comparability Comparability is the degree to which data can be compared over time and domain. Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar.
Comparability: Over time: The statistical releases cover trends since April 2015, the first period for which data are published by DfT. Over this period, the number of civilian bases grew from April 2015 onwards. The final two bases (Prestwick and Newquay) began to fully operate from January 2016 onwards. Trends over time, across all bases, can only be made in a comparable way from this point. In addition, the transition of data sources from iSAR to ARCC took place in June 2017. DfT compared data from the two sources where this was possible (for the first 3 months of 2017) which showed a close correspondence between the two sources; consequently the change of source is not considered to have impacted the trends shown in the statistics. By base: The Portland base only operated a 12 hour service compared to the other bases that operates 24 hours a day. Therefore, this should be taken into account when interpreting the results. Previous UK SARH statistics: Historically, the MOD produced monthly, quarterly and annual statistics releases covering search and rescue helicopters. The statistics covering the military operations were badged as National Statistics and provided in-depth coverage, whereas the statistics for the civilian operations were much more limited and were not badged as National Statistics (they covered a brief summary table at the end of the publication). The DfT statistics series is currently in the ‘experimental’ stage, with the aim being to badge it as a National Statistic once a stable series is in place. During the transition period, the MOD continued to produce quarterly and annual statistical publications as usual, for those bases that remained open, until they ceased operations. The last MOD base closed on 31 December 2015, and the final statistics were published in February 2016. Although similar, as the two sets of figures were collected in from different sources, comparisons between them are not made.
Search and Rescue Helicopter Statistics - Background Quality Report Page 7
Coherence: The statistical publication reports on all SARH activity in the United Kingdom, including the Channel Islands, Isle of Wight and Isles of Scilly. The Irish Coastguard is responsible for all search and rescue helicopter activity in Ireland, but do not publish official statistics in a comparable format; some information on the number of Irish helicopter taskings is available as part of the Coastguard end of year statement. Due to the various different methods used for reporting search and rescue activity in different countries, it is not possible to directly compare the UK SARH statistics to others internationally. DfT intend to explore data from other search and rescue organisations such as lifeboats and coastal rescue teams, and whether it is possible to expand the coverage of these statistics to a cover wider UK Search and Rescue statistics in the future. The following four sections cover additional principles which Eurostat asks official statistics producers to comment on when reporting on quality.
6. Trade-offs between Output Quality Components Trade offs between output quality components describes the extent to which different aspects of quality are balanced against each other.
The main trade-off is accuracy against timeliness and respondent burden. DfT have a documented process to follow from receiving the data from suppliers, cross checking results with iSAR to internal checks and queries. Since the transition to the new data source took place, around 15% of the data was queried to the MCA and ARCC in the first three months of data received, with additional queries resolved internally within the DfT statistics team (for example, using the iSAR system to fill in missing data, such as incident co-ordinates). The impact of this trade-off is reduced timeliness, in return for ensuring accuracy. If none of the data validation queries were undertaken, the statistics could potentially be published around a month earlier, but would be likely to contain less accurate data.
7. Assessment of User Needs and Perceptions Assessment of user needs and perception covers the processes for finding out about users and uses, and their views on the statistical products.
DfT regularly engages with users by social media, email and face to face methods when possible. This includes requesting feedback on the quarterly and annual statistics series, with contact details provided in every statistical release. Each publication is promoted via Twitter and often the MCA Facebook page. DfT is also in regular contact with the MCA who are responsible for SARH operations, to ensure that their needs continue to be met by the published statistics. More in-depth user consultation was carried out alongside the initial publication of the statistics, between November 2015 and January 2016. This provided DfT with a good understanding of user interests which led to the development of new dissemination techniques (including an interactive map) and publication of additional information.
Search and Rescue Helicopter Statistics - Background Quality Report Page 8
DfT also regularly analyses web page usage, ad-hoc requests and social media analytics to monitor activity overtime. In line with the Code of Practice for Official Statistics, users will be informed about any changes or revisions to the data series.
8. Performance, Cost and Respondent Burden Performance, cost and respondent burden describes the effectiveness, efficiency and economy of the statistical output.
The outputs are produced from the administrative data, managed by the MCA. The ARCC data is reviewed by both the MCA and DfT on a monthly basis where validations and checks are in place to ensure the correct number of taskings are recorded. Previous discussions between ARCC, DfT and MCA have resulted in more consistent recording and a set of trusted statistics that are produced to inform policy and decision making. There is considered to be a minimal burden associated with the data supplied by the ARCC and validated by MCA, since the data is also collected for their internal purposes.
9. Confidentiality, Transparency and Security Confidentiality, transparency and security refers to the procedures and policy used to ensure sound confidentiality, security and transparent practices.
All data is stored, accessed and analysed using MCA and DfT secure IT systems. Data protection regulations are adhered to throughout the SARH statistics production process, and any information provided to DfT by MCA and ARCC will be kept securely where access to data is controlled in accordance with departmental policy. The information used to compile these statistics provides details on individual events (for example a brief description of the circumstances of an incident, including details such as the age and gender of those involved), though sensitive personal or identifiable personal data is not sent to DfT. The published statistics do not therefore reveal any private information about any individual or organisation. No statistical disclosure control methods are applied to the outputs, as the variables included in the statistics are not considered to be sensitive. DfT aims to publish as much data as is possible whilst ensuring that confidentiality is maintained. In line with this, a spreadsheet of record level data containing high level variables for each tasking was published alongside the annual statistics in June 2017 (Table SARH0112). DfT adheres to the principles and protocols laid out in the Code of Practice for Official Statistics and comply with pre-release access arrangements. The pre-release access lists are available on the DfT website.
Search and Rescue Helicopter Statistics - Background Quality Report Page 9
Section 3: Summary and conclusions This Background Quality Report presents information for users of the search and rescue helicopter statistics covering different aspects of their quality. The Department concludes, on the basis of the assessment outline above, that the statistics are of a quality which is considered fit for the purposes for which the statistics are being used. Comments and feedback on this report, or any other aspect of these statistics are welcome, and can be provided by email to [email protected] Further information Search and rescue helicopter statistics: • Alongside this publication a series of data tables have been published, where users can find the key statistics: https://www.gov.uk/government/statistical-data-sets/search-and-rescue-helicopter. • In June 2017, DfT produced an interactive map of SARH taskings based on 2016/17 data: http://maps.dft.gov.uk/sarh-statistics/. • The document outlining the changes in data sources from the helicopter operating system to ARCC data can after April 2017 can be found here: https://www.gov.uk/government/uploads/ system/uploads/attachment data/file/618846/sarh-changes-to-data-series.pdf • A summary of the user feedback document, published in March 2016 can be found here: https:// www.gov.uk/government/uploads/system/uploads/attachment data/file/505975/user-feedbackrecommendations.pdf Statistical quality: • The European Statistical System (ESS) handbook for quality reports provides comprehensive guidance on measuring the quality of statistical processes. • The Code of Practice sets out common standards that should be followed by all UK organisations that produce official Code of Practice statistics: https://www.statisticsauthority.gov. uk/osr/code-of-practice/.
Search and Rescue Helicopter Statistics - Background Quality Report Page 10
