In September 2016, the Central Statistics Office (CSO) published its second review on the accuracy of data collated on An Garda Síochána’s PULSE (police using leading systems effectively) system.¹ The initial comprehensive review, published in 2015, was centred on data from 2011.

In Ireland, recorded crime statistics provide vital information regarding the frequency and type of crime that arises. However, in 2014, the Garda Inspectorate raised serious concerns regarding how crime was recorded on PULSE.² Issues included the non-recording of crimes, lack of timeliness in recording crimes, misclassifying crime incidents and non-crime incidents at initial stages, alterations of narratives, incorrect reclassification of incidents, and incorrect application of detection and invalidation status to certain crimes. Seeing that the CSO avails of PULSE data to produce crime statistics, it was considered pertinent to appraise the data received by the CSO to establish how accurate they were.³ Although publication of crime statistics restarted following this review, the CSO decided that the data would need to be monitored on a regular basis.

The aim of the second review was to determine whether issues raised in the first review still existed.

This analysis essentially retains the same methodology as the previous study.³ However, there are two exceptions: sample sizes have been adjusted and the reclassification section involves a more detailed analysis. Access was provided by An Garda Síochána to crime and non-crime data including CAD (command-aided dispatch), paper records and non-crime PULSE incident groups, incident narrative audit trails, and reclassified incidents. A random selection of CAD and paper records was checked against corresponding records on PULSE.

Non-recording of crimes

In 2015, validated data for crimes (e.g. assault, burglary, criminal damage, public order, robbery, theft) that occurred in Dublin and in non-Dublin areas reported on CADs were not reported on PULSE, 17% and 18% respectively. In addition, the proportion of validated paper records (16%) not reported on PULSE were similar to the 2011 analysis. Matching records was challenging; hence, the authors advised caution when interpreting results as some records believed not to be on PULSE may in fact be within the system.

Lack of timeliness in recording crimes

Within the PULSE system a creation date and report date are recorded. In 2015, the CSO found that following a crime being reported, there was a delay of over one week before 6% of incidents were recorded on PULSE. Between crime categories, the delay ranged from a low of 1% (robbery, extortion and hijacking offences; burglary and related offences) to a high of 46% of offences against government, justice procedures, and organisation of crime.

Alteration of narratives

The narrative field on PULSE, which records details of a crime, can be amended as further information becomes available. To determine whether narratives had been edited to justify classification decisions, the narrative lengths of a random sample of PULSE incident records (n=995) were analysed to see if any reductions in length had taken place in 2015, which would indicate unacceptable editing. Four cases were identified, which was only slightly higher than the analysis in the previous report (n=1).

Misclassification of incidents

There are approximately 300 crime classifications on PULSE. Although the majority are crimes, non-crime classifications also exist (e.g. attention and complaints). How crimes are classified is essential for accurate reporting. Similar to the previous analysis, the CSO focused on six classifications of serious crimes – assault minor; assault causing harm; criminal damages (not arson); theft from person; burglary; and robbery from the person. The analysis indicated that 3% of records were classified incorrectly and the classification of a further 2% was unclear.

To determine whether non-crime incidents should have been classified as crimes, narratives from attention and complaints (n=1200), property lost (n=500) and domestic dispute (n=300) were examined. Although the majority of records were classified correctly (95%–98%), a small number of records were either misclassified (1%–3%) or unclear (1%–2%). Further analysis indicated that 35 records from attention and complaints should have been classified as assaults or fraud/public order offences and one each in child pornography and robbery; two property lost should have been in thefts; and six domestic disputes should have been in assaults or harassment. Nonetheless, the percentage misclassified had decreased since the 2011 analysis.

Incorrect reclassification of crime incidents

In order to carry out a more in-depth analysis, the CSO obtained audit trails for all crimes that had been reclassified by An Garda Síochána in 2015 (n=3500). From these audit trails, four areas were examined: assault causing harm and assault minor (n=233), criminal damage (n=224), arson (n=52), robbery from person or institution (n=28) and burglary (n=272). The analysis indicated that 68% of reclassifications were justified, while 32% were not. Reclassifications were upwards (more serious) or downwards (less serious), 24% and 48% respectively. Within crime classifications, a higher percentage of upward reclassifications occurred in the assault minor category (69%). The highest percentage of downward reclassifications occurred in the burglary (90%), arson (87%), assault causing harm (70%) and robbery from person/institution (61%) categories.

A further analysis of ‘downgrades’ in the burglary category indicated that 55% were justified and 45% were not. Primarily, unjustified downgrades in this category were for reclassifications for attempted burglary to trespassing or criminal damage categories. Similarly, in the robbery from person/institution category, 59% were justified and 45% were not.

In relation to crimes reclassified into non-crime categories, for example 6% of reclassifications in 2015 were to attention and complaints, which was lower than the estimate for 2011 (17%). However, the CSO advises caution when interpreting this result as the samples and methodology may not be comparable. Notably, three groups ‒ Group 03 Attempts/threats to murder, assaults, harassments and related offences; Group 10 Controlled drug offences; and Group 12 Damage to property and to the environment ‒ accounted for the highest level of reclassifications to attention and complaints (11%).

Incorrect application of detection status

To determine whether ‘detected’ crimes resulted in criminal proceedings, 112 879 ‘detected’ crimes were examined. Sixty-three per cent, which illustrated an increase since 2011 (54%), were linked to a charge or summons, whereas 37% were not. However, the CSO advises caution on interpreting this, as the 2011 and 2015 data may not be comparable. A further analysis of the accuracy of ‘detection rules’ on ‘detected’ crimes with no charge or summons (n=1000) indicated that nearly one-fifth of crimes were wrongly assigned ‘detected’ (18%), which was nearly half of those ‘detected'. This accounted for a 10% (12 789) reduction in the total number of ‘detected’ crimes.

Incorrect application of invalidation status

Invalidation occurred when there was no crime or when ‘counting rules’ were wrongly applied (p. 28). Out of a sample of 1000 invalidated records in 2015, 21% were unjustified, which was slightly lower than 2011 (23%). Importantly, an unjustified invalidation does not mean a crime has not been dealt with correctly; often when a new case is created on PULSE, a previous crime linked to the same case becomes invalid due to lack of information.

Conclusions and recommendations

The CSO estimated the approximate impact of the problems identified in their review. The largest percentage changes in 2015 were:

• Group 13 Public order and other social code offences (50%)
• Group 03 Attempts, threats to murder, assaults, harassments and related offences (33%)
• Group 12 Damage to property and to the environment (32%)
• Group 11 Weapons and explosives offences (31%)

Alterations owing to misclassifications within groups were not considered; for example, assault causing harm and assault minor are both in Group 03.

Consistent with the Garda Inspectorate report and the previous analysis in 2011, the analysis carried out by the CSO on 2015 data illustrates that discrepancies between crimes recorded on CAD/paper and PULSE are ongoing. However, although there is evidence of non-recording of crimes on PULSE, it has to be acknowledged that crime misclassification on PULSE and inaccurate use of detection status has reduced since the previous report.

The CSO made a number of recommendations to further improve the quality of crime data.

• An area that requires immediate attention is the CAD narrative field, which should contain all necessary information about crimes prior to being closed, including a PULSE identity number (ID), and reasons for non-recording.
• RC1 forms need to be utilised in all non-CAD divisions.
• It is vital that Garda stations without an electronic recording system maintain proper paper records, including all crime information along with its associated PULSE record.
• Narratives on PULSE need to be more detailed such that information recorded supports crime classification applied and any subsequent invalidation/detection decisions that are made.
• To overcome quality issues that were delineated in this report, it is recommended that An Garda Síochána should put structures in place to ensure that crime-related data are monitored and controlled.

With the aim of increasing the reliability of the data, the CSO continues to collaborate with An Garda Síochána. Additionally, the CSO intends to continue monitoring the quality of the data by repeating this analysis on an ongoing basis.

Ciara H Guiney

______________________________

1       Central Statistics Office (2016) Review of the quality of crime statistics 2016. Cork: Central Statistics Office. Available online at: http://www.drugsandalcohol.ie/26176/

2       Garda Inspectorate (2014) Report of the Garda Síochána Inspectorate: crime investigation. Dublin: Garda Inspectorate. Available online at: http://www.drugsandalcohol.ie/22967/

3       Central Statistics Office (2015) Review of the quality of crime statistics. Dublin: Government of Ireland. Available online at: http://www.drugsandalcohol.ie/24887/