The Future in Focus

Today we're discussing one of the most interesting research studies that have been conducted in recent years on the organisational benefits of having a certified OHS management system. Can you first tell us something about the research study?

This is a study carried out by researchers at Harvard and Duke universities in the US, which examined whether organizations benefit from having a certified occupational health and safety management system. Now the study was actually based on the OHSAS 18,000 and one standard, the precursor to ISO 45,000 and one. And the data that the researchers have assembled. Comparing a population of certified and uncertified organizations in the US provides evidence that the adoption of an occupational health and safety management system standard serves both as a credible indicator of superior OHS performance and perhaps more importantly, as a means to improve that performance.

Now those conclusions may not come as a surprise to many of the people who've worked to implement the requirements of an occupational health and safety management system standard in their organizations, and perhaps gone on to obtain third party certification. But to date, there's been surprisingly little objective evidence to demonstrate these positive effects.

And what makes this study so important or so distinctive?

Well, there have been previous research studies in various parts of the world addressing more or less the same question, but what's distinctive and important about this latest study is that it's based on comparing actual data on the frequency and severity of workers. Injury and ill health.

In certified and uncertified organizations and in certified organizations in the periods before and after certification. Many of the previous studies have been based on comparing other factors or been based on more subjective evidence.So, by subjective evidence, I mean things like asking those who've implemented an OHS management system and obtained certification what effect they believe it has had on the OHS performance. Or perhaps the results of employee surveys in certified organizations. When I talk about studies that were based on comparing factors other than the frequency and severity of harm to workers. I'm talking about studies which compared for example, regulatory compliance violations or the extent of safety training or safety procedures in certified and uncertified companies.

Having said that, there have been some previous studies which have looked at actual performance, including two studies in Spain which found that organisations choosing to pursue and obtain certification tendered at the time they made that choice to have worse incident rates than those. But all these previous studies showing both positive and negative effects. Were based on much smaller numbers of organizations than the Harvard Duke study.

Another difficulty which has been faced in some of the previous studies, was that the available data was at the level of an entire organization or firm, parts of which may have been certified while other parts were not. Now, in all these studies, the biggest challenge is the availability of reliable data. And the key enabler for the Harvard Duke study was access to microdata on injury and illness incidents. Reported in the survey of occupational injuries and illnesses conducted by the US Bureau of Labor Statistics.

Now this is a survey sent to nearly 230,000 establishments each year, randomly selected according to location industry, number of employees. In order to provide a representative sample of all workplaces in the US. All private sector establishments are required by law to respond if they receive the survey. The resulting data set covers over 2,000,000 establishments and covers the period from 1995 to 2016, the most recent year available.

The other key to this work was that the researchers signed data sharing agreements with ten major international certification bodies. And obtained from them the name, address and certification date of every establishment in the US that they had certified to OHSAS 18,000 and one. Between 1995 and 2018, and that was a total of just over 1300 certifications. Now that enabled the research to identify Bureau of Labour Statistics Survey data provided by those certified companies both prior to and in the years following certification. The researchers also identified the equivalent data for a matched control group of organizations that were not certified.

So in summary, I think the Harvard Duke study is important because firstly, it's based on actual data which the companies concerned are required by law to report on the frequency and severity of harm to workers.

Secondly it's based on a much larger population of data, larger number of organizations than any previous studies. Thirdly, it's based on data at the establishment level rather than at the firm or organizational level, meaning the data reflects this specific establishments or sites within an organization that are or are not certified.

What questions did the research study seek to answer?

The focus of the study was to try to establish whether organizations which implement a certified occupational health and safety management system. Demonstrate superior OHS performance. In terms of the frequency and severity of harm to workers when compared to organizations which do not implement a certified OHS management system? Two specific aspects of this were examined. Firstly, whether the certified organizations already demonstrated superior performance in the years before becoming certified, and the researchers refer to this as the selection effect. And then secondly, whether after certification? Those organizations improve their OHS performance more than the improvement seen in comparable uncertified organizations. And the researchers refer to this as the treatment effect. Now it's interesting to think a bit more about these questions. Not second questions, quite straightforward. Does certification help and organization achieve greater improvement in OHS performance? Then comparable organizations which are not certified.

I guess for many of us, that's ultimately the most important question we might ask about the use of the standard and certification. But the first question is probably the more intriguing it. It's effectively asking whether organizations that choose to become certified. Are already better performers. In other words, does certification tend to appeal more to organizations which already have superior OHS performance? At this point, we should probably remind ourselves that certification to an OHS management system standards such as 80 uh OHSAS 18,000 and one. Doesn't of itself or require the achievement of any particular level of OHS performance. It only requires that the organization has put in place and operates a management system meeting the requirements of the standard. Indeed, you know, that's one of the criticisms we sometimes hear of management system certification. That it implies implies that the organization has the ability to deliver consistent and improving performance. But there's no requirement for that performance actually to be superior. So it's interesting to explore whether in reality there is this selection effect in which certification tends to attract organisations with better OHS performance. Because if if such an effect were to exist. It would imply that certification can be taken as an indicator of superior performance, despite the fact there's no such requirement in the certification criteria.

So what were the findings?

So let's start with the results regarding the selection effect. Is certification a signal of superior prior OHS performance? In other words, on average, do organizations that choose to adopt the standard already exhibit better OHS performance before becoming certified when compared to organizations which don't adopt the standard? Now the research showed a correlation between organisations, injury and ill health rates and the likelihood that they would go on to adopt OHSAS 18,000 and one. To be precise, what the researchers found was that each additional ill health or injury case. Was associated with a 21% decline in the odds or probability that the organization would go on to adopt OHSAS 18,000 and one within the subsequent two years. And each more severe case of ill health or injury involving lost time reduce those odds by 36%. So this research demonstrates that an average those organizations with a certified OHS management system tend to be better performing organizations. Now to that second question about the so-called treatment effect. Does certification help an organization to achieve greater improvement in OHS performance than comparable uncertified organizations? Here the research found that certified organizations experienced 20% fewer total cases of injury and ill health. In the years following their certification. Than those in a matched control group 20%. And the number of cases involving time away from work was 19% lower than the control group. In fact, there was some evidence of some reduction in the most severe cases of injury in ill health, but this was rather imprecise. But this research then demonstrates that certification helps an organization to achieve greater improvement in OHS performance over subsequent years. That would otherwise have been the case.

Are there any important limitations to the study or caveats that we ought to be aware of?

I think the first thing to say is that this research paper is currently undergoing peer review. And therefore maybe updated to reflect feedback. In fact, this provides an opportunity for anyone interested in fact, to access the paper online and to provide comments directly to the researchers. Secondly, we should remind ourselves that this study is specific to the United States. Taking advantage of that establishment level occupational injury and ill health data from the US Bureau of Labor Statistics. So we don't know from this study whether the same things would be observed in other geographies. You may recall that I mentioned previous smaller scale studies in Spain, which did not find the selection effect that was found in the Harvard Duke study. The finding in Spain being quite the opposite. Namely, that organizations choosing to pursue and obtain certification tended at the time they made that choice to have worse incident rates than those which chose not to perceive certification. The third thing to highlight is that this study is based on the adoption of OHSAS 18,000 and one the predecessor to ISO 45001. Now that was simply because at the time the study was carried out, the US Bureau of Labor Statistics Survey data. Only went up to 2016 and that's of course before the publication of ISO 45,000 and one. So the study had to be based on the standard that was in use at that time. Now I think anyone familiar with OHSAS 18,000 and one and ISO 45,000 and one would recognize that the two standards are very similar. It's tempting to suggest that the additional requirements introduced in ISO 45,000 and one might mean that. ISO 45,000 in one might be even more likely to help organizations improve their OHS performance. That that was certainly the standard writers intention in making those additions, but. To be clear, that's really just speculation, not something that has yet been proven.

Do you think there's scope for further studies of this sort to further examine these effects?

Give him the sparsity of independent research, firmly based on objective data. It would be hugely valuable to carry out further similar studies both when and where equivalent and equally reliable data can be obtained. I mean specifically it would be good to see first studies in other geographies where there's a regulatory requirement for reporting of OHS performance data. And then secondly, equivalent studies on the impact of ISO 45,000 and one. However, given the lag in data collection, I guess we may need to wait until ISO 45,000 has been in use for a few more years before this becomes practically possible. And that's perhaps one other area that would be interesting to explore.

This research shows that adoption of Osas 18,000 and one is associated with superior OHS performance. But it doesn't tell us how the adoption of the standard changes the way that work is carried out and subsequently results in that superior post certification performance. It would certainly be very valuable to understand the mechanism. In other words, exactly what changes in organizations that are certified to cause this improvement to occur.