Bravo Dr. Kucharski. This line makes me want to stand up and cheer.
"If one doesn't have quantitative information available, NO method will give you quantitatively accurate results"
Too many folks that I encounter want to use Baysian and call it quantitative because they are using a sort of quantitative method but the data itself is qualitative.
Qualitative data is still qualitative data no matter how it is manipulated.
There is all the difference in the world between looking at some leaves and arbitrarily deciding how green the leaf is compared to using light spectroscopy to make that determination.
Alicia
Frazier | 19100 Ridgewood Parkway | MPC | San Antonio, TX 78259
Direct: 210-626-6615
From: ACS Division of Chemical Health and Safety <DCHAS-L**At_Symbol_Here**PRINCETON.EDU>
On Behalf Of Kucharski, Timothy
Sent: Thursday, October 17, 2019 9:42 AM
To: DCHAS-L**At_Symbol_Here**PRINCETON.EDU
Subject: [EXTERNAL] Re: [DCHAS-L] Validity of the risk matrix
Good points, Rob.
My 2 cents:
This is intriguing, but I argue that 1) the essay author is oversimplifying the situation, and 2) is then generalizing it to all applications.
In any analysis, the "garbage in garbage out" applies. So yes, if one is just guessing at the likelihoods of certain events, the outcome will not be rigorously meaningful. While
that may be true in information security (difficult to get a good estimate on how likely a given security breach will be), that is most definitely NOT the case for all industries. The military and the process industries, among others I'm sure, do have documented,
reliable frequencies of initiating events, equipment failure rates, etc. Such information is critical input in determining, for example, what safety integrity level (SIL) is needed for a given aspect of a given (proposed) process within a given plant. This
comes up in the Layer of Protection Analysis (LOPA), wherein quantitative data is used to determine what safety measures are needed to be in place in order to reduce the overall risk of a given aspect to a tolerable level.
For example, the AIChE Center for Chemical Process Safety has published numerous books on how to do such a risk mitigation analysis quantitatively, such as their "Layer of Protection
Analysis: Simplified Process Risk Assessment" (2001) and their "Guidelines for Initiating Events and Independent Protection Layers in Layer of Protection Analysis" (2014). There are quantitative, location specific frequencies even for lightning strikes, earthquakes,
etc. out thereŅone just needs to research.
In a qualitative sense, you can use the risk matrix to at least prioritize what needs to be addressed, but this still carries the caveat of it only being as good as your input
for frequencies and impact. The essay is a bit disingenuous when complaining that "What's more, the matrix gives equal weight to probability and impact, so an incident with 1% probability and $200,000 impact has the same priority as one with 0.2% probability
and $1,000,000 impact." Yes. That's the point of quantitatively calculating risk. If the impact is catastrophic, one cannot just simply ignore it because one thinks 0.2% is "low". There's more nuance than that: In some cases, 0.2% is not tolerable (we blow
up the multi-billion dollar plant). In others (minor leak in the men's room) it may be.
So, it strikes me that the essay is generalizing the problems of not having reliable input information to the method in general, regardless of the industry. If one doesn't have
quantitative information available, NO method will give you quantitatively accurate results. I compare this to claiming that "my GPS system won't get me to my destination, even though I never loaded the maps into it."
Timothy J. Kucharski, Ph.D.
Lab Scientist
Chemical Hygiene Officer
Aramco Services Company
Advanced Materials Team
Aramco Research Center - Boston
Tel.: 713 432 5472 || 857 270 8308
EXTERNAL: This email came from the Internet. Report this message to
ASCSuspiciousEmail**At_Symbol_Here**aramcoservices.com as suspicious if it contains any suspicious content.
The flaw in the argument is that not all estimates of probability are unreliable. So there is still a value to the approach as long as you recognize its limitations. Which is true of any approach.
There is a very low probability I would ever fall for one of those as I have been dealing with such stuff since email was basically invented, probably well over a million emails to date, and am well aware that you don't click on a link
to download a file from somewhere (especially one with a goofy-looking "secure PDF" icon) or open an attachment without running it through a meta virus/malware engine (I recommend https://www.virustotal.com/gui/home/upload -
you will be amazed how many payloads are missed if you are relying on one popular antivirus/malware engine). In addition, my choice of OS also reduces the probability of damage and I call the client using a known phone number (not one in the suspicious email)
if necessary. And other measures I won't detail here.
However, if you have a bunch of younger or inexperienced employees (and don't forget, today's generation does not use email unless forced to for school or work), they are likely to bite on a "new purchase order" with previous email correspondence
no matter how well you have trained them. I just had to take an email security training module for the local university where I teach one class and I while it has good intentions, it's one of many training modules they throw at employees to check off boxes
and, because of this, it means that the average user is just hitting Next to get through it as quickly as possible.
My point here is that it's a pretty good bet that if your firm is a small business without a dedicated IT staff and/or a sophisticated threat system to block these things that you are going to by successfully hit by this kind of threat.
And therefore, at the very least, you should at least have a plan to deal with it if you are. The risk matrix here is very accurate.
However, that is not to say a Bayesian or other approach could not be more accurate or complementary. My observation above about virus checkers is proof positive of the dangers of relying on one system/method and thinking you're set.
Analyze in multiple ways, understand the limitations of each, look to overlap/complement, and proceed with caution.
======================================================
Safety Emporium - Lab & Safety Supplies featuring brand names
Fax: (856) 553-6154, PO Box 1003, Blackwood, NJ 08012
While stumbling around the web with regard to thinking about the risk matrix, I came upon an article that questioned its value:
The essence of the argument, I think, is that estimates of probability are very unreliable. I'd appreciate the wisdom of the list regarding this essay and its conclusion.
David C. Finster
Professor Emeritus, Department of Chemistry
Wittenberg University
--- For more information about the DCHAS-L e-mail list, contact the Divisional membership chair at membership**At_Symbol_Here**dchas.org Follow
us on Twitter **At_Symbol_Here**acsdchas
--- For more information about the DCHAS-L e-mail list, contact the Divisional membership chair at
membership**At_Symbol_Here**dchas.org Follow us on Twitter **At_Symbol_Here**acsdchas
--- For more information about the DCHAS-L e-mail list, contact the Divisional membership chair at
membership**At_Symbol_Here**dchas.org Follow us on Twitter **At_Symbol_Here**acsdchas
