Sommer Gentry Does The Math For The TSA
Another hard-hitting, evidence-based post by USNA Math Professor Sommer Gentry at the anti-TSA TSANewsBlog:
I was pleased recently to receive an email from Russell Wooten, the IT Strategy Branch Chief of the TSA. His email reached me through my membership in the Maryland chapter of the Institute for Operations Research and Management Sciences (INFORMS). For the uninitiated, operations research is the discipline of applying advanced analytical techniques to help make better decisions. Mr. Wooten was soliciting input on these questions:Do you have any ideas on how the Transportation Security Administration (TSA) can improve its capabilities in utilizing its current security technology, upgrading its security technology, or improving its security processes?What can/should we do to improve these technologies and their processes?
AIT, Advanced Imaging Technology
AT, Advanced Technology
BLS, Bottled Liquid Scanner
EDS, Explosives Detection System
ETD, Explosives Trace Detector
An excerpt from her response:
Explosive Trace Detection (ETD): The ETD technology is designed to detect materials used in explosives, though identical chemical compounds could be detected in medications, lotions, fertilizers, and other innocuous sources. As I understand the TSA's procedures (though of course the public is not permitted to know what the TSA's procedures are), when an ETD alarms that passenger is flagged for a secondary search in a private room.However, there is an extremely low base rate of passengers trying to bring explosives through a checkpoint - let's use 21 in 10 billion, though of course it should really be 2 in 10 billion, since the weapons used on 9/11 were boxcutters rather than explosives. Using Bayes' Rule, we can calculate that even if explosives detection technologies were nearly perfect: catching every actual explosive and only falsely alarming on one in every ten thousand passengers, then only one out of five million positive test results actually indicates presence of an explosive device.
The psychological impact of processing millions of false alarms without a single real hit guarantees that screeners implementing the secondary screening process will assume that every positive is a false positive, undermining the effectiveness of any secondary screening process that could possibly be designed.
In similar fashion, screener psychology defeats secondary bag searches. Passengers report that when a bag is flagged at the checkpoint, screeners search the bag looking for a prohibited item - and stop looking when one is found, even when multiple prohibited items are present. This is reported to be how one passenger was allowed to fly with a clearly labeled 5-pound block of C4 in his carry-on.
What to conclude? ETD and other technologies that flag passengers for extra screening based on any test can never be made to be effective, because the base rate fallacy dooms even a near-perfect flagging system. The false positive rate would need to be something like an unattainable 1 in 1 million chance of a false alarm on a non-threat passenger in order to achieve a sustainable ratio of false alarms to real alarms. Resources being spent on ETD should be directed elsewhere.
Another excerpt from her thoughts on AIT -- Advanced Imaging Technology:
A recent RAND study of airport vulnerabilities at LAX concluded that "small, portable explosives have been the most likely and most lethal means of attacks at airports" and that "The greatest risks for casualties for most types of attacks are in the high-density areas passengers encounter before reaching the security checkpoint, particularly lines for ticketing and for passing the security checkpoint." Thus, AIT is not only ineffective, it is actually dangerous because it leaves passengers waiting in long lines vulnerable.But surely the most glaring weakness of the AIT system is that one can plan one's flights to avoid AIT scanners, as I have been doing ever since they came into use. Thus, we can be utterly certain that AIT technology has never played even the slightest role in discouraging an attack, since any adversary who feared AIT might discover his plot could simply choose, say, Reagan's terminal A or Fort Lauderdale's Southwest terminal for his departure. From the Congressional Research Service's recent report, we know this wide-open door for anyone to fly sans AIT will remain open: "Even at full operating capacity, not all airports and not all screening lanes will be equipped with AIT under TSA's plan." Only innocent travelers will ever be screened with AIT - terrorists can evade it easily.
Neglected risks: The TSA appears to have blind spots for some threat vectors. The insider threat looms large after a number of high-profile arrests of screeners for smuggling drugs through the checkpoint. Bribing a screener to speed explosives through a checkpoint (perhaps unwittingly) would clearly be a winning strategy for a terrorist. Many passengers also report seeing large pallets of food and merchandise bound for secure-side vendors being waved through the checkpoint or delivered through unscreened corridors.
I've highlighted these two risks because there appear to be no fundamental barriers to addressing them. One way to address insider threat is to hire fewer part-time screeners, both because they may have a more tenuous sense of loyalty than full-time screeners, and because fewer screeners means fewer opportunities to find a compromisable person. Another good practice would be to search screeners arriving for their shifts, in checked baggage rooms as well as at passenger checkpoints. Searching vendor supplies is as straightforward as it sounds. In contrast, perhaps the TSA chooses not to address the risk of ground-launched missile attacks on planes because of a dearth of effective countermeasures.
The TSA has also entirely failed to consider the risk that TSA procedures will divert would-be flyers onto the roads. Blalock, Kadiyali, and Simon found that a decrease of 1 million emplanements leads to an increase of 15 driving-related fatalities. If negative publicity and the accumulated impact of negative passenger experiences at the checkpoint causes only a 1% drop in emplanements, then the TSA is responsible for 100 or more deaths each year. While it's true that the TSA will probably avoid blame for these deaths, it is simply unacceptable engineering practice to neglect this obvious side effect of the TSA's screening choices.
Well, there you go, "Knowing"... Jeff.
Bite it.
Radwaste at December 4, 2012 5:47 AM
*slow clap*
Sabrina at December 4, 2012 6:53 AM
As for diverting passengers onto the roads...
Didn't a BUNCH of people here say recently that parents with babies should be using their minivans and not flying to grandma's? Using this logic, isn't that diverting /them/ to their deaths?
If an airline raises ticket prices, and that leads people not to fly, are the airlines killing people?
I'm all for hating the TSA, but I hate this BS more. If you drive, it's because you chose to drive. You don't get to factor in everything that led you to drive a car that day and point fingers. ("If that guy from the DMV had only failed me on my driver's test..." "If the meeting hadn't been rescheduled to a rainy day...")
Insufficient Poison at December 4, 2012 7:23 AM
Amy Alkon
http://www.advicegoddess.com/archives/2012/12/04/sommer_gentry_d.html#comment-3501893">comment from Insufficient PoisonOr they can fly grandma to come see them. The point is consideration -- not irritating the hell out of people on a six-hour flight with your kid, if he's still in the screaming stage.
The minivan thing was just an example. Flying grandma to you, assuming she doesn't wail and bang others' tray tables with a plastic cup, is preferable.
Amy Alkon at December 4, 2012 7:49 AM
One thing I'll say for TSA is you can't measure the effectiveness of a detection mechanism solely by how much it finds. This is due to the deterrence value of scanning. Metal detectors can go years without finding a gun because it keeps people from trying. Turn them off and you'll have a gun on a plane soon.
That is not to defend the TSA. they are a collosal waste of money and the world is not served by screening shoes and banning liquids. I'm just saying we must measure them correctly. Instead of saying how many guns or explosives were found, we should ask what is the likelihood it will happen if we stop scanning, banning liquids, or making passangers go shoeless.
Trust at December 4, 2012 8:04 AM
Amy Alkon
http://www.advicegoddess.com/archives/2012/12/04/sommer_gentry_d.html#comment-3501956">comment from TrustTurn them off and you'll have a gun on a plane soon.
We've had many guns on planes, thanks to overscreening and dimwits doing it.
So what.
A gun isn't going to bring down a plane.
And if somebody really is motivated to do it, they will. But, a mall or the line of coerced sheep waiting to be screened before the TSA checkpoint are easier targets.
Amy Alkon at December 4, 2012 8:26 AM
"One thing I'll say for TSA is you can't measure the effectiveness of a detection mechanism solely by how much it finds." Insufficient Poison
Entirely correct... but if you look at airport ops you find that the detection mechanisms are entirely focused on the wrong thing, AND the TSA isn't even in charge of the most likely vectors like runway incursions, ramp workers, and the non passenger side of the operations.
The ramp workers and TSA smuggling drugs, could have just as easily smuggled an operational 1ED. Being poked and prodded by security theatre does not remove the most likely avenues for causing damage.
Neither the shoebomer nor the pantybomer would have brought down a plane, AND THEY CAME IN FROM OUTSIDE THE US.
Groping grandma for her trip from Boise to Omaha does nothing but accustom her to getting in line and keeping quiet.
Importantly, the guy in the times square try a couple a years ago WAS AN AIRPORT BUS DRIVER, and yet he decided that nothing at DIA was high enough profile... not like times square.
In the end, the bad guys may think it's enough to get us all focused on airports, regardless if we think the security theatre is working or the worst thing ever, we are focused on it... so where would the best way to hit be, while we are all looking the wrong way?
SwissArmyD at December 4, 2012 10:36 AM
Radwaste:
Try sneaking a mocked up bomb onto an airplane.
Why easy and valuable are different, and your suspect constitutional reasoning here.
Bollocks. They really think they can count 100 deaths out of 33,000?
Put it another way. Presume the TSA becomes so awesomely awful that it reduced airline travel by 100% for an entire year. Accordingly, then, there would be 10,000 more highway deaths, an increase of more than 30% over the current rate.
Right?
Before citing such numbers as fact, try doing a little bit of a sanity check on them. (The last similar study you cited was just as transparently ridiculous.)
Jeff Guinn at December 4, 2012 11:01 AM
Make sandals out of C4 - dumb fucks at the xray machine will see two sandals reflecting at the same density and think nothing of it.
Worried about explosive detection tech? put the C4 in a carryon bag under mounds of clothes, and put that bag in the closet for a few months before taking it thru security
lujlp at December 4, 2012 11:25 AM
lujlp:
A bomb consists of more than just explosive, BTW.
Jeff Guin at December 4, 2012 11:55 AM
C4 doesnt require much of a detonater to get it going.
And thats assuming you wanted to cause enough damage to harm a plane in an attempt to crash it,
If you just wanted to sow terror, an could of etch a sketch animators, a magnesium fire starter, a couple of rusty nails, and a dinner knife given to you by a flight attendent and you could put on a light show that would have the US governemnt shutting down all domestic flights while they tried to figure out what was going on.
Or you could just overcharge your laptop battery and flush it down the airplanes toliet
lujlp at December 4, 2012 3:30 PM
More on the math behind the "base rate fallacy" here:
http://en.wikipedia.org/wiki/Base_rate_fallacy
Lobster at December 4, 2012 5:36 PM
I just want to put a few thoughts in everyone's minds and ask how unreasonable you think they are.
Radio Controlled airplanes have been around since the early seventies. In the early eighties I flew them a little. They could get up to about 1500 feet and 2-3 miles out from the ground pilot. What would happen if you flew several of them into the engines of a large aircraft that had "short" hopped from DC to NYC on it's way to London. That plane will be almost full of gas and have twin engine failures. The odds of a Sullenberger level pilot are slim.
You can buy a Barrett M82 .50 cal relatively easily in the U.S. for several thousand dollars. The range is about 1-1.5 miles. Enough training and you can get multiple people to hit an 8 foot wide, moving, target (size of an engine intake) with multiple shots at a distance. Especially if you know where the target is coming in, like an airport. That will destroy the engine(s). Remember how long it took to find the D.C. sniper?
I had a friend that lived under an airport landing path that the aircraft were at about 250 feet above his roof.
As for security in the airport itself -- most airports have a "civilian" for small aircraft. Do you know how easy it is to get in on that side? Then, from there to walk to the commercial side wearing coveralls from your favorite airline is a few hundred yards, carrying whatever the hell you want is a few minutes.
If you are depending on the TSA for your security -- then you aren't looking at reality. If you don't believe me, ask how the jet skier wearing a bright yellow life vest made it onto JFK.
So giving up our liberty for a theater of security is a total waste of time. I'd rather fly knowing that about a sixteenth of the people on the flight were CCW licensees and carrying a firearm.
Jim P. at December 4, 2012 8:13 PM
Jeff,
As you've been shown - well, it's there in the news if you can read - I don't have to walk through the passenger line to do that. All I have to do is hand the right baggage handler a thousand dollars.
And now you're on bombs. Why, now, would my late Dad's pocketknife be taken from me?
You gonna talk about "null" some more, even as you point at no results? That's always fun.
As is your continued efforts to have yourself presumed guilty of something because you want to ride an airplane.
That's sick. You should get help.
Radwaste at December 5, 2012 2:15 AM
"Bollocks. They really think they can count 100 deaths out of 33,000?"
As you may know, this is an argument from personal incredulity. Having not addressed the method, one has not addressed, much less defeated the point.
And once again, the basics of safety analysis seem to have slipped your grasp.
That's an indication of religious zeal, not of reason or logical assessment.
Radwaste at December 5, 2012 2:52 AM
What argument are you trying to make? That there is no threat — the Islamist desire to attack the West is completely delusional? Or that Islamists are impenetrably stupid, because they can't figure out cheap and easy foolproof ways to do it?
Yes, you are right, that was an argument from incredulity — I think it is an extraordinary claim on its face, and extraordinary claims require extraordinary proof.
So extraordinary, in fact, that I actually went to the paper that was the source of the statistic to see how they had come up with a number so amazing that any sentient being any more numerate than my dog — who can only count to four — should have immediately questioned it.
Took me about five seconds and there it was. The very first result returned the abstract to the paper, which said they come up with that number by "Back of the envelope calculations."
In case you find "back of the envelope calculations" convincing analysis — and hey, why not, — then make sure you re-read what I said after my argument from incredulity.
And just as I thought the irony of this post's title, Sommer Gentry Does The Math For The TSA was already unbeatable, you go and top it. You accept a number at face value, without even a hint of skepticism, and then accuse me of being religious?
Your reading comprehension is abysmal. I have never discussed anything but bombs. As it happens, I think the TSA should ignore anything but guns and bombs.
However, even though Jim P thinks he would rather fly with 10% of the passengers armed, he forgets that he won't be going anywhere, because all concealed carry passengers must prevent themselves to the Capt, and unless they are in law enforcement, they aren't bringing the gun on the airplane. Period.
Yes, I am going to continue to talk about null, so long as you remain completely silent about what you would do instead.
Jeff Guinn at December 5, 2012 2:09 PM
Well, this is going nowhere. Since you have no idea what has gone before, here's some of it. But as I said before, I have no hope for anyone who doesn't recognize safety analyses cover deliberate acts and completely misses the basics.
Like the 4th Amendment. That's the biggest shame - and shame it should be. An American should be asking what the minumum level of intrusiveness is effective, not simply and blindly defending the existing system.
Is it really necessary to keep the guy from bringing his leaf blower with him? No it is not. He's not doing anything to the plane with it, or the passengers, who, as Jim P has pointed out repeatedly, will be the real stopper in any hijacking. The short story is that we can go back to letting people bring what they want. You already have ID methods, etc., and you could try waiting periods on tickets; nobody buys cash, nobody buys within two weeks of flight. Hey, you can charter for emergent travel, right? It's already happening.
But if you're a fan of central government authority and power, you wouldn't want the public to defend itself at any time.
But, here are some "giggles" for the spectators:
If searching passengers is so effective, then a "no-fly" list is not necessary. Even a person on that list cannot bring a weapon on board. Put ALL OF THEM on one plane, they still can't do anything. If searches are really effective.
If searching passengers is actually effective then fighter escorts are simply not necessary. (A good question about that is almost never asked: when do you want the USAF to shoot? They don't miss.)
If searching passengers protects the rest of America from terrorist attacks because no terrorist attacks occur on airplanes, then searching anyone else, anywhere else, serves no purpose. Conversely, we have only to search football stadiums and that will magically protect American aircraft from terror. The principle claimed is that denying a terrorist one avenue of attack removes all the others.
Enjoy your next patdown. I hope they change gloves nowadays.
Radwaste at December 5, 2012 6:15 PM
@Jeff,
I'm not going to bother citing anymore of your text, because you have no facts on your side.
Working from the Wiki on hijacking says that the high was 82 attempts in a single year[1]. Since the first hijacking in 1931 to now at 82 per year would be a total of 6642 hijacking attempts.
From airline bombings[2] there have been 86 known attempts both pre/post 9/11. Doing the same math {86 * (2012 - 1931) = } 6966 attempts to bomb aircraft.
So 6642 hijacking attempts + 6966 bombing attempts comes out to 13608 attempts. Let's double the attempts and suggestions to target a "high value" target to 27,216 since 1931.
In the U.S. alone there are over 50,000 thousand commercial launches per day currently. Again let's just average the number of commercial launches from 1931 to today {((2012 - 1931) * 365) * 2000 = 5,9130,000}. So that number of flights is about 6 Billion. The percentages of a terrorist threat are 27216 bombings/hijackings by the number of commercial flights (59,130,000) the percentage of flights that were attacked is 0.046027397%.
Would I rather suffer with pre-9/11 "non-security" for a sub .04% chance of being on an aircraft that was attacked; or rather go through security that is not effective and suffer through the problem of C-Store clerk violating my rights?
BTW, Jeff, you never told me where I was wrong in my other posts about why there will never be another 9/11 and how the TSA was not needed. I'll post it in a moment to give you a chance to respond to it.
[1] en.wikipedia.org/wiki/Aircraft_hijacking
[2] en.wikipedia.org/wiki/Timeline_of_airliner_bombing_attacks
Jim P. at December 5, 2012 10:13 PM
For all you regular readers of the Goddess' blog you can skip past this post. I'm going to post my regular rant about not needing the TSA. For all you new readers, please read it carefully and refute any statement or misstatement. ;-)
=================================================
The TSA was not needed one hour and one minute after Tower II was hit!
The paradigm, the norm, the expected, what everyone was taught to do was to sit down, shut up and wait for the plane to land and the negotiations happen. That was the model from Entebbe onward.
The passengers on board did not really know what was about to happen on September 11, 2001 at 8:46:30 when Flight 11 struck Tower I.
Even the passengers on Flight 175 probably didn't realize what was about to happen when they struck Tower II at 9:03:02.
The Pentagon crash of Flight 77 at 9:37:46 may have been still a matter of ignorance.
At 10:03:11 on September 11, 2001, United Airlines Flight 93 crashed after the brave souls counter-attacked and caused the hijackers to crash the plane.
The time difference is 60 minutes and 9 seconds from Tower II being struck to the crash of Flight 93. The shoe bomber and panty bomber were taken down by fellow passengers as well. Recently, JetBlue's Flight 191 pilot was taken down by the passengers once he was out of the cockpit. Additionally how many times have you heard of passengers' concerns and diverted flights?
The TSA is and has always been a joke, no make that a total stupidity, that has wasted our country's fortune going down a rabbit hole.
If you don't believe me look at the 9/11 timeline.
There will never be another 9/11 style attack unless the attackers can arrange planes full of geriatrics, and even then it would be doubtful.
Oh, and someone brought bombs being an issue. If bombs were effective and simple then the Lockerbie bombing would have been repeated multiple times between 21 December 1988 and 11 September 2001. That's 4647 days or 13 years. Where was the TSA in that time? There was one successful bombing that was done in Colombia and two unsuccessful attempts in that time. The bombing in Colombia was a drug dealer assassination and not a terrorist attack.
=================================================
Jim P. at December 5, 2012 10:14 PM
Jeff Guinn at December 7, 2012 8:46 AM
Oh by the way, I take it we all agree that this
is transparent nonsense.
Jeff Guinn at December 7, 2012 8:51 AM
Jeff Guinn at December 7, 2012 8:56 AM
Jeff, here's what you don't get:
TSA's not finding terrorists doesn't mean they are even there.
And you aren't showing a reason they aren't. Further, here's another basic fact of risk analysis that doesn't fly: no matter who brings it up, comparing the risk to that of another activity is a FALLACY. Colorectal cancer vs. patdowns or the scanner exposure? Total nonsense. As invalid as comparing military deaths in the Middle East with automobile death in the USA.
That's why this isn't going anywhere.
Now, who is using the term, "Constitutional blather" here? Someone so excited by their ideas that they can't see what they're doing.
Radwaste at December 8, 2012 4:51 AM
Would you mind parsing that sentence?
If that was what I was doing, it would be. However, if you took on board what I said, which was [funny] how you are completely willing to get anally violated by strangers when it is your skin in the game you would have realized it isn't about risk at all.
Amy, and you, et al, pitch a fit when asked to spend three seconds in a scanner, but when it is your skin in the game, you willingly undergo all manner of indecencies, even for a tiny reduction in risk.
That begs all manner of ethical questions.
Yes I have, it's called deterrence. You do know what deterrence is, don't you?
And it works in precisely the same way this does. It is wholly constitutional, and an effective deterrent, exactly analogous to checkpoint security, and decimates your argument.
Of course, if you really believe what you say, then you would sneak fake bomb on board, or, since it is so awfully easy, bribe ground staff to do it for you, and have media waiting to show how completely porous the TSA is. Heck, if you pulled it off, I'll bet there isn't a crime you could be charged with. (I'm surprised Amy hasn't already done it; after all, there is no such thing as bad publicity.)
But you don't.
I wonder why.
Jeff Guinn at December 8, 2012 8:55 PM
Amy Alkon
http://www.advicegoddess.com/archives/2012/12/04/sommer_gentry_d.html#comment-3508556">comment from Jeff GuinnWhat, exactly, is this incoherence?
"but when it is your skin in the game, you willingly undergo all manner of indecencies, even for a tiny reduction in risk"
Your attempts at arguments here have been pulverized into a fine dust -- over and over and over -- by many. It's almost humorous (but for the fact it's so tedious) that you are unable to recognize it.
Amy Alkon at December 9, 2012 12:00 AM
This "incoherence" is posing an ethical question. You will undergo colonoscopies, mammograms, and pelvic exams when it is your skin in the game. Even if the payoff is quite small: 0.8% chance of preventing your death from colorectal cancer, and far smaller payoffs for mammograms and pelvics.
So we know what you will do when it is all about you.
But when it is about someone else, you can't be arsed to stand in a scanner for several seconds.
That can't possibly be true, because my arguments haven't even been addressed.
To recap: Islamists are a threat. Islamists are engaged in asymmetric warfare. Their main enemy is the U.S., most lucrative target is airliners, and most effective tactic is suicide bombing.
Therefore, unless we have sufficiently effective measures to keep bombs big enough to down airliners off them, they will get blown up.
Which part is wrong?
If you grant the threat exists, instead of checkpoint screening, what?
Then there is your ongoing rant about the Constitution. You are wrong, and not in an opinion kind of way, but factually. You simply don't understand what the 4th Amendment means. It prohibits unreasonable searches. That means the constitution has nothing to say about reasonable searches.
As it happens, the FAA prohibits pilots flying with alcohol and other prohibited drugs in their blood stream. Seems reasonable — the presence of certain drugs increases the likelihood of pilot error. There is no way to reliably enforce the prohibition without some kind of testing. Given the risk and consequences, since 1995 airline pilots have been subjected to random urinalyses without probably cause (a positive result — BAC > .04% — means immediate firing, permanent loss of license, and possibly jail time). That search is constitutional because the prohibition is reasonable, and the means are commensurate with the prohibition.
So if you want to argue that checkpoint screening is unconstitutional, you either have to argue that the prohibition isn't reasonable — good luck with that — or that the means to implement the prohibition are more intrusive than required.
(BTW, there haven't been any mishaps attributable to alcohol since mandatory random testing started. That means it's useless, right?)
—
In the last couple months, you have put up a post that incorrectly cited evidence about scanner effectiveness; twice (including this thread) posted about highway death numbers that are transparently ridiculous; another that showed your favorite security expert knows nothing about explosives; repeatedly shown you aren't aware of constitutional decisions on reasonable searches; seemingly have no awareness of deterrence and even less about ethical issues. Jim P keeps posting the same hilariously irrelevant rant about hijackings, and Radwaste can't distinguish between an act of war and carelessness.
Further, you are two bloggers in one. Checkpoint security is ineffective. Checkpoint security is insane because it detected an artist carrying bomb components. We need to do it the way the Israelis do. The TSA is fascist when they try to do it the way the Israelis do.
You'll forgive me for not feeling the least bit pulverized.
Jeff Guinn at December 9, 2012 1:15 PM
Leave a comment