mostly useless crap from me

It would have been nice to get Web 1.0’s security fixed first before starting on Web 2.0. And before Web 1.0 was … the mainframe.

In my time with health care providers, at one of the world’s largest telcos, at various largish Australian banks, and over the last few weeks teaching mainframe folks about secure coding techniques, I’ve come to a conclusion: we’re in a crisis. Not only have we not really solved the necessary problems in Web x.0 land, we haven’t been keeping an eye on the ball in the most important area of all: where the value lies.

Critical nationally protected infrastructure all use mainframes to process the value of our lives:

• Banking (money – a necessity, which allows you to buy housing)
• Logistics (getting stuff from A to B)
• Defense (making sure we’re protected)
• Food chain (eating – a necessity)
• Government (unfortunate necessity)

Pretty much covers Maslow’s Hierarchy of Needs in every way – modern life would not be possible without mainframes. We cannot:

• house ourselves (buy houses with a mortgage),
• eat (we no longer farm or hunt ourselves and rely on getting access to food at supermarkets (who use mainframes to keep stock levels and understand what their customers are buying) which requires shipping – and most large logistics firms use mainframes for everything from load optimization through billing and scheduling
• or even get water depending on your water utility.

I look back wistfully at seminal security papers and works from the 1960’s and 1970’s. I feel miffed that this 40 year head start has been squandered. Almost everything we’ve “discovered” in web application security is old news to these researchers. Their programmer contemporaries are still out there, building mainframe apps, and they do not know anything about the old research, let alone the new. “Web apps? Not my job, governor!”

Today’s mainframe devs do not know the core principles – confidentiality, integrity, and availability (except for performance reasons). Key security principles such as defense in depth, complete mediation, and validation are no longer practiced, even though the old code lived and breathed it.

In my discussions with mainframe devs, they struggle with basic security concepts, and even in some cases, the need for security at all. After all, the attackers know nothing of the mainframe and they can’t get on to the system. Right?

In the end, the fault partially lies with the application security community. We in web application security know no mainframe stuff. I don’t know anyone competent to review COBOL, RPG, PL/1, IBM assembly language (!), or a host of 4GLs. I don’t know anyone who knows a jot about RACF, ACF2, and all the security mechanisms available on mainframes since before I was born.

Very few know anything about TPF, MVS, z/OS, CICS, and VM. And yet, taking even one of these – TPF – could reward study for years. It powers all the airlines, the travel industry (Galileo, the world wide booking system used by everyone who travels, is a TPF application), and even 911 emergency response systems. And yet, I must admit I’ve never heard of it until last week, and yet I’ve worked around folks who used mainframes for the last 15 odd years. What chance does that newbie .NET coder who needs to have a secure ordering system involving a mainframe as a backend? We have no advice.

So we can’t properly review applications that use mainframes … except to treat them as black boxes. And that is wrong! Most large organizations have a 30-40 year investment in their applications and they’re not going to re-write in a Johnny come lately language like Java or C# just because we can’t review old code. There are literally billions of lines of COBOL out there, and it … runs the world. There should and MUST be a way we can review this code.

What needs to happen in the security research community?

We need to research the security concerns in commonly used mainframe languages (COBOL, RPG, PL/1, assembly) on top of common platforms (z/OS, others) using their common data stores (DB2, and so on). We will break old/new ground here. Probably this stuff was being thought about back in the late 70’s early 80’s, but was never published. I think our field’s brightest researchers simply stopped looking at commercial environments around this time.

We need to understand their native toolsets to provide key security architecture deliverables, such as using RACF to provide authC and authZ services. The mainframes have a rich set of security tools… but no one uses them any more.

We need to become competent in reviewing these older systems and applications, so we can provide reasonable advice

We need to be choosy about what to review – there’s simply so much code at every large firm and government agency and basically no one to review it

Although OWASP should be publishing the results of this work, maybe we as the security industry should have an extended residency with the IBM Redbook folks to get access to the real deal (do you have a mainframe?), peer review our work, document all our findings and get it back in the hands of the mainframe devs in a timely fashion.

What needs to happen on the mainframe side of the fence

Business requirements should reflect in the security architecture. We know that mainframe has almost all if not all of the necessary support of a “modern” security architecture:

Data protection

• Confidentiality. Most mainframes have an inbuilt HSM. Few sites are using this feature! What a waste. Let’s get this out there and encourage the use of hard core encryption
• Integrity. Most of the protocols in the SNA stack have inbuilt integrity controls, and yet once they are mated to TCP/IP, the common protocols such as ftp, x3270, and access to MQ is all unencrypted and has no integrity controls. This needs to change

Identity management

• RACF is basically as old as me. It supports everything a SSO vendor would happily sell a Java site and has done for over 30 years. It now supports modern pass phrases, LDAP support, and so on. These “advanced” features should be in use at more sites.

Event management

• I still need to do more reading, but I can’t imagine that the systems which power banks has no audit or logging features. Let’s learn how to differentiate between logging and auditing, and ensure these logs are protected from unauthorized use.

The folks on the mainframe need to understand the powerful tools and native platform support they already in their possession. This requires training and support from their vendor. IBM? Hear me.

We in the security research community can’t possibly understand every possible version of every language and platform out there. It’s time to start consolidating around a few common platforms and languages. I propose the security industry sticks to supported platforms and languages as IBM has a chance of rectifying any problems we find.

This will be painful for many, but it’s like the pains the accounting industry went through in adopting the GAAP. We can’t and should not audit a planet’s worth of loose receipts.

I am not hopeful that any of this will change any time soon. I am willing to have a dialog with any and all mainframe folks. Let’s talk.

Sad Andrew

Reading:

Writing:

I am at the SANS GSSP second face to face in Chicago (photos soon). SANS have chosen a nice hotel, the W Lakeshore right on Lake Michigan.

Until 10 pm tonight, it was awesome. But then at 10 pm… It was spoiled by the Richter level 4.0-4.9 bass drivers (seriously! – we’re feeling it in our waters – constantly – my diet Pepsi has ripples in it). It’s 1.30 am. I have to get up at 7.30 am – on a Sunday, a miracle not often seen – even with a good night’s sleep.

This hotel has forgotten its core duty: a good night’s sleep for ALL of its guests. We are the ones paying nearly US $400 per night, not the young things paying $10 for a drink at the nightclub.

Never come here – spread the word.

In 2035, I will be 65. Most likely, unless I was to take up photography or cat breeding, I will most likely still be in this industry doing pretty much what I’m doing today.

Why?

I submitted a bunch of “how to fix” talks to OSCON (the unconverted) and Black Hat (the converted). I’ve spoken at both before, and I know I don’t suck too badly at speaking. Knowing that you suck more than other folks is the first step to being a good speaker, and I learnt that many years ago and have been learning ever since. Nowadays, I get good reviews from my customers, got good reviews and write ups for my last talk at OSCON. Black Hat provided me with my feedback which indicate that most of the folks who returned the forms liked what I had to say and how I said it, although there is room for improvement. When I train professionally, I am probably my harshest critic. That said, everyone – including me – can always learn how to present better, and make presentations that don’t suck. But let’s put that aside for a moment, and look at our industry’s premier developer and security conferences.

Why you will not learn solutions at any major event this year

I know this might come across as sour grapes, but seriously, when the biggest “security” conference rejects my talk (which will show how to scale code reviews in large enterprises, a huge problem for the Fortune 500, government and defense types, who just happen to send a bunch of folks to said conferences) in favor of the same theoretical root kit talk as we saw last year and a meta-theoretical anti-root kit talk targeting that specific theoretical root kit talk, they’ve lost the plot. When the largest *developer* conference rejects three of my talk suggestions, two of which are teaching developers how to code more securely (including a advanced level 300 class – I’m sick of teaching “hey, this is htmlentities(). He’s your friend”), they’ve lost the plot, too*.

OSCON’s security track is a paltry seven talks, basically most of one day out of five. And only one, by my friend Chris Shiflett, will teach you how to avoid the most common problems in web apps and another reports on the use of a source scanning tool by the open source community. Each of those talks is less than an hour. The chance you’ll learn something you don’t already know about PHP security is pretty small. At Black Hat, so far, there’s plenty of announced talks, but it will take you until day two before you learn how to do something useful. There are no other how to fix talks at Black Hat. That’s very, very sad.

There are some fine speakers at each event, for sure. But some have been seen before. And before that, too. But when you’ve seen ten theoretical root kit talks, or the fiftieth hundred buffer overflow talk (the same attack since 1988? kill me now), or yet another XSS talk or eight, we get it. Software sucks.

How do we fix it? Show me the money!

Do I want to be fixing SQL injections, buffer overflows or cross-site scripting issues when I’m 65? Hell no. These are solved problems. We know the solution. They MUST be burnt into the APIs so that programmers (no matter what skill) CAN’T do it the wrong way. There are some fine researchers working in the field, and you’re not going to hear them talk about fixes at Black Hat or OSCON. It’s Fear Uncertainty and Doubt. Scare the punters so you’ll buy their products or services. That security sales method is so 1995 when we thought firewalls were kinda neat.

That sucks.

It’s the reason the security industry is little more than snake oil modulo a few gems here and there. Why don’t A/V vendors go white-list? Spend 10 minutes telling your computer about the programs you use and white list the behavior of those probably very common apps? No more virus infections as everything else is untrusted and doesn’t run. That’d kill their shakedown revenue stream.

To be a smart security vendor today, you provide value to the customer by showing them how to architect a secure solution, how to build secure software (by training their devs – we can’t write all the software), how to test and review software (or indeed provide these services as an external audit function), so they don’t have to worry about spending *more* money on useless controls or worse case, notifying the regulators and their customers that they’ve screwed up and “gee, we’re sorry! we tried our best. Here’s $100 bucks”. Value folks, value. We’re here to provide secure business, not scare money out of folks. Once the horse has bolted, it’s far, far too late. That’s why I think forensics and a lot of compliance is a total WAFTAM. Dead money.

Providing solutions is exactly what we’ve been doing at OWASP. We provide value. Some of the solutions are actually getting towards voting age. We just need to get it out there so you don’t make the same mistakes, time after time. I’ve dedicated the last four – five years to researching, describing and educating how to fix things at OWASP. And yet, we don’t get no love at major conferences. And here’s why – they don’t want to tell you how to fix it. They want headlines in the meeja. The meeja only know about attacks, “hackers”, and people losing money to organized crime gangs, or their daughters to the nasty pedos across state lines. So the conferences provide that. We all lose with this approach. Luckily, with OWASP, we run the conferences, so this year, I will speak, and hopefully it will be useful to those who attend.

But realistically, the folks we want to talk to are at BlackHat and OSCON, not at OWASP (yet). So let’s learn …

How to write a successful talk submission

First off, and foremost, be honest about why you’re going. You’re a conference whore, and so am I. The hallway track is their raison d’etre, and best experienced with booze and lots of it. But how to get there… write a submission!

0. The title must be snappy. “Attacking OMG PONIES!111 2.0″ All good talks have 2.0 in them somewhere.
1. Subject matter must ONLY be about attacks, exploits, or bragging. The more esoteric the subject of your attacks, the better. I’m talking to you, side channel attacks.
2. Reading poetry to the attendees is only acceptable if it’s accompanied by images of death and you’re dressed in a funny hat, so try to come up with a reasonable approximation of how much your new tool (P0NIE PWNER) haxxors the badness (OMG PONIES!!111) you claim to attack. You don’t need to provide the tool, just claim it exists. No tool / exploit == no attendance.
3. Don’t include anything – ever – about how to fix the problem. That’d ruin the the “hacker” image of the conference.
4. Profit!

Conclusion

So screw them. See you at Black Hat. I’m the one who looks like a trans-gender lady of negotiable affections and I’m lovin’ it.

–

* OSCON has a talk on PHP security, by Zeev Suraski, one of PHP’s founders. The talk (PHP Security: Fact and Fiction) which sounds pretty defensive. Hopefully, it will say something like “gee, sorry about that!” to all the attendees. I’m very hopeful about the claimed agenda – it talks about what is changing in PHP to fix their previous stupid insane security decisions and lack of a security architecture. PHP *must* move in that direction, and fessing up to current and past indiscretions is the first of at least 12 steps to resolving the issue. Look at ASP -> ASP.NET. Same thing.