mostly useless crap from me

Well, what an interesting weekend. A cold, working like a slave, and one of my co-workers is a father for the first time (Congrats, Ty!). But that’s not the most interesting news.

I will be taking sole ownership of my forum, Aussieveedubbers, sometime this week. This means that I will have to spend a bit more non-existent personal time attending to it.

This is good and bad news:

• For UltimaBB, the underpinning forum software, it’s fabulous news. UltimaBB was never released and is now effectively a dead project. My acquisition of AVDS gives me the impetus to make the forum software as good as I can make it. Once I’m complete, as it’ll be by far the most secure PHP forum out there. Very few open source programs ever get the chance of a top to bottom code review. Once I’ve fixed all those issues, I will think about possibly adding a few flashy features and integrate it with CPanel and others, so ISPs can easily deploy it. Obviously that integration will not come cheap. Hopefully, I can start to earn a bit of income from the forum, finally.
• For OWASP, this is not good news. I have two current projects, the Developer Guide and ESAPI for PHP. I need ESAPI for PHP to be complete to help UltimaBB, so you can guess which of the two projects will get my time.
• For my personal life, I hope Tanya will forgive me making her do some basic accountancy work. I think it’ll ease her eventual way back into the workforce in a few years, as she is terrific at accountancy, and I would hate to see her lose all her skills for the want of a bit of work here or there. However, it’s not just delegating the book work to my poor suffering wife, it’s also a bit of an ask for the few hours I have to give right now.

So if you want to help with the Developer Guide, please join the mail list and let us know how much time you have, and where your interests are.

The last time I talked about OWASP’s new Application Security Verification Standard, I had performed a Level 2B-3 review of my forum software, UltimaBB.

This time, I’m working on a real project for a real customer. It’s been interesting.

• Level 1A and in particular, 1B has been emasculated. I’m not really sure of the value of these reviews as they have 22 and 13 controls to review respectively. Taken together at Level 1, it might have *some* value, but I’m not sure you’re getting a whole lot of assurance. I would only recommend this level if you have like 1000 apps to review, and you need to see which are the most atrocious so you can target Level 2B / 3 / 4 reviews. Sure they can be done quickly, but I’m not sure they prove anything in particular. The good news is I reckon the current state of the art dynamic and static tools could produce reports absolutely compliant with this level with no real changes other than ASVS report format (R1-R4) and risk rating methodology. The problem is that there are no tools today that do both automated dynamic testing (a la IBM’s AppScan or HP’s WebInspect) and automated static code analysis (like Fortify’s SCA or Ounce O2) in the one tool, so combining the output of two different tools in the time available would be a challenge (whilst being incredibly boring and unrewarding to a skilled assessor).
• The basis of a Level 2B review is “manual” verification, most likely using the results of automated scanning. You don’t HAVE to do the automated scan – you can do it by hand. Based upon my experience of both UltimaBB and this review (a mixture of ASP and ASP.NET), the automated scan was more of a waste of time than a blessing. Yes, it found XSS and was quite specific to its location, but honestly, in terms of performing an ASVS review, as long as you know how XSS works (lack of input validation and output encoding), you should be able to find them with grep or your favorite search feature in far less time. The problem is that the scanners are asked to scan for 88 things at Level 2B, and I’m pretty certain that without AI, the scanners are only going to be able to do around 50 of the findings automatically. For example, I have a CSRF token in my forum. The scanner I use claimed that each of the forms had no CSRF protection – a false positive that took nearly all of the alloted 30 minutes to eliminate, and thus put me behind schedule.
• Time ~= money ~= quality. We tried to make Level 2B reviews work in five days as this is a current sweet spot of this rather depressed market. Five days was chosen on best practice productivity of 15 minutes per control + slop time + QA. With the new ASVS release, there’s now 88 controls to review. That’s about 1 control needing to be written up every 20 minutes, assuming four hours for QA (10%) and four hours for the Exec summary. Doesn’t sound like much, but there are many controls that simply take longer than that. Some don’t. I honestly think Level 2B reviews cannot be completed satisfactorily on large or complex code bases in five days.
• Level 3 reviews have 109 controls in 13 categories. This is at least 10 working days’ worth of work – each control gets 30 minutes (assuming 1 day for QA and four hours for Exec summary), which is about right. I personally think Level 3 is about the right level for most code reviews as it does nearly everything a Level 4 review does, but is realistic in terms of schedule, budget and outcomes.
• Level 4 Reviews have 121 controls in 15 categories. If anyone is offering a Level 4 review of any size code base in less than four weeks, take a very hard look at their methodology. I just don’t believe you can satisfy the need to look through each file for malicious content in less than that time frame and give it a “professional indemnity insurance proof” result. This level of review is painstaking and I doubt many people will end up asking for it.
• Report Length will be an issue. Using the reporting format (R1-R4), the average Level 2B report in 12 point fonts will be about 100-120 pages long. This is far too long for the paying customer (== execs, which is different to the consuming customer – the developer), so spending time on the Executive Summary and follow up report out is essential if you’re to communicate the results in any meaningful way. So you know those time lines I mentioned above, take out a good four+ hours out of the juice to write up a proper Executive Summary and other report out materials. This reduces your time per control down to 20 minutes for a five day report and about 30 minutes for a ten day report. Obviously, if you have a large or complex code base, you’re going to be hosed if you’re not on the top of your game every single work day. Put away the nerf guns – it’s work time!
• For those of you used to writing reports that eliminate sections that don’t apply, you’re going to get a shock with ASVS. You need to write up ALL 22, 13, 56, 88, 109 or 121 findings – regardless of if the code is awesome or awful. Leave time for it!
• The OWASP risk rating scheme is a monster. 16 elements per item x say 88 items = A BUCKET LOAD OF CALCULATING. If you’re still using Word to write your reports, you may want to write macros to automate the calculation and Executive Summary elements, or else you’ll be here next year working out what the risk is. You should also check the balancing on the OWASP risk ratings. I find they produce a lot of mediums and highs. I will talk to Jeff about making the scale 0, 1..5, and producing a universal 1, 3, and 5 set of elements to make it easier to produce a more balanced risk rating. Find some of your previously QA’d risks and try them out on the OWASP risk rating to see if you get similar results (and you should!). If you don’t, adjust the risk rating methodology and document it in your report. You don’t want to be known as the risk manager’s nightmare. Too many highs == less work in the future if they constantly (successfully) argue with your ratings for being unrealistic and too high.
• Missing controls. By design, ASVS does not have every control under the sun. Some missing controls are actually very surprising. As there’s so much work in an average ASVS review already, I doubt many folks would add back these missing controls. However, I think the ASVS team is making a mistake by not making Level 3 and 4 include some of these more common controls, particularly if the clients asking for Level 3 / 4 reviews probably already have these controls in their IT security polices and would like to know about the status in their apps. We’re talking things that are found in ISO 27001 and COBIT 4, so I’m not just being tin foil hat crazy here.

So what do I think about ASVS for code reviews? The more I use it, the better I feel that we’re meeting the our customers’ need to produce something that doesn’t produce HIGH risks for information disclosures like internal IP addresses (which are irrelevant). The customer is in control of the amount of looking at their code, and we’re producing developer ready results that affect the design and architecture of the code, which hopefully means much safer applications in a few months’ time.

Good news! Safari 4.0 has:

• Supports read only HttpOnly protection
• XMLHttpRequest read protection for set-cookie, set-cookie2, and GetAllResponseHeaders!

It does not protect against cookie writing.

Test script here: http://greebo.net/owasp/httponly.php

This is a great improvement! Now all major browsers support HttpOnly in some form.

thanks,
Andrew

A long, long time ago, I took on running Aussieveedubbers, a forum based around the love of Volkswagens. We were on EzBoard, where the adverts and performance sucked so bad, that free was no longer acceptable. Over many iterations, I now run UltimaBB, a derivative of XMB. I had various titles – including lead programmer – at both projects for a while, but my main claim to fame was to get XMB 1.9.1 out the door with Tularis, the lead dev at the time, and the primary early force behind UltimaBB along with John Briggs. John implemented the hacks, and I did the architecture changes required to make them suck a lot less and hopefully make them more secure. In a way, I have – UltimaBB does not suffer from ANY of the XMB Bugtraq failures. Therefore, I am a coding god and UltimaBB is secure, right?

I got going in forum development primarily through security fixes. By the time the UltimaBB effort died and XMB imploded in 2008, I thought I had gotten most of the security bugs out of the code. UltimaBB was a shining beacon of the bazaar approach. Or so I thought.

But like most in open source, there’s a huge difference between what I think is secure, and what is actually secure. Just because I think it is secure doesn’t make it so. I wish more folks in open source projects would get this very simple message. You suck, and as long as you know that, everything (in time) will be okay.

So, like all good scientists, I do not let my good opinion of myself get in the way of a solid slap session. I gave UltimaBB a thorough code review using OWASP’s forthcoming Application Security Verification Standard at Level 2B, which encompasses a manual code review for 83 controls with an optional use of automated static code review tools. I used Fortify’s awesome Secure Code Analyzer, which is almost certainly the best in the business today. It didn’t find everything, but it highlighted a number of deficiencies I already knew about quite reliably.

I can’t claim I did this review because I suddenly had a huge chunk of time on my hands, but because I am implementing a new service at Pure Hacking, and we’re using ASVS as our verification baseline. I’ve added a few more things in as I think they’re important, but the main thing is that I’ve now validated that the ASVS produces a reasonable outcome for an average program, one that should have been reasonably secure – because *I* wrote a great deal of it, and certainly the security bits.

The good news is that ASVS can be used as a security consultancy’s baseline for a partially automated, but primarily manual code secure code review. I think with a bit of tweaking, we can develop automated searches in most languages for about 50-60% of its requirements, which will dramatically improve the quality and reliability of secure code reviews.

The bad news is that my forum sucks. I have a bunch of extreme, high, medium and low risks to fix. Until I’ve fixed them (as there’s precisely one forum running this code out there – mine!), I can’t go into them too much, but let’s just say that I was disappointed with myself, and I’ve taken myself out the back and slapped me silly.

Luckily, I have a good client for this review (me), who will actually fix all the defects and lacks found so far.

OWASP EU 2009 is coming up! This year, it’s held in Kraków, Poland. Time to book!

Program highlights:

• Keynote: Ross Anderson from Cambridge University. I’ve wanted to meet Ross for many years. Those guys are legends!
• Keynote: Bruce Schneier. I bet there are groupies
• w3af – Andrés Riancho. This is one of the best free toolkits I’ve tried recently. It’s awesome.
• HTTP Parameter Pollution, Luca Carettoni, Independent Researcher & Stefano Di Paola
• OWASP Source Code Flaws Top 10 Project, Paulo Perego, Spike Reply
• O2 Advanced Source Code Analysis Toolkit, Dinis Cruz
• … many others!

Although I would love to be there – I had a blast at OWASP EU 2006, I can’t attend this year. Which is a shame, because OWASP AU 2009 was huge fun, and I can’t imagine OWASP EU 2009 would be anything less.Don’t make the same mistake as me! Book now!

I am appearing at OWASP Melbourne tonight. Come along and enjoy my take on protecting business value.

AccessReferenceMap, RandomAccessReferenceMap and IntegerReferenceMap, and enough of the other classes (FileBasedAuthenticator, StringUtilties, etc) are present and working:

This is very good news as although some of the other classes in Milestone 1 are complicated, these two classes were actually going to be some of the hardest to port as PHP does not have the equivalent of J2EE Set, List, HashMap, and many other basic data structures. What PHP does have is native associative arrays (somewhat like HashMaps), ArrayObject, and ArrayIterator from the SPL. The problem is that PHP doesn’t like sparse arrays with very long indexes:

$foo["THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX. THIS IS A VERY LONG INDEX."] = $value

So I had to make up a workable hash function and hope for zero collisions. I tried using spl_object_hash(), but that actually is too good. It uses the object’s value AND pointer position, such that:

$foo = “123″;

$bar = “123″;

spl_object_hash($foo) != spl_object_hash($bar)

I think I still need to add a few more test cases as my hash function WILL collide when there are two direct object references of the same value, and thus will not be safe for some uses.

I’ve been working on the essentials for OWASP ESAPI, and now it passes its first set of unit tests, in this case a 1:1 mapping of the ESAPI exceptions test class.

This is the first set of classes that fully passes a set of tests that is exactly equivalent to the J2EE trunk SVN. Yes, it’s one test, but it tests the exceptions thrown by every single one of the Exception classes.

This is key as ESAPI throws a lot of ESAPI exceptions when things go south. In addition to ESAPI exceptions, the PHP port will also throw SPL exceptions, such as InvalidArgument and so on as it makes sense to do so.

To get this far, I’ve had to hand hack the Authenticator, User, Logger, and Intrusion Detection classes – currently no errors are sent out by ESAPI for PHP, but give me a bit of time and it will happen. String Utilities is also partially there. Authenticator is interesting as it actually does generate strong passwords, and actually reads from the resources directory for the user’s file and decodes it into an array. However, some of these behaviors are hard wired to allow more of the Milestone 1 classes to pass tests, rather than be part of the Milestone 3 build.

I’ve started work on the RandomAccessReferenceMap class. It’s almost there; but unfortunately, I’ve got to go to bed as it’s 2 AM. It’s so close I can smell it. Once done, that class is a close relative of the IntegerReferenceAccessMap, and so there are likely to be two valid and useful ESAPI for PHP classes done soon. I’ll see if I can finish it and check it in before I have to go to work on Monday.

No posts for like a month or two, and two in one day? Time for some shameless crass philanthropy and some good natured commercialism.

In some exciting news:

• I’ve donated my one and a bit ESAPI / ASVS training deck I gave at OWASP AU 2009 to OWASP! It’ll be available as soon as the education project finds a home for it. I’ll come back and link to it when it’s ready. Very rarely does an entire 1+ day deck escape into the wild, so I hope the OWASP Education community runs with it, and constantly improves it.
• My deck is forming the basis of Pure Hacking’s new two day developer training! Obviously, we’ll be extending the deck and giving it the Pure Hacking spin, but fundamentally it’ll be the same focus on building secure applications and not breaking crap applications. Our new deck will be ready at the end of the month for your training pleasure!

In other news, Pure Hacking is holding a one day WebEx (i.e. remote) training session on Testing Web Applications with Ty Miller as your host. If you’re interested, please drop me a line.

Last night, I spoke to the phpMELB folks for an hour on ESAPI for PHP.

The talk went well, and they taped it. When the video appears, I will link to it.

More importantly, I worked on ESAPI for a couple of hours after returning last night, and finally have something to show everyone! ESAPI for PHP almost passes some tests:

This means that folks can start cutting code as the test framework and the main framework are fully stubbed out and ready to go.