Tuesday, 29 May 2012

10 facts about Security Code Reviews

1.      What is the scope of a security code review?

A security code review is a form of an assessment and it is all driven by analysis. The objective of this assessment to identify all the security flaws present in a software along with their root causes,  by understanding and analyzing its code. 

2.      How different is security code review from application security testing?

Even the application security testing activity aims at identifying the security vulnerabilities of the application but it differs from security code review in its methodology.
Application security testing is performed by sending different test inputs to the application and analyzing their response as required for identifying different security flaws present in it. However, in security code review as the access to the entire code base of the application is available here is no need to send any test data to the application/software, the flaws can be easily discovered by analyzing the code.

In a security code review, the reviewer gets exposed to entire code layout of the application and many other important sections of the code, which would never get analyzed in an application security test. Some of those crucial areas include:
  •  Entry points for different inputs
  • Internal interfaces and integrations
  • Data handling and validation logic
  • Use of external API’s and frameworks
Code reviews are thus able to uncover security vulnerabilities from all areas of the code can be considered to be a much more authentic form of assessment than application security testing. 

3.      Is application testing included as a part of a security code review?

NO, there is no explicit need to perform application testing in a security code review; all the security flaws can be easily identified through the code. 

However, it is useful to access a running application to understand all its features. This understanding is required for identifying application related test cases for the review. 

There could be a need to perform small amount of testing to verify the flaws discovered at the code level. But, the process of discovering flaws is not dependent on any application testing method. It is only used, if required as an aid to support the analysis. 
4.      Is every line of code being reviewed in a security code review?

NO, I am sure this answer may have come to you as surprise. But, it is very much true. To get you more clarity on it let me ask you a question. 

Do you think security flaws will be present in every line of the code?

Well, if your answer to that one was No, I am sure you would have got the catch!

Certain sections of the code will be more prone to security vulnerabilities than the rest. Security flaws may arise due to improper coding or lack of specific security control at places where they are required in the code.  Both, different kinds of improper coding practices for any platform and the sections of the code that requires necessary security controls are known to a considerable extent today. Thus while reviewing the code looking out for such vulnerable cases first is important. What remains then are application's feature specific flaws, which require a high level understanding of the business logic and not line by line analysis of the code.

Considering the limitation of time for carrying out this activity the better way to do this it is:
  • Understand application features and realize application specific test cases to be reviewed for different features 
  • Understand the code layout of the application. So that tracing the code for any feature is simplified
  • Understand different validation schemes and techniques (data and access control validations) 
  • Identify security flaws in the application design and  underlying platform/framework 
  • Look for all  insecure code instances across the application 
  • Feature wise look for application specific security vulnerabilities (lack of business validation)
However, there must be a robust approach to review the applications so as to blend the above checks in a manner that gives optimum accuracy in review results.
5.      Are functional bugs discovered during a security code review?

NO, functional bugs are not discovered in a security code review. However, nothing would stop someone from analyzing functional defects while performing security code review. But if it is done so, it may hamper the quality of the review. If the reviewer focuses on many things in a single review it may be difficult to achieve optimum results and chances of error will creep in. 

The objective of the security code review is well defined and it is only aimed at discovering all the security flaws in the application/software being reviewed.

Due to above factors the security code reviews must be carried out separately, which will make the process of the review more exhaustive and thorough.
6.      Is design of the application reviewed?

YES, the review of the application design for security flaws is included in the security code review of the application. Understanding the design gives a good insight of the code layout and the interaction of various components of the application

The design related flaws are insightful and if they are left uncaught they may pose a big risk to the application. It requires good deal of understanding and manual analysis to uncover design flaws in an application. And it is unquestionably a part of the security code review process.
7.      When should I perform security code review?

It is needless to say that code must be developed with the sense of security in mind. Application developers must be aware of insecure coding and development practices and must be cautious about avoiding them while developing applications. 

There is no ideal time to carry out a review, but it is advisable to carry security code review after completion of a feature, module or an application. Incomplete code will not give proper sense of the functionality to the reviewer and thus might hamper the reviewer’s understanding of the code.

Review of smaller chunks of code can be done easily if it is a part of the development process. It is good practice though to incorporate security checks at different stages of application development.
8.      I have reviewed my application once is that enough?

Well, good to know that the application has been reviewed once. But before I answer that question let me ask you that will the application’s code base not undergo any code change after it is developed?

Owing to different reasons the code changes are continuously introduced into the application. These code changes will also bring in some security flaws with them if they go without security review into the application. Hence, it is just not enough to review the applications code base once. Either there must be a continuous review process for all the code changes introduced into the application or a periodic reviews of the entire code base of the application.

Also, it is good to follow security best practices while developing applications so that the chances of security flaws introduced into the application keeps decreasing.
9.      Are there tools available for performing security code reviews?

Yes, there are many code scanners available that can perform good secure code analysis. As needed they are useful in identifying insecure code instances and API usage. However, there are many areas of the analysis that require manual efforts once like review of design and application specific cases, etc. It would be good to use scanners as an aid to make the review process comfortable. But they cannot be used as a complete replacement to manual analysis in this activity. (Check this)
10.      Who can perform a security code review?

It takes a good deal of application security and software programming knowledge to be able to understand the code and analyze it thoroughly. There are many application security companies that also offer “security code review” as a service who can be one of the choices to carry out this activity. However, external vendors may have less understanding about the application and internal security teams may lack a defined process to carry out such a review activity. Irrespective of this trade-offs it must be ensured that the best guys are picked up to perform this activity. (Check this to help you further.)

Sunday, 20 May 2012

How to evaluate an "Application Security Testing" company?

Having known the shortcomings of the scan based approach in the previous post, we can say that application security companies who just run scans do not deliver optimum output to the customers. Hence, we must evaluate them thoroughly to ensure if their testing process can deliver best results for an application and perform much more elaborate testing than the scanners. 

Now, the question arises how do we evaluate the testing process of an application security company? 

To solve this problem I have put forth these simple questions which can help us choose a good vendor in this space. If any testing company is satisfactorily able to answer the below questions, then it can be assured that it is doing a good job and is keen on identifying security vulnerabilities in the application, correctly.

Expected answers from a good vendor
Do you understand the application platform and the features?
Yes, we understand all the application features well, to know:
  •  Business rules
  •  Access levels
  •  Sensitive Data
Do you threat model the application?
Yes, before the test we analyze the application features and its design to realize all the probable threats applicable to the application.
Do you incorporate application specific test cases?
Yes, we include all possible application specific threats and test cases in our testing, in addition to the general ones that may be applicable to all the applications.
Do you include application platform or framework test cases?
Yes, we do include test cases specific to the technologies and frameworks used by the application. We research for such test cases prior to our testing.
Do you simply rely only on the automated scan result to know the vulnerabilities present in an application?
No, we do manual analysis to verify the scan results and cover the test cases left out by scans.
Do you tailor the test cases to understand and combat the security controls present in the application?
Yes, we attempt to understand or know the existing security controls like validations present in the application or as an inbuilt feature of its framework to launch advanced attack on the application.
Do you present general recommendations for the vulnerabilities or change them to suit the application?
Yes, we alter the standard solutions for the vulnerabilities to make them to suitable for the application being tested.
Do you look for advanced and new test cases?
Yes, we constantly keep updating our knowledge base with new attacks and also keep researching on advance attack vectors.

Application Security companies must use has a sound process to ensure that they are able to accurately look for maximum security test cases in the application in less time. Automated scans must be coupled with manual analysis to create a comprehensive test methodology. 

I am sure this helps you to find a good testing company.

Friday, 18 May 2012

Don't Just Scan!

Application security Testing should be process driven and not scan driven. Running a scanner and relying on its results for identifying security flaws is not a comprehensive way of testing an application. There is no denying that scanners are helpful. They aid in finding a wide variety of vulnerabilities present in the application. But in reality there are many issues that are left untouched by the scanners. Hence, if an application testing methodology is solely dependent on scanners, then it is probably incomplete. 

The scanners check only for some defined set of vulnerabilities for all kinds of applications and do not tailor the scans as per the application types. They neither understand business process of the application nor its architecture or the design. Thus they are incapable of threat modeling an application. And due to these reasons security flaws related to business logic of the applications are mostly left out by the scanners. 

Some of such flaws include:
  • Understanding role based access required by different features of the application and pointing out violations in the same
  • Pointing out instances of lack of enforcement of business rule related to different features, wherever applicable. For e.g. Transaction limit for funds transfer, etc.
  • Identifying all the crucial parameters present in the application that can be altered to gain access to authorized data and functions in the application
...and many more.

What the scanners follow is a standard test pattern which is generally dependent on sending predefined set of attack vectors in various input fields of the application and analyzing their responses. Such scans not only miss out covering application specific cases but they are also incapable of understanding any existing security control present in the application to make an attempt to modify the attack vectors accordingly to bypass it.

So, we can conclude that by just running scans for an application we would miss out on:
  • Application feature and platform related security flaws
  • Determining the correctness of the existing security controls (if any)
There are many good application security scanners available today that look at most of the well known application security vulnerabilities. However, the point I want to make here is that just running scanners is not enough. Security testing methodology must be capable of understanding the application and uncovering all kinds of security flaws in it even once related to its business logic, rules and processes.

Organizations must have a well defined application security testing methodology that includes both automated and manual analysis. Security scanners should be used to speed up the process of security testing and as an aid to find all the vulnerable instances of the generic vulnerabilities present in the application. Whereas manual efforts can go into verifying the scan results and looking at other security areas untouched by the scanners, ones that require business sense of the application.

So, let’s make our application security process comprehensive by just not scanning the application but by following a good application security methodology. At least until the scanners become matured enough to handle even the application feature specific scenarios…