Sunday, 20 May 2012

How to evaluate an "Application Security Testing" company?

Having known the shortcomings of the scan based approach in the previous post, we can say that application security companies who just run scans do not deliver optimum output to the customers. Hence, we must evaluate them thoroughly to ensure if their testing process can deliver best results for an application and perform much more elaborate testing than the scanners. 

Now, the question arises how do we evaluate the testing process of an application security company? 

To solve this problem I have put forth these simple questions which can help us choose a good vendor in this space. If any testing company is satisfactorily able to answer the below questions, then it can be assured that it is doing a good job and is keen on identifying security vulnerabilities in the application, correctly.

Expected answers from a good vendor
Do you understand the application platform and the features?
Yes, we understand all the application features well, to know:
  •  Business rules
  •  Access levels
  •  Sensitive Data
Do you threat model the application?
Yes, before the test we analyze the application features and its design to realize all the probable threats applicable to the application.
Do you incorporate application specific test cases?
Yes, we include all possible application specific threats and test cases in our testing, in addition to the general ones that may be applicable to all the applications.
Do you include application platform or framework test cases?
Yes, we do include test cases specific to the technologies and frameworks used by the application. We research for such test cases prior to our testing.
Do you simply rely only on the automated scan result to know the vulnerabilities present in an application?
No, we do manual analysis to verify the scan results and cover the test cases left out by scans.
Do you tailor the test cases to understand and combat the security controls present in the application?
Yes, we attempt to understand or know the existing security controls like validations present in the application or as an inbuilt feature of its framework to launch advanced attack on the application.
Do you present general recommendations for the vulnerabilities or change them to suit the application?
Yes, we alter the standard solutions for the vulnerabilities to make them to suitable for the application being tested.
Do you look for advanced and new test cases?
Yes, we constantly keep updating our knowledge base with new attacks and also keep researching on advance attack vectors.

Application Security companies must use has a sound process to ensure that they are able to accurately look for maximum security test cases in the application in less time. Automated scans must be coupled with manual analysis to create a comprehensive test methodology. 

I am sure this helps you to find a good testing company.


  1. Nice post, keep them coming.

    We can have some more specifics in some answers, for example vendors mentioning, about their contribution vulnerability research etc.

    Or give examples of application specific test cases to look for.

  2. Thanks for the comments, Sid.