The question of how to improve the IT security of software and information-processing systems is more pressing than ever: while a single cyber-attack costs more than €15,000 on average, the number of malware variants has increased by 22 percent in 2021 alone. The damage caused by hacker attacks amounts to €203 billion in Germany in 2021 alone. Professional cyber criminals are increasingly targeting companies where there is enormous potential for extortion through the distribution and scaling of their products. Against this background, there is now a general understanding that this danger must be addressed appropriately.

What are the options for making software and IT systems more secure? There are some tried and tested methods such as: Adhoc testing, managed penetration testing or hybrid approaches (e.g. exploratory testing). All these methods specifically search for vulnerabilities that a hacker could exploit. This sounds good, but it has decisive disadvantages.

Adhoc tests provide quick results and usually find vulnerabilities in software. However, the more carefully the software is developed in terms of security, the more difficult it becomes for testers to actually find vulnerabilities. Adhoc security testing is thus a suitable method for quickly finding vulnerabilities that attackers can easily identify and then fixing them. This can make sense to increase IT security in the short term - or say: To increase justified confidence in IT security.

Whilst procedures such as adhoc testing are comparatively inexpensive, they have their weak points and a crucial catch from the IT security perspective. The quality of ad hoc security tests depends to a large extent on the testers’ or test lab’s skills, knowledge and experience. These competencies allow them to identify vulnerabilities in the software and devise successful attack vectors, which can then be closed. However - and here's the catch - many potential attack vectors can be overlooked in the process. Side-channel attacks such as Spectre and Meltdown, which rose to sad notoriety in recent times, are examples of vulnerabilities that are typically overlooked because they fall outside the focus of the testing.

Security Certification

Security certification, on the other hand, takes a systematic approach to security testing.

Common Criteria is considered the gold standard among non-industry-specific security certifications, and its artifacts can conveniently be applied to industry-specific certifications. Independent experts (mainly from governmental information security agencies) are involved in the certification process, checking the results and accepting or rejecting them according to well-documented methods. In case that the results are inadequate, improvements can be made and, in the event of escalation, certification can also be denied.

The certification process involves three stakeholders: The client or developer, the testing laboratory or evaluator, and the independent certification body. At the beginning of the process, all three agree on what exactly is to be certified, what is to be achieved, and how it is to be implemented. A document called the Security Target (ST) is prepared as a basis for the process.  The ST ensures that the product being evaluated and the associated definition of the security problem are clearly defined. It also guarantees that all (i.e. functional and assurance) security requirements are expressed in the language of the Common Criteria, which is clearly understood by all stakeholders.

Unlike in the aforementioned IT security evaluation methods, justified trust must meet clearly defined criteria and express these as a trust level. Evaluation Assurance Levels (EAL) indicate how high the (justified) confidence in IT security is from the point of view of a certifying authority.

The Common Criteria specify seven levels. The lowest level (EAL 1) expresses that a system has been functionally tested. In practice, this level is practically meaningless, because the cost of security certification is significantly higher than for unsystematic or semi-systematic approaches, but does not deliver more in terms of results. The highest level (EAL 7) also has no practical significance, because at this level mathematical proofs of certain security aspects must be provided, which can extremely difficult or impossible in practice.

Software with high complexity, a large code base, and a high number of assets that need to be protected, is more difficult to certify. An evaluation object such as a small bare metal application that has small functionality is significantly easier to certify than an operating system with a network stack and generally large functionality.

The compromise in reasonable confidence for a comprehensive software, therefore, lies in a high level such as EAL4 or even EAL5 with an ST that includes not only partial aspects (this is also possible), but all important functionalities.

In the final stages of the Common Criteria certification process, evaluators will perform a thorough vulnerability analysis through penetration testing, much like an adhoc security approach, but based on all the knowledge of the product gathered in the earlier stages of the evaluation (review of production specifications and documentation with traceability to code, functional testing of security requirements, etc.).

Conclusion                      

Adhoc and semi-systematic approaches are well suited to providing a quick assessment of the obvious potential for attack. But they may be unable to identify problematic structures early in the product life cycle and to fix them permanently. In the worst case, it is conceivable and does happen that an existing, mature and already established product has irreparable security gaps.

Certification processes such as Common Criteria are more complex and expensive, but provide more actual protection. This is expressed by levels of justified confidence awarded by independent certification bodies. This systematic approach allows developers to take a 360° view of threats and process their assets accordingly. Vulnerabilities that are typically missed in adhoc testing are also found during the process of security certification.

Certification thus pursues a long-term, holistic commitment to increasing the architectural security of software and IT systems. This methodical approach to making software secure drastically reduces the risk of irreparable software architecture.