VOLUME 5

State of Software Security Report
The Intractable Problem of Insecure Software
APRIL 2013

Read Our Predictions for 2013 and Beyond

Dear SoSS Report Reader,
As some of you may know I have spent most of my 25 year career in the IT Security industry, more specifically, I’ve been focused on application security as the use of web and mobile applications has flourished. For the past five years I have been an active participant in the preparation of the report before you today—our annual State of Software Security Report, or as we fondly refer to it at Veracode, the SoSS Report. Throughout my career I have been evangelizing the need for more secure application development practices, and with the release of each new SoSS report I find myself of two minds. The optimist in me is proud of the vast improvement in general awareness of the importance of securing the application layer. But the pessimist remains very concerned that we are not seeing the dramatic decreases in exploitable coding flaws that I expect to see with each passing year. It’s as if for each customer, development team, or application that has become more secure, there are an equal number or more that do not. While the benefits of web applications are clear to organizations, the risks to their brands, infrastructure, and their data are seemingly not as clear, despite being more apparent than ever. It’s at this point of my letter that I could mention that a cyber-Vesuvius is about to bubble over and create a cyber-Pompeii as there are so many breaches reported; but I’ll resist that temptation. Instead, here are a few links to recently released reports that do a shockingly good job of telling the scary story: • 2013 Trustwave Global Security Report 1 • 2012 Verizon Data Breach Investigations Report 2 I only cite these examples because the reports illustrate the “after” scenario, evaluating what has happened when vulnerable systems are exposed to the threat space. We at Veracode see the SoSS report as different, using data to shine light on what is to come by understanding the latent vulnerabilities in software organizations are deploying. The “before” scenario means our SoSS reports have become great predictors about future data breaches. For example, this report shows 32% of applications analyzed by Veracode contain SQL injection flaws. Knowing that, you should not be surprised that Trustwave reported that SQL injection was the attack method for 26% of all reported breaches in 2012. I can tell you with confidence that malicious actors target the flaws that are easy to find and exploit—like SQL injection—therefore the instances of SQL injection attacks will surely increase in 2013. Put more bluntly, we must figure out a way to code more securely simply to keep up with attacks from the most basic attacker. As you read this report I urge you to consider your organization’s application portfolio and how you currently make decisions about the risks your organization is willing to take. The amount of risk an organization takes should be a strategic business decision—not the aftermath of a particular development project. If you’re learning about risks after a breach—be it yours or an industry counterpart—then the time to act is now. Use this SoSS report to estimate your current application risk landscape— particularly on applications that you have never tested or only tested manually. Consider how you can act now to improve the security posture of your organization, by addressing the applications that you currently have in development and/or in production. Hopefully by the time we release SoSS V6 in 2014, we’ll see that dramatic improvement I’ve been waiting for! I hope you enjoy the report.

Chris Wysopal
Co-Founder, CISO and CTO, Veracode

1 2

www2.trustwave.com/rs/trustwave/images/2013-Global-Security-Report.pdf www.verizonenterprise.com/resources/reports/rp_data-breach-investigations-report-2012_en_xg.pdf

Veracode State of Software Security Report: Volume 5

Table of Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Key Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Security of Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Compliance with Standard Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Remediation Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Security Quality Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Language Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 .NET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 PHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 ColdFusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Applications Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Mobile Threat Landscape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 State of Mobile Application Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Web Application Threat Landscape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 State of Web Application Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Non-Web Applications Threat Landscape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 State of Non-Web Application Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Appendix A: About the Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Appendix B: Understanding How the Veracode Platform Determines Policy Compliance . . . . . . . . . . . . . . . . 37 Whisker Plot Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 P-Value Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Generalized Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

1

Veracode State of Software Security Report: Volume 5

Introduction
For the past five years, the Veracode State of Software Security (SoSS) report has examined trends associated with vulnerabilities in applications. Our initial goal was to provide key insights to those charged with managing enterprise application security risk, to give them a series of benchmarks from which they could measure their own application security posture. After five years and five versions of SoSS our goal now is to highlight the slow progress in securing the application layer. Since insecure applications are a leading cause of security breaches and data loss for organizations of all types and sizes, we can’t continue to whistle past the graveyard. We want the readers of this report to leverage the data to build a business case for an application security program at their organization.
As with past SoSS reports, this analysis draws on continuously updated information in Veracode’s cloud-based application security platform. Unlike a survey, the data comes from actual application security assessments conducted to identify vulnerabilities and validate remediations. SoSS Volume 5 examines data collected over an 18 month period from January 2011 through June 2012 from 22,430 application builds uploaded and assessed by our platform (compared to 9,910 application builds analyzed in Volume 4, which was published in December 2011). This report examines application security quality, remediation, and policy compliance statistics and trends. The data analyzed represents multiple security testing methodologies (including static binary, dynamic and manual) on a wide range of application types (web, mobile and non-web) and programming languages (including Java, C/C++, .NET, PHP and ColdFusion). We also expanded our analysis of the mobile vulnerability landscape with sections on Android, iOS and Blackberry applications. The resulting intelligence is unique in the breadth and depth it offers. Readers of Volume 5 will notice an increased focus on vulnerability distribution trends for each language. Also, new to this Volume is an analysis of the percentage improvement in the vulnerability distribution between the first and second application builds. This metric should provide some perspective on which vulnerabilities organizations chose to fix upon receiving the results from their first submission. These new visualizations and analysis are in response to customer questions about demonstrating the impact of security programs on enterprise risk profiles. Veracode’s data analytics team is always looking for new perspectives on security metrics. We welcome reader questions, comments and ideas so that we can continually improve and enrich the coverage, quality and detail of our analysis.

2

Veracode State of Software Security Report: Volume 5

Executive Summary
The following are some significant findings in the Veracode State of Software Security Report Volume 5. Each finding is accompanied by a prediction for the next 12 to 18 months, where we sketch out the possible futures if the status quo continues. We also provide recommendations for altering our predicted trajectory, because we can change the future.

Key Findings
70% of applications failed to comply with enterprise security policies on first submission. This represents a significant increase in the failure rate of 60% reported in Volume 4. While the applications may eventually become compliant, the high initial failure rate validates the concerns CISOs have regarding application security risks since insecure applications are a leading cause of security breaches and data loss for organizations of all types and sizes. Prediction: Average CISO Tenure Continues to Decline. The average tenure of a CISO is 18 months, and more CISO jobs will be at risk given the current state of software security. The expansive threat profile associated with software means the likelihood of CISOs being negatively affected by a high-impact security event has never been greater. Recommendation: Driving up compliance with enterprise application security policies lowers the risk of high-impact events. To accomplish this, CISOs and security professionals must work closely with their counterparts in Development and Procurement to set security policies and enable internal and external developers to consistently comply with those policies. SQL injection prevalence has plateaued, affecting approximately 32% of web applications. The downward trend in SQL injection that we reported in Volumes 3 and 4 has flattened. For six consecutive quarters, from the first quarter of 2011 to the second quarter of 2012, the percentage of applications affected by SQL injection has hovered around 32%. This should be a concern, as three of the biggest SQL injection attacks in 2012 resulted in millions of email addresses, user names, and passwords being exposed and damaged the respective brands. Prediction: The Rise of the Everyday Hacker. Once the sole domain of technical experts, now a simple search for “SQL Injection Tutorial” enables anyone to exploit a serious vulnerability and wreak havoc. The data shows that everyday hackers are on the rise, as Trustwave reported SQL injection to be the attack method for 26% of all reported breaches in 2012. We predict that number to exceed 30% in 2013. SQL injection vulnerabilities are just too easy to find and exploit. Recommendation: Organizations should institute zero-tolerance policies for SQL injection vulnerabilities and employ routine monitoring to detect vulnerabilities as new applications are deployed.

3

Veracode State of Software Security Report: Volume 5

Eradicating SQL injection in web applications remains a challenge as organizations make tradeoffs around what to remediate first. The percentage of applications affected by SQL injection has hovered around 32% and cross-site scripting around 67% for the last six quarters. For the first time we are reporting improvement percentages by language to illustrate which flaws organizations are choosing to fix after receiving results from their first submission. Java, representing 56% of web applications, showed 16% improvement in SQL injection and 14% improvement in cross-site scripting between the first and second submission. .NET, representing 28% of web applications, showed a 25% improvement in SQL injection and 15% improvement in cross-site scripting. Prediction: Decreased Job Satisfaction/Higher Turn-over for Security Professionals. The challenge is daunting. Companies face a seemingly ever-expanding threat profile brought on by new applications and application updates containing easy to exploit flaws such as SQL injection (26% of all 2012 reported breaches according to Trustwave). This can create a very frustrating work environment for security pros. The desire to find roles where their efforts will bear more fruit and where success is apparent will drive increased turnover among security pros. There is some good news, however. According to the Bureau of Labor Statistics,3 the employment segment that includes information security analysts is projected to grow 22% between 2010 and 2020, faster than the average for all occupations. Recommendation: Making a difference as a security professional often means building relationships with development executives. Instead of taking a “scan and scold” approach, the program goal should be improving overall developer productivity by efficiently integrating security remediation into existing development methodologies. Getting development executives focused on process integration, knowledge transfer, remediation support and incentives for secure code creation as key success criteria would represent a significant breakthrough in the relationship. Cryptographic issues affect a sizeable portion of Android (64%) and iOS (58%) applications. Using cryptographic mechanisms incorrectly can make it easier for attackers to compromise the application. For example, cryptographic keys can be used to protect transmitted or stored data. However, practices such as hard-coding a cryptographic key directly into a mobile application can be problematic. Should these keys be compromised, any security mechanisms that depend on the privacy of the keys are rendered ineffective. Prediction: Default Encryption, Not “Opt-in,” Will Become the Norm. Eavesdropping on mobile communications can make it easier for attackers to design successful social engineering attacks against key employees. There is a staggering amount of transmitted data at risk, considering the growth of open (i.e. easy to eavesdrop) Wi-Fi networks in combination with the number of social network users (Facebook 1.2B; Twitter 190M tweets/day) and the number of mobile devices (Cisco4 predicts that by the end of 2013, the number of mobile devices will exceed the number of people on earth—7.1B). These concerns have prompted companies like Twitter and Facebook to encrypt all traffic by default, despite the additional computing power required to encrypt every connection. As more business is conducted through applications resident on personal mobile devices, we expect enterprises to insist on mobile applications that force encryption to protect data in motion. Recommendation: Developers and security professionals should expect data encryption to be involved in all aspects of designing the business user’s experience with mobile applications. From a data in motion perspective, this would include understanding the performance impact and incremental infrastructure costs of encrypting traffic between the mobile application and the server side application. From a data at rest perspective, additional attention should be paid to the cryptographic techniques used to protect the application itself from unintended data disclosure.

3 4

www.bls.gov/ooh/computer-and-information-technology/information-security-analysts-web-developers-and-computer-network-architects.htm www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.html

4

Veracode State of Software Security Report: Volume 5

Security of Applications
The evidence linking organizational intrusions and data breach events to application security issues continues to grow. Web-based intrusions and hacking in general account for 52% of the breaches in 2011 and 2012 tracked by Open Security Foundation’s DataLossDB (Figure 1).
While these categories are extremely broad, hacking and web-based intrusions often involve exploiting software vulnerabilities. Reports published by companies that conduct actual breach investigations provide additional insight. The Verizon Data Breach Report, released in March 2012, indicated that 81% of attacks utilized some sort of hacking. This section explores application compliance with standard policies, remediation submission rates and security quality scores to shed some light on why this connection exists.

Data Loss Breaches Categorized by Root Cause
45% Hack 15% Stolen Item 12% Fraud-SE 7% Web 5% Lost or Missing Item 5% Disposal Item 4% Unknown 2% Snail Mail and Fax 2% Email 2% Virus 1% Skimming and Snooping
Figure 1: Data Loss Breaches Categorized by Root Cause (Source: DataLossDB)

Compliance with Standard Policies Upon First Submission
Figure 2 illustrates the compliance upon initial application submission against two standard policies.5 Web applications are assessed against the OWASP Top 10 and only 13% complied on first submission. Non-web applications are assessed against the CWE/SANS Top 25 and 31% complied on first submission. Only 30% of applications complied with enterprise defined policies. Compliance with policies upon first submission of an application can be a good indicator of the success or failure of “building-in” security as part of the software development lifecycle (SDLC).

5

More details about how the Veracode platform determines policy compliance can be found in the Appendix.

5

Veracode State of Software Security Report: Volume 5

Because security flaws that are eliminated before deployment, or never created in the first place, are much less expensive to remediate, thus building remediation into the SDLC at an early stage is often a key goal for most organizations. Yet, with more than two thirds of the applications failing to comply, our results show that secure software development practices are still not as widespread as they should be. While applications may eventually become compliant, the high initial failure rate validates the concerns CISOs have regarding the business risks related to application security.

Compliance with Policies Upon First Submission
Compliant Out of Compliance

Enterprise Policy

30%

70%

CWE/SANS Top 25

31%

69%

OWASP Top 10

13%

29%

87% 71%

0%

20%

40%

60%

80%

100%

Figure 2: Compliance with Policies Upon First Submission

The OWASP Top 10 compliance rate did not change significantly from Volume 4 (14%). In contrast, the percentage of applications passing enterprise policies declined significantly from Volume 4 (40%). Similarly, the percentages of non-web applications that complied with SANS/CWE policy were respectively 42% and 31% in Volumes 4 and 5, which is highly statistically significant decrease. We decided to investigate whether language or supplier types where potential drivers of the decrease in SANS/CWE policy compliance. Our analysis6 suggests that the major contributors to this Vol4 to Vol5 decrease in compliance rate were as follows: • There is evidence that Language (99.9% confidence level) influences CWE/SANS compliance with C/C++ being the most significant factor. This means that C/C++ applications, which represent 29% of non-web applications in our dataset, had a significant impact on driving down the CWE/SANS compliance rate from Volume 4. • There is no compelling evidence that software supplier types (internally developed, commercial, outsourced, and open source) influence CWE/SANS compliance. Another possible factor in the decrease of SANS/CWE policy compliance could be the increase in the number of first submissions in our sample set. The Volume 5 data had 75% more first builds than Volume 4. This increase in first builds suggests more broad use of the service by a wider variety of companies, perhaps with higher variation in secure software development practices.

6

The analysis that we performed used a generalized linear model to perform logistic regression on a proportional response variable (SANS Compliance) with categorical explanatory variables (Volume, Flaw Category, Industry, Supplier, and Language). See Appendix for additional detail.

6

Veracode State of Software Security Report: Volume 5

Remediation Analysis
We frequently get questions from customers and analysts on whether discovered vulnerabilities are actually remediated and whether those remediations are validated through additional testing. To shed some light on this issue, we start by examining how frequently organizations resubmit applications following the initial analysis. These resubmitted applications typically contain a combination of security remediations for previously reported vulnerabilities. Resubmitted applications may also contain new or altered code components to address non-security issues and new code components representing new functionality. One might expect that more companies would resubmit higher percentages of their very high criticality applications than they would their medium criticality applications. If this expectation were true then one would anticipate the distribution pattern for medium and very high criticality applications to look very different (possibly a classic bell curve for the medium criticality applications and an exponential curve for the very high criticality applications). At the very least, one might expect variability in resubmission rate to decrease as application criticality increases. The data does not support those expectations. Figure 3 shows statistically insignificant differences in the distribution patterns. Roughly 45% of organizations resubmit 91-100% of their applications regardless of the business criticality. In Volume 4 we reported that the very high group was slightly different from the high and medium groups, since over 50% of companies resubmitting 91-100% of their very high criticality applications, however that slight difference has disappeared in Volume 5.

Percentage of Applications Resubmitted by Business Criticality
Medium 50 High Very High

PERCENT OF ORGANIZATIONS

40

30

20

10

0
0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100

PERCENT OF APPLICATIONS RESUBMITTED

Figure 3: Percentage of Applications Resubmitted by Business Criticality

7

Veracode State of Software Security Report: Volume 5

Security Quality Analysis
We continue to track the quarterly mean Veracode Security Quality Score (SQS) as a means of determining when security quality becomes a standard part of developing software. We expect that when most organizations have built security into their SDLC we will begin to see an upward trend in SQS developing. An upward trend would indicate that applications from new Veracode customers and newly developed applications from existing customers are a less significant force in dragging down the mean with very low scores. Figure 4 shows we still have a lot of work to do in building in security. The best fit line across our analysis timeline has a p-value 7 of 0.37 indicating that the trend is flat. This flat trend is consistent with the trends reported in Volumes 3 and 4—there has been no increase or decrease in the quarterly mean SQS since the fourth quarter of 2009.

Veracode Security Quality Score Trend p-value = 0.37
100 80 60 40 20 0 2011-1 2011-2 2011-3
QUARTERS

MEAN VERACODE SQS

2011-4

2012-1

2012-2

Figure 4: Veracode Security Quality Score Trend

Next we examine the progress an application makes build-over-build as the developers respond to findings and attempt to remediate flaws using the median value of the Veracode Security Quality Score (SQS) as a progress indicator. The distribution of the Veracode Security Quality Score by application build is shown as a whisker plot 8 in Figure 5. The data shows statistically significant build-over-build improvement from the first to third builds. Builds four through six remain statistically flat, followed by a marked improvement in builds seven and eight. The median score decreased in build nine, however, it is still above the plateau of builds four through six. This pattern suggests the security quality in applications with nine or more builds has been permanently improved even as functionality in the form of new code is being added in the later builds.

7 8

See Appendix for definition. See Appendix for definition.

8

Veracode State of Software Security Report: Volume 5

Veracode Security Quality Score by Build

VERACODE SECURITY QUALITY SCORE

100

80

60

40

1

2

3

4

5
BUILD NUMBER

6

7

8

9

Figure 5: Veracode Security Quality Score by Build

The pattern of statistically significant improvement in security quality scores for builds one, two and three seen in Figure 5 is consistent with the figures reported in Volumes 3. However, there are significant differences between Volumes 4 and 5 in the patterns reported for later builds. In Volume 4 we saw an oscillating behavior with peaks occurring at builds four, seven and nine. The Volume 4 pattern suggested that new functionality was introduced in the build after each peak, which resulted in a new set of security flaws found and the consequently lower score. It is not immediately clear what has caused this shift in pattern between Volumes 4 and 5. It could be that our dataset for later application builds is richer in Volume 5 and therefore more representative of the actual improvement pattern. It is also possible that developers are starting to introduce new code that does not suffer from the vulnerabilities in the old code. The developers have learned from the mistakes and do not repeat them.

9

Veracode State of Software Security Report: Volume 5

Language Analysis
In this section we dive deeper into each language. For each language we look at the distribution of each vulnerability category. We measure vulnerability distribution in terms of share of vulnerabilities found in each language group.
We calculate this by first filtering our data by language. For each language we determine the total number of vulnerabilities found and the number vulnerabilities that belong to a specific category. These values allow us to calculate the percentage share for each vulnerability category for that language. These vulnerability distribution calculations allow us to make statements such as, 3% of vulnerabilities found in Java applications are SQL injection vulnerabilities (Figure 6). The vulnerability distribution metrics also give us a historical perspective, since we have been reporting them since Volume 3. Another metric we explore is the vulnerability prevalence in terms of the percentage of applications affected by each vulnerability category. To calculate this metric, we also begin by filtering our data by language. Then we identify how many applications contain one or more vulnerabilities from each category, which allows us to calculate the percentage affected. These calculations enable us to make statements such as: SQL injection vulnerabilities affect 31% of Java applications (Figure 7). Vulnerability distribution and prevalence information can be useful for planning purposes, particularly when internal and/or industry-specific benchmarks9 are not readily available. Organizations can estimate the resource impact of implementing or changing application security policies. Consider the situation of a security team writing a policy aimed at eliminating SQL injection flaws and a development team writing their application in Java. The percentage affected data tells the teams there is a 31% chance that their application will have SQL injection flaw. The vulnerability prevalence data means that if the application does have SQL injection, it is likely that only 3% of the vulnerabilities found will be SQL injection. Finally, we investigate the percentage improvement in vulnerability distribution between an application’s first and second build. This metric should provide some perspective on which vulnerabilities organizations chose to fix upon receiving the results from their first submission. For each language, we looked at the subset of applications with their first and second builds occurring within the analysis timeframe for this report. This means we excluded applications with their first build occurring before January 2011 and applications with their second build occurring after June 2012. We also excluded applications with components written in more than one language. Then we calculated the change in vulnerability distribution from the first build to the second build. The percentage change will be affected by the volume of flaws. For example, consider the case of a development team that has fixed 10 flaws between the first and second build. If there were 20 flaws in the first build then the calculation would show a 50% improvement. However, if the first build contained 100 flaws, then the calculation would show a 10% improvement. To acknowledge this impact we indicate the top vulnerability categories in the percentage improvement charts. The percentage change may also be affected by improvements to the Veracode platform, and we’ll discuss those improvements where applicable.

9

The Veracode Analytics capabilities enable organizations to benchmark their internal application security metrics with industry benchmarks.

10

Veracode State of Software Security Report: Volume 5

Java
Figure 6 shows that vulnerability distribution in Java applications has not significantly changed since Volume 3. The cross-site scripting category consistently represents more than half of all vulnerabilities discovered in Java applications. In the Volume 5 dataset, SQL injection makes its first appearance in the top 5 list at fifth place, replacing cryptographic issues. Figure 7 shows code quality, CRLF injection and information leakage affecting the most applications with 82%, 68% and 58% respectively.

Vulnerability Distribution Trends for Java Applications (Share of Total Vulnerabilities Found)
Rank Volume 3 Volume 4 Volume 5

1 2 3 4 5 6 7

50%

56%

51%

Cross-Site Scripting (XSS)

17%

16%

21%

CRLF Injection

14%

10%

12%

Information Leakage

4%

4%

3%

Encapsulation

5%

3%

3%

SQL Injection

3%

3%

Directory Traversal

2%

Cryptographic Issues

Figure 6: Vulnerability Distribution Trends for Java Applications (Share of Total Vulnerabilities Found)

11

Veracode State of Software Security Report: Volume 5

Vulnerability Prevalence in Java Applications (Percentage of Applications Affected)

Code Quality CRLF Injection Information Leakage Cross-Site Scripting (XSS) Cryptographic Issues Directory Traversal Insufficient Input Validation Encapsulation API Abuse Credentials Management Time and State SQL Injection Session Fixation Race Conditions OS Command Injection 0% 9% 10% 20% 30% 40% 50% 60% 70% 18% 38% 37% 34% 34% 31% 29% 44% 49% 58% 57% 55% 68%

82%

80%

90%

100%

Figure 7: Vulnerability Prevalence in Java Applications (Percentage of Applications Affected)

Figure 8 indicates the untrusted search path category had the highest improvement percentage from first to second application build. Although this vulnerability category does not occur very often (it is absent from Figure 6 and Figure 7) it contains some very high severity flaws. For example, CWE-114 is defined as executing commands or loading libraries from an untrusted source, or in an untrusted environment, can cause an application to execute malicious commands (and payloads) on behalf of an attacker.10 Figure 8 also shows an improvement percentage of 45% for CRLF injection, which holds the second place in both Java vulnerability distribution (21%) and prevalence (68%).

CRLF injection, which holds the second place in both Java vulnerability distribution (21%) and prevalence (68%), showed an improvement percentage of 45% from first to second submission.

10

For the complete description see cwe.mitre.org/data/definitions/114.html

12

Veracode State of Software Security Report: Volume 5

Percent Improvement in Java Vulnerability Distribution from First to Second Submission
Indicates categories with the highest vulnerability distribution in Java

Untrusted Search Path CRLF Injection Untrusted Initialization Session Fixation Dangerous Functions Code Quality Encapsulation Credentials Management Cryptographic Issues API Abuse SQL Injection Insufficient Input Validation Time and State Cross-Site Scripting (XSS) OS Command Injection 0% 8% 10% 20% 30% 40% 50% 60% 70% 80% 18% 18% 16% 15% 15% 14% 23% 23% 28% 36% 45% 45% 44%

90%

90%

100%

Figure 8: Percent Improvement in Java Vulnerability Distribution from First to Second Submission

.NET
The vulnerability distribution for .NET applications has not changed significantly over the last three Volumes (Figure 9). Cross-site scripting (XSS) retains the highest share of vulnerabilities at 49%. However, the percentages have been changing over time. Cross-site scripting and directory traversal categories have been slowly increasing while information leakage and cryptographic issues have been slowly decreasing.

Cross-site scripting and SQL injection showed improvement from first to second build in terms of share of vulnerabilities discovered, but still affect 60% and 30% of all .NET applications respectively.

13

Veracode State of Software Security Report: Volume 5

In addition, 61% of .NET applications contain one or more XSS vulnerabilities (Figure 10). The high percentages in both metrics indicate that cross-site scripting is a pervasive vulnerability, i.e. it occurs many times in many applications. Figure 11 appears to indicate a fairly low percentage improvement (15%) between the first and second build for XSS. When taken together, these three data points demonstrate the enormity of the task of removing cross-site scripting from existing applications, because there are so many vulnerabilities to remediate. Significantly, the top five categories that showed the most improvement are comprised of less than 10% of all discovered flaws and affect at most 50% of all .NET applications (Figure 11). If you leave out SQL injection, the top four categories that showed improvement comprise at most 20% of all .NET applications. Cross-site scripting and SQL injection showed improvement in terms of share of vulnerabilities discovered, but still affect 60% and 30% of all .NET applications respectively.

Vulnerability Distribution Trends for .NET Applications (Share of Total Vulnerabilities Found)
Rank Volume 3 Volume 4 Volume 5

1 2 3 4 5 6

44%

47%

49%

Cross-Site Scripting (XSS)

23%

18%

14%

Information Leakage

11%

10%

11%

Directory Traversal

8%

9%

9%

Cryptographic Issues

6%

6%

6%

Insufficient Input Validation