Security in Software Engineering

by Nick Feamster

Introduction

Security in software engineering a broad topic. This article limits its scope to defining and discussing software security, software reliability, developer responsibility, and user responsibility.

Computer Systems Engineering

Software security applies information security principles to software development. Information security is commonly defined as "the protection of information systems against unauthorized access to or modification of information, whether in storage, processing or transit, and against the denial of service to authorized users of the provision of service to unauthorized users, including those measures necessary to detect, document, and counter such threats."

Many questions regarding security are related to the software life cycle itself. In particular, the security of code and software processes must be considered during the design and development phase. In addition, security must be preserved during operation and maintenance to ensure the integrity of a piece of software.

The mass of security functionality employed by today's networked world, might deceive us into believing that our jobs as secure system designers are already done. However, computers and networks are incredibly insecure. The lack of security stems from two fundamental problems. Systems which are theoretically secure may not be secure in practice. Furthermore, systems are increasingly complex; complexity provides more opportunities for attacks. It is much easier to prove that a system is insecure than to demonstrate that one is secure -- to prove insecurity, one simply exploits a certain system vulnerability. On the other hand, proving a system secure, requires demonstrating that all possible exploits can be defended against (a very daunting, if not impossible, task).

There is presently no single solution for secure software engineering. However, there are specific approaches which improve the likelihood that a system is secure. In particular, we can improve software reliability. We can also improve our understanding of what must be required to trust a given piece of software.

Good Practice

Security requires more managing and mitigating risk than it does technology. When developing software one must first determine the risks of a particular application. For example, today's typical web site may be subject to a variety of risks, ranging from defacement, to distributed denial of service (DDoS, described in detail later) attacks, to transactions with the wrong party.

Once the risks are identified, identifying appropriate security measures becomes tractable. In particular, when defining requirements, it is important to consider how the application will be used, who will be using the application, etc. With that knowledge, one can decide whether or not to support complex features like auditing, accounting, nonrepudiation, etc.

Another potentially important issue is how to support naming. The rise of distributed systems has made naming increasingly important. Naming is typically handled by rendezvous: a principal exporting a name advertises it somewhere, and someone wishing to use that name searches for it (phone books and directories are examples). For example, in a system such as a resource discovery system, both the resources and the individuals using those resources must be named. Often there are tradeoffs with respect to naming: while naming can provide a level of indirection, it also can create additional problems if the names are not stable. Names can allow principals to play different roles in a particular system which can also be useful.

Software Reliability

Software reliability means that a particular program must continue to work in the presence of faults. Faults can be related to design, implementation, programming, or usage faults. As systems become increasingly complex, however, the number of possible faults which must be defended against increases, and the likelihood of a failure increases. As mentioned, it is incredibly hard to show that a particular system is secure. Ross Anderson has said that computer security is much like programming Satan's computer. Secure software must work in the presence of faults which an attacker attempts to take advantage of. While most software has faults, most of which will never be seen under normal circumstances, an attacker will seek out these faults in an attempt to compromise the system.

Many of the security problems witnessed today are related to faulty code. For example, the Morris Internet worm used a buffer overflow in the UNIX fingerd program to gain root access to computers which ran the program. Buffer overflow attacks have been the most common attack over the past ten years, and involve overwriting instructions in the program being run by overflowing the input buffer. Specifically, a fixed amount of memory on the stack may be allocated for user input; if the user input is larger than this allocated space, the user can overwrite program instructions. If this is done carefully, the user can insert his own instructions into the program code, thus causing the target machine to perform arbitrary operations as dictated by the attacker. While such attacks can typically be prevented against with bounds checking (checking the size of the input before copying it to the input buffer), this is a matter of programming practice which we trust the programmer himself will follow. The difficult aspect of buffer overflows is that they can occur in a large number of places in any given program, and are difficult to prevent from happening everywhere. This has often been the case in the past, particularly over the past 10 years.

"Trusting Trust"

In particular, Ken Thompson's ACM Turing Award lecture "Reflections on Trusting Trust" raises an interesting issue where the integrity of a piece of software might come into question. Specifically, Thompson shows an example whereby a C compiler itself can be hacked with a Trojan horse in order to allow code to arbitrarily (and relatively undetectably) deliberately miscompile source code whenever a particular pattern is matched. For the purposes of demonstration, Thompson would insert his own version of the UNIX "login" code whenever a user attempted to compile the source code. The moral of Thompson's lecture is that "you cannot trust code that you did not totally create yourself." This includes code such as the compiler, the assembler, the loader, hardware microcode, etc. Thompson also points out that as the level of the bug gets lower, these "bugs" will be more difficult to detect.

In fact, we don't have to even be compiling software to be subject to Thompson's message. Whenever we download and install new software, RPMs, etc., we trust a number of things. First, we trust that the machine from which we are downloading the software is actually the machine it claims to be. Projects such as the Self-Certifying File System have attempted to address this issue. Although we trust that the machine which we are talking to is the one we think it is, we must still ascertain that these files were prepared (coded, compiled, packaged, etc.) appropriately. Thus, upon downloading such a file, we place ultimate trust in the integrity of the software development process.

While compromises of UNIX systems seem to generate more interest in academic communities, other operating systems are not immune. Buffer overflow attacks are widespread, from Windows ftpd servers to hidden processes which capture every user keystroke.

Regardless how much emphasis we place on secure software design and engineering, there must be some basic amount of software and hardware that we are not going to be able to trust completely. The level of threat present for a particular system (i.e., would someone actually take the trouble to hack hardware microcode on a consumer machine) and the amount of risk associated with placing trust in a particular piece of software will determine whether such software is trusted.

Distributed Denial of Service

Distributed denial of service (DDoS) attacks are often the cause of ethical concern. DDoS tools and kernel module hacks, are gaining prominence in the hacker community replacing the more traditional sniffers and rootkits. Some security measures have been taken against DDoS attacks, such as ingress filtering, IP traceback mechanisms, granting the FBI greater powers for search and seizure, etc., but most of these are geared more for standard denial of service attacks, rather than DDoS attacks.

DDoS attacks begin with one "master" which is responsible for compromising a number of "slave" machines. These slaves are then responsible for the attack. Often daemons are installed on multiple compromised hosts; a client identifies a target to the daemons, each of which launches a denial of service attack. The DDoS problem becomes much more serious as an increasing number of users are constantly connected to the Internet through cable modems or DSL lines. In general, these users are less likely to detect a system intrusion (not being vigilant hackers themselves), thus increasing their likelihood for becoming a slave in a DDoS attack. The advent of DDoS attacks has placed the burden of security on individual users.

Even with systematic software development resulting in "secure" software (which would, in an ideal world, consist of writing every piece of code yourself, or placing some amount of trust in someone else to do it appropriately), we, as users of computers in a networked world, do not operate in isolation -- we are heavily dependent on other users also doing their part in developing, designing, deploying, and using software which is secure.

Resources

1
http://web2.deskbook.osd.mil/valhtml/2/26/264/264J19.htm
2
"Distributed Tools" http://phrack.infonexus.com/search.phtml?view&article=p56-12
3
http://securityportal.com/articles/webdev20001103.html
4
Practical Support for IP Traceback, SIGCOMM 2000.
5
"A Stealthy Windows Keylogger" http://phrack.infonexus.com/search.phtml?view&article=p56-9
6
Thompson, K , "Reflections on Trusting Trust", Communications of the ACM, Vol. 27, No. 8, August 1984, pp. 761-763.
7
Yahoo DDoS

Biography
Nick Feamster (feamster@wind.lcs.mit.edu) is a graduate student in the Networks and Mobile Systems group at the Lab for Computer Science at MIT. His current research is in adaptive streaming of MPEG video and authentication and access control in wireless networks of devices. He has also worked on applications for secure video transmission and video transcoding.


Last Modified:
Location: www.acm.org/crossroads/xrds7-4/onpatrol.html