Maurice Elzas on Simulation Ethics

Introduction

Simulation is a fundamental tool used for network performance evaluation. For example, the performance evaluation of new and existing network protocols (such as TCP) is mostly done using simulation tools [3, 4]. In fact, several studies show that simulation is one of the most widely used techniques for evaluation and decision making, if not the most widely used. Given its widespread usage, results and conclusions based on simulation studies can be critical for technological advancement, decision making, and often real life. Yet, simulation can be employed without following "standardized" procedures. Take, for example, network performance evaluation where there is no single, standardized way for showcasing the performance advantage of a given proposal (see also [1]). In addition, some studies do not present the full details of the simulations performed. Furthermore, sometimes critical information about the exact methodology used in these studies is not disclosed to the reader.

But, what is simulation? Zeigler [6] notes that

"the phrase `modeling and simulation' designates the complex of activities associated with constructing models of real world systems and simulating them on a computer. [..] In particular, modeling deals primarily with the relationships between real systems and models; simulation refers primarily to the relationships between computers and models." (See Figure 1.)

Two terms that one should pay special attention to are verification and validation. Verification is the process of determining whether the conceptual simulation model (model assumptions) has been correctly translated into a computer program. Validation is the process of determining whether a simulation model (as opposed to the computer program) is an accurate representation of the system, for the particular objectives of the study [5].

Figure 1: Simulation and Modeling

Professor Maurice Elzas has led the effort to establish a code of conduct for simulationists. I met Elzas at Eurosim 2001, in Delft, The Netherlands, where he was the chair of a session on Simulation Ethics. I found the session very motivating and I asked him to share his insights and opinions on the issue with ACM Crossroads. He agreed, and we proceeded in the following interview, which was conducted via e-mail.

Maurice S. Elzas is Professor Emeritus of Computer Science, Wageningen University, The Netherlands. He was one of the founding members of the Department and served as the Chairman for several years. He also served as the President of the Council of European Professional Informatics Societies, and is a Senior Member of the IEEE and SCS. He is the author of more that 200 publications in the form of reports, articles, technical papers, chapters in books and an encyclopedia, and can speak seven languages. In the past, Elzas consulted some of the most prestigious organizations and companies including ABN-AMRO, DEC, IBM, Unilever, but also the Dutch Navy, the US Army Material Command, and several Dutch Ministries. He is married, father of three and grandfather of six.

I think we should start with the basics: What is simulation?

I would say that we all know implicitly what simulation is: it is the main method by which we learned almost everything when we were small children: by imitating our parents, we learned to walk, speak, show emotions, etc. While playing with toys, in fact some kind of models of real objects, we (re)enacted events that happened in the real world. When reading a book, our mind emulates a world created by the author and interpreted in our personal way. In other words, simulation is mimicry. Never as good or true as reality but as good as the basic model governing it (our synthesized perception of reality) can be. Notice that modeling and simulation are necessarily close-tied together. Simulation allows us to experiment with a "reality" that is, for example, too dangerous to let happen, not yet realized or hypothetical, in the course of development (where we want to know more about e.g., feasibility, performance, reliability, etc.), and so on.

Last year, at Eurosim 2001, I heard you mentioning the term "simulationist." Can you give us a definition of who a simulationist is?

In principle everybody, but - to be more practical and in view of the professional degree than can be achieved - the practical definition is everyone who applies or is involved in simulation as a service to others. In other words, a person that builds models, runs associated simulations and interprets results for a third party (e.g., colleagues, management, or other clients) as his main professional activity is a simulationist.

The Rise of the Simulationist

Most, if not all, graduate programs in computer science and applied mathematics offer a course in simulation and modeling, while several schools offer undergraduate level courses as well. Simulation techniques are also taught in other disciplines, including engineering, economics, management, and some of the humanities. Many schools offer Master's and PhD degrees highly focused on Simulation and Modeling, and there are (at the time of this writing) two institutions that offer graduate degrees in Simulation and Modeling:

The College of Engineering at the Old Dominion University has recently started a program that leads to a PhD in Engineering with focus on Modeling and Simulation
The Naval Postgraduate School offers the MOVES Ph.D. Program

Why is simulation important for network design and evaluation?

For many networking problems, we can find some form of semi-analytical solutions thanks to queuing theory and the work of Kleinrock, and others. In order to keep the analytical model mathematically tractable researchers often make a number of (simplifying) assumptions. Nonetheless, these solutions mostly apply to "steady state" cases. A steady state is achieved when all system parts have achieved equilibrium, that is, when the dynamics have disappeared to such an extent that the time of measurement does not influence the result of the measurement in a decisive way. (In most mathematical contexts, e.g., the solution of differential equations, a steady state is reached asymptotically, that is, for all practical purposes, at infinity. Many situations in practice (contention, collisions, momentary network overload, crash of a node, etc.) do not appear in steady state studies, the key of their evaluation lies in knowing their time-linked behavior intimately. As we cannot predict or measure such events accurately, we will have to know how to handle them (preferably beforehand) by simulating those situations after modeling our network as accurately as possible.

But the impact of simulation and modeling is more far-reaching, right? I mean, we hear about simulation studies regarding nuclear waste disposal, water management in developing countries, and so on. Of course, everybody knows that many professionals (including pilots) do most of their training using a simulator. How important is simulation for everyday life? Is it becoming more pervasive and more critical for all of us?

I think the answer is yes. The importance of modeling and simulation has become so great because:

the apparent complexity of our society has become so large that we can not solve some problems by conventional (mental) processes anymore, even if we were to be the greatest experts in the particular field addressed
decisions and measures in one field directly or indirectly affect issues in other fields. For example, decisions about economic growth have an impact on the environment, the relation between rich and poor countries, the growth of the human populations, etc.
many of these phenomena have intrinsic feedback loops (that we know about or are not even aware of) meaning that the coupling effects can not be neglected or put aside
many models of real life phenomena are chaotic, stiff and/or highly non-linear. This entails that small causes can have gigantic effects and vice versa. Remember the famous anecdote about the butterfly flipping its wings in Hawaii causing a hurricane in the Caribbean

All of these, and the fact that we have made great progress in the methodologies and tools for modeling and simulation, have as a consequence that most of us (who either make decisions or give advice for them) use these tools to play "what if" games ahead of time, to fathom more or less where we will end up if we go ahead and do it. A good example is the Federal Reserve Board: Mr. Greenspan does not make his decisions about interest rates on basis of his gut feelings but uses models to reason about the effects.

However, simulation is still considered by many as the "method of last resort." When all other methods fail, we use simulation. To go one step further, in some cases (flawed) simulation results are used as justification, or should I say alibi, for decisions that otherwise would not stand any criticism. Is it time to coin the expression "lies, damn lies, and simulation?"

The size of the lie depends on the knowledge that we can put in the model. In this way, we can cover the whole field from realistic performance assessment to pure conjecturing. The first applies to "hard" systems (technical like cars, airplanes or computers without taking human interaction into account); the second to "soft" systems (with living, e.g., human, components like we find in sociological models, economic models, interaction models, etc.) It all depends on how far we can get with validation, i.e., proving that the model or the simulation is "right" by comparing the results with real-life experiments. Unfortunately, in many cases, the required experiments are impossible to carry out, at least with the prerequisite degree of detail and accuracy that is needed for creating sufficient trust in the model. It is especially in these cases where the modeler/simulationist has to be candid about the liberties he has taken and the relative merits of his/her results.

So, do simulationists need a code of ethics?

I would rather talk about a code of conduct, as ethical behavior is taken for granted as the realm of churches, educational institutions and the law. A code of conduct is more practical and can concentrate on the ethical aspects of a specific (set of) profession(s).

Is there a need for a code of ethics?

To some, codes of ethics serve no good purpose whatever, because ethics should be open-ended and reflective, while relying on a code of ethics is to confuse ethics with law. Further, it is a mistake to assume that there are special ethics for professionals that are separate from the ethics of ordinary human beings within a moral society. Professionals have no special rights or duties separate from their rights and duties as moral persons, and therefore codes of ethics are pointless and possibly pernicious.

The people supporting the use of codes of ethics postulate that codes of ethics can:

Serve as a collective recognition by members of a profession of its responsibilities
Help create an environment in which ethical behavior is the norm
Serve as a guide or reminder in specific situations
Enrich a profession in the process of developing and modifying the code
Protect professionals from certain pressures and serve as a solution to coordination problems
Serve as an educational tool, providing a focal point for discussion in classes and professional meetings
Indicate to others that the profession is seriously concerned with responsible, professional conduct

Contributed by Iva Smit.

How will a code of conduct change common practices and how much time will be needed to achieve that?

A code of conduct, of course, will not automatically change common practice. Only if there is an understandable penalty on misbehaving (however small the penalty is, e.g., membership of professional society being rescinded after warning), will such a code have any influence. The other side of this is that as soon as the public starts up malpractice cases for professionals acting against the Code(s) of Conduct, the professionals will, in general, hurry to conform. It must be noted, however, that even in professions where this occurs already (medical doctors, accountants, etc.) there still are (groups of) individuals that do not toe the line. Human nature and "money is the root of all evil," I presume. In some cases, you need a federal (or global?) authority to force people to behave (case in point: the SEC in the US).

Can we change the mainstream view of simulation as the "method of last resort"? Will a code of conduct help in this respect?

We can change the popular view by being open to the public about what we do and how we do it, volunteering information on the subject, and presenting specific cases of its application. I once did a TV-show/course on simulation, together with a friend, and that had a great impact at the time, but if you don't keep doing it the knowledge will fade and the new generations will not be aware. I believe that publicity will help to promote the role of simulation as a problem solving technique, as will the public discussion about merits and disadvantages. Education plays an important role, especially adequate education of decision makers. The code of conduct could help in the sense that it shows that the profession has become mature and that it clearly thinks about its role in society.

The most important function of a code of ethics according to ACM is "its role as an aid to individual decision making" [2]. Do you agree?

That is just only one of its aspects. A useful one, but hard to monitor and prove. Certainly not the most important one: the most important, in my modest view, is the role it plays in making the professional feel responsible towards the public, his/her customers, peers, etc., and giving him/her a way to measure how he/she is doing.

How soon will we see the efforts to establish a code of conduct bear fruits?

Immediately after the Eurosim conference, Iva Smit, Tuncer Oren, Gene Shapiro, Grant Sheng, and I sat together and finalized the code of conduct as it was presented at the final session of the conference. It was then mailed to the Board of Eurosim for sending on to sister societies. We do not know if this has happened yet. Simultaneously I sent a copy to the Board of SCSI, but have had no response yet. I know from Oren that the issue would be discussed in a Board meeting, but I have not heard anything from them either. I was invited by the IIAS (International Systems and Cybernetics Society) to present our findings at their conference this year, so at least the message gets along.

Are there any particular parts from the code of conduct that you would like to point out?

Yes, I think the most important points are that a simulationist should be open and clear about the applicability of specific models and associated simulation results, and be open about and exercise extreme care in the interpretation of any simulation study results.

References

1: Allman, M., and Falk, A., "On the Effective Evaluation of TCP". In ACM Computer Communication Review, 29(5), October 1999.
2: Anderson, R. E., Johnson, D. G., Gotterbarn, D., and Perrolle, J., "Using the New ACM Code of Ethics in Decision Making". In Communications of the ACM, Vol. 36, No. 2, February 1993.
3: Breslau, L., et al., "Advances in Network Simulation". In IEEE Computer, 33 (5 ), pp. 59-67, May, 2000.
4: Floyd, S., and Paxson, V., "Difficulties in Simulating the Internet". In IEEE/ACM Transactions on Networking, Vol.9, No.4, pp. 392-403, August 2001.
5: Law, A., and Kelton, W., Simulation Modeling and Analysis, 3^rd edition. McGraw-Hill, 2000.
6: Zeigler, B. Theory of Modelling and Simulation. McGraw Hill, 1976.

Biography

Kostas Pentikousis (kostas@cs.sunysb.edu) is pursuing a Ph.D. in Computer Science at Stony Brook University. His research interests lie in the area of computer networks, including transport and application layer protocols, mobile computing and wireless communications, network and energy management. He received a B.Sc. (Honors) from Aristotle University of Thessaloniki, Greece, and a M.Sc. from Stony Brook University, both in Computer Science.

Crossroads

The ACM Student Magazine

Maurice Elzas on Simulation Ethics

Introduction

The Rise of the Simulationist

Further reading

References

Copyright 2004, The Association for Computing Machinery, Inc.