Exploring a National Cyber Security Exercise for Colleges and Universities (August 24, 2004) - 2004-08

8/7/2019 Exploring a National Cyber Security Exercise for Colleges and Universities (August 24, 2004) - 2004-08

1/74

The United StatesMilitary Academy

Exploring aNational Cyber Security Exercise

for Colleges and Universities

Lance J. HoffmanDaniel Ragsdale

This report provides an overview of existing cyber securityexercises, explores the feasibility of generalizing thoseexercises to a national exercise, describes the structural andresource-related issues of hosting a cyber security exercise, andoutlines the mission and goals of a potential governing body for

such exercises.

Report No. CSPRI-2004-08The George Washington University

Cyber Security and Policy Research Institute

Report No. ITOC-TR-04001United States Military Academy

Information Technology and Operations Center

August 24, 2004


2/74


3/74

Exploring a National Cyber Security Exercisefor Colleges and Universities

Lance J. Hoffman1

Daniel Ragsdale2

Abstract

This report provides an overview of existing cyber security exercises, explores thefeasibility of generalizing those exercises to a national exercise, describes the structuraland resource-related issues of hosting a cyber security exercise, and outlines the missionand goals of a potential governing body for such exercises.

1 Computer Science Department, The George Washington University, Washington, DC 20052,[email protected] of Electrical Engineering and Computer Science, United States Military Academy, WestPoint, NY 10996, [email protected]
mailto:[email protected]:[email protected]:[email protected]:[email protected]


4/74


5/74

Table of Contents

Introduction......................................................................................................................... 1What Is a Cyber Security Exercise?.................................................................................... 1

Organized Competition among Service Academies ....................................................... 1

Small, Internal, Continuous Capture the Flag Exercise .............................................. 2National Capture the Flag Exercise............................................................................. 2Semester-Long Class Exercise........................................................................................ 2

Goal and Benefits of Cyber Security Exercises.................................................................. 3A Uniform Structure for Cyber Security Exercises ............................................................ 4Rules and Guidelines .......................................................................................................... 4Legal Considerations .......................................................................................................... 6Structural Considerations for a Cyber Security Exercise ................................................... 7

Personnel/Participation ................................................................................................... 7Tools ............................................................................................................................... 8Other ............................................................................................................................... 8

Resources and Costs ........................................................................................................... 8Evaluating the Costs and Benefits ...................................................................................... 9Governance ....................................................................................................................... 10Conclusion ........................................................................................................................ 11Acknowledgments............................................................................................................. 11Appendices........................................................................................................................ 13

Appendix 1. Cyber Security Exercise Workshop Participants .................................... 14Appendix 2. Workshop Agenda................................................................................... 20Appendix 3. United States Military Academy Cyber Defense Exercise (CDX) ......... 22Appendix 4. University of Texas Cyber Security Exercise ......................................... 24Appendix 5. University of California, Santa Barbara, Cyber Security Exercise......... 25Appendix 6. Texas A&M Cyber Security Exercise..................................................... 27Appendix 7. The Cyber Defense Exercise: An Evaluation of the Effectiveness ofInformation Assurance Education................................................................................. 29Appendix 8. Model Legal Memo for Cyber Security Exercise Participants andOrganizers..................................................................................................................... 44Appendix 9. Related Ideas beyond the Scope of a Standardized Cyber SecurityExercise......................................................................................................................... 47Appendix 10. Cost Estimates....................................................................................... 48Appendix 11. Rules for 2004 Inter-Service Cyber Defense Exercise ......................... 49Appendix 12. Sample Authorization Memorandum for Attackers.............................. 57Appendix 13. Movements towards a Governing Board............................................... 58Appendix 14. Architecture of a Cyber Defense Competition...................................... 60Appendix 15. Sample Legal Liability Release Form................................................... 68


6/74


7/74

Introduction

On February 27 and 28, 2004, a group of educators, students, and government andindustry representatives gathered in San Antonio, Texas, to discuss the feasibility anddesirability of establishing regular cyber security exercises for post-secondary level

students similar to the annual Cyber Defense Exercise (CDX) held among the students ofthe various U.S. military service academies. The military model and other smaller effortswere described, and numerous ideas, opportunities, and challenges were brought forth.This report attempts to capture the concepts discussed at the workshop. It provides anoverview of existing cyber security exercises, opens questions related to generalizingthose exercises to a national exercise yet to be defined, describes the structural andresource-related issues of hosting a cyber security exercise, and outlines the mission andgoals of a potential governing body for such exercises.

What Is a Cyber Security Exercise?

There are at least four examples of what could be called a cyber security exercise.

Organized Competition among Service Academies

The U.S. military service academies CDX was designed in 2001 as an inter-academycompetition in which teams design, implement, manage, and defend a network ofcomputers (see Appendixes L3, L7, and L14). A team of security professionals fromvarious government agencies participate in the exercise as attackers.

Any offensive activity by an academy is heavily penalized. The event, now heldannually, stresses the application of skills learned in the classroom as students attempt tokeep their networks functional while a group of professional security experts attacksthe networks repeatedly over the course of several days. The participants must build asecure network including several legacy applications. The must both install and securethe applications they employ to meet service requirements, and build defensive measuresaround systems that may not be altered. By focusing on the defensive tasks in networksecurity, each student has the opportunity to truly understand the fundamental conceptsand can spend time conducting forensic analysis. This helps avoid an inadvertent attackthat spills outside the network sandbox. While many might argue that the likelihood ofsuch an occurrence happening is small, one such event can be catastrophic to theexercise.

The greatest drawback of the CDX is its rigid nature. Students are strictly limited in boththe time frame of the exercise and the actions that can be taken during the exercise. Thisstructure does, however, provide a strong reference from which to gauge the relativeperformance of each participant.

EXPLORING A NATIONAL CYBERSECURITY EXERCISE FORCOLLEGES AND UNIVERSITIES Page 1


8/74

Small, Internal, Continuous Capture the Flag Exercise

In contrast to the large-scale, multi-institution event the CDX represents, a student groupfrom the University of Texas at Austin has established a small-scale, internal, continuouscyber security exercise. The students created their own isolated network to practice

system defense, and it evolved into an ongoing, online, offense-oriented competition.Teams of attackers are assigned objectives and gain points when they achieve theobjectives by a designated scoring system. No time constraints are involved, soindividual participants can take part at any time (see Appendix 4). The hardware wasdonated, and the students are responsible for managing and maintaining both thehardware and the online exercise.

This structure offers maximum flexibility at minimum cost. However, it lacks integrationinto an established curriculum and thus misses the opportunity to be used as a formalcapstone exercise that provides a focal point for an advanced information assurancecourse. Additionally, any perception that students are using university resources to learn

to hack in an unsupervised environment might cause concern among the administrationand others.

National Capture the Flag Exercise

What began as a classroom exercise in a course on network security at the University ofCalifornia, Santa Barbara, grew into a competition among teams around the UnitedStates. Teams are given a system, configured by the organizers. The system contains anumber of undisclosed vulnerabilities. The teams have a limited time to set up their ownsystems and then are allowed to attack each others systems at will. Each team attemptsto find the vulnerabilities in the given system so that they can fix or protect their systemand, at the same time, exploit this knowledge to compromise the system of other teams. Asuccessful compromise allows a team to access and modify specific hidden informationon anothers system (i.e., the flag). This allows a scoring system to determine thecurrent status of the competition and assign points to each team. Points are also assignedto teams that maintain their services active and uncompromised. Therefore, each team hasto defend its own system to maintain functionality, such as web access and networkconnectivity. (See Appendix 5.)

This scenario shares some characteristics of the previous one. In particular, it requiresthe students to engage in offensive actions to win. Introducing students to the attackprocess and actually requiring them to employ such skills each raise legal concerns.Specifically, what happens if an attack unintentionally leaks outside the exercise network(since virtual private networks [VPNs] are not guaranteed to be secure)?

Semester-Long Class Exercise

At Texas A&M University, a graduate-level advanced security class engages in a cybersecurity exercise throughout the whole semester. Students are divided into teams ofattackers (hackers) and defenders (system administrators); a third group oversees the



9/74

exercise and imposes the same limitations on the students as the university networkimposes on all its users. Access is limited to a private network, and defenders must keepthe network running at all times. At the end of the semester, both teams disclose whatthey were able to accomplish. Grading is subjective and focuses on the successfulattempts of each team (see Appendix 6).

This exercise also has students engaging in attack activities, although in a supervisedscenario, and thus also raises the potential legal concerns cited above. In addition, eachstudent group only has the hands-on experience for its own mission. The exercise may besomewhat less competitive than if the school were competing against a rival school.

These different types of exercise are summarized in Table 1.

Table 1. Summary of Cyber Security Exercises

OrganizedCompetition

Among

ServiceAcademies

Small,Internal,

Continuous

Exercise

RegionalCapture

the Flag

Exercise

Semester-Long

Class

Exercise

Student offense component X X X

Student administrative component X XIsolated exercise network X X

VPN exercise network X X

Inter-school competition X X

Goal and Benefits of Cyber Security Exercises

All of the cyber security exercises described involve hands-on application of informationassurance skills; as such, they enhance students understanding of both theory andpractice. They provide students a laboratory in which to experiment, just as in otherfields of science. They fulfill the same role as capstone projects in a traditionalengineering program, i.e., projects that allow students to synthesize and integrateknowledge acquired through course work and other learning experiences into a projectusually conducted in a workplace (in this case, the defense, not the attacks). Theexercises combine legal, ethical, forensic, and technical components while emphasizing ateam approach. Such experiential education increases the knowledge and expertise offuture professionals who may be in a position to contribute to the secure design andoperation of critical information and its supporting infrastructure.

Therefore, the goal of a cyber security exercise might be described as follows:

To provide a venue for practical education in the implementation of all strategies,tools, techniques, and best practices employed to protect the confidentiality,integrity, authenticity, and availability of designated information and informationservices.



10/74

A Uniform Structure for Cyber Security Exercises

It has been suggested that a uniform structure for cyber security exercises be set up. Thegoals of creating a uniform structure for cyber security exercises might include the

following:1) Providing a template from which any educational institution can build a cybersecurity exercise

2) Providing enough structure to allow for competition among schools, regardlessof size or resources

3) Motivating more educational institutions to offer students an opportunity togain practical experience in information assurance

Rules and Guidelines

Workshop participants identified the following concerns that should be addressed by a

standard set of rules.

Eligibility: Workshop participants agreed that participation should be limited to post-secondary school students for the immediate future. Commercial or government agenciesshould have opportunities to play a supporting role, but the focus should remain on theacademic exercise for now. By limiting exercises to educational institutions, organizerswill be better able to gain support from faculty, university leaders, and nationaleducational and professional societies.

Resources: The guidelines should specify options for setting up networks for an exercise.Attention must be given to creating (a) level playing field(s) so institutions with greaterresources (e.g., hardware with fast processors and access to high bandwidth forcommunication) do not have an outright advantage. Software and tools that can be usedshould be available to all participants and limited to open-source or pre-approvedprograms from an approved software list. Participants should not be allowed to useevaluation copies of commercial software. This ensures all schools have access to thesame set of tools to employ. This does not imply that a school should disclose its list ofsoftware to other schools each participant is still required to conduct the researchneeded to employ the most secure network possible.

Legal issues: Guidelines should offer specific methods for recognizing and meeting legalobligations when planning and conducting an exercise. Various legal considerations arediscussed below.

Limitations: Rules should define in writing as thoroughly and clearly as feasible whatstrategies and practices are and are not allowed. Two distinct sets of rules should bedevised: one for attackers and one for defenders. Referees should also have clearguidelines. Referees should be independent of both the defending and attacking teamssince they may be used to ensure fairness of the conduct of a competition. They also,dependent upon their experience, may add value to the learning experience by providing



11/74

insight and guidance in the form of an After Action review. This is where much of thelearning occurs. (See Appendix 11.)

Scoring: A uniform method of scoring should allow teams of all sizes to compete. Anobjective and relatively simple scoring algorithm will allow teams or even individuals to

engage in an internal cyber security exercise and compare themselves with those takingpart in a more formal, competitive exercise. Both automated and manual scoringapproaches should be considered. If possible, additional points should be awarded forrealistic solutions that preserve functionality, e.g., those that allow other network users tocontinue working, use e-mail, and access the Internet at an acceptable speed. It may behelpful to implement an ongoing (or real-time) assessment mechanism and possibly postscores during the exercise. (At least one workshop participant felt that this type ofcompetition would not scale to a national level because of difficulties involved incoordinating referees and ensuring a level playing field, and suggested removing thecompetitive element at the national level, pointing out that individual schools couldalways set up isolated competitions with one another if they considered their students and

curricula to be roughly equivalent.)

Penalties: Consequences for violating the rules should be determined at the outset.Ethical considerations should be made clear. Participants should agree to adhere to thespirit, as well as the letter, of the rules.

Assessment: During the exercise, communication among all participants is critical.Because of the adversarial relation that develops between the attackers and the defenders,the referees should be the conduit for all information requests. Rules should address howand how much information should be shared among teams during an exercise. It may behelpful to consider incentives for sharing information.

The exercise must be assessed after completion. Specifically, where and when attacksoccurred, whether they were identified, and how they were addressed is important toknow, so that an accurate assessment can be made of the participants understanding ofthe network activity. Setting up a secure network is good only until the first compromise.After that, participants must demonstrate that through forensic analysis, they fullyunderstand and can document what happened. In general, the format and framework of apost-event assessment should be determined at the outset; how and how muchinformation learned should be shared after the event should be determined.

Post-event disclosure: Once an exercise is completed, teams should be required todisclose all the tactics they used during the exercise. Tactics and strategies from pastcompetitions should be readily available.

(Note: Numerous ideas were proposed throughout the workshop; some were thought tobe beyond the scope of a standardized cyber security exercise. Some of those conceptsare noted in Appendix 9.)



12/74

Legal Considerations

It may be assumed that the sole purpose of a cyber security exercise is training, andfederal laws allow agencies to conduct vulnerability assessments for the purpose ofsecurity. However, to the extent that exercises may involve some use of real data and

may affect real users of a real interactive system, organizers and participants should beaware of applicable state and local laws and regulations as well as institutionalregulations regarding the following:

Unauthorized intrusion

Unauthorized access to data in transmission

Unauthorized access to stored data

Fourth Amendment limitations on government actors

Individual privacy rights

Contractual obligations

Organizers must take every reasonable step to ensure that no protected information iseven put at risk, let alone compromised during any form of exercise. Functionally thisequates to segregating the networks used for the exercise from production or supportnetworks. Ideally, the only systems ever connected to the exercise network are thosedirectly involved in the exercise. If such separation is not possible, than additionalmeasures may be required to insure proper information protection.

A more realistic (and possibly more damaging) scenario is the use of exercise systems tointentionally or accidentally harm an innocent third party, potentially resulting indownstream liability. The concept of downstream liability is gaining interest andmomentum in the legal communities. Lawsuits have been filed (e.g., FTC v. Guess

Jeans: http://www.securityfocus.com/news/5968, FTC v. Eli Lily:http://www.ftc.gov/opa/2002/01/elililly.htm) and there are several white papers andarticles on the issue. More on this can be found at Downstream Liability for AttackRelay and Amplification at http://www.cert.org/archive/pdf/Downstream_Liability.pdf,Poor Tech Security Can Mean Lawsuits athttp://www.williamsmullen.com/news/articles_detail/122.htm, and DownstreamLiability The Next Frontier at http://www.nocinfragard.org/docs/rasch.ppt.

Organizers should assess their authority to access the system, manipulate the system, andaccess specific data. To do so, they should determine what systems, data, and authoritieswill be involved or affected. Organizers should seek permission to conduct an event

from responsible parties. The entire procedure of the exercise (from planning throughpost-event disclosure) should be explained clearly to ensure that responsible parties givetheir fully informed consent. Students who participate in information assurance coursesoften are required to sign such an understanding of the concerns involved. See Appendix15 for an example used in the Department of Engineering Management and SystemsEngineering at The George Washington University. See Appendix 12 for theauthorization memorandum issued by the United States Military Academy for itsattacking team.

http://www.securityfocus.com/news/5968http://www.ftc.gov/opa/2002/01/elililly.htmhttp://www.cert.org/archive/pdf/Downstream_Liability.pdfhttp://www.williamsmullen.com/news/articles_detail/122.htmhttp://www.williamsmullen.com/news/articles_detail/122.htmhttp://www.cert.org/archive/pdf/Downstream_Liability.pdfhttp://www.ftc.gov/opa/2002/01/elililly.htmhttp://www.securityfocus.com/news/5968


13/74

Organizers should screen participants and develop a plan to address civil liability orcriminal activity, should it arise. Before sharing or publishing information, organizersand participants should consider the level of sensitivity of the information.

The exercise offers hands-on experience in a competition important to learning how todefend computer systems. Its main focus is not training to attack systems. It is importantto point this out to university administrators and to the public in advance, during, andafter the exercise to avoid expectations by participating students of a fun hacking gameto defuse criticisms by those who may consider the exercise likely to cause more harmthan good.

Appendix 8 contains a memo to organizers, players, and sponsoring organizations fromlegal staff in preparation for a cyber security exercise. This memo may serve as anexample for organizers of future cyber security exercises.

Structural Considerations for a Cyber Security Exercise

There are at least four possible structural models for a cyber security exercise:

Participants are given requirements and services they are to provide and mustdevelop their own systems/networks to provide them.

Participants are given specific systems and services to provide and must developprotections for them.

Participants are given specific systems and a network configuration and mustprotect them.

A major decision is whether to conduct an event with multiple teams at one site(centralized) or at multiple sites (distributed). A distributed exercise requires fewerresources, but a centralized exercise enhances the excitement of competition. Because acentralized event would require establishing an isolated network for the exercise, it maymore successfully limit the likelihood of damaging or malicious information travelingoutside the realm of the exercise via the Internet. The availability of other universitycomputer systems will affect the scheduling of the event.

The logistical issues identified below should be considered by (an) institution(s)exploring the possibility of establishing a cyber security exercise.

Personnel/Participation Scope of participation, e.g., members of a club, all students in a class,

students across the university, or students from several universities

Minimum and maximum number of participants

Conditions of participation

Qualifications and affiliations of referees or mediators



14/74

Tools

Isolated network (if participants have access to the Internet, justify inwriting beforehand). Consider simulating connectivity, e.g., creating ashadow server that gives the appearance of the Internet.

Ensure equity of tools, advance notice, and hardware.

All teams should have equivalent bandwidth; the following questionsshould be addressed in advance:

o What bandwidth is required?o Are filters or rate limiters already in place?o Will bandwidth-oriented, application-specific denial of service

(DoS) attacks be allowed?o Will general DoS attacks be allowed?o Can additional bandwidth be purchased or rented for the duration

of the exercise?o Should organizers develop a list of approved websites that teams

can access during the exercise, e.g., sites with tools that can help

patch new vulnerabilities as they develop?o Will dedicated bandwidth conflict with Internet service provider or

carrier?

Other

Duration of preparation time

Parameters for pre-attack setup, intelligence gathering, and surveillance

Duration of the event

Active/inactive periods of attack

Types and areas of vulnerability

Ensuring consistency of attacks, so all defending teams are subject to the

same types and variety of attacks Definition of a functional system, i.e., participants should ensure the

system can be navigated by naive users and not just technical experts

Resources and Costs

The costs of a cyber security exercise can be separated into six areas:

Procurement

Maintenance

Internal personnel

External support

Management

Facilities

This section provides some general observations on related costs. Some more detailedtreatments of costs are provided in Appendix 10.



15/74

Procurement: For some institutions, the cost of obtaining appropriate hardware may bemore than they can absorb, especially if the hardware is dedicated for the exercise only.The costs increase linearly with the number of teams involved. In some cases, it may bepossible to borrow or rent equipment, establishing a central repository where participantscan pick up and return equipment. The use of virtual machines would cost significantly

less.

Maintenance: The cost and frequency of technical upgrades should be considered inbudgeting and planning.

Each institution should maintain archives documenting its exercise, which would involveonly negligible costs for the institution. The governing body will maintain technicalreports, documents, scores, etc.

Internal personnel: Faculty members typically require release time or support timeapproved by their departments to oversee cyber security exercise properly. Both

administrative and technical staff support are also needed.

External support: In some cases, obtaining the services of an external team ofprofessionals in information assurance to act as attackers, referees, and/or controllers maybe appropriate.

Management: If there is an overall governing body (local, national, or other), its costswould have to be covered. Fees or dues from the exercise and/or its participants, as wellas from possible sponsors, are likely sources of revenue.

Facilities: The cost of procuring laboratory space for the exercise should be considered; itis expected the cost would increase in relation to the number of teams involved at a givensite. Ancillary costs related to facilities include the cost of hooking the computers up tothe Internet for the duration of the exercise.

Evaluating the Costs and Benefits

While the costs may seem daunting, it should be remembered that many institutions havefound ways to minimize the cost of organizing exercises by obtaining donated resourcesand encouraging volunteer support. It may be helpful to initiate an exercise on a smallscale, such as through a group study project or in the context of a special topics course.

Institutions should carefully weigh the many benefits of such an exercise against thepotential monetary costs. Cyber security exercises provide an opportunity for students toapply their skills in a real-world scenario such as that likely to be found in a largecorporation, a military coalition, a government agency, or a university. The exercise alsooffers lessons in teamwork, leadership, and coordination, as participants may be forcedto react to change and to work with students or faculty from other departments.



16/74

Among the most significant costs is the time of faculty members involved. Great effort isneeded to prepare students for an exercise, set up laboratories, and oversee and mentorstudents working in the laboratories for the duration of an exercise. These efforts taketime away from other faculty responsibilities; therefore, faculty may require recognitionby or even permission from the department to plan and implement an exercise. The

exercise may be (and probably should be) integrated with one or more classes in acomputer security and information assurance curriculum. Eventually, if an exercisebecomes commonplace at an institution, the burden on faculty decreases, as fewerresources and innovations are required to maintain the exercise.

(Another factor in the equation would be whether the institution would keep the upgradedlaboratories and equipment for instruction, etc.)

Governance

A central governing body with broad expertise is needed to establish and disseminate

rules and framework. This body would be responsible for the following:

Collect information about existing cyber security exercises, evaluate the pros andcons of the various models, and make the findings available to others.

Define the goals and objectives of a structured cyber security exercise.

Develop a framework for a cyber security exercise in an academic setting.

Develop standard rules, parameters, and scoring mechanisms for cyber securityexercises with an eye toward growing from single-school or small regionalexercises to a national competition.

Issue initial guidance for cyber security exercises.

On a more general level, it would also be appropriate for the governing body (or a portionof it) to

facilitate resources,

seek financial or other support and sponsorship for regional or national cybersecurity exercises,

coordinate with external agencies to enable a cyber security exercise/event,

promote the educational benefits of cyber security exercises to academicinstitutions,

support and disseminate research that furthers the goal of initiating and growingcyber security exercises, and

explore the feasibility of developing a national-level competitive cyber securityexercise.

This organization could have members representing a wide spectrum of interests andexpertise, including technological, legal, academic, governmental, and commercial. Anon-voting advisory board might include representatives of the federal government,corporations, or others.



17/74

Such a board might explore affiliation with another national organization such as theInstitute of Electrical and Electronics Engineers (IEEE) or the Association for ComputerMachinery (ACM). This would provide several benefits. First, the parent organizationmay be able to provide resources for the event execution. Second, universityadministrators might be more willing to support such an activity if it is recognized by a

well-known and respected organization. An analogous event might be the ACMprogramming competition.

A number of workshop participants are already in the process of establishing a governingbody (see Appendix 13). Once board members are elected, the governing body will turnits attention to collecting detailed information about existing cyber security exercises anddeveloping rules and guidelines for a standardized cyber security exercise. Eventually,the governing body will explore how to link various individual exercises to createregional, national, or even international competitions.

A patent and trademark is being sought for the Cyber Defense Exercise (CDX) as

implemented by the Service Academies, which may have legal implications for othersorganizing their own cyber security exercises or for a national exercise. Dan Ragsdaleand Wayne Schepens filed the patent to protect the CDX as envisioned and implementedby the service academies and prevent misrepresentation of event sponsorship. Theywere both involved in the workshop described in this report and in its planning. Giventhe fluid legal situation here, organizations creating or describing a similar competitionshould probably avoid using the term Cyber Defense Exercise. This report usescyber security exercise throughout, except when specifically describing the CyberDefense Exercise participated in by the service academies.

Conclusion

The workshop identified the various approaches taken in structuring cyber securityexercises and illuminated the technical, legal, ethical, educational, and financialconsiderations involved. The consensus was that such exercises are worthy of theconsiderable effort required to plan and implement them. Creating a standard structurefor cyber security exercises would have multiple benefits: it would provide a frameworkthat would enable more institutions to initiate an exercise, allow students from schools ofall sizes to compete against one another, and pave the way for regional and nationalcompetitions. One key missing item was a governing body. The development of agoverning body will facilitate the creation of rules and guidelines; a governing body willalso foster communication, promote the benefits of cyber security exercises, and providesupport for institutions.

Acknowledgments

This workshop would not have taken place without the hard work of several individuals.A steering committee met well in advance of the event and then was involved in acontinuous email meeting to set the agenda (Appendix 2) for the workshop and todetermine and invite the individuals who ultimately attended. That committee was



18/74

composed of the co-principal investigators (Lance Hoffman and Dan Ragsdale), theircolleagues immediately supporting them (Tim Rosenberg and Ron Dodge), WayneSchepens, Doug Jacobson, and Venkat Pothamsetty. Their affiliations are given in theroster of attendees in Appendix 1. Tony Stanco of The George Washington Universityand Hun Kim of the Department of Homeland Security contributed as members of this

group also, but were unable to attend the actual workshop. Gale Quilter was in charge ofthe logistical arrangements, assisted by Kevin Guerrieri. Dana Trevas wrote the firstdraft of this report and also provided editorial support. Sujit Rathod coordinated the finalmanuscript preparation.

Work on this project was supported in part by National Science Foundation grant0342739.



19/74

Appendices



20/74

Appendix 1. Cyber Security Exercise Workshop Participants

PARTICIPANT LIST

February 26 28, 2004La Mansion del Rio HotelSan Antonio, TX

George BakosSenior Security ExpertInstitute for Security Technology StudiesDartmouth College45 Lyme Road, Suite 104Hanover, NH 03755

Phone: 603-646-0665Fax: 603-646-0666Email: [email protected]

Matt BishopAssociate ProfessorDepartment of Computer ScienceUniversity of California, DavisOne Shields AvenueDavis, CA 95616-8562Phone: 530-752-8060Fax: 530-752-4767Email: [email protected]

George ChamalesStudentUniversity of Texas at Austin711 B. W. 35thAustin, TX 78705Phone: 512-565-0507Fax: 512-475-6183Email: [email protected]



21/74

Keri ChisolmAssistant Network AdministratorMississippi State UniversityComputer Science and EngineeringPO Box 9637

Mississippi State, MS 39762Phone: 662-325-1518Fax: 662-325-8997Email: [email protected]

Art ConklinStudentUniversity of Texas at San Antonio6900 N Loop 1604 WestSan Antonio, TX 78249Phone: 210-379-3671

Fax: 210-458-6311Email: [email protected]

David A. DampierAssistant ProfessorMississippi State UniversityComputer Science and EngineeringPO Box 9637Mississippi State, MS 39762Phone: 662-325-8923Fax: 662-325-8997Email: [email protected]

Ronald DodgeDirector, Information Technology and Operations CenterDepartment of Electrical Engineering and Computer ScienceWest Point601 Thayer Road, Room 109West Point, NY 10996Phone: 845-938-5569Fax: 845-938-3807Email: [email protected]



22/74

Charles Wesley Ford, Jr.ChairmanUniversity of Arkansas at Little Rock2801 South UniversityLittle Rock, AR 72204

Phone: 501-569-8134Fax: 501-569-8134Email: [email protected]. FulpLecturerNaval Postgraduate School833 Dyer RoadMonterey, CA 93943Phone: 831-262-4855Fax: 831-656-2814

Email: [email protected]

Derek GabbardChief Technology OfficerCDXpertsPO Box 7904Ann Arbor, MI 48107Phone: 734-604-0204Fax: 734-367-0458Email: [email protected]

Seymour GoodmanProfessor, International Affairs and ComputingCo-Director, Georgia Tech Information Security CenterGeorgia Institute of Technology781 Marietta Street, NWAtlanta, GA 30332-0610Phone: 404-385-1461Fax: 404-894-1900Email: [email protected]

Lance J. HoffmanDistinguished Research ProfessorComputer Science DepartmentThe George Washington UniversityWashington, DC 20052Phone: 202-994-4955Fax: 202-994-4875Email: [email protected]

mailto:[email protected]:[email protected]


23/74

Doug JacobsonDirector, Information Assurance CenterIowa State University2419 Coover HallAmes, IA 50011

Phone: 515-294-8307Fax: 515-294-8432Email: [email protected] MartiAssociate Director for NetworkingTexas A & M UniversityTeague, MS-3142College Station, TX 77843-3142Phone: 979-845-0372Fax: 979-847-8643


Clifford NeumanDirector, Center for Computer Systems SecurityUSC Information Sciences Institute4676 Admiralty Way, Suite 1001Marina del Rey, CA 90292Phone: 310-822-1511Fax: 310-823-6714Email: [email protected]

Venkat PothamsettySoftware EngineerCisco Systems12515 Research BoulevardAustin, TXPhone: 512-378-1675Email: [email protected]

Daniel RagsdaleDirector, Information Technology ProgramDepartment of Electrical Engineering and Computer ScienceWest PointWest Point, NY 10996Phone: 845-938-4628Fax: 845-938-4628Email: [email protected]



24/74

Tim RosenbergAssociate Research ProfessorComputer Science DepartmentThe George Washington UniversityWashington, DC 20052

Phone: 202-994-9516Fax: 202-994-4875Email: [email protected]

Anthony RuoccoAssociate ProfessorSchool of EngineeringRoger Williams UniversityOne Old Ferry RoadBristol, RI 02809Phone: 401-254-3334

Fax: 401-254-3562Email: [email protected]

Wayne J. SchepensFounding PartnerCDXperts Inc.504 Heavitree GarthSeverna Park, MD 21146Phone: 410-987-4484Fax: 410-987-4484Email: [email protected]

Ryan SmithStudentUniversity of Texas2606 Rio Grande, Apt 203Austin, TX 78705Phone: 972-814-8968Email: [email protected]

Erich J. SpenglerPrincipal InvestigatorNSF Regional Center for Systems Security and Information Assurance10900 South 88th AvenuePalos Hills, IL 60465Phone: 708-288-5361Fax: 708-974-0078Email: [email protected]



25/74

Anthony V. TeelucksinghTrial AttorneyDepartment of Justice/CCIPS128 Overbrook RoadBaltimore, MD 21212

Phone: 202-514-1026Fax: 202-514-6113Email:[email protected]

Krizi TrivisaniChief Security OfficerThe George Washington University44983 Knoll Square Drive, Suite 339Ashburn, VA 20147Phone: 202-345-2182Fax: 703-726-3622


Giovanni VignaAssistant ProfessorUniversity of California, Santa BarbaraDepartment of Computer ScienceSanta Barbara, CA 93106Phone: 805-893-7565Fax: 805-893-8553Email: [email protected]

Donna WarwasComputer Security EngineerAir Force Information Warfare Center402 Greig Street, Building 179San Antonio, TX 78226Phone: 210-925-3749Fax: 210-925-5087Email: [email protected]

Gregory B. WhiteDirectorCenter for Infrastructure and SecurityUniversity of Texas6900 North Loop 1604 WSan Antonio, TX 78249Phone: 210-458-6307Fax: 210-458-6311Email: [email protected]



26/74

Appendix 2. Workshop Agenda

Thursday, February 26, 20041800-2100 Reception

Friday, February 27, 2004

0800-0810 WelcomeLance Hoffman and Dan Ragsdale, co-Principal Investigators0810-0815 Meeting logistics

Gale Quilter, meetingsguru.com0815-0845 Self-introductions0845-1000 Cyber Defense Exercise to Date: Lessons Learned

History of CDXPossible future directions for similar exercisesFormal assessmentDos and Donts

Dan Ragsdale

Legal issuesAnthony TeelucksinghTechnical issues

Wayne Schepens1000-1015 BREAK1015-1100 Reactions and Raising of Any Missed Issues1100-1130 Discussion1130-1145 Assignments to Working Groups1145-1200 Charges to Working Groups (chairs and reporters designated

before meeting)1200-1300 WORKING LUNCH1300-1500 Working Group Meetings

1. Venue, duration, and refereeing2. Use of the actual Internet3. Eligibility, governance, costs, and prizes

1500-1515 BREAK1515-1600 Presentations of WG meeting results (1-5 slides each, 10 min.each)1600-1700 Discussion of these results

Collection of slides published, available to attendees by 1900

Saturday, February 28, 2004

0800-0900 Reactions to Collection of slides and WG meeting results0900-0930 Reorganizing Working Group topics and composition0930-0945 Charges to New Working Groups0945-1000 BREAK1000-1200 New Working Groups (4-6) meet1200-1300 LUNCH1300-1345 Presentations of New Working Group meeting results



27/74

1345-1445 Reactions to new WG meeting results1445-1900 FREE TIME FOR MOST ATTENDEES (during which early draft

visual presentation of workshop results is created by SteeringCommittee)

1900-2200 WORKING DINNER

2000-2030 Early Draft Presentation of Workshop Report2030-2100 Feedback to Early Draft2200 Adjournment



28/74

Appendix 3. United States Military Academy Cyber Defense Exercise (CDX)

The CDX has developed into an extraordinary educational experienced for the studentsand midshipmen who take part in the exercise. It provides an excellent capstone exercisefor these students during which their knowledge of information assurance concepts and

their skills in protecting and defending information systems are assessed in the context ofrealist, true-to-life scenario. During the four years that this exercise has been conductedthree significant benefits emerged; education, leadership development, and researchopportunities.

The CDX provides three significant benefits; education, leadership development, andresearch opportunities. The comments provided during after action reports and summarypapers unanimously stated that the educational experience provided by the CDX was oneof the most rewarding experiences while in school. The participating students, seniors intheir fifth semester of concentrated study in Computer Science, begin the semester byanalyzing the problem and follow up with a network design, an implementation of that

network, their own vulnerability assessment, and then the four-day exercise. Theirimplementation includes major applications requirements (web pages, electronic mail,databases, video conferencing, desk top applications) as well as a robust infrastructure(DNS services, bridges and routers, a honeynet, a firewall, a proxy server, an intrusiondetection capability and a backup and recovery facility).

These student activities build on and use every aspect of their by-then five semestercomputer science education a program whose initial emphasis is on foundationalknowledge and skills that are then reinforced by numerous project-oriented applications.They have not been trained in the particular technologies they now confront. From Linuxto MAC OS X, from firewalls to DNS servers to file servers, from email to web servers,this exercise demands that they quickly learn the strengths and weaknesses of theirassigned network component, identify threats and vulnerabilities, assess risk, find andapply safeguards. They learn what they have learned in this curriculum: to "drop down"into an unfamiliar situation and learn what they have to learn, fast.

As computer science majors, the students had taken the list of required theoretical andprogramming courses but were never presented real world problems that were dynamic innature. For example, each student at some point was required to develop a database.While this is certainly a task they will perform in the real world at some point, it is verystatic and canned. The Cyber Defense Exercise presented the students with a dynamicenvironment where they needed to respond to the changing tactics and techniques of avery skilled live opponent.

As important as this exercise was as the application of their intensive five semestercomputer science education, perhaps it is more significant as the culmination of theireight semester education in leadership. Military academies place a heavy emphasis onleadership. As with the educational rewards, the exercise leadership was cited in afteraction reports as one of the participants most challenging leadership experiences. This isa significant statement given that the academies are designed to challenge the students



29/74

from the day they arrive to the day they graduate. In 2004, 32 students organizedthemselves in two short months to design, build and defend a complex network. Theintricacies involved in leading a large group of students in an exercise where most areapplying new skills are a large challenge even for experienced leaders.

The third area of tremendous usefulness and potential is in research. The exerciseprovides the opportunity to evaluate new and existing technologies and policies, conducthuman interaction and management research, and forensic analysis. The typical exerciseresults in a tremendous amount of data from application, host, network, IDS and firewalllogs.

The exercise also produced two unexpected benefits. First, the coordination during theexercise by members of the attack team, who normally do not work together, providedinsight into complimentary procedures. Second, since the skill and the knowledge levelsof the participants has improved so dramatically over the past four years, the CDX hasbecome an excellent testing ground for new and emerging concepts in information

assurance.

More information regarding the USMA CDX can be found at www.itoc.usma.edu/cdx.



30/74

Appendix 4. University of Texas Cyber Security Exercise

We believe that the best way to defeat your enemy is to think like your enemy, and thenuse that foresight to stay one step ahead of them. We have incorporated this ideal as the

main focus of our capture the flag exercises. Our exercises give us an opportunity toreinforce the security practices taught in our class lectures, by allowing students to gainfirst hand experience using blackhat tools and tactics to exploit security weaknesses in asecure and monitored environment.

The architecture of our exercise is setup to allow students access to the target networkthrough an ssh gateway. While ideally we would like the network to be completelyseparated from the Internet, we've found it just isn't practical. Depending on theircomplexity, our exercises can span anywhere from a week to two months or beyond.Also, all of our participants are all undergraduate students and participate on a completelyvolunteer basis. So they don't always have a lot of time to dedicate, but they can drop

into the network and work when they do have some free time. Our current setup has onegateway/firewall machine that only allows ssh in, and drops all outbound attempts. Eachteam has their own attack computer on the network. They have full control of thiscomputer, which they are also responsible for protecting from the other teams.

Once the competition starts, a team is provided the address of the target network and alist of objectives. The competition ends after all objectives are completed. Each teamearns points based on the objectives they've completed, and how well theyvedocumented and reported their activities. Invariably each round, students will come upwith different creative attacks on both the target network and the other teams' computers,and they will receive bonus points depending on the originality and difficulty of theattack. Administration and judging of each round is carried out by the same person whodesigned the round, an undergraduate senior.

The scenarios may range anywhere from a single host with a software vulnerability to acomplex e-commerce environment with a firewall, IDS, and honeynet. The vectors ofattack change with every round of our competition, but the topics we've covered haveincluded buffer overflows, heap overflows, SQL injection, weak passwords, directorytraversal, ssh vs. telnet, and the principle of least privilege, just to name a few.

Rules have been the toughest thing to evolve over the years. When the participants firstget the rules, they will hold them up to the light, find all the holes and use those holes totheir full advantage. There are two ways that we found to deal with this: try and find allthe holes and have a large comprehensive rule set, or have a very simple rule set whosespirit encompasses all the holes. We chose to do the latter of the two. We have made ourenvironment as "self-enforcing" as possible; our only rules are that you can't commit anydenial of service acts, and you can't try to circumvent the outbound restrictions of thenetwork to access the Internet.



31/74

Appendix 5. University of California, Santa Barbara, Cyber Security Exercise

The Capture the Flag contest is a multi-site, multi-team hacking contest in which anumber of teams compete independently against each other.

This exercise is the latest of a series of live exercises organized as part of the graduatecourse on "Network Security and Intrusion Detection" taught at UCSB by ProfessorGiovanni Vigna. Previous versions of this exercise are described in the paper: G. Vigna,"Teaching Hands-On Network Security: Testbeds and Live Exercises," Journal ofInformation Warfare, vol. 3, no. 2, pp. 8-25, 2003.

The most recent live exercise was different because instead of having the students of theclass compete against each other, it involved different teams at different universities andinstitutions. The exercise is loosely based on the DEFCON Capture the Flag contest.This exercise is different from the DEFCON contest because it involves several

educational institutions spread across the nation. The DEFCON contest includes locallyconnected teams only. In addition, the DEFCON contest has always involved a limitednumber of teams. We developed a new network solution that allows a large number ofteams to participate.

The goal of each team is to maintain a set of services available and uncompromisedthroughout the contest phase. Each team can (and should) attempt to compromise otherteams' services. The services to be provided are implemented as part of an operatingsystem installation running as a VMware image. Each service has a number of associatedflags. Initially, the flags are set to the flag of the team that set up the VMware host. Thegoal of each team is to keep their flag uncompromised, while trying to change the flags ofother teams to their own.

During the contest phase of the exercise, the scoring software connects periodically toeach service and checks the corresponding flag values. If the service is not available, theteam receives no points. If the service is up and the flag is the flag of the team managingthe host, the team gets some points. If the service is up but the flag is set to the one ofanother team, the other team gets some points.

Note that each time a flag is tested its value is substituted with a new value computed byapplying a secret hash function to the original value. Therefore, simply rebooting a hoston a regular basis will not grant points since the hash value will be restored to the originalvalue at each reboot.

Rules

It is not possible or feasible to list all the rules and the exceptions to rules that apply.When deciding if an attack/protection technique is fair or not, students are urged to thinkabout the fact that the goal of this exercise is to learn about protecting/attacking a systemin a live situation. They are encouraged not to focus on breaking the scoring system,



32/74

but instead to concentrate on developing/deploying effective (and realistic) defense andattack techniques.

Below is the current list of rules. These rules might be changed during a particularinstance of the competition, as more issues are raised by the participants.

It is forbidden to launch denial-of-service (DOS) attacks. This is particularlycritical, given the limited duration of the exercise (4 hours). No floods, no DNSpoisoning, no obviously destructive behavior.

Excessive traffic generation will be penalized, whether or not the traffic is part ofa DOS attack. Generating traffic from a host that a team has compromised topenalize the owner team is considered unfair practice.

It is possible to patch the services, provided that the patch is made available to theorganizers by sending an email to them. This will allow the organizers to makesure that a patch will not block the scoring system. If this is not done, the serviceswill be considered as non-functional.

The scoring mechanism will access random pages at random times, in addition tochecking for the flag values. Blocking access to the service functionality that isnot associated with flag verification is equivalent to having the service notavailable.

It is not possible to perform attacks outside the VPN. For example, attacking ateams VPN router using its routable address (i.e., the address that is visible onthe Internet) is not allowed. All the traffic for the exercise must be containedwithin the VPN.

It is allowed to attack any host of a team's subnetwork. The attacks are notnecessarily limited to the host system provided by the organizers. For example, ifone compromises the target system of Team 1, he/she may try to compromise the

host that is running the VMware application.

More information regarding the UCSB Capture the Flag Exercise can be found athttp://www.cs.ucsb.edu/~vigna/CTF/.

http://www.cs.ucsb.edu/~vigna/CTF/http://www.cs.ucsb.edu/~vigna/CTF/


33/74

Appendix 6. Texas A&M Cyber Security Exercise

Advanced Networks and Security, CPSC 665, is a graduate level course to educatestudents about aspects of computer security important to future administrators. Part ofthis course is a semester-long hands-on exercise that gives students greater insight and

understanding of the nature of computer security issues by allowing attempts to penetratea live, but isolated, network environment. Students role play as normal users or attackersor system administrators and class discussion emphasizes understanding from multiplepoints of view.

A single gold team sets up a network of hosts offering services consistent with thoseoffered by a university network. With this in place several black teams attempt tocircumvent security features of the network. The ultimate goal of a black team is to gaincontrol of a host with out being detected. Black teams are assigned hosts outside thecampus, but are also given user accounts on the department systems.

The platinum team, consisting of two faculty and support staff, serves as referees andguides for both black and gold teams. The gold team is formed by selecting keyindividuals in the semester prior to preparation. Additional members are added onceclass starts, based upon their expertise in network administration. The gold teamadministers the sandbox network and is responsible for defending the systems while stillproviding required services.

Law prohibits attempting to compromise hosts. Therefore special care must be takenwith this type of exercise so that actions taken by students do not affect hosts outside ofthe exercise. To facilitate this separation a sandbox network has been constructed in theNetwork Engineering Lab. This is a reference to the measures taken to enclose thenetwork in a manner that ensures safety and isolation. This access point is setup toprevent any actions in the sandbox from escaping the exercise. Network monitoring isalso done at this point to ensure students are acting within the guidelines of the class.These are important aspects, to maintain a continuation of this course.

It is important for students to be able to distinguish the transition from public oracademic networks where the activities encouraged in this class are not only prohibitedbut can raise criminal charges. Thus, mechanisms for protecting the students frominadvertently sending attacks to network nodes outside of the sandbox are critical, as wellas preventing real life hackers from using the sandbox as an attack platform. The sandboxis intended to emulate a computer science department on a typical campus and is brokenup into 3 logical networks: the Black (Internet), Campus, and Department networks. Notshown are systems used as traffic generators. This was done to lessen the artificiality ofall traffic being security related. At semesters end, each team presents its activities andlessons learned.



34/74

Figure 1 Network Layout



35/74

Appendix 7. The Cyber Defense Exercise: An Evaluation of the Effectiveness of

Information Assurance Education

THE CYBER DEFENSE EXERCISE: AN EVALUATION OF THE

EFFECTIVENESS OF INFORMATION ASSURANCE EDUCATION

Wayne J. Schepens Daniel J. Ragsdale, John R. SurduNational Security Agency United States ArmyInformation Technology Operations Center Information Technology and Operations Center

United States Military Academy United States Military Academy

West Point, NY 10996 West Point, NY [email protected] {dd9182 | dj6106}@usma.edu845-938-7674 845-938-2056/2407

Joseph SchaferUnited States ArmyU.S. Naval War CollegeNewport, [email protected] x3816

ABSTRACT

The US Military Academy at West Point issued a challenge to the five United Statesservice academies to participate in an inter-academy Cyber Defense Exercise (CDE).This exercise was initiated and implemented by faculty and cadets assigned to the USMilitary Academy, West Point, with funding and direction provided by the NationalSecurity Agency. The concept of defending the network was derived to evaluate cadetskills and the effectiveness of the Information Assurance (IA) education invoked at WestPoint. The Cyber Defense Exercise served as the final project for senior-level ComputerScience majors enrolled in the Information Assurance (IA) course. The IA - ServiceAcademy Group for Education Superiority (IA-SAGES), a group formed to plan, developand share IA curriculum, proposed that all US service academies teaching an IA courseparticipate in the exercise. The US Air Force Academy and US Military Academyaccepted the challenge to compete in 2001.

The distributedfacility in which this exercise will be conducted is known as the CyberDefense Network (CDN). It was designed and developed by a West Point cadet (student)team, and is an extension of the Information Warfare Analysis and Research (IWAR)Laboratory. To understand the function of the CDN, it is necessary to understand all theresources at the disposal of USMA for IA education.

The IWAR Laboratory is an isolated laboratory used by undergraduate students andfaculty researchers at the US Military Academy. It is a production-like, heterogeneous



36/74

environment and has become a vital part of the IA curriculum at West Point. The militaryrange analogy is used to teach the students in the class that the exploits and other toolsused in the laboratory are weapons and should be treated with the same care as rifles andgrenades. This paper describes the structure of the laboratory and how it is used inclassroom instruction. It describes the process used to create the IWAR and the Cyber

Defense Exercise (CDE). Finally, this paper describes the concept of the 2001 CyberDefense Exercise and expectations for future participation.

INTRODUCTION

The Information Technology and Operations Center (ITOC) is a focal point forInformation Assurance education at USMA. Soon after its creation in 1999, the ITOCbuilt the Information Warfare Analysis and Research (IWAR) Laboratory. This facilitywas designed to support undergraduate education and faculty research at West Point. Itwas developed with the thought in mind that, one day, each US Service Academy wouldhave similar resources and curriculum in which to train; therefore, representatives from

the service academies created the Information Assurance Service Academy Group forEducation Superiority (IA-SAGES) in June 2000.

The mission of this working group is to share IA curriculum, resources, and experiencesin order to align each academys IA program in a similar fashion. The service academiesare training the future leaders of America, who in their future roles will rely daily on theintegrity of information. The founders of the IA-SAGES conceived a Cyber DefenseExercise (CDE) in which participating academies would match information assurancewits against one another. Several hurdles had to be overcome to make this a reality;however, the concept was quickly accepted. This exercise serves as a real-worldeducational experience, and the inter-service rivalry generates interest in the growingfield of IA.

This report describes how the CDE became a reality, the development of the CyberDefense Network (CDN) to support the CDE, and the plans for its first execution. It

describes the vital rolethat the IWAR Lab playsin teaching informationassurance and preparingundergraduate studentsmajoring in computerscience to defend thenetwork againstprofessional securityevaluators, known asRed Teams.

Figure 1: IWAR Laboratory



37/74

BACKGROUND

The nation that will insist upon drawing a broad line of demarcation between the

fighting man and the thinking man is liable to find its fighting done by fools and its

thinking by cowards. - Sir William Butler, 1874

The U.S. military is rapidly changing to take advantage of information technology fromthe Army's Advanced Warfighting Experiments to the Navy's Network-Centric GlobalWargames. Tomes argues that we are so far ahead, no adversary will threaten us withinformation warfare for twenty years [1]. Carver counters that, although we have thetools to defend ourselves, we are not using them, and we are blundering toward anotherPearl Harbor [2]. The fact that nearly half of the nations employed in U.S. Y2Kremediation efforts have been identified as using offensive information warfare supportsCarver's pessimism [3]. George Surdu, Global Director of Information Systems,Technology, and Services at Ford Motor Company, said that most of Ford's Y2K codewas written in India and Israel [4]. The wide dissemination of hacker tools, lack of

designed-in security in virtually all Department of Defense (DoD) information systems,and increasing DoD use of commercial communications infrastructures makes theprospect of asymmetrical threats horrifying. Each day it becomes increasingly plausiblethat young hackers working for a foreign power could cripple critical informationsystems. Recently the Army has placed as much emphasis on defending its informationinfrastructure as it had spent on Y2K remediation [5].

History teaches us that "technology permeates warfare," but the technological advancesdo not necessarily govern or even influence strategy and tactics immediately [6]. Themission of the U.S. Military Academy is to prepare future military leaders. A basictechnical literacy is required of all cadets. For computer science majors, one of the mostpopular courses is the Information Assurance (IA) course. The goal of InformationAssurance education at West Point is to improve awareness of security issues associatedwith information system. To this end, cadets get a broad appreciation for the policy andethical considerations of Information Operations along with a strong grounding in thehands-on, technical aspects.

INFORMATION ASSURANCE COURSE OBJECTIVES

Upon graduation, all cadets are commissioned as officers in the U.S. Army. Many ofthem will be responsible for the security of critical Army information systems. The IAcourse, therefore, is designed to provide a firm foundation in the fundamentals ofinformation assurance. With this foundation, recently commissioned lieutenants have intheir toolbox the intellectual skills needed for continued self-education.

The protection and defense of physical locations is a notion with which all cadets arecomfortable. All cadets have had the benefit of no less than three years of militarytraining and education by the time they take the IA course. A tenant of military planningand operations from as long ago as Sun Tzu and Julius Caesar is that knowing the tools,tactics, vulnerabilities of ones opponent as well as oneself leads to victory [7]. To



38/74

establish an effective defense you must have a good understanding of your ownvulnerabilities. In addition, you must be aware of the techniques that your adversarymight employ to exploit those vulnerabilities. These ideas have direct applicability in thecyber domain.

In the IA course, cadets learn many offensive techniques. Cadets write malicious appletsand viruses. They use port scanners, network sniffers, and vulnerability scanners to findthe holes in a system's defenses. They use scripts, Trojan horses, and other tools to gainroot-level access to target hosts. The purpose of all this familiarization, however, is not tomake them hackers. The purpose is to give them an appreciation for the tools used bypotential adversaries as well as the vulnerabilities of currently fielded or commerciallydominant information systems and how those vulnerabilities might be exploited.Information ethics are emphasized throughout this learning process to strengthen moralcharacter.

For the IA course to be successful, it is necessary to provide an environment that

facilitates active learning and provides maximum opportunity for hands-on experiencesfor the cadets [8]. It was quickly determined, however, that nearly all of the tools andcapabilities needed for this hands-on experience could not be installed in any of thegeneral-purpose computer laboratories for both legal and practical reasons. This led to thecreation of an Information Warfare Range, like those used for conventional weaponstraining.

Once the IWAR Range was developed, it was time to create the sandbox for actualwargames to be held. Since the goal from the onset of this IA course has been to educatein the context of defense, defense of a network would be the objective for the wargame.The sandbox needed to consist of a network that would mimic the function, form, and fitof an information infrastructure used to support a base or organization in which a futurelieutenant might be assigned. After learning various offensive and defensive techniquesthroughout the semester, cadets would be assigned to defend the network, whileprofessional Red Teams would remotely access, attack, and identify vulnerabilitiesassociated with the system. This Cyber Defense Exercise would serve to not only testtheir defense skills but also to allow the faculty to evaluate the effectiveness of theireducation.

IWAR RANGE

As part of their training, cadets are taught the military concepts of offense and defense aswell as tactics like reconnaissance and "defense in depth." Additionally, by the time theyare eligible for the IA course they will have had significant basic classroom and fieldmilitary training experiences. This training includes familiarization and/or qualificationwith various weapons systems on weapons ranges. These ranges provide a safe andauthorized location to conduct training. Leveraging this knowledge, the IWARLaboratory is introduced to the cadets as an IWAR range. While the IWAR Laboratory(Range) also facilitates faculty research, this paper focuses on the laboratory itself andhow it supports the IA course.



39/74

By describing the IWAR as a range, instructors leverage several important concepts fromconventional weapons training. First, the range is a special, isolated space. Just as onemay fire automatic weapons on a rifle range at various targets and launch missiles atother targets, so too can cadets launch cyber attacks from their firing position (cadet

computer terminals) at the IWAR Range target computers (also within the isolatedlaboratory). Second, it is unthinkable to fire an automatic weapon at a crowd of peoplefrom one's barracks room; it should also be unthinkable to use any of the cyber attacksfrom one's barracks room - or anywhere outside the IWAR laboratory.

Recall that the IWAR is a completely isolated laboratory with no physical connection tothe outside world.

The IWAR Laboratory is divided into four networks. The Gray network is the "attack"side of the network. Cadets have their workstations on the Gray sub network. Each cadetteam has one host workstation, but each workstation uses VMware to run variousoperating systems on the same physical machine. These operating systems include

Window 2000, Windows NT, Window 98, and Redhat Linux. Cadets haveAdministrator and root accounts in each of these environments. They also have useraccounts on all other Gray sub network machines. An example of how these systems are

used for instruction is this: for an in-class exercise cadets use their Windows NTvirtual machines to download a malicious applet from their Linux virtual machine on thesame physical hardware. The malicious applet then does "bad things" to the Windows

NT machine. Also, on the Gray network are servers on which the cadet teams haveuser-level accounts. These "low-hanging fruit," fruit that is easy to take off the tree, allowthe cadets to launch "insider" attacks.

The Gold network hosts the target systems. These are a series of Unix (Solaris

andIrix), Linux, Windows NT, and Macintosh workstations and servers. Severalmachines are Gray/Gold, meaning that they are targets, but they are on the Gray subnetand thus "low-hanging fruit." Except for those machines that are also Gray, users do nothave accounts on Gold machines. This makes attacking these hosts harder. In addition,Gold machines are on the other side of routers, switches, and firewalls, again creating arealistic heterogeneous environment. The Gold network helps cadets appreciate thecapabilities and vulnerabilities of firewalls and routers. Also wrapped in the Gold subnetwork is the Green sub network on which tactical command and control systems areattached.

Faculty members use the Black network for information assurance research. Due to theplacement of the machines and the switch (shown in the topology), researchers can workon both offensive and defensive projects on the Black network.

Two machines in the laboratory are not connected to any of the IWAR networks. Cadetsuse these machines for hunting the Internet for offensive and defensive tools. They canthen copy these tools to disks and hand-carry them to an IWAR Range machine. Cadetsphysically remove these Internet connected boxes from the network when not in use. This



40/74

isolation, along with some other techniques, reduces the likelihood that external hackerswill compromise these machines. In this way the IWAR Range should avoid having thesesystems serve as launching points for attacks against other Internet resources.

Together the sub networks that make up the IWAR Range provide a valuable resource for

teaching cadets how to defend systems against attackers. The Gray network allows cadetsto get an appreciation for insider attacks while the Gold network gives them anappreciation for outsider attacks. The Green network allows cadets to explore thevulnerabilities of Army tactical systems. Finally, the Black network allows faculty toconduct research in the same isolated facility.

THE "MAKING OF" IWAR

All four of the isolated and non-routable networks comprising the IWAR form a realistic,production-like environment of heterogeneous systems. Initially four criteria constrainedthe design of the range. First, the design must allow minimal possibility of misuse for

damage to other systems. Second, on-hand resources should be used whenever possible.Third, time was limited. Finally, the laboratory needed to fit into one classroom.

After investigating several possible designs involving all manner of access controls andfirewalls, we decided that the most expedient and least risky method of reducing thepossibility of misuse would be to electrically and physically isolate the range from allother networks. In our worst nightmares we envisioned a New York Times headline,"Network Attack Lab at West Point used to destroy XX," where XX is your favoriteexternal site.

On-hand resources were used because of constraints on both time and money. Theprimary means of achieving these goals was to use "rescued machines." These machineswere those that were five to ten years old and that the administrators had removed frommain production use after replacing them with newer models.

The West Point Department of Electrical Engineering and Computer Science maintains a"Tech Area" where many of these old machines awaited turn-in and donation to otherorganizations. We rescued several of these machines to form the core or our initialIWAR. Typical of these machines were a dozen generic, 60MHz Pentium boxes with oldmonitors and four SUN IPC and IPX boxes.

This rapid initial success helped identify several "underutilized" machines with which toaugment the IWAR. These machines consisted of three old SGI computers that had beenearly Web and graphics servers and two old, dual-processor, Pentium servers that had

been used for domain controllers and file servers on the Gray and Gold Windows NTdomains. Support personnel located some equipment that had been procured for oldprojects, such as networking components and an Imac, that were transferred into the lab.

Since the IWAR Range is completely isolated, a more secure method for the students toaccess resources on the Internet was needed. The goal was that the cadets should be able



41/74

to search for and download information from even the most untrusted of sites withoutrisking damage to any other systems. Two 90 MHz Gateway PCs, loaded with a verylimited and secure version of Linux serve this purpose. Forcing the user shell to Netscapeand requiring the presence of a Zip disk as the home directory further secured thesecomputers. In addition, these two Search boxes are connected to the Academy network

through a production firewall donated by the Academy's Directorate of InformationManagement.

Of greater concern was the risk that the IWAR network would be compromised and usedto attack external sites than the possibility that someone would gain access to the limitedresources on these search boxes. The search boxes are easily rebuilt from a ghostimagesince there are no home directories on the hard drive. The Zip disk was chosen since itwould allow a relatively simple method of transferring files downloaded from hackersites into the isolated IWAR range. Zip disks are also not in widespread use throughoutthe rest of the Academy, thus reducing somewhat the chance that someone would transferthese weapons to the main networks.

Early enthusiasm and achievements in the IWAR garnered some scarce dollars that wereused to upgrade some of the rescued machines and procure essential networking,upgrading, and space-saving components. Rescued or redirected networking componentsincluded mostly inexpensive hubs. Primarily due to space considerations each cadet teamuses a single hardware system, loaded with a variety of operating systems running invirtual machines.

Running many virtual machines on a single hardware platform significantly consumesmemory and CPU cycles. New motherboards, memory, and Zip drives in the Graymachines helped to improve the performance of these machines from dismal toacceptable.

The classroom in which IWAR Range resides had been previously separated into twosides by a divider with a door to the hallway from each side. The attackmachines werelocated on one side of the solid room divider and the target machines were located on theother side. This close proximity but isolation of the attack and target machines simplifiedadministration and setup of the lab. Additional administrative simplification wasachieved by ghostingmost of the systems and using Sun Microsystems administrativeservers and tape backups to allow rapid reconstruction of the systems.

The most important space, power, and heat saving components were the use of KVM(Key, Video, and Mouse) switches for nearly all of the Gold target systems. In addition tospace, heat and power proved to be huge constraints on the number of systems that couldbe reasonably set up in one classroom. With KVM switches, four sets of Keyboards,Mice, and Monitors provide interfaces for all 25 gold systems, significantly reducing thespace, power, heat, and clutter on the Gold network.

In addition to a heterogeneous hardware environment, the IWAR provides a wide varietyof production quality network applications and services. These include Domain Name



42/74

Service (DNS), WINS, authentication and replication with Domain Controllers,Network Information Service (NIS), and NIS+. Also provided are web servers, mail

servers, Network File System (NFS), Samba, LanMan, and additional services.Common production configurations were adopted. For example we ran Microsoft Internet

Information Server (IIS) and Exchange on the Windows NT servers and Apache on

the Linux and Sun servers.

The Gray/Gold servers were configured with old and unpatched versions of the

operating systems (e.g., Redhat 2.1 and Windows NT 4 with no service packsapplied) and applications. Additionally, these boxes were located on the Gray subnet onthe same hub with the attack machines. The students also had user accounts on theseservers. Thus, the students could log onto the Gary/Gold servers and easily sniff thenetwork and attempt well-known exploits to upgrade their privileges from user to root oradministrator. The Linux boxes and Linux virtual machines on the student's boxesparticipated in the Sun NIS Domain. The attack boxes were members of the Gray NTdoma

Documents

Exploring a National Cyber Security Exercise for Colleges and Universities (August 24, 2004) - 2004-08