Analysis framework of network security situational awareness … · 2019. 8. 13. · REVIEW Open Access Analysis framework of network security situational awareness and comparison

REVIEW Open Access

Analysis framework of network securitysituational awareness and comparison ofimplementation methodsYan Li1* , Guang-qiu Huang2, Chun-zi Wang1 and Ying-chao Li1

Abstract

Information technology has penetrated into all aspects of politics, economy, and culture of the whole society. Theinformation revolution has changed the way of communication all over the world, promoted the giant developmentof human society, and also drawn unprecedented attention to network security issues. Studies, focusing on networksecurity, have experienced four main stages: idealized design for ensuring security, auxiliary examination and passivedefense, active analysis and strategy formulation, and overall perception and trend prediction. Under the backgroundof the new strategic command for the digital control that all countries are scrambled for, the discussion of networksecurity situational awareness presents new characteristics both in the academic study and industrialization. In thisregard, a thorough investigation has been made in the present paper into the literature of network security situationalawareness. Firstly, the research status both at home and abroad is introduced, and then, the logical analysis frameworkis put forward concerning the network security situational awareness from the perspective of the data value chain. Thewhole process is composed of five successive stages: factor acquisition, model representation, measurementestablishment, solution analysis, and situation prediction. Subsequently, the role of each stage and themainstream methods are elaborated, and the application results on the experimental objects and thehorizontal comparison between the methods are explained. In an attempt to provide a panoramic recognitionof network security situational awareness, and auxiliary ideas for the industrialization of network security, thispaper aims to provide some references for the scientific research and engineering personnel in this field.

Keywords: Network security, Network situational awareness, Big data network security, Intrusion detection,Data fusion analysis

1 IntroductionThe information technology revolution has made greatchanges in the way of human communication in the worldtoday. Especially in recent years, in-depth studies of theindustrialization concepts of cloud computing, large data,Internet of Things, and mobile terminals have made thecontrol of digital information become a new strategic com-manding point, and the problem of network security hasalso received more attention in a wider range. The exposureof “prism plan” in June 2013 brought information securityfrom economic interest to the level of national security. InFebruary 2014, the establishment of the “central networksecurity and information group” marked the awakening of

the national consciousness of the Internet in China andhighlighted the importance of the national information se-curity strategy. However, the ability of network overalldefense at the national level to attack risk is still relativelyweak [1]. How to prevent organized malicious network at-tack has become a hot topic in the field of security.Studies on network security have started since the

birth of information networks. The exponential growthof network size and application, especially the randomdynamic access relationship built on the static Internetphysical connection network based on OSI model, makesthe study of network security more complicated. Beforethe 1960s, the focus on the network security research ishow to build an absolute security system and reduce de-sign vulnerabilities to ensure the confidentiality, integrity,and availability of the system, which can be regarded as

© The Author(s). 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made.

* Correspondence: [email protected]’an Polytechnic University, Xi’an 710048, Shaanxi, ChinaFull list of author information is available at the end of the article

Li et al. EURASIP Journal on Wireless Communications and Networking (2019) 2019:205 https://doi.org/10.1186/s13638-019-1506-1

http://crossmark.crossref.org/dialog/?doi=10.1186/s13638-019-1506-1&domain=pdfhttp://orcid.org/0000-0001-5326-9327http://creativecommons.org/licenses/by/4.0/mailto:[email protected]

the first stage of network security research. However,people soon realized the impossibility in practical oper-ation [2]. The existence of malicious intrusion provokesthe thought to build a security assistant system with anaim to detect the intrusion in time and take correspondingmeasures. The most typical application is the intrusiondetection system (IDS) [3]. The intrusion detection is orig-inated from Anderson’s Technical Research Report [4],and the subsequent researches can be divided into twocategories: anomaly detection and misuse detection. Atpresent, the IDS of most research institutions and com-mercial organizations is based on these two categories.Intrusion detection technology provides predictive warn-ing information to ensure network security when networkattacks occur, but it is too weak to do anything about thewall-around stealth attack and multi-step compoundattack. Such a passive defense technology is unsatisfactoryin the real-time detection. On this basis, the focus of thethird stage research after the 1990s shifted from passivedefense to active analysis [5, 6], which is originated fromthe development of hacker technology. The intent is tocarry out an integrated safety assessment before the occur-rence of network attacks, formulate a defense strategy, orstill provide predetermined service function given the dam-aged network. In 1990, Bass first proposed the concept ofCyber Situation Awareness CSA [7, 8], which intends toperceive elements in the time and space environment, sothat people can better grasp the overall network securitysituation and predict future trends, which to a certain ex-tent promotes the integration of network security researchand other disciplines. The development, especially thecombination with some advanced stochastic models, hasmade theoretical progress (such as stochastic algebra [9],game theory [10], Bayesian network [11]). However, mostof them are based on CSA conceptual model to optimizethe evaluation algorithm with few breakthroughs in thepractical application and systematic expositions (Table 1gives a brief summary of the four main stages of the devel-opment of network security studies).This paper gives a systematic introduction to the field

of network security situational awareness, with an aim

to provide insightful guidance for understanding therelated concepts, promoting their application in practiceand carrying out large-scale network expansion. Inaddition, a general analysis framework of network secur-ity situational awareness is proposed from the perspec-tive of value chain. The framework divides the process ofnetwork security situational awareness into five stages:factor acquisition, model representation, measurement es-tablishment, solution analysis, and situation prediction,which summarizes the current research progress in eachstage and discusses the practical application results of typ-ical methods. Moreover, this paper also elaborates thevisualization of perception analysis results and situationalawareness in the large data environment and prospectsthe key issues and research trend of this topic.

2 Research status at home and abroadSituational awareness is first seen in the study of militaryacademia. The human factor analysis of Theureau [12]in aviation has greatly promoted the application of thisfield in human-machine interaction, medical emergencyscheduling, and real-time battlefield command. In 1988,Endsley [13] defined situation awareness as the three-level model of situation factor acquisition, situationunderstanding, and situation prediction, and the applica-tion framework of situational awareness in dynamicdecision making was proposed in 1995 [14]. On thisbasis, the case study of the practical application ofsituational awareness is started, for example, Boyd con-trol cycle model [15], Tadda JDL data fusion model [16]based on Endsley’s three-level model, cognitive fusioncontrol model [17], and so on.Inspired by the air traffic control (ATC) situational

awareness, Bass [7] of the US Air Force Communica-tions and Information Center first proposed the conceptof network situational awareness, in an attempt to applythe ATC data fusion to network situational awareness.Since then, the attention of most studies is paid to thedata fusion analysis with the ignorance of the essentialdefinition of cybersecurity situational awareness. Atpresent, there is no clear and unified expression of

Table 1 Four main stages of network security research

Li et al. EURASIP Journal on Wireless Communications and Networking (2019) 2019:205 Page 2 of 32

network security situational awareness. However, con-firmation is made that network security situationalawareness and situational awareness belong to the rela-tionship between instance and type instead of that ofsubset, which means the relevant theory of situationalawareness and the method can be applied in the field ofnetwork security situational awareness after the specificprocessing. The literature [19] has a systematic explan-ation for the definition of network security situationalawareness and the understanding of the basic concept.Based on the explanation above, this paper offers thebasic operation mechanism of network security situ-ational awareness and illustrates the role of each link inthe cognitive process of network security status in themechanism.

2.1 Network security situation awareness and intrusiondetectionThe general model of the intrusion detection system(IDS) is first proposed by Denning [20]. Its core idea isto set up a regular set of rules that can be updated andmodified under the condition of a unified clock. There-after, information is collected by an agent from the net-work process records and compared with the definedrules, and then, determination is made whether theactivity set exists, which is trying to break the integrity,confidentiality, and availability of resources. The struc-ture of IDS can be mainly divided into three types: host-based detection [21], network-based detection [22, 23],and agent-based detection [24]. The host-based detec-tion mainly matches the process record information ona single host. This obviously does not meet the securityrequirements under the network environment; thus, thenetwork-based detection is built after adding some ele-ments on the host-based detection, such as network traf-fic and protocol information; however, with the gradualuse of distributed systems, IDS on distributed hosts alsoneeds information interaction, which contributes to theformation of agent-based detection. Technically, IDS ismainly divided into two types [25], abnormal intrusiondetection and misuse intrusion detection. Abnormal be-havior is the opposite of normal or harmless behavior, sothe rule set in abnormal behavior detection is the modeof the normal operation of the system. When detectingthe deviation from the normal model, the alarm signal isgenerated. The advantage of this method is that any ex-ploratory behavior will be recorded in addition to theprescribed “normal” action. But there will be a higher“false alarm rate” because the normal mode of the sys-tem is dynamic and cannot be completely normalized atthe beginning of the establishment of the detectionmodel; misuse behavior is abnormal or harmful behavior,so the rule set of misuse behavior detection is a modelof system harmful behavior. When it detects the behavior

that matches the harmful pattern, it produces an alarm. Inthe case of clear matching, this method has high accuracy,especially for the typical known attack model. But there is abig “rate of missing report” because it is almost impossibleto passively carry out the whole sample summarization ofharmful behavior under the background of diverse aggres-sive behaviors.Through the brief summary of IDS, there are two main

bottlenecks: passive response and false alarm rate/miss-ing report rate, and the researchers have done a greatdeal of improvement on these two points. The mainimprovement of the passive response mode is on theautomatic or semi-automatic response mode [26]. Themain reason for false alarm rate or missing report rate isthat there is a gray area between normal and abnormal,for which the IDS system and administrators cannot beanalyzed in a unified perspective. Therefore, the im-provement of this aspect is mainly the multi-level fusionanalysis of more information [27–29], which is consist-ent with the summary of the four main stages of net-work security research in Table 1. In fact, the initialresearch on network situational awareness is also basedon IDS. Bass [7, 30] proposed a multi-sensor integrationintrusion detection framework after the concept of situ-ation awareness, and literature [31, 32] also put forwarda similar framework. On this basis, lots of influentialsecurity situational awareness applications appeared,such as NVisionIP [33], VisFlowConnect-IP [34], andUCLog+ [35].It can be seen that the network security situational

awareness is a more advanced research stage and devel-opment direction to make up the defects of IDS. On theone hand, the existing results of IDS are the basis of thein-depth study of the network security situational aware-ness, and the latest methods and results of the networksecurity situational awareness can relieve the contradic-tions of IDS. As shown in Table 2, there are differencesand strong connections between network security situ-ational awareness and IDS. First of all, the focus of IDSis the presence or occurrence of attacks (or exceptions)in the network, and network security situational aware-ness is concerned with the security trend of a whole net-work. The analysis of attack behavior in networksecurity situational awareness plays a fairly importantpart, and attack behavior is carried out step by step innormal behavior steps. Furthermore, the results of fusionanalysis in network security situational awareness willalso make IDS better explain and describe the rules ofabnormal behavior or misuse behavior; secondly, beforerule comparison, the core information acquisition resultsof IDS is the attack precursor and post which is in thenetwork management audit category. However, the fu-sion analysis of network security situational awareness isdefinitely the element information abstraction of the


http://dict.cnki.net/dict_result.aspx?searchword=%e8%af%af%e6%8a%a5%e7%8e%87&tjType=sentence&style=&t=false+alarm+ratehttp://dict.cnki.net/dict_result.aspx?searchword=%e6%bc%8f%e6%8a%a5%e7%8e%87&tjType=sentence&style=&t=missing+report+ratehttp://dict.cnki.net/dict_result.aspx?searchword=%e6%bc%8f%e6%8a%a5%e7%8e%87&tjType=sentence&style=&t=missing+report+ratehttp://dict.cnki.net/dict_result.aspx?searchword=%e8%af%af%e6%8a%a5%e7%8e%87&tjType=sentence&style=&t=false+alarm+ratehttp://dict.cnki.net/dict_result.aspx?searchword=%e6%bc%8f%e6%8a%a5%e7%8e%87&tjType=sentence&style=&t=missing+report+ratehttp://dict.cnki.net/dict_result.aspx?searchword=%e7%bc%93%e8%a7%a3&tjType=sentence&style=&t=relief

whole network. With the elaborate study, the inputinformation of IDS also has a great expansion, but theinput of IDS must be a subset of the input of the net-work security situational awareness, and the output ofthe IDS can also be used as the input element of the net-work security situational awareness. In turn, the resultof the network security situational awareness will makeIDS’s information collection more precise and effective.Thirdly, at the functional level, the core function of IDSis to intercept suspected attack behavior through abnor-mal/misuse detection comparison and guide networkadministrators to take measures to defend the next at-tack. The core purpose of network security situationalawareness is to carry out the security situation predic-tion, which is intended to guide the administrator totake configuration measures before the attack, whichwill certainly improve the detection efficiency of IDS.The pre/post-rule detection method based on standardIDS is also the most effective and reliable predictionmethod of network security situational awareness;fourthly, the analysis of IDS mainly focuses on attack be-havior, but it is not capable of multi-step attack or attackaround the wall. Most fusion analysis of network secur-ity situational awareness also deals with the analysis ofaggressive behavior or abnormal behavior, because suchbehavior produces more benefit than normal access be-havior. However, the overall analysis results includingother behaviors will give IDS guidance both in particlesize and in the accuracy of description; fifthly, in theearly warning period, IDS carries out the acquisitionanalysis and warning based on audit information afterattack, and the passive response mode is difficult toguarantee the network security in real time. Network se-curity situational awareness does the active security situ-ation perception before the attack, and it does not aim

to eliminate the attack but to ensure that the networksystem is still safe or can still provide a predeterminedfunction under the conditions of a certain attack. At last,in the detection efficiency, the core breakthroughs ofIDS are high rate on false alarm/missing report andweak real time. If the configuration is too strict, the as-sertion of “suspect is wrong” will affect the effectivenessof the system. Loose configuration means “only heavyperson should be judged” will miss the report. The com-promise state between the two extremes requires thesystem to have the human gray perception ability, ratherthan the computer cognitive logic which means one orthe other. The fusion process of network security situ-ational awareness (NSSA) is easier to cross boundarieswith artificial intelligence and other multidisciplinaryresearch results for further improving the flexibility ofdetection, and the fusion analysis of flow data in largedata environment will greatly promote the real-time per-formance of detection.

2.2 Status of foreign researchThe study of situational awareness comes from a seriesof studies and elaborations of more than 15 articles byEndsley [13, 14, 36]. Bass [7] proposed the concept ofnetwork situational awareness for the first time andcombined it with cyberspace. Driven by the new tech-nologies such as the Internet of Goods, big data, andmobile applications, the innovation and promotion ofthe Internet application level have expanded rapidly, andthe topology has become increasingly complex. As thepublic information shows (Fig. 1), all countries haveraised their network security awareness to the nationalstrategic level. From the summary of the cybersecuritystrategies, publicized in various countries in recent years,it can be seen that although countries have different

Table 2 The difference and connection between IDS and network security situational awareness


http://dict.cnki.net/dict_result.aspx?searchword=%e8%af%af%e6%8a%a5%e7%8e%87&tjType=sentence&style=&t=false+alarm+ratehttp://dict.cnki.net/dict_result.aspx?searchword=%e6%bc%8f%e6%8a%a5%e7%8e%87&tjType=sentence&style=&t=missing+report+rate

understandings of cybersecurity and strategy implemen-tation, countries are aware of the need to take action toprotect the key information and related infrastructure,as well as to achieve the prediction of intelligent net-work security situation with new methods andtechnologies.The great emphasis from governments can bring more

financial support in terms of the fund. Besides, the spon-taneous and continuous attention of many researchersto this field has made the researches on cybersecuritythe top hot issue. In order to fully understand the re-search status of network security situational awareness,this paper firstly searched and reviewed articles on thistopic in the past 10 years in the core database inSeptember 2017, and sorted out a total of 10 large cita-tions of review literature [16, 37–45]. Based on theactor-network theory, Kopylec et al. [37] explored thecritical relationship between physical and network infra-structure, and demonstrated the results of situationalawareness through visual cascading. From the viewpointof network’s key equipment administrators, he managedto maximize the understanding of the process of the riskpropagation, thus providing systematic guidance in

related planning and emergency response. Based on thecombination of computer automation technology withhuman irregularity (abnormal or new mode) processingcapabilities, literature [38] describes the research ideasand tools provided by the VizSec R&D community,which enables network managers to better identify thepotential cyber threats. With aspect to the multidiscip-linary integration, Jajodia et al. [39] conducted the re-search in relation to the questions and methods ofnetwork situational awareness in 2010 with an excellentconclusion and analyzed the key problem of the networksituational awareness, as well as summarizes the mainreasons for the lack of network situational awareness.Tadda and Salerno [16], Giacobe [40], and Schreiber-Ehle and Koch [42] inquired into the application processof JDL model in the field of situational awareness, espe-cially in literature [40] for the favorable induction andsummary of the data source information at level 0/1 inJDL model. In addition, Klein et al. [41] and Vincent[45] et al. applied the OODA loop model [15] to thenetwork situational awareness and some stages in themodel are prerequisites for others. Through such a classdecision paradigm, the various activities in network

Fig. 1 Departments and public security strategies for network security in time series of countries


defense are integrated. Much emphasis in literature [43]is attached to the information security of industrialnetworks. The difference between industrial networksand general computer networks makes the commonlyused “detection/repair” methods in general computernetworks not fully applicable. In light of this, the currentstate of distributed computing systems has been evalu-ated in the present paper, and the key elements in defen-sive countermeasures can help to reduce the risks to anacceptable threshold. In 2014, Franke and Brynielsson[44] conducted an effective summary of 102 articles inthe four major scientific databases, being regarded asone of the best researches in the past 3 years, where 11sub-categories were compared and the current status ofthe research was discussed according to the researchfield or content. The literature [46] provides an overviewof the problems, challenges, threats, and solutions insocial network security. In a strict sense, computer net-work security is an integral part of social network secur-ity. Therefore, some of the methods mentioned providea meaningful reference, and the logic of their inductionand comparison has greatly inspired this current paper.By summarizing the literature review, it can be found

that the main thread of foreign research is to instantiatethe situational awareness model and method in the fieldof network security situational awareness, and continu-ously test and optimize the process in practice. In orderto effectively analyze the research details of networksecurity situational awareness, this paper concludes 75papers in the core database in recent years and the re-search points of these articles are mainly concentratedon 9 aspects (the key points in these 9 aspects are shownin Table 3). The research content is mapped with the

traditional Endsley model [36], the JDL model [40], andthe logical phase of the OODA model [45]:

� The concept of the model (integration with otherdisciplines) [16, 18, 39, 45, 50–57, 63, 67, 68]

� The completeness and regularization of dataacquisition variables [40, 42, 45, 47, 73, 87]

� The optimization of related algorithms [58–67]� The information fusion analysis [40, 42, 53, 69–74]� The automation of process tools [33–35, 73, 75, 84, 85, 87]� The visualization of work at each stage [5, 11, 55,

61, 76–79, 86]� Practice testing and efficiency gains in large-scale

real-world networks [80–82]� The software engineering implementation of sensing

methods [42, 83–85, 88]� The practical application of analysis and prediction

results in specific fields [42, 47, 73, 79, 87, 89, 90]

(1). In the research for the concept of model, somepapers are aimed at explaining interpretations oftraditional situational awareness models in networksecurity situational awareness (such as literature[16, 39, 45]). Some papers focus on thecombination of situational awareness with securityissues in specific fields. For example, Ralston et al.[47] summarize the safety perception problem ofdistributed control system and data acquisitioncontrol system. Barford et al. [48] defines andexplains the scope, background, and researchobjectives of network-aware defense. Alexandros

Table 3 A statistical classification summary of 75 foreign languages based on literature abstracts


et al. [49] summarizes the security threats anddetection technologies in the field of wirelessnetworks. Literature [50] has incorporated sensitivedevices into the priority perception area andshowed that how the DPI is installed at theboundary of the network perceives the health of thesystem; some literatures try to integrate the conceptsof other disciplines into network security situationalawareness, such as the combination with gametheory in literature [51–53, 63], the combination withPetri network [54], and the combination with theBayesian network [55]; also, some other articles try toprovide a more general operational model (such asthe literature [56, 57, 67]).

(2).Data acquisition is the basis of network securitysituational awareness. The attention now is paid tohow to ensure that the collected information is acomplete set for the fusion analysis in next step(completeness) and to standardize the collectedinformation to promote the mutual call betweendifferent systems (regularization). Giacobe [40] haseffectively combed the scope of source data andentities. In literature [45], the categories of sensorsare divided into three categories: activity,configuration, and topology. In addition, in thespecific field, the scope or type of collectedinformation may be different [42, 47, 73, 87].

(3).The research on perception algorithm orarchitecture accounts for the largest part in allliterature, with a proportion of more than 70%.Most of the articles give the logic of the algorithmand the demonstration effect in the experimentalapplication. Literature [58] divides the commonmethods in situation awareness into five categories:Bayesian approach, knowledge-based approach,artificial neural systems approach, fuzzy logicapproach, and genetic algorithm approach. In thealgorithm for network security situationalawareness, there are algorithms for data sources(such as the algorithm for the attacker [59], thealgorithm for intrusion detection data [60], and thealgorithm for the vulnerability logic associationanalysis [61]). Some algorithms are targeted at thebehavior analysis of attackers or defenders (forexample, hidden Markov chains are used to predictinternal attack threats in document [62], combinedwith game theory [63], machine learning method[64], and honeypot technology [65], etc.); also, thereare many algorithms for improving efficiency andenabling them to be extended in large scale networks(such as real-time decision analysis method [66], andfast calculation method for static statistical data [67]).

(4).The fusion analysis ability on the relatedinformation is the advantage of network security

situational awareness. The core method is to derivethe hidden knowledge from the data from differentsources. The related literatures are divided into threeparts: one is the instantiation of data fusion model intraditional situational awareness in the networksecurity situational awareness (such as [40, 42]); oneis to propose a specific fusion technology or ideabased on the characteristics of network security data.For example, Paffenroth et al. [70] and Mathews et al.[71] have designed data models or coordinateworking systems to integrate data from differentnetwork sensors. Literature [69, 72] discuss theuncertainty in the network security situation.Sanfilippo [73] design a multi-sensor fusionframework to improve the perception ability; otherliteratures attempt to promote the efficiency ofinformation fusion (e.g., [53, 74]).

(5).Automation based on the full use of the computingpower of the computer is one of the effective waysto improve efficiency. In the IDS phase (the secondstage of Table 1), the working mechanism of IDS isautomated, but it also becomes the bottleneck ofthe system in turn, since the rule of the computer isnot consistent with the perspective of human fuzzyevaluation. At present, the research on automationis mainly focused on information collection (such asliterature [33–35, 75]). In addition, systematicimplementation of the overall application effect hasrealized automatic processing to a certain extent(such as [84, 85]); the automation ability is also aprerequisite for the practical application of large-scale networks (e.g., [73, 87]).

(6).Visualization is undoubtedly an important part ofnetwork security situational awareness [86].Tamassia et al. [76] give a clear statistical result onthis aspect. Most of the current literature focuseson the friendly interaction between human andmachine. Beaver et al. [77] effectively filter theanalysis process and data in IDS and present themto administrators in a visual way. In literature [78],with the help of the unique professional knowledgeof the participants, a real-time evaluation visualframework is designed to allow network managers toparticipate in the analysis loop manually; somearticles focus on machine learning methods for visualrendering (such as artificial neural network [79] andcluster analysis [77]). In addition, most active analysismodels such as attack graphs are combined withvisualization technology [5, 11, 55, 61].

(7). Effect test constitutes the core of the modelconstruction. In most of the articles, there is achapter for the simulation experiment, but most ofthese experiments are analyzed with a brief abstracttopology, for the verification of the correctness of


the model. There are two aspects of research in thissegment. One is the construction of basic data setsthat can be used for horizontal comparison amongmultiple models (for example, the data setproduced by the security contest held in literature[80] in 2010. Fink [81] collates the data set by eachteam in the competition). The other is the practiceof wide area environment (at present, the attentionto this aspect is little; literature [82] has made apreliminary attempt on this).

(8).Consideration for the overall logic rather than acertain segment is the consensus view of thescholars [83], in view of the fact that the overalllogic means that it should be designed from theperspective of software engineering. Only on thisbasis, the process and result of perceptual analysiscan become effective tools. D’Amico and Whitley[84] design the overall analysis process based on thedifferent roles and present it visually; literature [85]gives a task flow chart according to processes, goals,and concerns. There is a long way to go now, andthe design and realization of network securitysituational awareness can be done from theperspective of instrumentalism software, whichintegrates the characteristics of all kinds of users inthe network, and give a friendly targetunderstanding method when human-machineinteraction with necessary attention [42].

(9).There are some articles concerning the analysismethod of network security situational awarenessand the practical application of prediction results inspecific fields. The present statistical results mainlyconcentrate on three parts: one is the application ofindustrial control networks [47], especially in thefield of power grid control [79, 87]; one is for theemergency management of the key equipment, suchas the shared situational awareness metamodelingproposed in Literature [89] and the operationalarchitecture proposed by Adams [90]; and anotheris in the military field [42], such as the practiceapplication of nautical training [73].

2.3 Status of domestic researchWhen it comes to the dominance of policies China, greatimportance is attached to the network security from top tobottom. As a consequence, China has established the emer-gency response mechanism related to network security atall levels, which is similar to European and American coun-tries, such as CCERT(China education and scientific re-search network computer emergency team), set up in May1999, and CNCERT/CC (National Computer NetworkEmergency Technology Processing Coordination Center,referred to as the “National Internet Emergency Center”),established in September 2002, as well as the central

network security and information leading group, formed onFebruary 27, 2014. On April 19, 2016, General Secretary XiJinping emphasized the importance, task, and goal of net-work security in his speech at the Symposium on NetworkSecurity and Informatization [91], and clearly put forwardthat perceiving network security situation is the most basicand basic work. Due to the limited space, this paper doesnot make too much interpretation of China’s network se-curity policies and industrial development.Domestic scholars have devoted great interest and

enthusiasm to academic research. Almost every relevantcore journal has dealt with the topics related to “networksecurity.” In order to summarize the current research situ-ation in China and keep in line with the research ideas offoreign literature, this paper firstly sorted out the reviewliterature based on the author’s accumulation and effectivesearch in this field. A total of 9 [17, 19, 92–98] compre-hensive literature has a large number of citations or strongreference significance. In literature [92], the research anddevelopment of cryptography, trusted computing, networksecurity, and information hiding in information securitytheory and technology are introduced. Especially in Sec-tion 4, Professor Feng Dengguo summarizes the researchstatus and development trend of network information se-curity and points out that the network-based securitytechnology is the future trend of the development of theinformation security technology. Almost all network at-tacks are implemented by using the security flaws in sys-tem software or application software. Based on thispremise, Liu and other scholars [93] conclude the researchstatus at home and abroad from three aspects: malicioussoftware, software vulnerabilities, and software securitymechanisms from the perspective of software design forensuring safety (study of the first stage in Table 1). Litera-ture [94] provides an interpretation from the concept,necessity, structure of system, and basic model of intru-sion detection and points out the development directionof intrusion detection system. In recent years, the researchon the intrusion detection system probes further into theexisting problems. Yingxu et al. [95] analyzes the charac-teristics and detection difficulties of industrial controlsystem attacks. The performance and characteristics ofdifferent detection techniques are compared in order toprovide theoretical support for researchers in the field ofindustrial control security. In 2005, Professor Lin Chuangof Tsinghua University [96] discusses the researchmethods and evaluation techniques used in the stochasticnetwork security model which can be employed for theactive evaluation and improves the network survivability.The analysis shows that most of the active evaluationmodels in the last 10 years (the third stage in Table 1) areextended on the basis of the models described in this art-icle. In the study of situational awareness, literature [97]introduces the basic concepts of network situational


awareness and expounds the relationship between situ-ational awareness and IDS. Gong et al. [98] put forward alogical research framework on the basis of full under-standing of situational awareness and attached emphasison the method of network assessment. Based on the fu-sion algorithm of cross-layer swarm optimization, Liu etal. [17] puts forward a cognitive sensing and controlmodel. Under the background of the transition of networkdevelopment from perceptual network to perceptual net-work, the related algorithms of quantitative perception aregiven. Gong et al. [19] discuss the relationship betweennetwork security situational awareness and situationalawareness at the conceptual level and further proposesthe definition and explanation of network security situ-ational awareness. Based on Endsley’s three-stage model[14], the stages of network security situational awarenessare divided, and the specific analysis methods of eachstage are compared.In light of the comparison between the domestic and

foreign literature, it is found that the time Chinesescholars pay attention to network security situationalawareness is close to that of foreign scholars, but mostof them are in the state of “following,” with few originaland innovative articles. Most of the high-cited articles inESI are aimed at the breakthrough of the model algo-rithm optimization and application level [96, 99], espe-cially in the aspect of situation quantitative computingperception [115, 117, 124, 129], which can be regardedas the main line of domestic research in this field. At thesame time, after a careful screening of domestic researchliterature, it can be found that a considerable number ofarticles on the topic of “information fusion, situationalawareness” only stay at the micro-cognitive level (whichis generally different from foreign literature based on theimprovement of Endsley’s model [36], JDL model [40],

and OODA model [45]), that is, more data sources areintegrated from the bottom up instead of the top down.However, these first partial then overall studies have alsomade remarkable progress and have played an obviousrole in promoting the whole field. By summarizing about100 articles among core journals in the CNKI, the re-search focus of these articles is mainly concentrated onfive aspects (the summary of the key research contentsin these five aspects and the typical article representa-tives are listed in Table 4):

� The definition or explanation of concept [17, 19, 97,98, 100–102]

� The intrusion detection data fusion [103–107]� The active evaluation model attempt [96, 101, 108–

114, 124–126, 128, 129, 132, 143, 153–155, 159–162, 177]

� The systematic evaluation after quantification [102,109, 115–117, 121–124, 173]

� The implementation of design and application inspecial fields [92, 118–120]

(1) The research on the definition or interpretation ofthe concept mainly focuses on two aspects: one is thebasic conceptual explanation, and the other is the prac-tical significance of network security situational aware-ness in special field after merging with other subjects.The basic conceptual explanation is mostly found in thesummary literature, such as the definition of the basiccontent and research category in literature [100], thedescription of the concept of intrusion detection in lit-erature [19], and the definition of the network securitysituation perception by the literature [17, 19, 97, 98].Prior to achieving multisensory integration with otherdisciplines, it is necessary to do the abstract definition,

Table 4 Statistical classification of about 100 Chinese literature based on titles and abstracts


which can explain whether the integration is effective,and the effect after the combination, such as the defin-ition of color Petri net (CPN) in literature [101] and thedefinition of risk propagation model in [102].(2) The fusion and utilization analysis on IDS includes

two aspects: the collection of more complete datasources and the integration and utilization of multipletypes of data. In the collection of multi-data sources,there is a good display in the evaluation framework ofliterature [103]. Li and Lan [104] combine data attri-butes with time attribute and space attribute, which isbeneficial to the evidence fusion of subsequent data;there are lots of articles for multi-type data fusion; litera-ture [105] combines multiple IDS and manual surveytechniques, and studies its optimal allocation and strat-egy based on game theory. Ren et al. [106] puts forwardan intrusion detection model based on data mining andontology, which can cluster and classify the underlyingalerts, discover and filter attacks, and then based on theestablished ontology attack knowledge model, correlatethese attacks to identify, track, and predict the effect ofmulti-step attacks, such as the fuzzy clustering anomalyintrusion detection method in literature [107].(3) The attempt of the active evaluation model mainly re-

volves around the attack model, and each article usuallycontains three components: model definition, model solv-ing algorithm, and solution result. The definition of themodel is generally combined with other disciplines,such as Petri network [96, 153–155], game theory[108, 124, 159–162], and Bayesian network [114, 132],and some articles also focus on the improvement ofmodel description ability [125, 126]; the solution algo-rithm depends on the definition of the model, and itis generally shown together with the solution result.There are lots of literature [109–114] trying to improveon this point, such as the reachable path analysis based onattack graph [101, 128, 129, 143, 177], defense strategyanalysis [111, 124, 161], and survivability analysis [126].(4) There are three main parts in the systematic evalu-

ation after quantifying: systematization of evaluationindex, index quantification, and quantified results andits application. The research on the systematization ofevaluation index and the quantification of correspondingindicators mainly proceed from two angles: security at-tribute and attack behavior. From the perspective of se-curity attributes, it is more focused on the definition andinterpretation of network security. For example, Wanget al. [121] propose an attack technology classificationmethod to meet the Amoroso classification standard;from the perspective of attack behavior, most of theresearches take the attack as the center to quantify theimportant factors in the attack process. According to thestatistics and analysis of the existing literature, the quantifi-cation of the 3 elements (attack severity, attack occurrence/

success probability, and attack income) has basically formeda certain standard [102, 122–124]. On the basis of indexsystem and index quantification, risk assessment algo-rithm can be developed to get the perception orevaluation result [109, 115–117].(5) The active participation of all parties will definitely

promote the production of relevant research results anddeepen the application in the industry. The emergencyresponse of China’s network security follows thePDCERF methodology (the preparation, detection, eradi-cation, suppression, recovery, and tracking of 6 stages).A large number of practical products and systems havebeen put into use, such as information sharing and ana-lysis center, large network security events coordinationearly warning positioning and rapid isolation control, se-curity event planning system, large-scale network secur-ity state simulation platform, linkage system, and backupand recovery system [92]; on the combination of indus-try applications, similar to foreign countries, it mainlyfocuses on two aspects: ICS [118, 119] and ECPS [120].

2.4 Summary of the present researchThis section summarizes the research history, develop-ment stage, and present situation at home and abroad ofnetwork security situational awareness. In general, in thebackground of winning the commanding heights ofnetwork security strategy for all countries, the researchon this aspect is of great significance and has madeconsiderable progress, but the result of the study is stillon the path of exploration, and the main problems areconcentrated in three aspects.Firstly, there is no comprehensive analytical perspec-

tive in terms of concept and ideology. Foreign researchesmainly focus on the instantiation of situational awarenessin this field, and domestic researches concentrate more onthe integration of more information and efficiency im-provement. However, according to the summary of Table 1in this paper, network security situational awareness is amore advanced stage of network security research. It isnot a model or a method. It should be a more valuableframework from all the existing network security conceptsor means.Secondly, there is no practical deep integration at the

level of model and algorithm. Both foreign and domesticarticles on models and algorithms are over 70%. Althoughmultidisciplinary integration is an important breakthroughin this field, after the groundbreaking formulation, mostof the articles begin to model and algorithm optimizationsblindly. This is incorrect since these improvements shouldbe carried out on the basis of integration practice. Inaddition, fusion perception must be a process of multiplecycles between information and decision-making. Most ofthe existing models are unidirectional, and the level of


feedback effect should be effectively embodied in themodel after perception decision.Thirdly, there is no meaningful horizontal comparison

in terms of effectiveness and application level. Every art-icle or model will be verified by experiments, but few ar-ticles are compared as a whole. The existing andprevious literature are more compared in the complexityof the algorithm, and the result of perception is a com-prehensive synthesis of intelligence. It is different tojudge directly for so many constraint factors, and thecurrent application value comparison should focus onthe horizontal comparison within a certain stage basedon a standard data set.The following chapters are arranged as follows: The

second section abstracts the experiment object from theactual network topology and configuration of a mediumscale software company to ensure the accuracy verifica-tion and relative comparison in the following chaptersunder the same standard. In the third section, from theperspective of system engineering, the network securitysituational awareness analysis is divided logically andgives out a new reasonable frame. From the fourth tothe eighth, each segment of the whole framework isexpounded, focusing on the role of this segment, themainstream method, the application results on the ex-perimental network, and the horizontal comparison be-tween the methods within a certain segment. The ninthsection briefly introduces the research dimension anddirection of network security in a big data environment.The tenth section is the summary of the full text.

3 Experimental basisIn order to effectively compare and summarize the dif-ferent methods in different stages of the proposedframework, this section first briefly introduces the ex-perimental environment used in this paper as the basisfor subsequent chapters. A medium-sized software

development company is chosen as an experimental ob-ject. Figure 2 is the network topology graph of the enter-prise. The network God is used as the monitoring devicebetween the internal and external networks through thededicated telecommunication lines and the external net-work links. 10.10.0.10 is a web server which provides thefunction of publicity website and product demonstration.10.10.0.140 is a log server that can be accessed from theexternal network (because company personnel are oftenon business trips, both internal and external network ac-cess are required to go through the external network).10.10.0.15 is the company’s database server, runningSQL Server, Oracle, the two relational databases, and anon-relational database MongoDB. 10.10.0.16 is the testserver, and the products the company has delivered andis developing have the latest version of the deploymenton the test server. 10.10.0.11 is the internal developmentserver. All the company’s source code and importantproject solutions, process information, etc. are all on thisserver. The company has a development team of about100 people, which is mainly divided into two categoriesdue to the different development technologies.10.10.0.58 represents the technical team developedby.net, and 10.10.0.59 denotes the technical team devel-oped by Java.

4 Logical analysis frameworkNetwork security situational awareness usually involvesmultiple different phases, and the systematic approach ispreferred to process the data related to cybersecurity.There are two main methods for logical division: thefirst method is the engineering hierarchical method(such as Figure 2 in literature [45], Figure 3 in literature[97], Figure 1 in literature [103], and Figure 4 in litera-ture [126]) and the second is the conceptual hierarchy(such as Figure 3 in literature [45] and Figure 1 in litera-ture [14]), but neither of these methods can provide an

Fig. 2 The graph of experimental network topology


easy-to-understand architecture from the perspective ofthe data processing stage. From the perspective of datavalue chain, the present paper adopts the systematicengineering method which is widely accepted by indus-try to decompose the typical cybersecurity situationalawareness process into five continuous processingstages, including element acquisition, model representa-tion, metric establishment, solution analysis, and situ-ation prediction, as is shown in Fig. 3 below.

(1) The element acquisition phase is concerned withhow to effectively obtain the security-related data asmuch as possible, which is mainly divided into twotasks: data acquisition and data preprocessing. Dataacquisition refers to the effective storage processincluding collecting configuration information inthe network, behavior information in the log, andvulnerability information which can be achieved byusing a scanner, a sensor, or a specially written tool.Data preprocessing is a process of regularizingoriginal data before data modeling or analysis andutilization.

(2) The model representation stage is focused on thecorrelative expression of the effective elements,which is mainly divided into two tasks: elementreduction and formal representation. According tothe purpose of the analysis, it is necessary to reducethe acquired objects effectively during the elementacquisition process in order to achieve the efficientanalysis. The formal expression refers to the processof precision abstraction including the attributes ofthe reduced elements, the relationship between theelements, and the order relationship.

(3) The metric establishment stage is the process ofrefining the value of each element object before the

solution analysis, mainly including the quantitativeclassification and evaluation index system todetermine two tasks. The quantitative process is aprocess of numerically assigning the attribute valuesof each element (in this present paper, thequalitative classification is treated as a specialquantitative classification without specialexplanation), and the confirmation of the evaluationindex system is the process to regularize the logicalrelationship between the attribute values of theelements.

(4) Solution analysis is the algorithmic process basedon the first three stages mentioned above, whichmainly includes three tasks: the determination ofthe solution algorithm, the verification of thecorrectness of the algorithm, and the improvementof the algorithm. The solution algorithm is theprocess of effectively combining the target with themodel and the metric to ascertain the analysis step.The correctness verification of the algorithm is tovalidly correspond to the input and output of thealgorithm. On this basis, the efficiency of thealgorithm should be considered to improve in orderto expand in the true scale network environment.

(5) Situation prediction is a process of comprehensiveevaluation and decision-making based on the analysisresults, which mainly includes two tasks: resultvisualization and decision-making after knowledgeapplication. The result visualization is the process ofpresenting and constructing the solution results in aneasy-to-understand way. After the analysis anddecision-making, the feedback loop will be applied tothe current network for cybersecurity reinforcement(such as vulnerability repair and configurationupgrade) to complete a perceptual loop.

Fig. 3 Network security situational awareness operation mechanism


5 Phase I: Element acquisitionThe function of the element acquisition phase is to ef-fectively capture the key data used in each phase of thecybersecurity situational awareness. In general, elementacquisition refers to the collection of all the elementsrelated to cybersecurity. In the narrow sense, elementacquisition refers to the collection of the elementsinvolved in a certain perception process. The purpose ofthis present paper is to sort out the basic framework ofcybersecurity situational awareness, and the core imple-mentation methods of each stage are compared horizon-tally, so the element acquisition in this current paperrefers to the generalized element acquisition.Undoubtedly, element acquisition is the premise of cy-

bersecurity situational awareness. Other subsequentstages are unable to work without basic data collection.Most of the documents collected so far have clearlydefined the functions and important impacts of thisstage in the logical description of the framework. How-ever, as for the implementation, most of them only men-tion the data acquisition through automated scanningtools or sensors, and according to the following-upmodel to directly stipulate or preprocess, there are alsosome literature introducing the way to obtain data ortools [33–35] and so on. Strictly speaking, element ac-quisition is divided into three parts: data generation,data acquisition, and data preprocessing. In light of thedivision of logic analysis framework in Section 3, datapreprocessing is generally carried out after the modeldefinition or measurement establishment phase. Data ac-quisition is generally completed by combining manualand automatic methods. The focus is generally on thedevelopment of automated tools. This section focuses onthe classification of data from the perspective of datageneration.In the existing cybersecurity situational awareness lit-

erature, the basic data collection part is mostly accord-ing to the needs of model analysis to reverse the dataused (narrow element acquisition), which is not condu-cive to data standard unification and model-to-modelcomparison verification. According to the logic of engin-eering, this present paper briefly summarizes and classi-fies the data in cybersecurity analysis from theperspective of data generation.Here, the data is divided into two categories: static

data and dynamic data. Static data refers to data thatdoes not change substantially in a cybersecurity situ-ational awareness analysis cycle shown in Fig. 3. Dy-namic data refers to changes in the cybersecuritysituational awareness analysis cycle shown in Fig. 3 asthe analysis process going on. As shown in Table 5, thestatic data mainly includes host information (such ashost IP address or MAC address unique identifier, run-ning service or program, file, data and other confidential

assets, operating system, hardware composition, systemconfiguration, and permission configuration), networkinformation (such as network device information, net-work topology information, protocol information,firewall information, and network configuration informa-tion), and IDS information (such as basic information ofintrusion detection system, expert knowledge base, andalarm information), and the dynamic information mainlyincludes activity information (such as source address,destination address, and activity description), behaviorinformation (such as source address, destination address,protocol in use, transmission data size, and compressionalgorithm), vulnerability information (such as vulnerabil-ity name, logo, basic information such as release time,vulnerability host information, attack methods, attack ef-fects, and repair methods), attack information (such asattack source address and attack method), and perceivedresult information (e.g., perceptual result information ofthe last perceived loop and the action information afterperception).

6 Phase II: Model representationFormal modeling is the key link in the cybersecuritysituational awareness operation mechanism. The de-scription ability in the modeling stage of reduced stateand formalization will directly affect the subsequent per-ceptual analysis results. Through the summary of theexisting literature, the cybersecurity situational aware-ness model is mainly divided into three categories:mathematical model, stochastic model, and biologicalheuristic model. The core concepts and typical represen-tatives of each classification are shown in Table 6 below.

6.1 Mathematical modelThe mathematical model is used to analyze the cyberse-curity situational awareness. The main idea is to usemathematical language or mathematical symbols tosummarize or approximate the security-related featuresor quantity dependencies of computer network systems.The mathematical model here refers to the mathematicalmodel in the narrow sense, that is, the mathematicalexpression of the relationship between variables in thecybersecurity system. Therefore, the perceptual analysismethod based on a mathematical model is more biasedtowards the form of quantitative analysis. It mainly in-cludes analytic hierarchy model, Bayesian model, fuzzyset/rough set model, reliability/survability model, etc.The Analytic Hierarchy Process (AHP) was proposed by

Professor T.L. Saaty and is now widely used in decision-making. Chen et al. [99] proposed a hierarchical securitythreat assessment model (Fig. 4 is the model results ob-tained by the experimental network according to themethod in literature [100]), and Fig. 5 is Tomcat service,FTP service, and the overall security situation of each host


and local area network are security situation quantificationresults, based on the subjective quantization method in lit-erature [99]; Tomcat service, FTP service, and the overallsecurity situation of each host and local area network aresecurity situation quantification results. The hierarchicalmodel is consistent with the decision-maker’s thinkingprocess in both the analysis and the calculation process,which ensures the results are intuitively understandable (forexample, the security situation index is relatively high inFig. 5 at around 17:30; because most people fill in the logsaround this moment, the frequent external network map-ping will lead to higher security risks). The construction ofan effective hierarchical structure is the key to the applica-tion of this model, and some literature has studied the in-stantiation of the hierarchy [127], but the current element

quantization process basically adopts the subjective experi-ence value method, which cannot be compared and quanti-fied between every two factors in the classical analytichierarchy process, thus leading to the lack of objectivity,and the current hierarchical structure is only suitable forthe local area network which contributes to the difficulty incarrying out large-scale promotion, as well as no effectiveprediction of the future situation.In order to effectively reflect the uncertainty and sub-

jective elements in the cybersecurity situational awarenessanalysis, the probabilistic method is usually used for quan-titative description [128, 129], in which Bayesian logic isthe most commonly used model. The relationship rulesand mathematical reliability of Bayes are very similar tothose of human thinking reasoning. Bayesian calculation

Table 5 Classification results of entity and data in element collection

Table 6 The main model and its classification of network security situational awareness


can synthesize the latest evidence information and priorinformation to ensure that the calculation results maintaintwo important characteristics: continuity and accumula-tion. There are literatures adopting Bayesian mathematicalmethods for cybersecurity situational assessment [131], butmost of them are used as quantitative computing tools incombination with other models, especially the combinationof Bayesian and attack graphs [114, 130, 132], combininggraph theory and probability theory to complete a Bayesiannetwork, using graph theory to show the structure andinterdependence at the qualitative level, and using probabil-ity theory to carry out quantitative expression and reason-ing at the quantitative level. Some progress has been madein this perspective, but the Bayesian network is a decom-position form of the joint probability distribution at the the-oretical level. The variables in the actual solution are notindependent from each other, and the joint probability istoo complex to suit the large-scale networks.The fuzzy set contraposes the traditional set. In the

traditional set, the relationship between the object andthe set is clear (either one or the other), but in reality,some objects do not have a clear affiliation of the set,There exists an interval of degree of membership

(membership function). Some literatures apply fuzzysimilarity and fuzzy comprehensive evaluation in cyber-security situational awareness analysis [133, 134]; therough set extends the classical set theory, which uses theupper and lower approximations to approximate any set,and it can analyze incomplete information such as in-accuracy, inconsistency, and incompleteness withoutprior knowledge, discover hidden knowledge, and revealpotential laws. Zhao and Xue [135] and Kong et al. [136]utilized the idea of rough concentration mode classifica-tion in the cybersecurity situational assessment, usingeach security evaluation index as the condition attributeset C, and determining the decision attribute D of theload situation assessment result according to C and thenaccording to the D synthesis comprehensive securitysituation network. However, the current research in thisarea is limited to describe the uncertainty in the processof fuzzy sets or rough sets, and it is impossible to com-bine the target or core problem of cybersecurity situ-ational awareness with the fuzzy set or rough setmethod. The practicability and the continuity of researchare limited. In combination with other models ormethods, it is generally carried out at a certain point in

Fig. 4 Hierarchical structure partition diagram of experimental network

a b

c d

Fig. 5 Hierarchical security situational awareness results of experimental network. a Threat situation of Tomcat on server. b Threat situation of FTPon server. c Threat situation of host level. d Threat situation of system level


the analysis process and adopted more as a quantitativetool for uncertainty.Feng et al. [137] combined the reliability theory with

the vulnerability analysis process to quantify the securityof the distributed system. It is intended to ascertain thesystem maintenance probability of the security stateunder the specified conditions and the specified cost cthrough the reliability function Rs(c). Figure 6 below isthe vulnerability state modeling result of the attack onthe Ftp service on 10.10.0.11 (internal developmentserver) according to the literature [137], and the averageattack cost for this service is E(C) = 1/λ1 +1/λ2 +1/λ3 +1/λ4 +1/λ5. In literature [138], the mathematical condi-tions are used to obtain the criteria of complete prob-ability control or partial probability control of complexattack networks. It is theoretically proved that if thereare effective defense nodes in the network, the complexnetwork can still provide normal service when it isattacked and destroyed and suggests ways to defendagainst node selection and control networks. The ad-vantage of adopting the reliability or survivability modelfor cybersecurity situational awareness analysis is thatthere is a mathematical derivation process to ensurethe rigor of the analysis, but the preconditions of theseformulas also greatly limit its large-scale network con-ditions of actual perceptual analysis, the diversity of in-fluencing elements in the real network often makes thecalculation result unsatisfied, and the model generallycannot provide the repair method after confirming thenetwork insecurity state, so that the system has theability of active defense.

6.2 Stochastic modelThe stochastic analysis model is a non-deterministicmodel. Its main feature is that the exogenous variablesin the model will change with specific conditions, whichhas a high degree of fit with the occurrence ofcybersecurity-related behaviors. During the attack, thechoice of the attacker’s assault means the choice of thedefender’s resist strategy and the normal user’s operation

are random. Using a stochastic model for cybersecuritysituational awareness, it is possible to describe the lo-gical relationship between the random behaviors and be-haviors of various elements of the system more clearly,and thus, it is easier to fully describe the network status,and it can also include the influence of unknown behav-ior, based on Stochastic model cybersecurity situationalawareness is the focus of current academic circles, in-cluding attack tree/graph model, Petri net, game theory,and Markov’s model.The attack tree model was proposed by Scheier [139]

in 1999. It can be seen as an extension of the fault tree,which is intuitive and easy to understand, but the de-scription capabilities are limited. The attack graph modelwas first proposed by Swiler and Phillips [5] in 1998. Itis currently the most widely used method. Sheyner et al.[140] adopt the model detection method to generate theattack graph, and Ammann et al. [61] generate an attackgraph through the idea of graph theory which startsfrom the initial state and searches forward. The litera-ture [141] focuses on the attack, and a tool for generat-ing an attack graph is given. There are also literaturefocuses on large-scale construction and visualization ofattack graphs [142, 143]. Early attack graphs tend toconstruct state attack graphs [5, 61, 140–143], but it iseasy to cause the explosion of state space. As the re-search progresses, it tends to construct the causality dia-gram [144], and its edges represent the connectionrelationship between nodes or the logical relationship ofatomic attacks, which is more scalable and easier to usefor large-scale networks. Figure 7 is the result of theattack graph of attacker Eve attacking the FTP servicelocated on the development server (10.10.0.11) in the ex-perimental network in Section 2. Figure 7a is a graphicaldescription, and Fig. 7b is a formal description of theattack step. The advantages of attack graph model is dir-ectness and descriptive and is easy to combine withother methods which are the currently basic model ofcybersecurity situational awareness analysis; the currentresearch focuses on the refinement of the original [125]

Fig. 6 Modeling results of reliability quantitative model of experimental network


or improved model [145] to enhance the descriptionability and fusion with other disciplines [11, 146] andthus to enhance the analytical ability.Models similar to the attack graph also include privil-

ege graphs and state transition graphs. Dacier [147]abstracted the nodes in the graph into the permissionstate and proposed the privilege graph model. Ortalo etal. [148] established the Markov model based on theconcept of privilege graph and presented the securityevolution process. Dr. Wang Lidong [149] refined thisprocess, but the privilege graph model is difficult todescribe the dependencies between states or randomevents, so subsequent research on the extension of thismodel has little influential results; Porras and Kemmerer[150] proposed the intrusion detection method based onstate transition graph for the first time. Each node in thegraph represents a temporary state of the system, andthe edge represents the state transition and transferprocess. The probabilistic model in literature [151], thesemi-Markov process model in literature [152], and soon are all the extensions based on it. The advantage ofthe state diagram is that it is more descriptive, but thereare problems of state space explosion under large-scalenetworks, and the solutions to this problem [128, 143]are still not satisfied.Petri Net (PN) was first proposed by Karl A. Petrie in

1962 to perform effective mathematical simulations ofdiscrete parallel systems. It consists of three elements:place, transition, and the directed arc (Arc); N = (P, T; F)can have any number of tokens in the place to representthe resource (Token), and the initial application scenariois through the flow of Token in the place to detect theprotocol Error (deadlock state). In the combination ofPetri net and cybersecurity situational awareness, theplace P usually represents the descriptive local state ofthe system. The transition T represents an attack event

or normal activity that can change the state of the sys-tem. The directed arc F effectively associate the localstate and the event. On the one hand, it refers to thelocal state that can cause the change to occur, and onthe other hand, it points to the change of the local statecaused by the change. The following Fig. 8 shows theexperimental network in the second section which is thePetri net model modeling result of the FTP serviceattack for 10.10.0.11. Compared with the classic Petrinet, the place is not a Token, but the probability of atransition occurring in a local state. The number at-tached to the transition represents an attack or successprobability, on this basis, qualitative reachable identityanalysis or quantitative analysis by correlation matrix,state equation, etc., for example, using the “or” principleof maximum risk estimation (maximum probabilitybetween different paths) and the probability of the inter-mediate place P7 is max (0.4 × 0.4, 0.7 × 0.5, 0.8 × 0.1) =0.35. It can be seen that Petri net not only has the char-acteristics of intuitive and vivid of graphical modeling,but also is more suitable for asynchronous and parallelattack process. The research progress in this directionincludes coloring Petri nets with increased modeldescription ability [153], a stochastic Petri net with in-creasing random occurrence time for transitions [154], afuzzy Petri net described for uncertainty in the modelingprocess [155], etc.With the deepening of cybersecurity situational aware-

ness research, researchers have realized two problems:First, the cybersecurity confrontation process is not sim-ply a technical matter, and different people who apply indifferent scenarios will produce the opposite result withthe same technology implementation means; Second, theanalysis of cybersecurity must not be the behavior of oneparty. In an environment with active defense, the secur-ity situation will variate on the choice of two or more

a b

Fig. 7 An attack map for FTP on 11 servers in the experimental network. a A graphical description of an attack graph. b Formal description ofApache attacks in attack steps


parties, which has a very high degree of agreement withthe strategic dependence of game theory. Once pro-posed, it has become a hot topic of research [156]. Trad-itional research on intrusion detection or aggressionbehavior is based on a game analysis [157]. Consideringthe application in the real environment, it is certainly arepetitive multistage incomplete information dynamicgame [158], and there is a refined Bayesian Nash equilib-rium. Each cybersecurity situational awareness modelbased on game theory contains at least five parts: N = {1,2,…,n} is a collection of people in the game (generallycombines multiple similar objects and divides them intoattackers, defender, and normal user |N| = 3). S = {S1,S2,…,Sk} is the set of game states in the offensive and defen-sive process. θ = {θA, θD}is the set of action strategies ofboth offense and defense. P is the transition probabilitybetween game states S. Rn = Si × θ × Sj∈(−∞,+∞)), whichrepresents the income function of the person n in thestate Si transitioning to the state Sj; GM = {N,S,θ,P,R},according to this basic definition, after a finite-step (k-step) game process, the system transforms between dif-ferent states to form a tree structure, the goal of theplayer is to make their function maximized, and themodel’s Nash equilibrium strategy f* can be obtained bymeans of Shapley algorithm or problem transformationsolution [162]. The combination of game theory makesthe focus of cybersecurity situational awareness rise fromtechnology to management strategy and can portray thepsychological activities of each participant, which greatlyimproves the description ability of the model and thescientific nature of the analysis results. The improve-ment direction focuses on static game turning to dy-namic game [159], model-related element quantification[10], or combining with other methods [160] and pre-sents practical application effects [108, 124, 161], etc.

The basic idea of the Markov model is that the transi-tion of the next state is only related to the current statebut not the historical state. The Markov model consistsof three elements: S is the set of non-empty states com-posed of all possible states of the system, P is the systemstate transition probability matrix, and Q is the initialprobability distribution of the system, M = {S,P,Q}. Theintention of applying the Markov model to the cyberse-curity situational awareness is to predict the attack anddefense evolution effectively when the initial conditionsare met, but there will be a large number of camouflageattacks or covert attacks during the attack. Forcing theapplication with inefficiency will lead to the extreme resultof statistics (overexaggerating the impact of a certain acci-dent or neglecting the impact of a key step), so Markov isgenerally combined with other models [53, 109, 162]. Toobtain causal knowledge through Markov’s method, andto simplify the operation process by one-step transitionprobability matrix, the model can be performed efficientlyunder large-scale networks.The risk communication model is proposed by Zhang

et al. [102], whose core idea is that the risk of a networksubject will spread to the object with non-vulnerabilityor even the whole network because of the high relevanceof the network system, so it needs effective means toeffectively evaluate the risk state of the whole networkinformation system. The risk communication model(vulnerability diffusion model) is generally composed oftwo parts: network abstraction and propagation algo-rithm. The network abstraction describes the logicalaccess relationship structure of the system, and thepropagation algorithm describes the rules of risk diffu-sion. Figure 9 below is the result of abstract modeling ofthe risk diffusion logic access to the development server(10.10.0.11), the database server (10.10.0.15), and the test

Fig. 8 Petri net modeling results for FTP attacks on 11 servers in the experimental network


server (10.10.0.16) in the second section of the experi-mental network, after the attacker’s attack on the webserver (10.10.0.10), in which the weight of the directededge represents the attack revenue. If we use the cumu-lative effect algorithm that ensures the optimal result ofthe final risk diffusion to determine the diffusion value

between the nodes λuv, that is λuv ¼ wðu;vÞXm∈NðvÞ

wðm; vÞ where

w(u,v) represents the weight between nodes u and v; wecan get the results shown in Table 7 below. From the re-sult, we can see that the risk state of the network is notonly related to the object with vulnerabilities, but alsorelated to the logical access structure and the distribu-tion state of the vulnerabilities, and the risk propagationmodel can be used to identify the most security threatsor risk propagation paths.

6.3 Biological heuristic modelThe intelligent computing method, which is inspired bythe natural phenomena or processes of nature, is calledthe biological heuristic calculation method. The basicprinciple is to explore the solution of a problem

combined with the known information and to effectivelyrecord and accumulate related information during theexploration process and guide the next move and correctthe previous steps, and then get better overall results.The attacker’s attack process and the defender’s defenseprocess are also the same. They are all based on thecurrent knowledge state to seek the maximum benefit atthe least cost. This promising approach can be regardedas the specific application of artificial intelligence in thefield of cybersecurity situational awareness. At present,the research is in its infancy, the high-dimensional andnon-linear data in the offensive and defensive processare abstracted, and the results of the solution throughheuristic calculation are tested and improved in terms offeasibility and optimality. Models that have made someprogress include neural network models and artificialimmune models.The general method based on neural network is to use

the collected real-time security status indicators (such asvulnerability information, attack methods, and defensemethods) as the input vector X, and regard the indica-tors of situation awareness results (such as confidential-ity and integrity) as the output vector Y. In this regard, anon-linear mapping from X to Y is constructed by effect-ive training [163, 164]. Literature [165] introduced theneural network learning method in IDS research, whichgreatly improved the accuracy of the alarm effectively.The literature [166] integrated the self-encoding networkand deep belief network structure technology into therisk identification model and proposed a lightweight in-trusion detection model which can reduce training timeand test time to a certain extent and reduce the falsealarm rate.Computer immunology, which imitates the biological

immune system [167], has been widely used in cyberse-curity situational awareness analysis. The literature [168]proposed an immune model that applies the dynamicclonal selection algorithm to the network intrusion de-tection system. Based on the correspondence betweenthe changes of antibody concentration in the human im-mune system and the invasion intensity of pathogens, LiTao proposed an immune-based cybersecurity risk de-tection model [169], and an immune-based networkmonitoring model was established by the dynamic modelof immune memory and the recursive equation of re-sponse [170]. In literature [171], the artificial immune al-gorithm is used as a multi-objective solution method forrisk assessment, which shows the change of cybersecu-rity status under different attack strategies to some ex-tent. However, as a new approach to cybersecuritysituational awareness analysis, the immune model mustfully mimic the mechanism of immunology to function.The complexity and agnostic of immunology will makethe modeling and solving process more complicated.

Fig. 9 Logic access modeling results of risk diffusion forexperimental network


Whether it can effectively reflect the evolution of the se-curity situation remains to be tested.

6.4 Combination and comparison between modelsTable 8 shows the classification results of each model in9 dimensions. It can be seen that there is no model thatcan meet the high standard requirements of more than 5dimensions at the same time, which also indicates thatthe research on network security situational awareness isstill in the exploration stage. For the formal modelingphase of model representation, there are two mainimprovement aspects: one is to improve or enhance theresearch for a certain model, such as in-depth analysisbased on attack graph [101, 143, 146] and the applica-tion of fuzzy set ideas in the field of perception [107].Most of them belong to the second category, that is,through the combination of models, the purpose of analysiscan be achieved by means of the advantages of multiplemodels, such as Bayesian attack map [114, 128, 129], fuzzyPetri net [155], and Markov game [162].

7 Phase III: Establishment of metricsThe core purpose of metric establishment is to refine orquantify the value of each element object involved in cy-bersecurity situational awareness before solving the solu-tion. According to the cybersecurity situational awarenessoperation mechanism in Fig. 3, the metric establishmentphase may occur after the formal representation of themodel, or directly on the basis of element acquisition, sothis phase is mainly divided into two cases: one is modelelement quantification and the other is the evaluation sys-tem and index.

7.1 Model element quantificationIn the process of formal modeling in Section 5, the rele-vant elements have been defined in detail. To conductthe solution analysis needed for cybersecurity situationalawareness, it is also necessary to quantify each elementin the model (from the perspective of model descriptionability, the process of quantifying the value of elementsis also the process of describing the refinement of cap-abilities). Therefore, this stage has a strong correlation

Table 7 The λuv calculation results of each node’s in Fig. 10

Table 8 Comparison results of each model


with the idea of model construction. Through the existingliterature statistics and analysis, it is found that the modelsare focused on different points, but each model contains adescription of the attack behavior. The quantification ofthe three elements of attack severity, attack occurrence,success probability, and attack revenue has basicallyformed certain standards or norms.The metrization premise of serious attacking is the

qualitative classification of attack types. The variety ofcyber attacks leads to different types of attack. Atpresent, the six-member representation method, pro-posed by Christy [122], has strong practicality and hasbeen accepted by most people. Based on the qualitativeclassification method, it is divided into several levels toquantify the severity of the threat [102, 124]. Thismethod is generally associated with the alarm mechan-ism of IDS and is widely used in intrusion detection.The widely used method in the attack model is CVSSvulnerability evaluation mechanism [10, 123], which isdivided into three aspects: basic evaluation criteria, lifecycle assessment, and environmental assessment. Thefinal result is 0~1. The higher score indicates the greaterthreat to the vulnerability.The purpose of quantifying the occurrence of attack/

successive probability is to measure the authenticity ofthe attack or the possibility of successful attack. The net-work attack process is filled with a large amount of falseand useless information. The information provided byeach host and security device is often inaccurate; thisbrings great difficulty to the comprehensive estimationof the information fusion model. Currently, the subjectiveprobability estimation method of experts is mainly used ineach experimental model [10, 128, 162] (Tables 9 and 10are the quantitative criteria used in the follow-up analysisof this article [124]), and the Bayesian network can effect-ively express the probabilistic reasoning of uncertaintyknowledge, and thus in this research, Bayesian-based esti-mation methods [55] have also made some progress.The quantification of the attack revenue is an important

part of the attack effectiveness evaluation. Generally, thedestructive size of the attack is qualitatively measured (forexample, the attack acquires the root permission of a ser-vice [5, 6], etc.), and then the quantitative value of thedamage degree is given according to the qualitative classi-fication. The quantitative research can be carried out from

the perspective of the attacker and the defender. From theview of the attacker, the quantitative research refers to thereturn obtained by the attack under a certain attack cost,while the defender refers to the loss of the system at a cer-tain defense cost. In general, the attack revenue is lessthan the network system loss. For the sake of simplicity,the defense loss is used as the attack benefit in mostmodels [124]. This method is also adopted in the subse-quent analysis of this paper.

7.2 Indicator system and indexThe indicator system is used to evaluate and reflect acertain situation in a certain field and is widely used atall levels. Different from the point-based quantificationof each element in the model, the cybersecurity situ-ational assessment index system should proceed fromthe whole, intending to exhaustively classify the attri-butes related to the cybersecurity situational evaluation,giving the clear meaning of each class; the quantitativeoperation is carried out based on mutual related andcomplementary systematic indicators, and through themathematical calculation method to obtain the cyberse-curity situational index value to be evaluated, throughthe change of the index value to reflect the change of cy-bersecurity status.The cybersecurity situational indicator system and

index distract the network administrator’s concerns freefrom the scattered or massive log data monitoring; facili-tate the intuitive response to the cybersecurity state,especially the relative number of changes help to findabnormalities better; and then confirm the main influen-cing elements and achieve effective protection. It mainlyincludes two aspects of work: one is to comprehensivelyand systematically ascertain the elements related to cy-bersecurity situational awareness (the evaluation systemin Fig. 3 and the quantified parts of each metric element)and the second is to establish a mapping model betweensystemic elements and result index (mathematicalanalysis method and solution analysis part are confirmedin Fig. 3).Based on the effective synthesis of the explanation of

the specific meaning of network security and the studyof reliability, Lin

Documents

Analysis framework of network security situational awareness … · 2019. 8. 13. · REVIEW Open Access Analysis framework of network security situational awareness and comparison