When Wireless Security Meets Machine Learning:Motivation, Challenges, and Research Directions

Yalin E. Sagduyu1, Yi Shi1, Tugba Erpek1, William Headley2, Bryse Flowers3, George Stantchev4, and Zhuo Lu5

1Intelligent Automation, Inc., Rockville, MD 20855, USA2Virginia Tech, Blacksburg, VA 24060, USA

3University of California, San Diego, CA 92093, USA4Naval Research Laboratory, Washington, DC 20375, USA

5University of South Florida, Tampa, FL 33620, USA

Abstract—Wireless systems are vulnerable to various attackssuch as jamming and eavesdropping due to the shared andbroadcast nature of wireless medium. To support both attack anddefense strategies, machine learning (ML) provides automatedmeans to learn from and adapt to wireless communicationcharacteristics that are hard to capture by hand-crafted featuresand models. This article discusses motivation, background, andscope of research efforts that bridge ML and wireless security.Motivated by research directions surveyed in the context of MLfor wireless security, ML-based attack and defense solutions andemerging adversarial ML techniques in the wireless domain areidentified along with a roadmap to foster research efforts inbridging ML and wireless security.

Index Terms—Wireless security, machine learning, adversarialmachine learning, attack, defense.

I. INTRODUCTION

A. Motivation

The research area of radio frequency machine learning(RFML) has had extremely strong growth in recent years.RFML solutions have been proposed to solve many problemsin the areas of wireless communications, networking, andsignal processing such as cognitive radio, spectrum sensing,spectrum coexistence, jamming/anti-jamming, emitter identi-fication, and intrusion detection [1]. While machine learning(ML) is getting traction in wireless security applications suchas detecting conventional attacks, little work has been doneso far in investigating how vulnerable wireless systems areto ML-based security and privacy attack vectors that haverecently been considered in other modalities such as imagerecognition and natural language processing. In addition, com-mercial applications for the internet of things (IoT) and 5Gcommunications and recent interest by government entities indefense mechanisms to protect ML applications have shownthat security and privacy concerns of ML systems are ex-tremely timely and relevant.

B. Scope and Background

ML offers invaluable tools for a diverse and far-reachingset of applications ranging from image recognition and nat-ural language processing to cyber security and autonomousnavigation. In recent years, applications of ML have emergedin the wireless communications domain, forming a major

ingredient of a more general topic area, colloquially referred toas RFML. In particular, ML systems based upon state-of-the-art deep learning architectures, powered by the ever-increasinghardware accelerations for computing, have been consideredfor spectrum sensing applications (signal detection, estimation,classification, and identification), channel estimation, emitteridentification, cognitive jamming and anti-jamming, amongmany others. The success of these research thrusts are expectedto play an increasingly prevalent role in the development offuture generations of commercial wireless networks [2].

In the more established ML domains, recent research hasdemonstrated the efficacy of utilizing adversarial ML to neg-atively impact the performance of ML systems. Additionally,vulnerabilities to the privacy and security of these systems,and the data used to train the systems, have been exposed.However, the impact of these concepts to RFML technologiesis currently underdeveloped. Therefore, it is a timely researcheffort to investigate the interaction of RFML with wirelesssecurity, privacy, robustness, and resilience. Given these facts,it is imperative to combine research efforts in ML, RFML,privacy, security, and wireless communications to further ad-vance the state-of-the-art in security techniques, architectures,and algorithms for ML in wireless systems.

Research efforts that are needed to address ML for wirelesssecurity are diverse and include (but not limited to) thefollowing directions:

• Adversarial ML Techniques: evasion attacks (adversarialexamples), poisoning (causative) attacks, Trojan (back-door or trapdoor) attacks, generative adversarial learning.

• Privacy Issues of ML Solutions: membership inferenceattacks, model inversion, physical layer privacy.

• ML Hardening Solutions: privacy-preserving learning,secure learning, hardware and software implementations,edge computing, testbeds and experiments, datasets.

• Relevant ML Applications for Wireless Security: deviceidentification, spectrum monitoring, RF fingerprinting,smart jamming, smart eavesdropping, localization, covertcommunication, authentication, anonymity, intrusion de-tection, IoT security.

This article surveys the recent efforts to address ML for

arX

iv:2

001.

0888

3v1

[cs

.NI]

24

Jan

2020

wireless security in Section II, documents the challenges thatare still left to solve in Section III, and concludes withrecommendations for future research directions in Section IV.

II. RESEARCH EFFORTS

In this section, we highlight the research results presentedat the ACM Workshop on Wireless Security and MachineLearning (WiseML 2019) that was held in conjunction withthe 12th ACM Conference on Security and Privacy in Wirelessand Mobile Networks (ACM WiSec). The goal was to bringtogether members of the ML, RFML, privacy, security, andwireless communications communities in order for them toshare the latest research findings in the emerging and criticalarea of wireless security and ML.

The workshop consisted of a keynote, three sessions onML applications, two sessions on adversarial ML, and twosessions on defense with ML. The keynote [3] highlighted theapplications of ML in wireless security with examples rangingfrom conventional attacks formulated with ML to new types ofattacks enabled with ML. There were 13 paper presentationsand 8 invited talks in three main areas that are discussed inthe following three subsections.

A. Adversarial Machine Learning in Wireless Systems

Adversarial ML is an emerging field that studies learningin the presence of adversaries [4], [5] and has received majorattention in other data domains such as computer vision [6].

In this subsection, we highlight potential uses of adversarialML in wireless domain.

• A generative adversarial network (GAN) was used in [7]to generate spoofing wireless signals that cannot be dis-tinguished from real signals. By accounting for channel,waveform, and radio effects, an adversary was shownto train a deep learning based generator that transmitsspoofing signals over the air, whereas the defender’s goalwas to train another deep learning based discriminator todetect spoofing signals.

• A GAN was also applied in [8] to generate synthetic RFsignals that can be used to confuse spectrum users byjammers, spoofers, and other attackers in wireless radioenvironments. In particular, GAN models were fit to LongTerm Evolution (LTE) and Frequency Modulation (FM)broadcast signals to demonstrate the feasibility of GANsto generate realistic wireless signals.

• Defense mechanisms were developed in [9] to mitigatetargeted evasion attacks against deep learning-based RFsignal classifiers. Two different datasets were considered,one on modulation recognition and the other one on WiFi802.11n, Bluetooth and ZigBee classification.

• Another adversarial example was studied in [10] for tar-geted attacks against RF signal classifiers. Direct accessto the inputs of a deep learning classifier was shown tobreak down the classifier and the adversarial perturbationpower needed to cause source-target misclassificationcould be used as a proxy for the model’s estimation oftheir similarity.

• Adaptation of radio parameters such as power control wasconsidered in [11] against deep learning based predic-tive adversaries in wireless communications. Lyapunovoptimization and virtual queues were used to satisfydata transmission reliability in the presence of wirelessadversaries while minimizing power consumption.

• In addition to these papers, [12] surveyed research intothe threats that adversarial ML poses to deep learningenabled cognitive radios. The primary focus was on eva-sion attacks by providing a threat model that characterizesattacks by their prior knowledge, their goals, and thelocation the attack is launched from.

B. Machine Learning Applications in Wireless Security

In this subsection, we highlight various applications of MLtechniques in wireless security.

• Wireless virtualization (WiVi) was discussed in [13] forhigh speed and quality-of-service (QoS) aware commu-nications while reducing the deployment cost of wirelessinfrastructure. A Blockchain was used to eliminate over-committing of wireless resources of Wireless Infrastruc-ture Providers such as RF channels, while ML optimallypredicts the QoS requirements.

• A survey of attacks on ML based wireless systems waspresented in [14]. These attacks ranged from modelextraction, model inversion and leak of information toevasion and poisoning attacks that aim to manipulate theintegrity of test and training data, respectively.

• Based on siamese convolutional neural networks (CNNs),a noise-robust signal classification solution was presentedin [15]. Trained on compressed spectrogram images,siamese CNNs were shown to effectively distinguishwireless signals by accounting for frequency offsets.

• ML for RF signal processing was discussed in [16] interms of catching the so called “Third Wave of AI”,which refers to the paradigm shift in artificial intelligencefrom statistical learning and deep neural frameworks, tocontextually adaptive, explainable, and generalizable cog-nitive systems. Recent research programs in Academia,Industry, and Government have been key indicators ofthe growing interest in applying ML to wireless securitywhere the conceptual framework of the Third Wave is ofcritical importance.

• The focus of [17] was on how to launch wireless jammingattacks with the assistance of deep learning. The keypoint was in exploratory attacks that aim to infer thefunctionality of deep learning classifiers for spectrumsensing followed by subsequent evasion and poisoningattacks to misguide the target classifiers in test andtraining phases, respectively.

• The rate and level of performance degradation that oc-curs when CNNs are subjected to bit-flip errors, whichcould result from single event upsets that occur in harshenvironments, were discussed in [18]. The discussionprovided a foundation for ongoing research that enhancesthe overall resilience of neural network architectures

and implementations in space under both random andmalicious error events, offering significant improvementsover current implementations.

• Initiatives were presented in [19] to introduce studentsto research in RFML applications. Different types ofadversarial ML attacks were discussed to highlight threatsto cognitive radio systems that increasingly rely on MLto make data-driven automated decisions.

• Contextual combinatorial bandit learning was discussedin [20] for online decision making under uncertainty. Analgorithm was presented to account for issues raised byvolatile arms and submodular reward functions and asublinear regret bound is proven.

• The focus of [21] was ML-defined 5G New Radio Net-works. Reflecting recent advances in commercial systems,robustness and security were discussed in the applicationof ML to 5G systems.

C. Wireless Defense with Machine Learning

In this subsection, we highlight various cases of defendingwireless systems with ML techniques.

• Defense strategies were studied in [22] for mobile crowd-sensing (MCS) that is vulnerable to illegitimate taskinjection. ML was shown to eliminate illegitimate taskswith high accuracy and save energy of battery-orientedmobile systems.

• A software-defined radio (SDR) implementation waspresented in [23] to detect jammers using deep neuralnetworks. Wavelet transform was used as preprocessingto CNN and recurrent neural network (RNN) classifiersthat were shown to detect jammers with high accuracy.

• Deep learning was applied in [24] to detect adversarialand unintentional communication collisions in a sharedspectrum. CNN was used for classification purposes andtransfer learning provides the necessary means to scalethe training process.

• Intrusion detection in IoT systems was studied [25]in the context of anomaly detection. Cumulative sumcontrol chart (CUSUM) was used for timely and accurateanomaly detection followed by in-depth analysis of k-Nearest Neighbors (KNN) distances.

• ML was applied in [26] to detect accurately and quicklywhether a drone is flying or lying on the ground. Randomforest and neural network classifiers were run on featuresthat are based on packet size and inter-arrival time ofcommunications between the drone and its remote con-troller (RC).

• A quick and accurate ML-based detection and mitiga-tion solution against IoT-empowered cyber attacks wasdiscussed in [27]. The attack space covered volumetricDistributed Denial of Service (DDoS), synchronization-based DDoS, protocol-based DDoS, Byzantine attack,and false data injection attack.

TRUST IN RFML

Explainable AIWhy? & How?

Adversarial MLInference, Evasion,

Poisoning, Trojan, …

ML ConfidenceTraining vs. Testing

Trust in radio/HWFrom Code Design to

Fabrication

Wireless UncertaintiesChannel, Interference,

Traffic, …

Fig. 1: Trust in ML for RFML.

III. CHALLENGES AND GAPS

One fundamental issue regarding the use of ML in wirelesscommunications and security is whether it is possible to trustML for these applications. While there are many challengesregarding trust in RFML (with different components shown inFig. 1), that are universal (independent of data domain) suchas explainability and confidence, some challenges are specificto the wireless domain such as the characteristics of the radiohardware used and the inherent uncertainties associated withthe wireless medium.

A. Development Environment

One important challenge is how RFML should be deployed.A multi-level development environment will be beneficialwhile designing RFML solutions that rely on various wave-form, channel and radio hardware characteristics. Simulationtests mainly represent virtual hardware and channel effects thatmay not reflect the real-world scenarios. Simulations need tobe followed with emulation tests that include both real trafficand real radios. Emulation tests use actual radios making realtransmissions over emulated (virtual) channels [28]. Emulationtests can be repeated by controlling channel effects such aspath loss, fading, and delay [29], [30]. On the other hand,over-the-air (OTA) testbeds provide the opportunity to test thesystem with real channels along with real hardware typicallyin fixed settings.

Development environment for RFML should involve nec-essary training and test datasets, and also reflect potentialchanges between training and test conditions (e.g., indoorvs. outdoor channel effects) under which data is collected(see Section III-B for more detailed discussion of RFMLdatasets). In addition, embedded computing should be ex-plored to support edge applications and training data collectionfor RFML (see Section III-B for more detailed discussionof embedded implementation). Different aspects regarding theneed for multi-level development environment for RFML areillustrated in Fig. 2.

B. Training and Test Datasets

ML requires relevant data for both test and training pur-poses. In wireless security applications, the data needs toreflect the underlying channel, waveform, radio, interference,

Need: Multi-level Development of RFML

Simulations

• Virtual HW effects • Virtual channel effects

OTA Testbeds

• All real effects• Typically fixed setting • Not repeatable

Emulations• Real traffic • Real radios• Virtual channels

Edge Computing

• CPU/GPU clusters vs. embedded devices

• Speed vs Power vs Accuracy• Federated & decentralized

learning

Training Data

• Simulated: large but not realistic• Real captured: incomplete• Real + Augmentation: GAN or

domain specific effects such as phase, frequency or AWGN

Fig. 2: Development environment for RFML.

and adversary characteristics. While simulated data can becontrolled and configured to the needs of the ML problem athand, it typically lacks fidelity as it is difficult (if not impossi-ble) to model all channel and radio effects reliably. Therefore,real datasets captured over the air are useful resources for MLproblems in the wireless domain.

While there is a trend of releasing real datasets (e.g., see[31]), there are not enough datasets available yet to accom-modate various scenario needs, e.g., spectrum characteristicsin the presence of intelligent jammers. This paradigm raisesthe danger that research efforts are tailored to the availabilityof datasets, e.g., there is a plethora of research focused onmodulation recognition, where efforts pursuing other signalclassification tasks are lagging behind. Therefore, there is aneed to produce and publicly release real wireless datasets. Inparticular, it is of paramount importance to build a wirelesssecurity database similar to National Vulnerability Database[32] in the cyber domain.

GAN and other synthetic data generation approaches can befurther applied for data augmentation and domain adaptationto satisfy the large data needs of ML algorithms, in particulardeep learning. While GAN has been successfully applied togenerate synthetic data samples for computer vision [33] andtext analytics [34] that can be further used to strengthen attackson target systems [35], there is only limited work (e.g., [36])yet to generate synthetic wireless signal samples to supportdata augmentation and domain adaptation for wireless systems.

The typical practice is to apply pre-trained ML models towireless security. Online ML update mechanisms are neededto continue with the optimization of attack and defense spacein test time. To that end, reinforcement learning is promising tosecure wireless communications in the presence of adversariessuch as jammers and eavesdroppers [37]. ML security inwireless domain differs from its counterpart in other datadomains in terms of performance measures. ML can be used tooptimize the quality of communications measured by various

performance metrics such as throughput and delay subjectto power consumption requirements. These metrics are oftencoupled with each other and need to be jointly optimized [38].

Spectrum is shaped by various effects such as channel,inference, waveform, and traffic [39]. Due to the dynamic andrandom nature of traffic flows and links in a wireless network,it is imperative to maintain queue stability (i.e., queue lengthremains bounded) [40], [41] while optimizing performance viaML. One related performance metric that has gained interestand may benefit from ML is the age of information (AoI) thatmeasures the information freshness (the time elapsed since thelast received update) [42].

C. Repeatability, Hyperparameter Optimization, and Explain-ability

It is important not only to share the datasets but also theML code. This will help other researchers quickly extendthe proposed solutions to other wireless security problems orcompare their own solution to the proposed ones. A commonlibrary of algorithmic solutions to ML-based wireless attacksand defenses is ultimately needed. Deep learning has startedfinding more applications in wireless security as it can capturephysical signal characteristics better than conventional MLalgorithms such as support vector machines.

There are four major issues to be solved with deep learningbefore it can be practically used in wireless applications:

1) Deep learning requires large datasets to train. GAN isone way for data augmentation. Additional methods canbe borrowed from computer vision and other domains ornew ones can be developed to meet this need.

2) Deep learning has many parameters to optimize such asthe number of layers and neurons. Hyperparameter op-timization techniques are needed to match deep learningperformance to wireless security needs. These techniquesshould reduce the computational complexity of parametersearch. For example, Hyperband [43] iteratively selectsand evaluates the performance of hyperparameter con-figurations with increasing run times. Such techniquesreduce the time to select hyperparameters compared toexhaustive search that becomes easily intractable as theneural network structures become deeper.

3) Deep learning is typically used as a black box. However,wide adoption of wireless security solution requires jus-tification of the performance. For example, preprocessingof spectrum data is crucial to extract features that can bedone through statistical, temporal, or spectral methods,and deserves more attention from the research commu-nity. In addition, future work should account for thefact that wireless domain further increases discrepancybetween test and training data due to

• spectrum dynamics (channel, traffic, interference, andradio effects change over time),

• model changes (data labels (e.g., signal types) maychange over time),

• noisy data (features, e.g., I/Q samples, are hidden innoise, fading, and interference),

• distributed computing (training and inference may bedistributed at edge devices), and

• adversarial learning (wireless signals are vulnerableto over-the-air attacks based on adversarial ML).

4) In a wireless medium, it is possible that new signal types(such as jammers) may emerge over time. However, it iscostly to retrain the signal classifier, whereas accountingonly for new signal types may result in catastrophicforgetting. Continual learning such as using elastic weightconsolidation (EWC) based loss function can be usedto strike a balance between old and new tasks to learn.In particular, EWC slows down the learning process onselected ML model parameters to remember previouslylearned tasks. In wireless domain, EWC was applied tothe problem of wireless signal classification in [44] whenthe classifier encounters new signal types over time.

D. Embedded Implementation

The typical practice is offline computation using centralprocessing unit (CPU) or graphics processing unit (GPU)resources for ML tasks. However, embedded solutions areneeded for ML-driven wireless security to support onlinedecision making in size, weight, and power (SWaP) con-strained radio platforms. While computational resources suchas ARM processors, embedded GPUs and field-programmablegate arrays (FPGA) are available with a wide range of ca-pabilities, research efforts supported by embedded platformimplementations are largely missing. Therefore, it is hard toclaim whether the current state of the art can match the lowlatency and low power requirements of wireless security. Asexamples of embedded implementation of ML solution, ML-based signal authentication was implemented on embeddedGPU in [45] and ML-based modulation classification wasimplemented on Android smartphone in [46].

An alternative implementation is on FPGA. Initial studiesshowed that FPGA provides major improvements in latencyand energy consumption for ML-based RF signal classificationcompared to embedded GPU [47], [48]. However, it is knownthat it is difficult to program FPGAs compared to CPUs andGPUs. Therefore, automated means are needed to convert soft-ware codes of ML solutions (especially deep neural networks)to FPGA implementation.

Table I qualitatively compares expected latency, powerconsumption and programmability of different embedded com-puting resources. To this end, more efforts should be put tobridge the gap between theory and practice by implementingthe proposed solutions on embedded platforms and quantifyinglatency and energy consumption to run ML solutions.

E. Adversarial Machine Learning

There is a growing interest in applying adversarial ML to thewireless domain. The idea is to attack the training and/or test-ing processes of ML developed for various application tasks.When applied to wireless domain, such attacks operate withsmall footprints and are stealthier and more energy efficientcompared to conventional attacks that directly target wireless

TABLE I: Performance of different of embedded computationplatforms.

Measure \ Platform ARM Embedded GPU FPGALatency High Medium Low

Power Consumption Medium High LowProgrammability High Medium Low

transmissions such as jamming [49], [50], while accounting foruncertainties regarding wireless communications [51]–[53].

Different types of adversarial ML are illustrated in Figure 3.Below, we discuss evasion attacks (adversarial examples), ex-ploratory (inference), poisoning (causative) attacks, and Trojanattacks in the context of wireless security:

• Evasion attack: In an evasion attack, the adversary ma-nipulates the test data to fool a receiver into making errorsin classification decisions. Most of the studies in wirelessdomain have focused on evasion attacks against modu-lation classifiers [54]–[58]. Various defense mechanismsare proposed against these attacks [59], [60]. There arealso efforts to apply evasion attacks to other ML-basedwireless applications such as spectrum sensing [61]–[63]and end-to-end communications with an autoencoder [64](note that instead of using conventional communicationblocks, an autoencoder is trained with two deep neuralnetworks, one as an encoder at the transmitter and theother one as a decoder at the receiver [65]).

• Exploratory attack: Exploratory attack aims to infer howan ML algorithm of a victim system works. Typically, thisinvolves building a surrogate model that is functionallyequivalent to the victim ML system with the same typesof input and outputs [66]. In wireless domain, exploratoryattack was studied in [67], [68] to learn transmit patternsof a communications system and build more efficientjamming strategies based on the inferred transmit pattern.Exploratory attack is typically the first step before launch-ing subsequent attacks and benefits from methods likeactive learning [69] that can help reduce the number ofdata samples needed to reliably infer the inner workingsof an ML algorithm.

• Poisoning attack: Poisoning attack manipulates the train-ing process of an ML algorithm such that the MLalgorithm does not produce outcomes as intended [70].Wireless applications include spectrum data falsificationto fool classifiers that are built for spectrum sensing[62], [63] and cooperative spectrum sensing [71]. Theseattacks correspond to the ML-generalization of spectrumsensing data falsification (SSDF) attacks that have beenextensively studies in the context of cognitive radiosecurity [72].

• Trojan attack: Trojan attack manipulates training databy inserting Trojans (i.e., triggers) to only few trainingdata samples in the training phase and then activatesthose Trojans in the test phase to fool the ML modelinto making wrong decisions [73]. To selectively fool awireless signal classifier, a Trojan attack was proposed in

Deep Learning

Algorithm

Test Data Output

Labels

Adversary

Evasion Attack

Poisoning

(Causative)

Attack

Exploratory AttackData

InferenceML Model

Learned Model

QPSK16QAM

QPSK 16QAM

QPSK

Trojan

Attack

Training Data

Fig. 3: Taxonomy of adversarial ML: evasion, poisoning(causative), exploratory (inference), Trojan attacks.

[74], where the adversary modifies phases of few samplesand changes corresponding labels in training data, andlater transmits signals with the same phase shift that wasadded as a trigger during training.

Adversarial attacks have been also considered in reinforcementlearning algorithms with various applications in computervision [75]. Studies of adversarial ML in wireless domain arestill in their early stages and more work is needed to fullycharacterize the attack and defense spaces by accounting forunique features of wireless applications such as differences infeatures and labels observed by adversaries and defenders dueto different channel and interference effects.

IV. CONCLUSION AND RECOMMENDATIONS

Based on challenges and gaps identified in the previoussection, the following conclusions are made:

• While ML is critical to secure wireless systems, theattack vectors have not yet been fully understood. Moreefforts are needed to characterize the vulnerabilities ofwireless systems against adversaries that are employingML techniques. A clearly-defined ML-based attack modelis the precondition to formal analysis for ML-relatedwireless security.

• As adversaries are getting smarter, defense schemesshould become more agile and adaptive; ML providesthese capabilities and is thus a powerful means to detectand mitigate wireless attacks.

• Adversarial ML is important to understand, disrupt orprotect the ML process in the presence of wirelessadversaries. The wireless medium provides new means

for the adversaries to manipulate both test (inference)and training phases of ML. To that end, novel techniquesare required to determine the wireless attack surface ofadversarial ML.

• The duel between both the attacker and the defender usingML is an evolving, interactive process. Efforts are neededto understand the dynamic evolution of this process withquantification of performance metrics obtained by theattacker and the defender.

• More high-fidelity and diverse datasets that are publiclyavailable are needed to support ML efforts for wirelesssecurity. Embedded implementation is crucial to meet la-tency, power and computational complexity requirementsof ML-based attacks and defenses in SWaP-constrainedenvironments.

REFERENCES

[1] T. Erpek, T. O’Shea, Y. E. Sagduyu, Y. Shi, and T. C. Clancy, “DeepLearning for Wireless Communications,” Development and Analysis ofDeep Learning Architectures, Springer, 2019.

[2] K. B. Letaief, W. Chen, Y. Shi, J. Zhang, Y.A. Zhang, “The Roadmapto 6G: AI Empowered Wireless Networks.” IEEE CommunicationsMagazine, Aug. 2019.

[3] K. P. Subbalakshmi, “AI/ML and Wireless Security,” Keynote at ACMConference on Security and Privacy in Wireless and Mobile Net-works (WiSec) Workshop on Wireless Security and Machine Learning(WiseML), 2019.

[4] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial Machine Learn-ing at Scale,” arXiv preprint arXiv:1611.01236, 2016.

[5] Y. Vorobeychik and M. Kantarcioglu, “Adversarial Machine Learning,”Synthesis Lectures on Artificial Intelligence and Machine Learning, Aug.2018.

[6] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, and I.Goodfellow, and R. Fergus, “Intriguing Properties of Neural Networks,”arXiv preprint arXiv:1312.6199, 2013.

[7] Y. Shi, K. Davaslioglu, and Y. E. Sagduyu, “Generative AdversarialNetwork for Wireless Signal Spoofing,” ACM Conference on Securityand Privacy in Wireless and Mobile Networks (WiSec) Workshop onWireless Security and Machine Learning (WiseML), 2019.

[8] T. Roy, T. O’Shea, and N. West, “Generative Adversarial Radio Spec-trum Networks,” ACM Conference on Security and Privacy in Wirelessand Mobile Networks (WiSec) Workshop on Wireless Security andMachine Learning (WiseML), 2019.

[9] S. Kokalj-Filipovic, R. Miller, and J. Morman, “Targeted AdversarialExamples against RF Deep Classifiers,” ACM Conference on Securityand Privacy in Wireless and Mobile Networks (WiSec) Workshop onWireless Security and Machine Learning (WiseML), 2019.

[10] S. Bair, M. Delvecchio, B. Flowers, A. J. Michaels, and W. C. Headley,“On the Limitations of Targeted Adversarial Evasion Attacks AgainstDeep Learning Enabled Modulation Recognition,” ACM Conferenceon Security and Privacy in Wireless and Mobile Networks (WiSec)Workshop on Wireless Security and Machine Learning (WiseML), 2019.

[11] E. Ciftcioglu and M. Ricos, “Efficient Power Adaptation against DeepLearning Based Predictive Adversaries,” ACM Conference on Securityand Privacy in Wireless and Mobile Networks (WiSec) Workshop onWireless Security and Machine Learning (WiseML), 2019.

[12] B. Flowers, “Adversarial RFML: Threats to Deep Learning EnabledCognitive Radio,” Invited Talk at ACM Conference on Security andPrivacy in Wireless and Mobile Networks (WiSec) Workshop on WirelessSecurity and Machine Learning (WiseML), 2019.

[13] A. Adhikari, D. B. Rawat, and M. Song, “Wireless Network Virtual-ization by Leveraging Blockchain Technology and Machine Learning,”ACM Conference on Security and Privacy in Wireless and MobileNetworks (WiSec) Workshop on Wireless Security and Machine Learning(WiseML), 2019.

[14] L. Pajola, L. Pasa, and M. Conti, “Threat is in the Air: Machine Learningfor Wireless Network Applications,” ACM Conference on Security andPrivacy in Wireless and Mobile Networks (WiSec) Workshop on WirelessSecurity and Machine Learning (WiseML), 2019.

[15] Z. Langford, L. Eisenbeiser, and M. Vondal, “Robust Signal Classi-fication Using Siamese Networks,” ACM Conference on Security andPrivacy in Wireless and Mobile Networks (WiSec) Workshop on WirelessSecurity and Machine Learning (WiseML), 2019.

[16] G. Stantchev, “Machine Learning for RF Signal Processing: Catching theThird Wave,” Invited Talk at ACM Conference on Security and Privacyin Wireless and Mobile Networks (WiSec) Workshop on Wireless Securityand Machine Learning (WiseML), 2019.

[17] T. Erpek, “Deep Learning for Wireless Jamming Attacks,” Invited Talkat ACM Conference on Security and Privacy in Wireless and MobileNetworks (WiSec) Workshop on Wireless Security and Machine Learning(WiseML), 2019.

[18] A. Michaels, “Testing the Resilience of CNN Implementations,” InvitedTalk at ACM Conference on Security and Privacy in Wireless and MobileNetworks (WiSec) Workshop on Wireless Security and Machine Learning(WiseML), 2019.

[19] W. C. Headley, “Introducing Students to Research in Radio FrequencyMachine Learning Applications,” Invited Talk at ACM Conferenceon Security and Privacy in Wireless and Mobile Networks (WiSec)Workshop on Wireless Security and Machine Learning (WiseML), 2019.

[20] J. Xu, “Contextual Combinatorial Bandit Learning for Online DecisionMaking Under Uncertainty,” Invited Talk at ACM Conference on Securityand Privacy in Wireless and Mobile Networks (WiSec) Workshop onWireless Security and Machine Learning (WiseML), 2019.

[21] M. Yao, “Artificial Intelligence Defined 5G New Radio Networks:Perspective of Industry,” Invited Talk at ACM Conference on Securityand Privacy in Wireless and Mobile Networks (WiSec) Workshop onWireless Security and Machine Learning (WiseML), 2019.

[22] Y. Zhang, M. Simsek, and Burak Kantarci, “Machine Learning-basedPrevention of Battery-oriented Illegitimate Task Injection in MobileCrowdsensing,” ACM Conference on Security and Privacy in Wirelessand Mobile Networks (WiSec) Workshop on Wireless Security andMachine Learning (WiseML), 2019.

[23] S. Gecgel, C. Goztepe, and G. Kurt, “Jammer Detection based onArtificial Neural Networks: A Measurement Study,” ACM Conferenceon Security and Privacy in Wireless and Mobile Networks (WiSec)Workshop on Wireless Security and Machine Learning (WiseML), 2019.

[24] H. N. Nguyen, T. Vo-Huu, T. Vo-Huu, and G. Noubir, “Towards Adver-sarial and Unintentional Collisions Detection Using Deep Learning,”ACM Conference on Security and Privacy in Wireless and MobileNetworks (WiSec) Workshop on Wireless Security and Machine Learning(WiseML), 2019.

[25] K. Doshi, M. Mozaffari, and Y. Yilmaz, “RAPID: Real-time Anomaly-based Preventive Intrusion Detection,” ACM Conference on Security andPrivacy in Wireless and Mobile Networks (WiSec) Workshop on WirelessSecurity and Machine Learning (WiseML), 2019.

[26] S. Sciancalepore, O. A.Ibrahim, G. Oligeri, and R. Di Pietro, “DetectingDrones Status via Encrypted Traffic Analysis,” ACM Conference on Se-curity and Privacy in Wireless and Mobile Networks (WiSec) Workshopon Wireless Security and Machine Learning (WiseML), 2019.

[27] Y. Yilmaz, “Quick and Accurate Detection and Mitigation of IoT-empowered Cyberattacks,” ACM Conference on Security and Privacy inWireless and Mobile Networks (WiSec) Workshop on Wireless Securityand Machine Learning (WiseML), 2019.

[28] J. Yackoski, B. Azimi-Sadjadi, A. Namazi, J. H. Li, Y. E. Sagduyu,and R. Levy, “RF-NEST: Radio Frequency Network Emulator SimulatorTool,” IEEE Military Communications Conference (MILCOM), 2011.

[29] K. J. Kwak, Y. E. Sagduyu, J. Yackoski, B. Azimi-Sadjadi, A. Namazi,J. Deng, and J. Li, “Airborne Network Evaluation: Challenges and HighFidelity Emulation Solution,” IEEE Communications Magazine, Oct.2014.

[30] S. Soltani, Y. E. Sagduyu, Y. Shi, J. Li, J. Feldman, and J. Matyjas, “Dis-tributed Cognitive Radio Network Architecture, SDR Implementationand Emulation Testbed,” IEEE Military Communications Conference(MILCOM), 2015.

[31] “Datasets & Competitions,” IEEE Comsoc Machine Learning ForCommunications Emerging Technologies Initiative. [Online]. Available:https://mlc.committees.comsoc.org/datasets.

[32] “National Vulnerability Database,” NIST. [Online]. Available:https://nvd.nist.gov/vuln.

[33] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,”Advances in Neural Information Processing Systems, 2014.

[34] Y. Shi, Y. E. Sagduyu, K. Davaslioglu, and J. Li, “Generative AdversarialNetworks for Black-Box API Attacks with Limited Training Data,”IEEE International Symposium on Signal Processing and InformationTechnology (ISSPIT), 2018.

[35] Y. Shi, Y. E. Sagduyu, K. Davaslioglu, and R. Levy, “VulnerabilityDetection and Analysis in Adversarial Deep Learning,” Guide to Vul-nerability Analysis for Computer Networks and Systems, Springer, 2018.

[36] K. Davaslioglu and Y. E. Sagduyu, “Generative Adversarial Learning forSpectrum Sensing,” IEEE International Conference on Communications(ICC), 2018.

[37] N. Abu Zainab, T. Erpek, K. Davaslioglu, Y. E. Sagduyu, Y. Shi, S.Mackey, M. Patel, F. Panettieri, M. Qureshi, V. Isler, A. Yener, “QoSand Jamming-Aware Wireless Networking Using Deep ReinforcementLearning, IEEE Military Communications Conference (MILCOM), 2019.

[38] E. Ciftcioglu, Y. E. Sagduyu, R. Berry, and A. Yener, “Cost-Delay Trade-offs for Two-Way Relay Networks,” IEEE Transactions on WirelessCommunications, Dec. 2011.

[39] S. Wang, Y. E. Sagduyu, J. Zhang, and J. H. Li, “Spectrum Shapingvia Network coding in Cognitive Radio Networks,” IEEE INFOCOM,2011.

[40] L. Tassiulas and A. Ephremides, “Stability Properties of ConstrainedQueueing Systems and Scheduling Policies for Maximum Throughputin Multihop Radio Networks,” IEEE Transactions on Automatic Control,Dec. 1992.

[41] Y. E. Sagduyu and A. Ephremides, “On Broadcast Stability Regionin Random Access through Network Coding,” Allerton Conference onCommunication, Control, and Computing, 2006.

[42] S. Kaul, R. Yates, and M. Gruteser, “Real-time Status: How OftenShould One Update?” IEEE INFOCOM, 2012.

[43] L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar,“Hyperband: A Novel Bandit-based Approach to Hyperparameter Opti-mization, Journal of Machine Learning Research, 2018.

[44] Y. Shi, K. Davaslioglu, Y. E. Sagduyu, W. C. Headley, M. Fowler, andG. Green, “Deep Learning for Signal Classification in Unknown andDynamic Spectrum Environments,” IEEE International Symposium onDynamic Spectrum Access Networks (DySPAN), 2019.

[45] K. Davaslioglu, S. Soltani, T. Erpek, and Y. E. Sagduyu, “DeepWiFi:Cognitive WiFi with Deep Learning,” IEEE Transactions on MobileComputing, 2019.

[46] N. Soltani, K. Sankhe, S. Ioannidis, D. Jaisinghani, and K. Chowdhury,“Spectrum Awareness at the Edge: Modulation Classification UsingSmartphones” IEEE International Symposium on Dynamic SpectrumAccess Networks (DySPAN) 2019.

[47] S. Soltani, Y. E. Sagduyu, R. Hasan, K, Davaslioglu, H. Deng, andT. Erpek, “Real-Time and Embedded Deep Learning on FPGA forRF Signal Classification,” IEEE Military Communications Conference(MILCOM), 2019.

[48] S. Soltani, Y. E. Sagduyu, R. Hasan, K, Davaslioglu, H. Deng, andT. Erpek, “Real-Time Experimentation of Deep Learning-based RFSignal Classifier on FPGA,” IEEE International Symposium on DynamicSpectrum Access Networks (DySPAN), 2019.

[49] W. Xu, W. Trappe, Y. Zhang, and T. Wood, “The Feasibility ofLaunching and Detecting Jamming Attacks in Wireless Networks,” ACMInternational Symposium on Mobile Ad Hoc Networking and Computing(MobiHoc), 2005.

[50] Y. E. Sagduyu and A. Ephremides, “A Game-Theoretic Analysis of De-nial of Service Attacks in Wireless Random Access,” Wireless Networks,July 2009.

[51] Y. E. Sagduyu, R. Berry, and A. Ephremides, “MAC Games forDistributed Wireless Network Security with Incomplete Information ofSelfish and Malicious User Types,” IEEE International Conference onGame Theory for Networks (GameNets), 2009.

[52] Y. E. Sagduyu, R. Berry and A. Ephremides, “Wireless Jamming Attacksunder Dynamic Traffic Uncertainty,” IEEE International Symposium onModeling and Optimization in Mobile, Ad Hoc, and Wireless Networks(WIOPT), 2010.

[53] Y. E. Sagduyu, R. Berry, and A. Ephremides, “Jamming Games inWireless Networks with Incomplete Information,” IEEE Communica-tions Magazine, Aug. 2011.

[54] M. Sadeghi and E. G. Larsson, “Adversarial Attacks on Deep-learningBased Radio Signal Classification,” IEEE Wireless CommunicationsLetters, Feb. 2019.

[55] B. Flowers, R. M. Buehrer, and W. C. Headley, “Evaluating AdversarialEvasion Attacks in the Context of Wireless Communications,” arXivpreprint, arXiv:1903.01563, 2019.

[56] M. Z. Hameed, A. Gyorgy, and D. Gunduz, “Communication withoutInterception: Defense Against Deep-learning-based Modulation Detec-tion,” arXiv preprint, arXiv:1902.10674, 2019.

[57] B. Flowers, R. M. Buehrer, and W. C. Headley, “CommunicationsAware Adversarial Residual Networks,” IEEE Military CommunicationsConference (MILCOM), 2019.

[58] B. Kim, Y. E. Sagduyu, K. Davaslioglu, T. Erpek, and S. Ulukus,“Over-the-air adversarial attacks on deep learning based modulationclassifier over wireless channels,” Conference on Information Sciencesand Systems (CISS), 2020.

[59] S. Kokalj-Filipovic and R. Miller, “Adversarial Examples in RF DeepLearning: Detection of the Attack and its Physical Robustness,” arXivpreprint, arXiv:1902.06044, 2019.

[60] S. Kokalj-Filipovic, R. Miller, N. Chang, and C. L. Lau, “Mitigationof Adversarial Examples in RF Deep Classifiers Utilizing AutoencoderPre-training,” arXiv preprint arXiv:1902.08034, 2019.

[61] Y. Shi, T. Erpek, Y. E Sagduyu, and J. Li, “Spectrum Data Poison-ing with Adversarial Deep Learning,” IEEE Military CommunicationsConference (MILCOM), 2018.

[62] Y. E. Sagduyu, Y. Shi, and T. Erpek, ”IoT Network Security from thePerspective of Adversarial Deep Learning,” IEEE International Confer-ence on Sensing, Communication and Networking (SECON) Workshopon Machine Learning for Communication and Networking in IoT, 2019.

[63] Y. E. Sagduyu, Y. Shi, and T. Erpek, “Adversarial Deep Learningfor Over-the-Air Spectrum Poisoning Attacks,” IEEE Transactions onMobile Computing, 2019.

[64] M. Sadeghi and E. G. Larsson, “Physical Adversarial Attacks AgainstEnd-to-end Autoencoder Communication Systems,” IEEE Communica-tions Letters, Feb. 2019. 847-850.

[65] T. J. OShea and J. Hoydis, “An Introduction to Deep Learning for thePhysical Layer,” IEEE Transactions on Cognitive Communications andNetworking, Dec. 2017.

[66] Y. Shi, Y. E. Sagduyu, and A. Grushin, “How to Steal a Machine Learn-ing Classifier with Deep Learning,” IEEE Symposium on Technologiesfor Homeland Security (HST), 2017.

[67] Y. Shi, Y. E Sagduyu, T. Erpek, K. Davaslioglu, Z. Lu, and J. Li, “Adver-sarial Deep Learning for Cognitive Radio Security: Jamming Attack andDefense Strategies,” IEEE International Conference on Communications(ICC) Workshop on Promises and Challenges of Machine Learning inCommunication Networks, 2018.

[68] T. Erpek, Y. E. Sagduyu, and Y. Shi, “Deep Learning for Launching andMitigating Wireless Jamming Attacks,” IEEE Transactions on CognitiveCommunications and Networking, 2019.

[69] Y. Shi, Y. E. Sagduyu, K. Davaslioglu, and J. H Li, “Active DeepLearning Attacks under Strict Rate Limitations for Online API Calls,”IEEE Symposium on Technologies for Homeland Security (HST), 2018

[70] Y. Shi and Y. E Sagduyu, “Evasion and Causative Attacks with Ad-versarial Deep Learning,” IEEE Military Communications Conference(MILCOM), 2017.

[71] Z. Luo, S. Zhao, Z. Lu, J. Xu, and Y. E. Sagduyu, “When Attackers MeetAI: Learning-empowered Attacks in Cooperative Spectrum Sensing,”arXiv preprint arXiv:1905.01430.

[72] Y. E. Sagduyu, “Securing Cognitive Radio Networks with DynamicTrust against Spectrum Sensing Data Falsification,” IEEE MilitaryCommunications Conference (MILCOM), 2014.

[73] Y. Liu, S. Ma, Y. Aafer, W.-C. Lee, J. Zhai, W. Wang, and X. Zhang,“Trojaning Attack on Neural Networks, Network and Distributed SystemSecurity Symposium, 2018.

[74] K. Davaslioglu and Y. E. Sagduyu, “Trojan Attacks on Wireless SignalClassification with Adversarial Machine Learning,” IEEE Workshop onData-Driven Dynamic Spectrum Sharing of IEEE DySPAN, 2019.

[75] S. Huang, N. Papernot, I. Goodfellow, Y. Duan, and P. Abbeel,“Adversarial Attacks on Neural Network Policies,” arXiv preprintarXiv:1702.02284, 2017.