Applying neural network basedalgorithms in communication

technologyTheses excerpt

Daacutevid Tisza

Scientific advisorsDr Jaacutenos Levendovszky

dr Andraacutes Olaacuteh

Paacutezmaacuteny Peacuteter Catholic UniversityFaculty of Information Technology and BionicsRoska Tamaacutes Doctoral School of Sciences and

Technology

Budapest 2018

1 Technological motivation

Due to recent and vast improvement in sensor technology and to the yet unfailingtrend set by Moorersquos law multitude of novel fields opened up for new applicationsThe platform carrying these novel applications are at the same time required to supportmobility and flexibility at the end user side As the applications become evermorecomplex their supporting systems have to deal with more demands These (suchas their own communication networking subsystem central processing units or thedevices in the background network which they communicate with) have to act moreintelligently and adapt to the new demands arriving from the upper layers Also themajority of the end user devices are portable and use battery as a power source itbecomes increasingly important to take this into consideration at the widest range ofsystem design possible

Typically into these areas we could count in the IoT based applications monitoringand intervening systems that are based on WSNs [2] peer-to-peer and ldquobroadcasttype relaying and processing systems dealing with multimedia streams (let that bevideo or audio) or most of the cloud based services [40] These services and systemshave the common aspect of providing a certain QoS while their resources are timeand location dependent and also limited [45] On inherent shared resources likeon the radio subsystem these constraints appear even more stringent Furthermorescheduling tasks efficiently in these distributed environments are imperative

Networking technologies used today are following a layered structure [16] and mostof them form packet switched communication networks In these networks typicallythere are no separate resource dedicated for signaling and controlling but withoutthese processes providing a required QoS can be mostly done in a best effort mannerif possible at all Requiring these crucial processes to be present means that they haveto isolate an additional portion from the resources bearing the payload to themselvesThus the total capacity from the end user perspective further diminishes in contrastto their signaling-less counterparts The best effort like structures which build upthe traditional packet switched networks do not or rarely use signaling to governtheir internal mechanisms in respect to QoS These structures consist of protocolsdefined by OSI in several layers Examples for these in the lower OSI layers (physicaldata-link network) are the Ethernet [1] the 80211 [15] or the IP Although theirdesign does not incorporate providing QoS directly [32 20] they are widespreadbecause of their reliability and usability [20] One of the base questions of the packetswitched networks is to find an appropriate path and scheduling for the data packetsto traverse through the network from the source to the destination [39 20 28 98] Applications using wireless technologies in the physical layer face even moreconstraints due to the shared nature of the radio media which has to be accounted forif one requires to provide QoS

2

In the light of this the open questions that this dissertation aims to answer are

bull How can one find an appropriate path in a packet switched network whichprovides a QoS (QoS unicast multicast routing)

bull How can one perform scheduling tasks in communication networks efficientlywith algorithms which lend themselves to parallelization (scheduling)

bull How can one solve at the physical layer which use wireless technologies nearoptimally the MUD problem in a parallelized fashion (Multiuser Detection)

11 Applied methodologyThis dissertation uses the following general approach to investigate the problems

and bring up possible solutions as theses

Problemsbull QoS routingbull MUDbull scheduling

mathematical modelling bybull Graph theory methodsbull Queuing theorybull Quadratic Programmingbull Nonparametric Bayesian decision methods

methods used for the solutionsbull Information theoretic metricsbull Large deviation theorybull Hopfield networksbull Feed forward nonlinear networks

performance analysisbull implementing reference algorithmsbull implementing the proposed new algorithmsbull numerical performance evaluation and comparison

evaluating and ranking the results

The summary of the problems the used algorithms theories and used tools can befound at Table 1

technologicalchallenges

modelformulation andproblemcategory

theoretical performance used methods andalgorithms

relatedtheses

Unicastrouting

using randomlink descriptorsand LASreducing theproblem toadditive andbottleneck typemetrics

The path of choice is capable ofproviding a QoS

Queuing models Markov modulatedPoissondistributionsGaussianapproximationLarge deviationtheory

Thesis I1Thesis I2

OptimizingLAS

Applyinginformationtheoretic metrics(Link Entropyand SignalingEntropy) asconstrains inoptimization

Using the resulting LAS thebandwidth of the signaling process inthe network can be bound and keptunder a predefined value while at thesame time the uncertainty of the linkstates in the network is also keptunder a well defined value

Informationtheoretic measuresexhaustive searchgeneral nonlinearconstrainedoptimizationmethods

Thesis I4

- continuing on the next page -

3

Table 1 ndash continued from previous page ndash

technologicalchallenges

modelformulation andproblemcategory

theoretical performance used methods andalgorithms

relatedtheses

Multicastrouting

using randomlink descriptorssearch for aCGSMT in thenetwork andreformulating thesearch of theCGSMT as anUBQP

Sending streams through the pathsfrom the source and destinationpoints provide the prescribed QoSwhile they perform optimally in thesense that they strain the chosenresource minimally Theseconstraints posed on the resourcesare like minimal energy consumptionor minimal total used bandwidth todeliver the payload

Gaussianapproximationlarge deviationtheory Hopfieldnetwork binaryquadraticprogrammingmethods

Thesis I3

MultiuserDetection(MUD)

UBQP andnon-parametricBayesiandetection

The exact solution of the UBQP isthe optimal solution of the problemThe proposed UBQP ldquosolvers lendthemselves to parallelization wellsince they are made up from simplenonlinear computing units Theygive well performing sub-optimalsolutions with fast convergence timeIn case of the non-parametric modelsthe proposed algorithm canarbitrarily approximate the optimaldecision at the expense of increasingits complexity

Algorithmsoperating on ahypergraph baseddimensionreduction anddimension additionHopfield networkbinary quadraticprogrammingmethods FeedForward NeuralNetwork (FFNN)logarithmic search

Thesis II1Thesis III1

Scheduling UBQP Exact solution to the UBQP is theoptimal solution to the problem Theproposed ldquosolvers give sub-optimalsolutions at fast convergence speeds

Hopfield networksbinary quadraticprogrammingmethods

Thesis II1

Table 1 Problems investigated used algorithmic tools and the related theses

4

2 New scientific results and theses of the dissertation

This dissertation presents the problems and the theses in three groups in threesections

I) In the first group I elaborate on the route searching problem providing QoSin packet switched networks both for unicast and multicast cases and proposenovel methods to solve or approximate them In this section I also investigatethe link scaling problem and propose a method for the optimization usinginformation theoretical measuresThe theses in this group are Thesis I1 Thesis I2 Thesis I3 and Thesis I4

II) In the second group I elaborate on the Multiuser Detection (MUD) and schedul-ing problems in communication technologies I formulate these tasks as BinaryQuadratic Programming (BQP) and I propose a family of algorithms which actas a suboptimal solver to the BQP problem These algorithms lend themselvesto parallelization well because of their inherent structureThesis II1 of this group can be found on page 23

III) In the third group I propose a FFNN based algorithm which performs compa-rably to the non-parametric optimal Bayesian decision for the MUD problemdefined for the wireless networks PHY layerThesis III1 of this group can be found on page 29

Thesis group I - routing with incomplete information in unicastand multicast scenarios

In this thesis group I propose novel solutions to the problem of unicast and multicastQoS routing with incomplete information These new methods are capable of findinga sub-optimal path and a multicast route set in polynomial time Also in this thesisgroup I provide a solution to the OLS task by using information theoretical measuressuch as ldquosignaling entropyrdquo and ldquolink entropyrdquo The corresponding publication ofthe author is titled ldquoMulticast Routing in Wireless Sensor Networks with IncompleteInformationrdquo [48]

In networks where is no dedicated channel for signaling or controlling purposes(typical in IoT networks using IP) these procedures consume additional resourcesAs a result it diminishes the capacity available for information transfer however atthe same time services heavily demand the speed and reliability of the underlyingcommunication stack This gives rise to the problem of finding QoS fulfilling pathsFor example in the IoT vision every device can send and receive information and mightact as an intermediate node From an angle these devices can be seen forming a WSNIf the network contains battery operated devices then the applications also requirereliable communication while keeping energy consumption at a minimal level (eg

5

consider a smart home application where the user should not be forced to change thebatteries frequently or a smart agricultural application where battery change might notbe feasible at all) On the other hand in an application where the energy consumptionmight not be a problem (eg a smart fridge automotive application or a factorywith smart production appliances) other types of reliability criteria exist that the(sub)networks must meet Among others these can be robustness to communicationshortage redundancy efficient use of the communication bandwidth responsivenessetc One can also consider peer-to-peer applications eg video on demand servicesor voice over IP services where large quantity of data needs to be reliably transportedto the peers In these scenarios both unicast and multicast type communication iscommon Data is to be transmitted to a single or a set of destination nodes with thepackets routed in a multi-hop manner where intermediate nodes are also used forpacket forwarding

Legacy network structures which were not designed for QoS but mostly for besteffort usually have no dedicated channel for signaling Examples of these protocolsin the lower OSI layers (physical data-link network) are the Ethernet 80211 orIP Despite they can not provide QoS natively by their design[32 20] they arewidespread because of their reliability and interoperability[20] Nevertheless thereis a need to run QoS communication over these best effort platforms One of themajor challenges in IP networking is to ensure QoS routing which selects paths tofulfill end-to-end delay or bandwidth requirements [39 20 28 9 8] as opposed totraditional shortest path routing Because of the manifold optimization criteria[17]even in the unicast case QoS routing can prove to be much harder than the problemof finding optimal paths based on merely the hop count [17] Protocols striving toprovide QoS need to have some knowledge about the state of the network Thisinformation has to be propagated also (signaling) QoS-aware routing protocolsoften propose a method to reduce the amount of state information that have to bekept synchronized among routers called topology aggregation (eg in QOSPFPNNI) Thus routing information describing a certain domain of the network have tobe aggregated which acts as another source of uncertainty of resource availabilityinformation Another source of uncertainty is introduced when multiple hierarchicallevels are connected Such a pattern is inevitable from the base rules of the networkdesign These hierarchical levels are introduced because these networks connectsthrough inter-domains

Incorporating QoS into routing has been long studied[20] For example an extensionto the classical IP over ATM system to support application level QoS is studied in [32]Also QoS aware routing algorithms exist in many levels such as in inter-domain level[19] in MPLS based network parts [31] inside ASs [39] but routing is done mostly bya hierarchical manner In networks following the OSI model the routing is done in the3rd (network) layer The OSPF routing protocol offers two while PNNI offers many

6

levels where routing can be performed in a hierarchical manner [13] Subnetworkson a given level of the hierarchy are abstracted as ldquonodesrdquo for a higher layer and delayinformation in those subnetworks are aggregated into an average QoS parameter Onthe other hand randomly fluctuating traffic load on links can also result in randomdelays Thus link delays are periodically advertised when the delay surpasses a giventhreshold (eg in PNNI and QOSPF standards see [3 13]) These thresholds aredefined in advance This prompts us to take delays into account as random variablescharacterized by their probability distribution functions over the interval between tworeported thresholds [14 43 27] The distribution of these thresholds (and thereforethe length of the intervals over which the link delay is not fully characterized) can beequidistant or non-equidistant In practice non-equidistant thresholds are used sincein this case the impact on network utilization by sending only signaling information(part of the available bandwidth is used for carrying information about link delays) isminimized

The phenomena described above give rise to the task of routing with incompleteinformation Namely how to select paths to fulfill end-to-end delay requirements inthe lack of the exact values of link delays [14 9 8] Routing is then perceived asan optimization problem to search over different quality paths where the quality ofa path is determined by the probability of meeting the end-to-end QoS requirement[26 14] Unfortunately routing with incomplete information in general cannot bereduced to the well-known Shortest Path Routing (SPR) thus it cannot be solved inpolynomial time in general

The multicast scenario can be seen as an extension of the previous problem andcan be treated as the well-known GSMT problem which has proved to be NP-hardeven for deterministic link descriptors and cost functions[23] The CGSMT problemis even more difficult where the minimal cost tree is sought by guaranteeing a givenQoS at the same time This has proved to be NP-hard as well[24] On the other handcommon multicast routing algorithms utilize stored network state information[18]which can quickly become obsolete due to the changing fading radio environment ortraffic pattern Link state information in clustered networks can also be incompletedue to aggregation Heuristic algorithms for finding Multicast Trees are published in[42] however the computational complexity becomes overwhelming as the numberof nodes increases in the network That is why we would like to benefit from the fastoptimization properties of the HNN [38] The execution time of such algorithms onlydepends on the interconnectivity of the network because every neuron represents anedge in the graph

The QoS over an obtained path however strongly depends on the ldquoincompletenessrdquoof link descriptors which is determined by the thresholds initiating Link State Adver-tisement (LSA)s These thresholds are referred to as a Link Advertisement Scheme(LAS) The process of defining LASs over the network is called link scaling which

7

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

1 Technological motivation

Due to recent and vast improvement in sensor technology and to the yet unfailingtrend set by Moorersquos law multitude of novel fields opened up for new applicationsThe platform carrying these novel applications are at the same time required to supportmobility and flexibility at the end user side As the applications become evermorecomplex their supporting systems have to deal with more demands These (suchas their own communication networking subsystem central processing units or thedevices in the background network which they communicate with) have to act moreintelligently and adapt to the new demands arriving from the upper layers Also themajority of the end user devices are portable and use battery as a power source itbecomes increasingly important to take this into consideration at the widest range ofsystem design possible

Typically into these areas we could count in the IoT based applications monitoringand intervening systems that are based on WSNs [2] peer-to-peer and ldquobroadcasttype relaying and processing systems dealing with multimedia streams (let that bevideo or audio) or most of the cloud based services [40] These services and systemshave the common aspect of providing a certain QoS while their resources are timeand location dependent and also limited [45] On inherent shared resources likeon the radio subsystem these constraints appear even more stringent Furthermorescheduling tasks efficiently in these distributed environments are imperative

Networking technologies used today are following a layered structure [16] and mostof them form packet switched communication networks In these networks typicallythere are no separate resource dedicated for signaling and controlling but withoutthese processes providing a required QoS can be mostly done in a best effort mannerif possible at all Requiring these crucial processes to be present means that they haveto isolate an additional portion from the resources bearing the payload to themselvesThus the total capacity from the end user perspective further diminishes in contrastto their signaling-less counterparts The best effort like structures which build upthe traditional packet switched networks do not or rarely use signaling to governtheir internal mechanisms in respect to QoS These structures consist of protocolsdefined by OSI in several layers Examples for these in the lower OSI layers (physicaldata-link network) are the Ethernet [1] the 80211 [15] or the IP Although theirdesign does not incorporate providing QoS directly [32 20] they are widespreadbecause of their reliability and usability [20] One of the base questions of the packetswitched networks is to find an appropriate path and scheduling for the data packetsto traverse through the network from the source to the destination [39 20 28 98] Applications using wireless technologies in the physical layer face even moreconstraints due to the shared nature of the radio media which has to be accounted forif one requires to provide QoS

2

In the light of this the open questions that this dissertation aims to answer are

bull How can one find an appropriate path in a packet switched network whichprovides a QoS (QoS unicast multicast routing)

bull How can one perform scheduling tasks in communication networks efficientlywith algorithms which lend themselves to parallelization (scheduling)

bull How can one solve at the physical layer which use wireless technologies nearoptimally the MUD problem in a parallelized fashion (Multiuser Detection)

11 Applied methodologyThis dissertation uses the following general approach to investigate the problems

and bring up possible solutions as theses

Problemsbull QoS routingbull MUDbull scheduling

mathematical modelling bybull Graph theory methodsbull Queuing theorybull Quadratic Programmingbull Nonparametric Bayesian decision methods

methods used for the solutionsbull Information theoretic metricsbull Large deviation theorybull Hopfield networksbull Feed forward nonlinear networks

performance analysisbull implementing reference algorithmsbull implementing the proposed new algorithmsbull numerical performance evaluation and comparison

evaluating and ranking the results

The summary of the problems the used algorithms theories and used tools can befound at Table 1

technologicalchallenges

modelformulation andproblemcategory

theoretical performance used methods andalgorithms

relatedtheses

Unicastrouting

using randomlink descriptorsand LASreducing theproblem toadditive andbottleneck typemetrics

The path of choice is capable ofproviding a QoS

Queuing models Markov modulatedPoissondistributionsGaussianapproximationLarge deviationtheory

Thesis I1Thesis I2

OptimizingLAS

Applyinginformationtheoretic metrics(Link Entropyand SignalingEntropy) asconstrains inoptimization

Using the resulting LAS thebandwidth of the signaling process inthe network can be bound and keptunder a predefined value while at thesame time the uncertainty of the linkstates in the network is also keptunder a well defined value

Informationtheoretic measuresexhaustive searchgeneral nonlinearconstrainedoptimizationmethods

Thesis I4

- continuing on the next page -

3

Table 1 ndash continued from previous page ndash

technologicalchallenges

modelformulation andproblemcategory

theoretical performance used methods andalgorithms

relatedtheses

Multicastrouting

using randomlink descriptorssearch for aCGSMT in thenetwork andreformulating thesearch of theCGSMT as anUBQP

Sending streams through the pathsfrom the source and destinationpoints provide the prescribed QoSwhile they perform optimally in thesense that they strain the chosenresource minimally Theseconstraints posed on the resourcesare like minimal energy consumptionor minimal total used bandwidth todeliver the payload

Gaussianapproximationlarge deviationtheory Hopfieldnetwork binaryquadraticprogrammingmethods

Thesis I3

MultiuserDetection(MUD)

UBQP andnon-parametricBayesiandetection

The exact solution of the UBQP isthe optimal solution of the problemThe proposed UBQP ldquosolvers lendthemselves to parallelization wellsince they are made up from simplenonlinear computing units Theygive well performing sub-optimalsolutions with fast convergence timeIn case of the non-parametric modelsthe proposed algorithm canarbitrarily approximate the optimaldecision at the expense of increasingits complexity

Algorithmsoperating on ahypergraph baseddimensionreduction anddimension additionHopfield networkbinary quadraticprogrammingmethods FeedForward NeuralNetwork (FFNN)logarithmic search

Thesis II1Thesis III1

Scheduling UBQP Exact solution to the UBQP is theoptimal solution to the problem Theproposed ldquosolvers give sub-optimalsolutions at fast convergence speeds

Hopfield networksbinary quadraticprogrammingmethods

Thesis II1

Table 1 Problems investigated used algorithmic tools and the related theses

4

2 New scientific results and theses of the dissertation

This dissertation presents the problems and the theses in three groups in threesections

I) In the first group I elaborate on the route searching problem providing QoSin packet switched networks both for unicast and multicast cases and proposenovel methods to solve or approximate them In this section I also investigatethe link scaling problem and propose a method for the optimization usinginformation theoretical measuresThe theses in this group are Thesis I1 Thesis I2 Thesis I3 and Thesis I4

II) In the second group I elaborate on the Multiuser Detection (MUD) and schedul-ing problems in communication technologies I formulate these tasks as BinaryQuadratic Programming (BQP) and I propose a family of algorithms which actas a suboptimal solver to the BQP problem These algorithms lend themselvesto parallelization well because of their inherent structureThesis II1 of this group can be found on page 23

III) In the third group I propose a FFNN based algorithm which performs compa-rably to the non-parametric optimal Bayesian decision for the MUD problemdefined for the wireless networks PHY layerThesis III1 of this group can be found on page 29

Thesis group I - routing with incomplete information in unicastand multicast scenarios

In this thesis group I propose novel solutions to the problem of unicast and multicastQoS routing with incomplete information These new methods are capable of findinga sub-optimal path and a multicast route set in polynomial time Also in this thesisgroup I provide a solution to the OLS task by using information theoretical measuressuch as ldquosignaling entropyrdquo and ldquolink entropyrdquo The corresponding publication ofthe author is titled ldquoMulticast Routing in Wireless Sensor Networks with IncompleteInformationrdquo [48]

In networks where is no dedicated channel for signaling or controlling purposes(typical in IoT networks using IP) these procedures consume additional resourcesAs a result it diminishes the capacity available for information transfer however atthe same time services heavily demand the speed and reliability of the underlyingcommunication stack This gives rise to the problem of finding QoS fulfilling pathsFor example in the IoT vision every device can send and receive information and mightact as an intermediate node From an angle these devices can be seen forming a WSNIf the network contains battery operated devices then the applications also requirereliable communication while keeping energy consumption at a minimal level (eg

5

consider a smart home application where the user should not be forced to change thebatteries frequently or a smart agricultural application where battery change might notbe feasible at all) On the other hand in an application where the energy consumptionmight not be a problem (eg a smart fridge automotive application or a factorywith smart production appliances) other types of reliability criteria exist that the(sub)networks must meet Among others these can be robustness to communicationshortage redundancy efficient use of the communication bandwidth responsivenessetc One can also consider peer-to-peer applications eg video on demand servicesor voice over IP services where large quantity of data needs to be reliably transportedto the peers In these scenarios both unicast and multicast type communication iscommon Data is to be transmitted to a single or a set of destination nodes with thepackets routed in a multi-hop manner where intermediate nodes are also used forpacket forwarding

Legacy network structures which were not designed for QoS but mostly for besteffort usually have no dedicated channel for signaling Examples of these protocolsin the lower OSI layers (physical data-link network) are the Ethernet 80211 orIP Despite they can not provide QoS natively by their design[32 20] they arewidespread because of their reliability and interoperability[20] Nevertheless thereis a need to run QoS communication over these best effort platforms One of themajor challenges in IP networking is to ensure QoS routing which selects paths tofulfill end-to-end delay or bandwidth requirements [39 20 28 9 8] as opposed totraditional shortest path routing Because of the manifold optimization criteria[17]even in the unicast case QoS routing can prove to be much harder than the problemof finding optimal paths based on merely the hop count [17] Protocols striving toprovide QoS need to have some knowledge about the state of the network Thisinformation has to be propagated also (signaling) QoS-aware routing protocolsoften propose a method to reduce the amount of state information that have to bekept synchronized among routers called topology aggregation (eg in QOSPFPNNI) Thus routing information describing a certain domain of the network have tobe aggregated which acts as another source of uncertainty of resource availabilityinformation Another source of uncertainty is introduced when multiple hierarchicallevels are connected Such a pattern is inevitable from the base rules of the networkdesign These hierarchical levels are introduced because these networks connectsthrough inter-domains

Incorporating QoS into routing has been long studied[20] For example an extensionto the classical IP over ATM system to support application level QoS is studied in [32]Also QoS aware routing algorithms exist in many levels such as in inter-domain level[19] in MPLS based network parts [31] inside ASs [39] but routing is done mostly bya hierarchical manner In networks following the OSI model the routing is done in the3rd (network) layer The OSPF routing protocol offers two while PNNI offers many

6

levels where routing can be performed in a hierarchical manner [13] Subnetworkson a given level of the hierarchy are abstracted as ldquonodesrdquo for a higher layer and delayinformation in those subnetworks are aggregated into an average QoS parameter Onthe other hand randomly fluctuating traffic load on links can also result in randomdelays Thus link delays are periodically advertised when the delay surpasses a giventhreshold (eg in PNNI and QOSPF standards see [3 13]) These thresholds aredefined in advance This prompts us to take delays into account as random variablescharacterized by their probability distribution functions over the interval between tworeported thresholds [14 43 27] The distribution of these thresholds (and thereforethe length of the intervals over which the link delay is not fully characterized) can beequidistant or non-equidistant In practice non-equidistant thresholds are used sincein this case the impact on network utilization by sending only signaling information(part of the available bandwidth is used for carrying information about link delays) isminimized

The phenomena described above give rise to the task of routing with incompleteinformation Namely how to select paths to fulfill end-to-end delay requirements inthe lack of the exact values of link delays [14 9 8] Routing is then perceived asan optimization problem to search over different quality paths where the quality ofa path is determined by the probability of meeting the end-to-end QoS requirement[26 14] Unfortunately routing with incomplete information in general cannot bereduced to the well-known Shortest Path Routing (SPR) thus it cannot be solved inpolynomial time in general

The multicast scenario can be seen as an extension of the previous problem andcan be treated as the well-known GSMT problem which has proved to be NP-hardeven for deterministic link descriptors and cost functions[23] The CGSMT problemis even more difficult where the minimal cost tree is sought by guaranteeing a givenQoS at the same time This has proved to be NP-hard as well[24] On the other handcommon multicast routing algorithms utilize stored network state information[18]which can quickly become obsolete due to the changing fading radio environment ortraffic pattern Link state information in clustered networks can also be incompletedue to aggregation Heuristic algorithms for finding Multicast Trees are published in[42] however the computational complexity becomes overwhelming as the numberof nodes increases in the network That is why we would like to benefit from the fastoptimization properties of the HNN [38] The execution time of such algorithms onlydepends on the interconnectivity of the network because every neuron represents anedge in the graph

The QoS over an obtained path however strongly depends on the ldquoincompletenessrdquoof link descriptors which is determined by the thresholds initiating Link State Adver-tisement (LSA)s These thresholds are referred to as a Link Advertisement Scheme(LAS) The process of defining LASs over the network is called link scaling which

7

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

In the light of this the open questions that this dissertation aims to answer are

bull How can one find an appropriate path in a packet switched network whichprovides a QoS (QoS unicast multicast routing)

bull How can one perform scheduling tasks in communication networks efficientlywith algorithms which lend themselves to parallelization (scheduling)

bull How can one solve at the physical layer which use wireless technologies nearoptimally the MUD problem in a parallelized fashion (Multiuser Detection)

11 Applied methodologyThis dissertation uses the following general approach to investigate the problems

and bring up possible solutions as theses

Problemsbull QoS routingbull MUDbull scheduling

mathematical modelling bybull Graph theory methodsbull Queuing theorybull Quadratic Programmingbull Nonparametric Bayesian decision methods

methods used for the solutionsbull Information theoretic metricsbull Large deviation theorybull Hopfield networksbull Feed forward nonlinear networks

performance analysisbull implementing reference algorithmsbull implementing the proposed new algorithmsbull numerical performance evaluation and comparison

evaluating and ranking the results

The summary of the problems the used algorithms theories and used tools can befound at Table 1

technologicalchallenges

modelformulation andproblemcategory

theoretical performance used methods andalgorithms

relatedtheses

Unicastrouting

using randomlink descriptorsand LASreducing theproblem toadditive andbottleneck typemetrics

The path of choice is capable ofproviding a QoS

Queuing models Markov modulatedPoissondistributionsGaussianapproximationLarge deviationtheory

Thesis I1Thesis I2

OptimizingLAS

Applyinginformationtheoretic metrics(Link Entropyand SignalingEntropy) asconstrains inoptimization

Using the resulting LAS thebandwidth of the signaling process inthe network can be bound and keptunder a predefined value while at thesame time the uncertainty of the linkstates in the network is also keptunder a well defined value

Informationtheoretic measuresexhaustive searchgeneral nonlinearconstrainedoptimizationmethods

Thesis I4

- continuing on the next page -

3

Table 1 ndash continued from previous page ndash

technologicalchallenges

modelformulation andproblemcategory

theoretical performance used methods andalgorithms

relatedtheses

Multicastrouting

using randomlink descriptorssearch for aCGSMT in thenetwork andreformulating thesearch of theCGSMT as anUBQP

Sending streams through the pathsfrom the source and destinationpoints provide the prescribed QoSwhile they perform optimally in thesense that they strain the chosenresource minimally Theseconstraints posed on the resourcesare like minimal energy consumptionor minimal total used bandwidth todeliver the payload

Gaussianapproximationlarge deviationtheory Hopfieldnetwork binaryquadraticprogrammingmethods

Thesis I3

MultiuserDetection(MUD)

UBQP andnon-parametricBayesiandetection

The exact solution of the UBQP isthe optimal solution of the problemThe proposed UBQP ldquosolvers lendthemselves to parallelization wellsince they are made up from simplenonlinear computing units Theygive well performing sub-optimalsolutions with fast convergence timeIn case of the non-parametric modelsthe proposed algorithm canarbitrarily approximate the optimaldecision at the expense of increasingits complexity

Algorithmsoperating on ahypergraph baseddimensionreduction anddimension additionHopfield networkbinary quadraticprogrammingmethods FeedForward NeuralNetwork (FFNN)logarithmic search

Thesis II1Thesis III1

Scheduling UBQP Exact solution to the UBQP is theoptimal solution to the problem Theproposed ldquosolvers give sub-optimalsolutions at fast convergence speeds

Hopfield networksbinary quadraticprogrammingmethods

Thesis II1

Table 1 Problems investigated used algorithmic tools and the related theses

4

2 New scientific results and theses of the dissertation

This dissertation presents the problems and the theses in three groups in threesections

I) In the first group I elaborate on the route searching problem providing QoSin packet switched networks both for unicast and multicast cases and proposenovel methods to solve or approximate them In this section I also investigatethe link scaling problem and propose a method for the optimization usinginformation theoretical measuresThe theses in this group are Thesis I1 Thesis I2 Thesis I3 and Thesis I4

II) In the second group I elaborate on the Multiuser Detection (MUD) and schedul-ing problems in communication technologies I formulate these tasks as BinaryQuadratic Programming (BQP) and I propose a family of algorithms which actas a suboptimal solver to the BQP problem These algorithms lend themselvesto parallelization well because of their inherent structureThesis II1 of this group can be found on page 23

III) In the third group I propose a FFNN based algorithm which performs compa-rably to the non-parametric optimal Bayesian decision for the MUD problemdefined for the wireless networks PHY layerThesis III1 of this group can be found on page 29

Thesis group I - routing with incomplete information in unicastand multicast scenarios

In this thesis group I propose novel solutions to the problem of unicast and multicastQoS routing with incomplete information These new methods are capable of findinga sub-optimal path and a multicast route set in polynomial time Also in this thesisgroup I provide a solution to the OLS task by using information theoretical measuressuch as ldquosignaling entropyrdquo and ldquolink entropyrdquo The corresponding publication ofthe author is titled ldquoMulticast Routing in Wireless Sensor Networks with IncompleteInformationrdquo [48]

In networks where is no dedicated channel for signaling or controlling purposes(typical in IoT networks using IP) these procedures consume additional resourcesAs a result it diminishes the capacity available for information transfer however atthe same time services heavily demand the speed and reliability of the underlyingcommunication stack This gives rise to the problem of finding QoS fulfilling pathsFor example in the IoT vision every device can send and receive information and mightact as an intermediate node From an angle these devices can be seen forming a WSNIf the network contains battery operated devices then the applications also requirereliable communication while keeping energy consumption at a minimal level (eg

5

consider a smart home application where the user should not be forced to change thebatteries frequently or a smart agricultural application where battery change might notbe feasible at all) On the other hand in an application where the energy consumptionmight not be a problem (eg a smart fridge automotive application or a factorywith smart production appliances) other types of reliability criteria exist that the(sub)networks must meet Among others these can be robustness to communicationshortage redundancy efficient use of the communication bandwidth responsivenessetc One can also consider peer-to-peer applications eg video on demand servicesor voice over IP services where large quantity of data needs to be reliably transportedto the peers In these scenarios both unicast and multicast type communication iscommon Data is to be transmitted to a single or a set of destination nodes with thepackets routed in a multi-hop manner where intermediate nodes are also used forpacket forwarding

Legacy network structures which were not designed for QoS but mostly for besteffort usually have no dedicated channel for signaling Examples of these protocolsin the lower OSI layers (physical data-link network) are the Ethernet 80211 orIP Despite they can not provide QoS natively by their design[32 20] they arewidespread because of their reliability and interoperability[20] Nevertheless thereis a need to run QoS communication over these best effort platforms One of themajor challenges in IP networking is to ensure QoS routing which selects paths tofulfill end-to-end delay or bandwidth requirements [39 20 28 9 8] as opposed totraditional shortest path routing Because of the manifold optimization criteria[17]even in the unicast case QoS routing can prove to be much harder than the problemof finding optimal paths based on merely the hop count [17] Protocols striving toprovide QoS need to have some knowledge about the state of the network Thisinformation has to be propagated also (signaling) QoS-aware routing protocolsoften propose a method to reduce the amount of state information that have to bekept synchronized among routers called topology aggregation (eg in QOSPFPNNI) Thus routing information describing a certain domain of the network have tobe aggregated which acts as another source of uncertainty of resource availabilityinformation Another source of uncertainty is introduced when multiple hierarchicallevels are connected Such a pattern is inevitable from the base rules of the networkdesign These hierarchical levels are introduced because these networks connectsthrough inter-domains

Incorporating QoS into routing has been long studied[20] For example an extensionto the classical IP over ATM system to support application level QoS is studied in [32]Also QoS aware routing algorithms exist in many levels such as in inter-domain level[19] in MPLS based network parts [31] inside ASs [39] but routing is done mostly bya hierarchical manner In networks following the OSI model the routing is done in the3rd (network) layer The OSPF routing protocol offers two while PNNI offers many

6

levels where routing can be performed in a hierarchical manner [13] Subnetworkson a given level of the hierarchy are abstracted as ldquonodesrdquo for a higher layer and delayinformation in those subnetworks are aggregated into an average QoS parameter Onthe other hand randomly fluctuating traffic load on links can also result in randomdelays Thus link delays are periodically advertised when the delay surpasses a giventhreshold (eg in PNNI and QOSPF standards see [3 13]) These thresholds aredefined in advance This prompts us to take delays into account as random variablescharacterized by their probability distribution functions over the interval between tworeported thresholds [14 43 27] The distribution of these thresholds (and thereforethe length of the intervals over which the link delay is not fully characterized) can beequidistant or non-equidistant In practice non-equidistant thresholds are used sincein this case the impact on network utilization by sending only signaling information(part of the available bandwidth is used for carrying information about link delays) isminimized

The phenomena described above give rise to the task of routing with incompleteinformation Namely how to select paths to fulfill end-to-end delay requirements inthe lack of the exact values of link delays [14 9 8] Routing is then perceived asan optimization problem to search over different quality paths where the quality ofa path is determined by the probability of meeting the end-to-end QoS requirement[26 14] Unfortunately routing with incomplete information in general cannot bereduced to the well-known Shortest Path Routing (SPR) thus it cannot be solved inpolynomial time in general

The multicast scenario can be seen as an extension of the previous problem andcan be treated as the well-known GSMT problem which has proved to be NP-hardeven for deterministic link descriptors and cost functions[23] The CGSMT problemis even more difficult where the minimal cost tree is sought by guaranteeing a givenQoS at the same time This has proved to be NP-hard as well[24] On the other handcommon multicast routing algorithms utilize stored network state information[18]which can quickly become obsolete due to the changing fading radio environment ortraffic pattern Link state information in clustered networks can also be incompletedue to aggregation Heuristic algorithms for finding Multicast Trees are published in[42] however the computational complexity becomes overwhelming as the numberof nodes increases in the network That is why we would like to benefit from the fastoptimization properties of the HNN [38] The execution time of such algorithms onlydepends on the interconnectivity of the network because every neuron represents anedge in the graph

The QoS over an obtained path however strongly depends on the ldquoincompletenessrdquoof link descriptors which is determined by the thresholds initiating Link State Adver-tisement (LSA)s These thresholds are referred to as a Link Advertisement Scheme(LAS) The process of defining LASs over the network is called link scaling which

7

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Table 1 ndash continued from previous page ndash

technologicalchallenges

modelformulation andproblemcategory

theoretical performance used methods andalgorithms

relatedtheses

Multicastrouting

using randomlink descriptorssearch for aCGSMT in thenetwork andreformulating thesearch of theCGSMT as anUBQP

Sending streams through the pathsfrom the source and destinationpoints provide the prescribed QoSwhile they perform optimally in thesense that they strain the chosenresource minimally Theseconstraints posed on the resourcesare like minimal energy consumptionor minimal total used bandwidth todeliver the payload

Gaussianapproximationlarge deviationtheory Hopfieldnetwork binaryquadraticprogrammingmethods

Thesis I3

MultiuserDetection(MUD)

UBQP andnon-parametricBayesiandetection

The exact solution of the UBQP isthe optimal solution of the problemThe proposed UBQP ldquosolvers lendthemselves to parallelization wellsince they are made up from simplenonlinear computing units Theygive well performing sub-optimalsolutions with fast convergence timeIn case of the non-parametric modelsthe proposed algorithm canarbitrarily approximate the optimaldecision at the expense of increasingits complexity

Algorithmsoperating on ahypergraph baseddimensionreduction anddimension additionHopfield networkbinary quadraticprogrammingmethods FeedForward NeuralNetwork (FFNN)logarithmic search

Thesis II1Thesis III1

Scheduling UBQP Exact solution to the UBQP is theoptimal solution to the problem Theproposed ldquosolvers give sub-optimalsolutions at fast convergence speeds

Hopfield networksbinary quadraticprogrammingmethods

Thesis II1

Table 1 Problems investigated used algorithmic tools and the related theses

4

2 New scientific results and theses of the dissertation

This dissertation presents the problems and the theses in three groups in threesections

I) In the first group I elaborate on the route searching problem providing QoSin packet switched networks both for unicast and multicast cases and proposenovel methods to solve or approximate them In this section I also investigatethe link scaling problem and propose a method for the optimization usinginformation theoretical measuresThe theses in this group are Thesis I1 Thesis I2 Thesis I3 and Thesis I4

II) In the second group I elaborate on the Multiuser Detection (MUD) and schedul-ing problems in communication technologies I formulate these tasks as BinaryQuadratic Programming (BQP) and I propose a family of algorithms which actas a suboptimal solver to the BQP problem These algorithms lend themselvesto parallelization well because of their inherent structureThesis II1 of this group can be found on page 23

III) In the third group I propose a FFNN based algorithm which performs compa-rably to the non-parametric optimal Bayesian decision for the MUD problemdefined for the wireless networks PHY layerThesis III1 of this group can be found on page 29

Thesis group I - routing with incomplete information in unicastand multicast scenarios

In this thesis group I propose novel solutions to the problem of unicast and multicastQoS routing with incomplete information These new methods are capable of findinga sub-optimal path and a multicast route set in polynomial time Also in this thesisgroup I provide a solution to the OLS task by using information theoretical measuressuch as ldquosignaling entropyrdquo and ldquolink entropyrdquo The corresponding publication ofthe author is titled ldquoMulticast Routing in Wireless Sensor Networks with IncompleteInformationrdquo [48]

In networks where is no dedicated channel for signaling or controlling purposes(typical in IoT networks using IP) these procedures consume additional resourcesAs a result it diminishes the capacity available for information transfer however atthe same time services heavily demand the speed and reliability of the underlyingcommunication stack This gives rise to the problem of finding QoS fulfilling pathsFor example in the IoT vision every device can send and receive information and mightact as an intermediate node From an angle these devices can be seen forming a WSNIf the network contains battery operated devices then the applications also requirereliable communication while keeping energy consumption at a minimal level (eg

5

consider a smart home application where the user should not be forced to change thebatteries frequently or a smart agricultural application where battery change might notbe feasible at all) On the other hand in an application where the energy consumptionmight not be a problem (eg a smart fridge automotive application or a factorywith smart production appliances) other types of reliability criteria exist that the(sub)networks must meet Among others these can be robustness to communicationshortage redundancy efficient use of the communication bandwidth responsivenessetc One can also consider peer-to-peer applications eg video on demand servicesor voice over IP services where large quantity of data needs to be reliably transportedto the peers In these scenarios both unicast and multicast type communication iscommon Data is to be transmitted to a single or a set of destination nodes with thepackets routed in a multi-hop manner where intermediate nodes are also used forpacket forwarding

Legacy network structures which were not designed for QoS but mostly for besteffort usually have no dedicated channel for signaling Examples of these protocolsin the lower OSI layers (physical data-link network) are the Ethernet 80211 orIP Despite they can not provide QoS natively by their design[32 20] they arewidespread because of their reliability and interoperability[20] Nevertheless thereis a need to run QoS communication over these best effort platforms One of themajor challenges in IP networking is to ensure QoS routing which selects paths tofulfill end-to-end delay or bandwidth requirements [39 20 28 9 8] as opposed totraditional shortest path routing Because of the manifold optimization criteria[17]even in the unicast case QoS routing can prove to be much harder than the problemof finding optimal paths based on merely the hop count [17] Protocols striving toprovide QoS need to have some knowledge about the state of the network Thisinformation has to be propagated also (signaling) QoS-aware routing protocolsoften propose a method to reduce the amount of state information that have to bekept synchronized among routers called topology aggregation (eg in QOSPFPNNI) Thus routing information describing a certain domain of the network have tobe aggregated which acts as another source of uncertainty of resource availabilityinformation Another source of uncertainty is introduced when multiple hierarchicallevels are connected Such a pattern is inevitable from the base rules of the networkdesign These hierarchical levels are introduced because these networks connectsthrough inter-domains

Incorporating QoS into routing has been long studied[20] For example an extensionto the classical IP over ATM system to support application level QoS is studied in [32]Also QoS aware routing algorithms exist in many levels such as in inter-domain level[19] in MPLS based network parts [31] inside ASs [39] but routing is done mostly bya hierarchical manner In networks following the OSI model the routing is done in the3rd (network) layer The OSPF routing protocol offers two while PNNI offers many

6

levels where routing can be performed in a hierarchical manner [13] Subnetworkson a given level of the hierarchy are abstracted as ldquonodesrdquo for a higher layer and delayinformation in those subnetworks are aggregated into an average QoS parameter Onthe other hand randomly fluctuating traffic load on links can also result in randomdelays Thus link delays are periodically advertised when the delay surpasses a giventhreshold (eg in PNNI and QOSPF standards see [3 13]) These thresholds aredefined in advance This prompts us to take delays into account as random variablescharacterized by their probability distribution functions over the interval between tworeported thresholds [14 43 27] The distribution of these thresholds (and thereforethe length of the intervals over which the link delay is not fully characterized) can beequidistant or non-equidistant In practice non-equidistant thresholds are used sincein this case the impact on network utilization by sending only signaling information(part of the available bandwidth is used for carrying information about link delays) isminimized

The phenomena described above give rise to the task of routing with incompleteinformation Namely how to select paths to fulfill end-to-end delay requirements inthe lack of the exact values of link delays [14 9 8] Routing is then perceived asan optimization problem to search over different quality paths where the quality ofa path is determined by the probability of meeting the end-to-end QoS requirement[26 14] Unfortunately routing with incomplete information in general cannot bereduced to the well-known Shortest Path Routing (SPR) thus it cannot be solved inpolynomial time in general

The multicast scenario can be seen as an extension of the previous problem andcan be treated as the well-known GSMT problem which has proved to be NP-hardeven for deterministic link descriptors and cost functions[23] The CGSMT problemis even more difficult where the minimal cost tree is sought by guaranteeing a givenQoS at the same time This has proved to be NP-hard as well[24] On the other handcommon multicast routing algorithms utilize stored network state information[18]which can quickly become obsolete due to the changing fading radio environment ortraffic pattern Link state information in clustered networks can also be incompletedue to aggregation Heuristic algorithms for finding Multicast Trees are published in[42] however the computational complexity becomes overwhelming as the numberof nodes increases in the network That is why we would like to benefit from the fastoptimization properties of the HNN [38] The execution time of such algorithms onlydepends on the interconnectivity of the network because every neuron represents anedge in the graph

The QoS over an obtained path however strongly depends on the ldquoincompletenessrdquoof link descriptors which is determined by the thresholds initiating Link State Adver-tisement (LSA)s These thresholds are referred to as a Link Advertisement Scheme(LAS) The process of defining LASs over the network is called link scaling which

7

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

2 New scientific results and theses of the dissertation

This dissertation presents the problems and the theses in three groups in threesections

I) In the first group I elaborate on the route searching problem providing QoSin packet switched networks both for unicast and multicast cases and proposenovel methods to solve or approximate them In this section I also investigatethe link scaling problem and propose a method for the optimization usinginformation theoretical measuresThe theses in this group are Thesis I1 Thesis I2 Thesis I3 and Thesis I4

II) In the second group I elaborate on the Multiuser Detection (MUD) and schedul-ing problems in communication technologies I formulate these tasks as BinaryQuadratic Programming (BQP) and I propose a family of algorithms which actas a suboptimal solver to the BQP problem These algorithms lend themselvesto parallelization well because of their inherent structureThesis II1 of this group can be found on page 23

III) In the third group I propose a FFNN based algorithm which performs compa-rably to the non-parametric optimal Bayesian decision for the MUD problemdefined for the wireless networks PHY layerThesis III1 of this group can be found on page 29

Thesis group I - routing with incomplete information in unicastand multicast scenarios

In this thesis group I propose novel solutions to the problem of unicast and multicastQoS routing with incomplete information These new methods are capable of findinga sub-optimal path and a multicast route set in polynomial time Also in this thesisgroup I provide a solution to the OLS task by using information theoretical measuressuch as ldquosignaling entropyrdquo and ldquolink entropyrdquo The corresponding publication ofthe author is titled ldquoMulticast Routing in Wireless Sensor Networks with IncompleteInformationrdquo [48]

In networks where is no dedicated channel for signaling or controlling purposes(typical in IoT networks using IP) these procedures consume additional resourcesAs a result it diminishes the capacity available for information transfer however atthe same time services heavily demand the speed and reliability of the underlyingcommunication stack This gives rise to the problem of finding QoS fulfilling pathsFor example in the IoT vision every device can send and receive information and mightact as an intermediate node From an angle these devices can be seen forming a WSNIf the network contains battery operated devices then the applications also requirereliable communication while keeping energy consumption at a minimal level (eg

5

consider a smart home application where the user should not be forced to change thebatteries frequently or a smart agricultural application where battery change might notbe feasible at all) On the other hand in an application where the energy consumptionmight not be a problem (eg a smart fridge automotive application or a factorywith smart production appliances) other types of reliability criteria exist that the(sub)networks must meet Among others these can be robustness to communicationshortage redundancy efficient use of the communication bandwidth responsivenessetc One can also consider peer-to-peer applications eg video on demand servicesor voice over IP services where large quantity of data needs to be reliably transportedto the peers In these scenarios both unicast and multicast type communication iscommon Data is to be transmitted to a single or a set of destination nodes with thepackets routed in a multi-hop manner where intermediate nodes are also used forpacket forwarding

Legacy network structures which were not designed for QoS but mostly for besteffort usually have no dedicated channel for signaling Examples of these protocolsin the lower OSI layers (physical data-link network) are the Ethernet 80211 orIP Despite they can not provide QoS natively by their design[32 20] they arewidespread because of their reliability and interoperability[20] Nevertheless thereis a need to run QoS communication over these best effort platforms One of themajor challenges in IP networking is to ensure QoS routing which selects paths tofulfill end-to-end delay or bandwidth requirements [39 20 28 9 8] as opposed totraditional shortest path routing Because of the manifold optimization criteria[17]even in the unicast case QoS routing can prove to be much harder than the problemof finding optimal paths based on merely the hop count [17] Protocols striving toprovide QoS need to have some knowledge about the state of the network Thisinformation has to be propagated also (signaling) QoS-aware routing protocolsoften propose a method to reduce the amount of state information that have to bekept synchronized among routers called topology aggregation (eg in QOSPFPNNI) Thus routing information describing a certain domain of the network have tobe aggregated which acts as another source of uncertainty of resource availabilityinformation Another source of uncertainty is introduced when multiple hierarchicallevels are connected Such a pattern is inevitable from the base rules of the networkdesign These hierarchical levels are introduced because these networks connectsthrough inter-domains

Incorporating QoS into routing has been long studied[20] For example an extensionto the classical IP over ATM system to support application level QoS is studied in [32]Also QoS aware routing algorithms exist in many levels such as in inter-domain level[19] in MPLS based network parts [31] inside ASs [39] but routing is done mostly bya hierarchical manner In networks following the OSI model the routing is done in the3rd (network) layer The OSPF routing protocol offers two while PNNI offers many

6

levels where routing can be performed in a hierarchical manner [13] Subnetworkson a given level of the hierarchy are abstracted as ldquonodesrdquo for a higher layer and delayinformation in those subnetworks are aggregated into an average QoS parameter Onthe other hand randomly fluctuating traffic load on links can also result in randomdelays Thus link delays are periodically advertised when the delay surpasses a giventhreshold (eg in PNNI and QOSPF standards see [3 13]) These thresholds aredefined in advance This prompts us to take delays into account as random variablescharacterized by their probability distribution functions over the interval between tworeported thresholds [14 43 27] The distribution of these thresholds (and thereforethe length of the intervals over which the link delay is not fully characterized) can beequidistant or non-equidistant In practice non-equidistant thresholds are used sincein this case the impact on network utilization by sending only signaling information(part of the available bandwidth is used for carrying information about link delays) isminimized

The phenomena described above give rise to the task of routing with incompleteinformation Namely how to select paths to fulfill end-to-end delay requirements inthe lack of the exact values of link delays [14 9 8] Routing is then perceived asan optimization problem to search over different quality paths where the quality ofa path is determined by the probability of meeting the end-to-end QoS requirement[26 14] Unfortunately routing with incomplete information in general cannot bereduced to the well-known Shortest Path Routing (SPR) thus it cannot be solved inpolynomial time in general

The multicast scenario can be seen as an extension of the previous problem andcan be treated as the well-known GSMT problem which has proved to be NP-hardeven for deterministic link descriptors and cost functions[23] The CGSMT problemis even more difficult where the minimal cost tree is sought by guaranteeing a givenQoS at the same time This has proved to be NP-hard as well[24] On the other handcommon multicast routing algorithms utilize stored network state information[18]which can quickly become obsolete due to the changing fading radio environment ortraffic pattern Link state information in clustered networks can also be incompletedue to aggregation Heuristic algorithms for finding Multicast Trees are published in[42] however the computational complexity becomes overwhelming as the numberof nodes increases in the network That is why we would like to benefit from the fastoptimization properties of the HNN [38] The execution time of such algorithms onlydepends on the interconnectivity of the network because every neuron represents anedge in the graph

The QoS over an obtained path however strongly depends on the ldquoincompletenessrdquoof link descriptors which is determined by the thresholds initiating Link State Adver-tisement (LSA)s These thresholds are referred to as a Link Advertisement Scheme(LAS) The process of defining LASs over the network is called link scaling which

7

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

consider a smart home application where the user should not be forced to change thebatteries frequently or a smart agricultural application where battery change might notbe feasible at all) On the other hand in an application where the energy consumptionmight not be a problem (eg a smart fridge automotive application or a factorywith smart production appliances) other types of reliability criteria exist that the(sub)networks must meet Among others these can be robustness to communicationshortage redundancy efficient use of the communication bandwidth responsivenessetc One can also consider peer-to-peer applications eg video on demand servicesor voice over IP services where large quantity of data needs to be reliably transportedto the peers In these scenarios both unicast and multicast type communication iscommon Data is to be transmitted to a single or a set of destination nodes with thepackets routed in a multi-hop manner where intermediate nodes are also used forpacket forwarding

Legacy network structures which were not designed for QoS but mostly for besteffort usually have no dedicated channel for signaling Examples of these protocolsin the lower OSI layers (physical data-link network) are the Ethernet 80211 orIP Despite they can not provide QoS natively by their design[32 20] they arewidespread because of their reliability and interoperability[20] Nevertheless thereis a need to run QoS communication over these best effort platforms One of themajor challenges in IP networking is to ensure QoS routing which selects paths tofulfill end-to-end delay or bandwidth requirements [39 20 28 9 8] as opposed totraditional shortest path routing Because of the manifold optimization criteria[17]even in the unicast case QoS routing can prove to be much harder than the problemof finding optimal paths based on merely the hop count [17] Protocols striving toprovide QoS need to have some knowledge about the state of the network Thisinformation has to be propagated also (signaling) QoS-aware routing protocolsoften propose a method to reduce the amount of state information that have to bekept synchronized among routers called topology aggregation (eg in QOSPFPNNI) Thus routing information describing a certain domain of the network have tobe aggregated which acts as another source of uncertainty of resource availabilityinformation Another source of uncertainty is introduced when multiple hierarchicallevels are connected Such a pattern is inevitable from the base rules of the networkdesign These hierarchical levels are introduced because these networks connectsthrough inter-domains

Incorporating QoS into routing has been long studied[20] For example an extensionto the classical IP over ATM system to support application level QoS is studied in [32]Also QoS aware routing algorithms exist in many levels such as in inter-domain level[19] in MPLS based network parts [31] inside ASs [39] but routing is done mostly bya hierarchical manner In networks following the OSI model the routing is done in the3rd (network) layer The OSPF routing protocol offers two while PNNI offers many

6

levels where routing can be performed in a hierarchical manner [13] Subnetworkson a given level of the hierarchy are abstracted as ldquonodesrdquo for a higher layer and delayinformation in those subnetworks are aggregated into an average QoS parameter Onthe other hand randomly fluctuating traffic load on links can also result in randomdelays Thus link delays are periodically advertised when the delay surpasses a giventhreshold (eg in PNNI and QOSPF standards see [3 13]) These thresholds aredefined in advance This prompts us to take delays into account as random variablescharacterized by their probability distribution functions over the interval between tworeported thresholds [14 43 27] The distribution of these thresholds (and thereforethe length of the intervals over which the link delay is not fully characterized) can beequidistant or non-equidistant In practice non-equidistant thresholds are used sincein this case the impact on network utilization by sending only signaling information(part of the available bandwidth is used for carrying information about link delays) isminimized

The phenomena described above give rise to the task of routing with incompleteinformation Namely how to select paths to fulfill end-to-end delay requirements inthe lack of the exact values of link delays [14 9 8] Routing is then perceived asan optimization problem to search over different quality paths where the quality ofa path is determined by the probability of meeting the end-to-end QoS requirement[26 14] Unfortunately routing with incomplete information in general cannot bereduced to the well-known Shortest Path Routing (SPR) thus it cannot be solved inpolynomial time in general

The multicast scenario can be seen as an extension of the previous problem andcan be treated as the well-known GSMT problem which has proved to be NP-hardeven for deterministic link descriptors and cost functions[23] The CGSMT problemis even more difficult where the minimal cost tree is sought by guaranteeing a givenQoS at the same time This has proved to be NP-hard as well[24] On the other handcommon multicast routing algorithms utilize stored network state information[18]which can quickly become obsolete due to the changing fading radio environment ortraffic pattern Link state information in clustered networks can also be incompletedue to aggregation Heuristic algorithms for finding Multicast Trees are published in[42] however the computational complexity becomes overwhelming as the numberof nodes increases in the network That is why we would like to benefit from the fastoptimization properties of the HNN [38] The execution time of such algorithms onlydepends on the interconnectivity of the network because every neuron represents anedge in the graph

The QoS over an obtained path however strongly depends on the ldquoincompletenessrdquoof link descriptors which is determined by the thresholds initiating Link State Adver-tisement (LSA)s These thresholds are referred to as a Link Advertisement Scheme(LAS) The process of defining LASs over the network is called link scaling which

7

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

levels where routing can be performed in a hierarchical manner [13] Subnetworkson a given level of the hierarchy are abstracted as ldquonodesrdquo for a higher layer and delayinformation in those subnetworks are aggregated into an average QoS parameter Onthe other hand randomly fluctuating traffic load on links can also result in randomdelays Thus link delays are periodically advertised when the delay surpasses a giventhreshold (eg in PNNI and QOSPF standards see [3 13]) These thresholds aredefined in advance This prompts us to take delays into account as random variablescharacterized by their probability distribution functions over the interval between tworeported thresholds [14 43 27] The distribution of these thresholds (and thereforethe length of the intervals over which the link delay is not fully characterized) can beequidistant or non-equidistant In practice non-equidistant thresholds are used sincein this case the impact on network utilization by sending only signaling information(part of the available bandwidth is used for carrying information about link delays) isminimized

The phenomena described above give rise to the task of routing with incompleteinformation Namely how to select paths to fulfill end-to-end delay requirements inthe lack of the exact values of link delays [14 9 8] Routing is then perceived asan optimization problem to search over different quality paths where the quality ofa path is determined by the probability of meeting the end-to-end QoS requirement[26 14] Unfortunately routing with incomplete information in general cannot bereduced to the well-known Shortest Path Routing (SPR) thus it cannot be solved inpolynomial time in general

The multicast scenario can be seen as an extension of the previous problem andcan be treated as the well-known GSMT problem which has proved to be NP-hardeven for deterministic link descriptors and cost functions[23] The CGSMT problemis even more difficult where the minimal cost tree is sought by guaranteeing a givenQoS at the same time This has proved to be NP-hard as well[24] On the other handcommon multicast routing algorithms utilize stored network state information[18]which can quickly become obsolete due to the changing fading radio environment ortraffic pattern Link state information in clustered networks can also be incompletedue to aggregation Heuristic algorithms for finding Multicast Trees are published in[42] however the computational complexity becomes overwhelming as the numberof nodes increases in the network That is why we would like to benefit from the fastoptimization properties of the HNN [38] The execution time of such algorithms onlydepends on the interconnectivity of the network because every neuron represents anedge in the graph

The QoS over an obtained path however strongly depends on the ldquoincompletenessrdquoof link descriptors which is determined by the thresholds initiating Link State Adver-tisement (LSA)s These thresholds are referred to as a Link Advertisement Scheme(LAS) The process of defining LASs over the network is called link scaling which

7

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

can be subject to further optimization in order to economize on signaling bandwidthThe smaller these intervals are the smaller is the measure of ldquoincompletenessrdquo underwhich a path is selected As a result the routing performance is higher On theother hand small intervals forces more frequent announcements of the values of thelink descriptors throughout the network which implies that considerable portion ofbandwidth is used up for transmitting signaling information Thus the optimizationof the size of these intervals is a crucial task for network management This problemis referred to as OLS As can be seen OLS is a constrained optimization problem inwhich one has to maximize routing performance under the constraint of keeping thesignaling bandwidth bellow a given threshold

The communication network is modeled by a graph

G(VEδ(uv)F(uv)(x)

)

where nodes are denoted by index u isinV and links referred to as node pairs (uv) isin EEach link (uv) isin E has a QoS link descriptor δ(uv) which is assumed to be a randomvariable subject to a CDF F(uv)(x) = P

(δ(uv) lt x

) The random variable δ(uv) is

referred to as bottleneck measure if the smallest link measure determine the qualityof the route (δ(uv) is a bandwidth-like variable) It is also referred to as bottleneckmeasure if the largest link measure determine the quality of the route (δ(uv) isa energy consumption-like variable) The random variable δ(uv) is referred to asadditive measure if the sum of link measures contained in the path determine thequality of the path (δ(uv) is a delay-like variable)

u v

u rarr v

v

s

d

(a) unicast case

uv

s

m1m2

m3

m4

mN

m5

m6

regular node

multicast node

(b) multicast case

Figure 1 graph model of the network

For the unicast case the objective is to find an optimal path R from all pos-sible paths Rsrarrd which most likely fulfills the given QoS criterion namely

8

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

R = argmaxRisinRsrarrd

P(

min(uv)isinR

δ(uv) ge B)

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)

for the bottleneck and additive measures respectively

For the multicast case the information source typically a Base Station is denotedby s isin V and the set of multicast destination nodes by M = m1m2 mN subV In this case the objective is to find an optimal tree A from all multi-cast trees A srarrM which most likely fulfills the given QoS criterion namely

A1 = argmaxAisinA srarrM

P(

max(uv)isinAE

δ(uv) lt P)

A2 = argmaxAisinA srarrM

P

max

RsmisinRTsum(uv)isin

Rsm

δ(uv) lt T

for the bottleneck and additive measures respectively Note that for the multicastbottleneck case the problem is formulated differently than in the unicast case becausethe link descriptor is typically power consumption like quantity in WSNs thus thebottleneck in this case is the most consuming element But the derived conclusionsare also applicable to a bandwidth-like link descriptor with minor modifications

An advertisement of a link state change happen when the current value of linkdescriptor steps out from an interval defined by two threshold values which is calledLink State Advertisement (LSA) Figure 2 depicts an LSA in case the link descriptoris of delay type When a link advertisement is received the receiver could assume

delay on link (uv)

t0 t1 timinus1 ti ti+1current delay

last advertised LAS entry

delay on link (uv)

t0 t1 timinus1 ti ti+1

delay at at some future time

newly advertised LAS entry

Figure 2 LSA in case the link descriptor is of delay type

that the link descriptor of the sender node is within the new interval The finer thethreshold grid is defined on the link descriptors the more precise the receiver canbe about the current state of the sender but on the other hand it also means the

9

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

frequency of the advertisement to increase Here our objective is to strike an optimalbalance between the signaling bandwidth and routing performance To capture theunderlying phenomena two information theoretical measures are introduced theSignaling Entropy (SE) and the Link Entropy (LE)

Since link state advertisement occurs when randomly jumping over one (or several)advertisement threshold it can be regarded as an information theoretical source de-noted by κ(uv) with values t0 t1 1Zminus1 covering the axis of the link descriptor withprobabilities p0 p1 pZminus1 The source does not emit any symbol (inactive) withprobability P = 1minussumZminus1

i=0 pi These probabilities are governed by the randomly fluctu-ating link traffic (ie by the varying queue length in the buffer as each link is perceivedas a buffer) Assuming optimal source coding the necessary bandwidth to distributethe link state information is related to the conditional entropy H

(κ(uv)

∣∣χ(uv) = 1)

of κ(uv) where the Bernoulli random variable χ(uv) describes whether the source isactive or not The link entropy on the other hand is the measure which quantifies theuncertainty about state of link (uv) if [ti ti+1) interval was advertised for this link(ν(uv) = i) Using these two quantities the bandwidth and the quality of the signalingprocesses can be kept at bay

link entropy H(δ(uv)

∣∣ν(uv))

signaling entropy H(κ(uv)

∣∣χ(uv))

THESIS I1 (unicast routing with incomplete information by Gaussian approxima-tion) I gave a mapping for the link descriptors under the condition that the linkdescriptors have normal distributions with parameters m and σ(uv) =

radicm(uv) andalso the LAS follows m =

ti+1+ti2 in Theorem 1

Theorem 1 If δ(uv) is a subject to a normal distribution with parameters σ(uv) =radicm(uv) then the solution of ARII

R = argmaxRisinRsrarrd

P

(sum

(uv)isinRδ(uv) lt T

)(2 revisited)

is equivalent to minimizing the objective function

R = argminR

sum(uv)isinR

m(uv) (22)

by using the Bellman-Ford algorithm in polynomial time

Using these assumptions the ARII problem can be reduced to a deterministictraditional SPR

10

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

THESIS I2 (unicast routing with incomplete information by recursive path finderalgorithm) I gave procedures that can find routes in a packet switched networkwhich satisfy the required QoS parameter with a given probability in Algorithm 1 andAlgorithm 2 (restating below)

The algorithms are based on a transformation of the random link descriptors usingthe large deviation theory which is described in Theorem 2Theorem 2 Using the logarithm of the moment generating function (log-momentgenerating function)

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

int infin

minusinfinexp(sx)dF(uv)(x) (23)

or in case of a discrete random variable

micro(uv)(s) = lnE(exp(sδ(uv)

))= ln

infin

sumi=1

exp(sxi) pi (24)

the solution of the ARII is equivalent with minimizing the objective function

R = argminR

sum(uv)isinR

micro(uv)(s) (25)

where the optimal s parameter is

s = infs sum

(uv)isinR

micro(uv)(s)minus sT (26)

Where the two algorithms are the following

11

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Algorithm 1 Exhaustive-s algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Define a grid on the set of possible values of s denoted by S =si si gt 0 i = 1 Mfor all i = 1 M do

Pick si isin S and perform path selection Ri by an SPR algorithm with linkmeasures micro(uv)(si) = lnE

(exp(siδ(uv)

))

Based on the selected path Ri determine

si = Solve

sum(uv)isinRi

dmicro(uv)(s)ds

= Ts

(27)

and calculate the bound

Bi = exp

(sum

(uv)isinRi

micro(uv)(si)minus siT

) (28)

end forFind the path which belongs to minimal bound

R j j = argmini

Bi (29)

Output R j chosen path between src and dst

Algorithm 2 The Recursive Path Finder - s Finder Algorithm

Input G(VEδ(uv)F(uv)(x)

)srcdst

Pick slarr a positive starting value and compute the path independent micro(s)repeat

Associate measure micro(uv)(s) to each link (uv) isin EPerform the SPR algorithm to find the optimal path R(s) for parameter sFor the obtained R determine s by expression

s =

micro primeminus1

(T minus sum

(uv)isinRa(uv)

)

|R|

slarr suntil R(s) 6= R(s)

Output R(s) chosen path between src and dst

12

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Performance analysis In order to compare the algorithms I introduce a perfor-mance measure for comparing two paths Ra and Re both starting from node src andending at node dst This is defined as the ratio of the probability of the path Re(found by the exhaustive search) and the probability of the path Ra (found by a givenalgorithm) fulfilling the end-to-end QoS criterion given as follows

η(RaT ) = P

(sum

(uv)isinRe

δ(uv) lt T

)minusP

(sum

(uv)isinRa

δ(uv) lt T

) (210)

Given that Re is the best route in the sense that it fulfills any T QoS criterion withthe largest probability η(RaT ) ge 0 and it measures the ldquoperformance droprdquo inprobability for given T QoS value

The normalized area under these curves are defined as

χ(Ra) =

Tmaxint

0

η(RaT )dT Tmaxint

0

P

(sum

(uv)isinRe

δ(uv) lt T

)dT (211)

where Tmax is the investigated maximum value for T χ corresponds to the perfor-mance drop of a path compared to the best path It is easy to see that the ldquoworst pathrdquoR0 which can fulfill any T le Tmax only with 0 probability corresponds to the valueχ(R0) = 1 and the best path Re corresponds to χ(Re) = 0 The closer this functionapproximates the value 0 the better the performance of the corresponding route is

Furthermore to obtain more general numerical results the algorithms were runnot only on one graph with particular link states but on a set of graphs denoted byG Each graph had 10 nodes having cardinalities at least 3 and the random graphgenerator made sure that all nodes has belonged to the same component Let usdenote all possible routes (from all possible src to all possible dst src 6= dst) in agraph G with R(G) I characterize the ensemble performance by metrics χ(G) andχe given as follows

χ(G) =1

|R(G)| sumRisinR(G)

χ(R) χe =1|G | sum

GisinGχ(G) (212)

The performance of four algorithms will be shown and compared to the perfor-mance of the exhaustive search ldquoOSPFrdquordquoGaussian approximationrdquordquoexhaustive-srdquordquorecursive-srdquo The ldquoOSPFrdquo is a simple shortest path algorithm having a metric asthe advertised LAS entry The ldquoGaussian approximationrdquo had its metric according toTheorem 1 the ldquoexhaustive-srdquo is based on Algorithm 1 and ldquorecursive-srdquo is based onAlgorithm 2

13

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Figure 3 GEANT network topology used to evaluate the proposed algorithms

On the GEANT topology I have chosen Island ldquo1ISLrdquo for the src node and Romanialdquo37ROrdquo for the dst node as this pair offers a wide variety of routes in between themAll the routes from src to dst can be labeled with an index (1 to 4937) in an increasingorder according to χ(Ra) This labeling can be seen in Figure 4 Using the fixed and

0 1000 2000 3000 4000 5000

ith path

0

02

04

06

08

1

0 5 10 15 20

ith path

0

001

002

003

004

005

Figure 4 Path index vs its performance χ Tmax=3000 The left figure depicts all thepaths while the figure on the right zooms in to the first 20 best path

known link descriptor distribution I sampled the network When the OSPF algorithmfound the best route (path 1) so did all other algorithms For this particular linkdistribution ensemble the relative frequency for the OSPF not choosing path 1 wasP = 04148 so almost half of the time there were better performing paths To quantifythe improvement of the introduced algorithms I chose 10000 sample points when theOSPF algorithm did not choose path 1 The frequency of the routes chosen by thealgorithms are depicted in Figure 5 The total performance metric χe on the graphensemble can be found in Figure 7

14

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

1716

7763

147

245

70

19 1914

6

1

36962539

3193

171

271

69

22 26

12

1

8431

741 717

98

8

1

3

1

36452483

3300

171

271

69

22 26

12

1

1 2 3 4 5 6 7 8 10 11 15

i th path

10 0

10 2

10 4t

imes

pat

h ch

osen

OSPFGaussianexhaustive-srecursive-s

Figure 5 Path choice frequency out of 104 samples when the OSPF did not choosepath 1 in the GEANT topology

0 0005 001 0015 002 0025 003 00350

50

100

150

200

250

300OSPFGaussianexhaustive-srecursive-s

Figure 6 The distributions of the ensemble performance metric χ(G) over all graphsG in the random graph ensemble G for all for all introduced algorithms

OSPF Gaussian exhaustive-s recursive-s0

0005

001

0015

X OSPFY 001551 X Gaussian

Y 001359

X exhaustive-sY 0004268

X recursive-sY 001217

Figure 7 The total ensemble performance metric χe for all introduced algorithms

15

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

THESIS I3 (multicast routing with incomplete information with HNN) I definedalgorithm to find a sub-optimal solution to the multicast routing problem with randomlink descriptors in Algorithm 3 (restating)

Algorithm 3 Find optimal tree for end-to-end requirement

Input G(VEδ(uv)F(uv)(x)

) κ = 1 T gt 1 srcm

repeatA = find tree with HNN(GκT)if A is found then

decrease κ

elseincrease κ

end ifuntil no significant increase in performance

Output A is the multicast tree between src and m

The procedure transforms the random link descriptors into deterministic ones byusing results from large deviation theory which I formulated

A2 argminAisinA

sum(uv)isinA

Cuv

st microRsrcm(s)lt ln(κ)+ s middotT

The transformed problem can be seen as a CGSMT which is still NP-hard but Ipropose a sub-optimal solution by using HNN where the corresponding parametersare described at sub-subsection 243

E (y) = α1

(2ytrb(1)

)+α2

(ytrW(2)y+2ytrb(2)

)

Performance analysis The A1 and the A2 objective functions were evaluated byexhaustive search and the HNN based algorithm on a graph with the following param-eters The size of the network N = 8 the Rayleigh channel parameters were chosento typical or better indoor environment γ = 3g = 1θ = 10s2 = 1 The positions ofthe nodes were randomly generated according to iid uniform distributions in theunit square The group of the multicast nodes consisted 3 randomly chosen nodes

I have performed the exhaustive search by enumerating all the possible trees andevaluating the objective functions on the trees I have compared the results of theHNN algorithm to the exhaustive solution For the A2 objective function I haveevaluated the performance given by the Chernoff bound and also the correspondingtheoretical probability by performing convolutions on the known distributions

16

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

0 5 10 15 20 25 300

05

1

HNNExhaustive

Figure 8 A typical evaluation of the A1 objective function

0 5 10 15 200

05

1

0 5 10 15 20

5

10

(a) Near optimal solution for A2 obj func

0 5 10 15 20 25 300

05

1

0 5 10 15 20 25 300

10

20

(b) A typical evaluation of the A2 obj func

Figure 9 Performance of the multicast tree finder algorithm in case of additivemeasures

17

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

THESIS I4 (optimizing link scaling using MAPM1) I formulated a constrainedoptimization problem which connects the information about the random link descrip-tors (Link Entropy) and the appropriate bandwidth of the signaling process to supportthat information (Signaling Entropy) at a certain probability

min∆t

H(δ(uv)

∣∣ν(uv)(∆t))

st H(κ(uv)(∆t)

∣∣χ(uv)(∆t))lt α

I proposed a computable solution to this problem by modeling the dynamics of thelink descriptors as MAPM1 described as

H(κ(uv) (πππD0D1micro∆t)

∣∣χ(uv)(∆t))

H(δ(uv) (πππD0D1micro)

∣∣ν(uv) (πππD0D1micro∆t))

Consequently the information theoretical quantities can be obtained analytically andthe optimal solution can be found

Performance analysis I have based my traffic models on the publicly availableDISCO data-set from the Measurement-lab data-set [34] I choose the switch con-nected to the ldquomlab1dub01measurement-laborgrdquo server Since June 2016 M-Labhas collected high resolution switch telemetry for each M-Lab server and site uplinkI have used the ldquoBytes received by the machine switch portrdquo and ldquoUnicast packetsreceived by the machine switch portrdquo to gather the statistics for the traffic modelsThe data-set contains these metrics with sampling time of 10s I assumed that within

May 04 May 05 May 06 May 07 May 08 May 09time

0

50

100

150

200

[Mbp

s]

traffic speed on local rx port (moving average over 30min)

1400-15001830-1930average traffic speed

Figure 10 Daily self similarity pattern in the data from mlab1dub01measurement-laborg Moving averaged data was plotted from 2018 May 4-9

an hour interval the traffic somewhat stays the same (relative to the daily regularfluctuations) Based on this I have chosen two time periods over which I have ag-gregated the necessary statistics to derive the traffic models An average traffic load

18

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

from 1400-1500 each day in 2018 May 1 to Jun 30 Traffic situation 1 has amean packet rate of 6066341pps std686561pps max7594403pps and mean speed62Mbps std7828Mbps max88366MbpsA more intensive traffic load from 1830-1930 each day in 2018 Sep 1 toNov 30 Traffic situation 2 has a mean packet rate of 1361381pps std122783ppsmax1371807pps and mean speed 1435Mbps std1415Mbps max160545Mbps

I have used 16 state MAP-s in both cases to model the sequence of events From theidentified models I also generated sequences of events which then were mapped backto the average packet rate metric for comparison It can be seen that the identifiedmodels are detailed enough to reproduce the statistics of the real life data OnFigure 13 and Figure 14 one can see the SE and the LE plotted against the link scalingHere the link scaling means the number of divisions over the link descriptor

10 0 10 1 10 2 10 3 10 4 10 5

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

May 01 - June 30 1400-1500

mlab datadata generated byidentified MAP traffic model

10 -2 10 -1 10 0 10 1 10 2 10 3

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 11 Traffic statistics for 1400-1500 each day in 2018 May 1 to Jun 30

10 1 10 2 10 3 10 4 10 5 10 6

avg packet rate[pkts]

0

0005

001

prob

abili

ty o

f bin

Sep01-Nov30 1830-1930 each day

mlab datadata generated byidentified MAP traffic model

10 -1 10 0 10 1 10 2 10 3 10 4

avg traffic speed [Mbps]

0

0005

001

prob

abili

ty o

f bin

Figure 12 Traffic statistics for 1830-1930 each day in 2018 Sep 1 to Nov 30

19

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 19709

X 315961Y 25865

X 372364Y 19649

X 372364Y 34545

Figure 13 Information theoretic metrics for traffic situation 1

10 0 10 1 10 2 10 3 10 4

Number of emittable symbols |LAS|

2

4

6

entr

opie

s

SE equidistant LASLE equidistant LAS

10 15 20 25 30Number of emittable symbols |LAS|

1

2

3

entr

opie

s

SE exponential LASLE exponential LAS

X 315961Y 26813

X 315961Y 17829

X 436126Y 17838

X 436126Y 36281

Figure 14 Information theoretic metrics for traffic situation 2

20

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Thesis group II - a heuristic solver based on hypergraphs forUBQP and its applicability in ICT

This chapter is organized around the combinatorial approach and tractability ofseveral ICT problems since they can be formulated as an Unconstrained BinaryQuadratic Programming (UBQP) These applications include load balancing a wideclass of scheduling problems MUD VLSI design Steiner tree problemsetc Asurvey of these applications can be found in [22]

In cloud computing environments and in IoT the efficient scheduling [30 52]and distribution of tasks plays a central role in performance and scalability Severalapproaches exist [41 51 44] - usually metaheuristics - to address these problems butat their core almost each of them contains a method for approximating a solution ofa constrained optimization problem This core step can be usually formulated as aUBQP therefore the proposed algorithms can be utilized on it Recent surveys onscheduling in IaaS cloud computing environments and load balancing can be found in[41 51 44]

Furthermore other problems under linear constraints can also be transformed intoUBQP as demonstrated in [21 6 35 50 10 29] Unfortunately UBQP has proved tobe NP-hard [12] but in some special cases it can be solved in polynomial time [6 3637 4] In general though there is still a great need for developing fast methods whichcan reach near-optimal solutions when the size of the problem goes beyond a givenlimit Thus the aim of this chapter is to present some novel approaches to UBQPwhich are based on recursive dimension reduction (or addition) techniques Althoughthe more complex applications are more relevant the proposed algorithms and theirperformance will be presented in detail on simpler applications for traceability

bull large scale problems listed in ORLIBbull simple schedulingbull Multiuser Detection

Based on the performance analysis the new algorithms prove to be superior to theknown heuristics regarding both the quality of the achieved solution and the conver-gence time The corresponding publication of the author is titled ldquoNovel algorithmsfor quadratic programming by using hypergraph representationsrdquo [47]

The UBQP is a quadratic COP where each component of vector y can have twodistinct values which are taken to be -1 and +1

L (yWb) = yT Wyminus2yT b

y isin plusmn1N b isin RN W isin RNtimesN

yopt = minyisinplusmn1N

L (yWb)

21

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

The state space of the N dimension UBQP problem can be represented by a weightedgraph where the vertices of the graph correspond to the state vector y the weightsof the vertices are the values of the objective function L (yWb) while the edgesbetween the vertices are defined by a given neighborhood function

q = L (yWb)

y =

plusmn1_to_dec(y) =

-1-1-1-1

+1+1+1+1

-1-1+1-1

+1-1-1-1

-1+1-1-1

-1+1+1-1

+1+1+1-1

+1+1-1+1

-1-1-1+1

-1+1+1+1

+1+1-1-1

+1-1-1+1

+1-1+1-1

+1-1+1+1

-1-1+1+1

-1+1-1+1

0 15 2 8 4 6 14 13 1 7 12 9 10 11 3 5

Figure 15 Search space of a 4 dimension UBQP - vertices of a 4 dimension hypercube

We can also extend the graph to a hypergraph based representation of the problemIn this the search space is extended by including all possible points from the subspacesThe extension of the search space on one hand is motivated with the reduced run timeie searching for a candidate solution is performed in a smaller space and on theother hand since we reduced the dimension we can get out from certain local minimaThe new sub space points will be the intersections of the cutting plane with the edgesof the hypercube

If we do this in all possible combination we get 2N sub spaces for search One canconnect these sub spaces with a hypergraph where the vertices of this hypergraphare the graphs defined over the sub spaces of the original problem In the followingpicture a hypergraph corresponding to the previous example is shown

22

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Figure 16 discarding the 2nd dimension from the 3 dimension hypercube

Figure 17 A hypergraph of a 4 dimension UBQP

THESIS II1 (A heuristic solver family based on hypergraphs for UBQP) In Algo-rithm 4 I have given a hypergraph based easily parallelizable algorithm family tosub-optimally solve the UBQP problem

The algorithms project the original search space into a hypergraph representationand use a HNN based internal solver to find a solution I have given four instancesof which two employs dimension reduction and two dimension addition Table 3(restating) summarizes the operation modes of the instances (The precise descriptionof the algorithms can be found in Appendix

23

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Table 3 Categorization of the algorithmsgreedy opportunistic

dim reducer L01 D01dim adder DA02 DA01

I have tested the performance on three different problem sets on the standardORLIB UBQP benchmark set on a scheduling problem and on a simulated MUDproblem I have shown that the proposed methods perform near optimal on theinvestigated ICT problems

Algorithm 4 Pseudo code of the general UBQP solver algorithm

1 function INNER_SOLVER(W isin Rktimeskb isin Rky(init) isin plusmn1k)2 an arbitrary UBQP minimizer3 return y isin plusmn1k

4 end function5 function Ψ(u isinVH y isin uV )6 choose u isinVH choose the next hypernode and7 choose y isin uV choose a state in that hypernode8 return u y9 end function

Input W b and u(init) the problem and the starting hypernode10 ularr u(init) isinVH start hypernode of the alg11 choose y isin u(init)V init state in the hypernode12 repeat13 define L(Wby) objective function14 ularr u and ylarr y15 Wblarr parameters from u = G(VEQ(Wb))16 if SHOULD_EMPLOY_INNER_SOLVER( )then17 ylowastlarr INNER_SOLVER(Wby)18 else19 ylowastlarr y20 end if21 u ylarr Ψ(uylowast)22 until STOP_CRIT( )Output y the best solution found by the alg

24

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Performance analysis For the ORLIB [5] test set we have implemented all thealgorithms in the same programming environment and performed the tests within thesame computational environment as well Besides the new algorithms we also use thefollowing traditional algorithms for the sake of comparison

bull ldquoHNNrdquo - a discrete time and discrete valued Hopfield type recurrent networkbull ldquo1-opt LSrdquo - a 1-opt local search type algorithmbull ldquoBLSrdquo an algorithm presented by Beasley [6] based on a 1-opt local search

methodbull ldquoBTSrdquo a taboo search algorithm also presented by Beasley [6]bull ldquoDDTrdquo the well-known DDT algorithm by Boros Hammer and Sun [7]bull ldquoSDRrdquo an SDR type algorithm without randomization presented by Luo et al

[29]bull ldquoSDR with randomizationrdquo is the same algorithm but with randomization [29]

Table 4 Performance comparison of BTS vs DA01Esolution best solution relative freq of best solution mean run time mean run time until best solution found

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=50

K=1

0000

1 2160 20554 2160 2160 1 01052 23223 1 2322286 950572 3658 35886 3658 3658 1 01316 25105 1 2510545 7598783 4650 46804 4650 4778 1 04924 24036 1 2403645 2030874 3472 34007 3472 3472 1 01275 23453 1 2345311 7843145 4152 40985 4152 4152 1 07053 23499 1 2349933 1417846 3842 38236 3842 3842 1 05405 23971 1 2397059 1850147 4588 45351 4588 4588 1 02746 24111 1 241107 3641668 4222 41952 4222 4222 1 08435 23371 1 2337129 1185549 3862 38299 3862 3862 1 06273 22899 1 2289912 15941310 3496 34508 3496 3496 1 02163 23626 1 236264 462321

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=100

K=1

0000

1 7910 7631 7910 7910 1 00039 65814 1 658142 256412 11178 11030 11178 11178 1 00192 75619 1 7561899 5208333 12956 12875 12956 12956 1 01854 73601 1 7360066 5393744 10606 10493 10606 10606 1 00722 70157 1 7015745 1385045 8994 87772 8994 8996 1 00173 70659 1 706588 5780356 10470 10362 10470 10486 1 0011 71521 1 715213 9090917 9980 98776 9980 10030 1 00182 72419 1 7241868 5494518 11380 11240 11380 11380 1 01357 71742 1 7174218 736929 11340 11246 11340 11340 1 0269 79918 1 7991803 37174710 12438 12348 12438 12438 1 0032 79959 1 7995852 3125

probalg BTS DA01 BTS DA01 BTS DA01 BTS DA01 BTS DA01

dim

=500

K=1

000

1 116526 114780 116526 116532 1 0001 13642 1 136418 10002 128678 127801 128678 128678 1 0002 14043 1 140425 5003 131084 130013 131084 131084 1 0006 13294 1 132944 1666674 129794 128646 129794 129784 1 0001 13209 1 132092 10005 125008 123859 125008 125062 1 0001 11694 1 116937 10006 121868 120189 121868 121868 1 0001 13336 1 133355 10007 122730 121163 122730 122756 1 0001 13989 1 13989 10008 123454 121958 123454 123428 1 0001 13811 1 138106 10009 121622 120026 121622 121668 1 0001 13889 1 138885 100010 130900 130219 130900 131374 1 001 13865 1 138647 100

When applying the algorithms to the scheduling problem we compared to thetraditional Earliest Deadline first Weighted Shortest Processing Time first and itswariants The Total Weighted Tardiness is displayed as the performance measure

In this case the best TWT values achieved by the different algorithms are HNN84

25

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Figure 18 TWT performance of algorithms versus heuristic parameter α for a specificcase J = 10

DA0181 D0184 L0179 TS79 EDD130 WSPT131 WSPTR97 WSPTS95respectively For this specific case L01 and Taboo Search perform the best but theirperformances are nearly identical with the others

For the Multi User Detection application the following detectors were used [25]ldquothresholdrdquo is the plain sign decision rule following the matched filter which filters thesymbols with the appropriate code The ldquoinvFilterrdquo is the generalized Zero-Forcingdetector trying to equalize the channel and cancel the multi user interference The ldquoQRrdquoand the ldquoDFrdquo variant of this algorithm uses the QR factorization and reformulationof the problem and a decision feedback respectively The ldquoMMSErdquo is the minimummean square detector and the sphere detector is denoted with the abbreviation ldquoSDrdquoIn the following figures the performance measures are shown for an unsaturated multiuser configuration We used M = 7 users to communicate simultaneously It can beseen that two of our solvers perform as well as the Sphere Detector and the othertraditional methods perform very poorly under this condition From the other figure itcan be seen that in terms of objective function value for almost all SNR values theldquoD01rdquo algorithm and the ldquoSD performs best but the ldquoDA01rdquo algorithm performsalmost identically to them Referring back to the previous chapter we recall thatldquoDA01 needs much smaller time to reach its candidate solution than the ldquoD01rdquo

26

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Figure 19 BER performance of the algorithms for 7 users

Figure 20 Objective function performance of the algorithms for 7 users

The sphere detector gives a very good quality solution however it converges muchmore slowly than the DA01 Furthermore our proposed algorithms lend themselvesto easy parallel implementation

27

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

Thesis group III - near Bayesian performance non-parametric de-tection with Feed Forward Neural Networks

This thesis group elaborates on developing novel encoding techniques for imple-menting non-parametric neural network based detectors for pattern recognition onnoisy input This fundamental problem appears in many real world applications like ininformation extraction in big data context automated surveillance speech recognitioncontent based search or in legacy systems using CDMA to name a few

I propose an FFNN based algorithm which I present on the MUD problem but dueto the nature of the method it can be easily generalized In the MUD scenario it iscapable of achieving near optimal performance with relatively limited complexityThese new encoding methods on the one hand can increase the processing speed andreduce the complexity of the FFNN based detector on the other Furthermore wedemonstrate that an asymptotically optimal detection performance can be achievedby the proposed algorithms Due to the increased processing rate the new schememay further improve SE Extensive simulations and the corresponding numericalanalysis demonstrate that the proposed algorithms yield near optimal performanceon real channel models (COST-207) The corresponding publication of the authoris ldquoMulti-user detection using non-parametric Bayesian estimation by feed forwardneural networksrdquo [46]

In the most general case the optimal decision after receiving a sequence x issymbol y which is the most probable that had been sent through the channel as it willminimize the BER [49 10] This is also called the Bayesian or MAP decision whichis given as follows

yopt = fopt(x) = argmaxyisinplusmn1L

P(Y = y|X = x) (213)

In other words the MAP decision can be carried out by searching for the maximumamong the components of this vector p(y|x) Please also note that computing thisprobability vector directly is also of exponential complexity since N = 2L

yopt = y(i) i = argmaxnisin1N

p(y(n)|x) (214)

The block diagram of this exponential complexity optimal detector can be seen inFigure 21

28

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

+Hy

ν

x yp(y|x)

optimal detector

decision

search formax component

computeall

conditional

Figure 21 Flow graph representation of the optimal detector

THESIS III1 (blind detection by interval halving and FFNN) I have defined anFFNN based blind detector for the MUD problem which lends itself to easy parallel-

Net (xw)

mE(s|x)x

x E(s|x) =

= S middotp(y | x)

conditional prob-ability for allpossible y

Sp(y|x)

Figure 22 Equivalence of the FFNN with an encoding

Hy

ν

x yE(s|x)Net (xw)

decision

detector utilizing arbitrary coding

g( )

s(i) = Coding(

y(i))

S = [s(1)s(2) s(N) ]T (l) =

(x(k)s(k)

)k = 1 l

Figure 23 flow graph representation of the detector using an arbitrary encoding

29

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

ization and can perform optimally under the constraint

j = maxiisin1N

p(s(i)|x)

if j isin E

sumkisinE

p(s(k)|x)gt sumiisin1NE

p(s(i)|x)

In the following I give the linear encoding based on interval halving which is usedto generate a training set for an FFNN

Si j = s( j)i = sgn

(minussin

(2π middot2(iminus1) middot j

N +1

)) i = 1 L j = 1 N

and the low complexity decision function which is to be employed on the output of thenet is

y = sgn(E(s|x))

I have shown that the detector performs near optimally on the investigated MUDscenarios

Performance analysis For comparison beside the new methods I also show a fewtraditional detector (MMSE ZF SDR) and also the theoretical limit The channelmodel for each user was computed based on the COST 207 models [11] Four modelswere used namely ldquoCOST207 Hilly Terrain 6 tap alternativerdquo ldquoCOST207 Rural Area6 taprdquo ldquoCOST207 Typical Urban 12 taprdquo and ldquoCOST207 Bad Urban 12 taprdquo models

The following figures show the performance of the methods for M = 7 K = 2and L = 14 for different radio channel models One can see when the channelchanges from the one measured on hilly terrain to another one measured in urbanarea practically can achieve the optimal performance The difference betweenthe theoretical performance and the actual one is due to the asymptotic nature oflearning Namely the FFNN failed to capture the conditional expected value perfectlyHowever it achieves similar BER than the best performing SD Note that while theSD is a parametric detection which needs the channel characteristics as opposed toour proposed method which does not One can see that ldquoFFNN I2rdquo produces onlyslightly worse performance than the MAP decision But even in this case the novelmethod provide low BER with respect to SNR

30

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

(a) BER using channel COST 207 Hilly Ter-rain 6 tap alternative

(b) BER using channel COST 207 Rural Area6 tap

(c) BER using channel COST 207 TypicalUrban 12 tap

(d) BER using channel COST 207 Bad Urban12 tap

Figure 24 Performance curves with parameter L = 14 for the four channel models

(a) SNR loss curves with parameter L= 10 forchannel COST 207 Bad Urban 12 tap channel

(b) SNR loss curves with parameter L= 14 forchannel COST 207 Bad Urban 12 tap channel

Figure 25 SNR loss curves for the Bad Urban channel model

31

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

3 Summary of the dissertation and closing remarksIn this dissertation I have given the following answers to the posed questions

One can efficiently find an appropriate path or tree in a packet switched networkwhich provides a QoS (either bottleneck or additive type) This can be achieved byexploiting the statistical properties of the traffic and transforming the models in sucha way that they become a natural fit for the traditional route finding algorithms orneural networks Furthermore the precision with which the QoS is met can be scaledat the expense of some bandwidth by applying information theoretic measures

One can solve efficiently and near optimally the UBQP problem which is presentin relevant ICT applications the scheduling tasks in communication networks (IoT orcloud computing environments) load balancing and MUD for wireless technologiesThis problem can be treated in a parallel fashion with the aid of both Feed Forwardand Recurrent type neural networks To achieve this I have demonstrated how toreformulate the problems to fit these algorithms For the Recurrent type neuralnetwork I posed these problems as an ldquoenergy basedrdquo optimization problem For theFeed Forward neural networks I exploited their general approximation capabilities

One can solve efficiently and near optimally a general pattern recognition problemwith the aid of Feed Forward neural networks and a linear encoding technique I havedemonstrated the efficiency of the algorithm on the MUD problem but it is applicableon a wide range of problems including automated surveillance applications contentbased search speech recognition

The numerical examples presented back up my conjecture that these algorithmsare in deed applicable and perform efficiently

Although a lot of aspects were not addressed this work gives sufficient detailssuch that it can be used as a basis for further investigation For example how aphysical implementation of such neural networks could speed up finding sub-optimalsolutions for these problems Furthermore it gives a common numerical referencefor comparison to other types of algorithms A possible natural extension of theproposed methods would be to change the currently used neural networks with ldquodeep-learningrdquo based variants compare the performance and investigate the gains andlosses Certainly if a particular application is to be considered these algorithms needtailoring Also note that the appropriate physical architecture may not exist at thetime of writing but this work might point to such possible directions

32

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

4 AcknowledgementFirst I would like to give thanks to my supervisor Dr Jaacutenos Levendovszky to

prof Tamaacutes Roska to Dr Aacuterpaacuted Csurgay to Judit Nyeacutekyneacute dr Gaizler to Dr PeacuteterSzolgay and to all the present and past directors of the doctoral school to MartinHaenggi and prof Geacuteza Kolumbaacuten for their encouragement advises and critics ofhigh standards which all modulated the trajectory I traveled so far

I would like to express my special gratitude to dr Andraacutes Olaacuteh and to Peacuteter Vizidr Korneacutel Neacutemeth Jaacutenos Paacutesztor Joacutezsef Somogyi Daacuteniel Suumltto and Daacutevid Kaukerfor their support endurance and help without which this work would not have beencompleted

I would like to thank my fellow doctoral school members and co-workers for theircommunity and support inter alia Andraacutes Bojaacuterszky dr Gergely Treplaacuten dr KaacutelmaacutenTornai dr Csaba Joacutezsa dr Reguly Istvaacuten Mikloacutes Koller Imre JuhaacuteszGaacutebor Abonyi Peacuteter Paacutel Poraacutezik Tamaacutes Kosztolaacutenczi Attila Feheacuter Marcell TibeacutelyMaacuteteacute Nagy Ferenc Varjasi Alexander Dvorzsaacutek Milaacuten Gyoumlrki

I would like to give thanks to Katinka Vida Tivadarneacute for her precise and attentivesupport throughout my years in the doctoral school and beyond

To the members of the finance and registrars department the members of the deansoffice the members of the IT department and for all fellow members of the facultyfor their work which are often not as visible but equally important

I would also like to give thanks to the members and founders of the SP CEEScholarship program for their friendship guidance patience and financial support

And I am deeply grateful to my parents brothers and to the other members of myfamily Kaacutelmaacuten Tisza Kaacutelmaacutenneacute Tisza born Katalin Adorjaacuten Kaacutelmaacuten and LeventeGergelyneacute Busa Ildikoacute also to my magisters Jaacutenos Gutai Maacuteria Kovaliczky andteachers namely Katalin Szaboacute Kaacutelmaacutenneacute and Erika Szeitzneacute Viski for building thefoundations on which I could set a firm stand

33

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

References[1] 8023-2012 2012 DOI 101109ieeestd20126419735 URL http

dxdoiorg101109IEEESTD20126419735[2] IF Akyildiz W Su Y Sankarasubramaniam and E Cayirci ldquoWireless sensor

networks a surveyrdquo In Computer Networks 384 (2002) pp 393 ndash422 ISSN1389-1286 DOI httpdxdoiorg101016S1389-1286(01)00302-4 URL httpwwwsciencedirectcomsciencearticlepiiS1389128601003024

[3] G Apostolopoulos S Kama D Williams R Guerin A Orda and T Przy-gienda QoS Routing Mechanisms and OSPF Extensions RFC 2676 (Experi-mental) 1999 URL httpwwwietforgrfcrfc2676txt

[4] F Barahona ldquoA solvable case of quadratic 0-1 programmingrdquo In DiscreteApplied Mathematics 131 (1986) pp 23ndash26

[5] J E Beasley ldquoOR-Library distributing test problems by electronic mailrdquo InJournal of the Operational Research Society 4111 (1990) pp 1069ndash1072

[6] JE Beasley ldquoHeuristic algorithms for the unconstrained binary quadratic pro-gramming problemrdquo In London UK Management School Imperial College(1998)

[7] E Boros P Hammer and X Sun ldquoThe DDT method for quadratic 0-1 mini-mizationrdquo In RUTCOR Research Center RRR 39 (1989) p 89

[8] Shigang Chen and Klara Nahrstedt ldquoAn overview of quality of service routingfor next-generation high-speed networks problems and solutionsrdquo In IEEENetwork Magazine Special Issue on Transmission and Distribution of DigitalVideo 126 (Nov 1998) pp 64ndash79 ISSN 0890-8044 DOI 10110965752646

[9] Shigang Chen and Klara Nahrstedt ldquoOn finding multi-constrained pathsrdquo InCommunications 1998 ICC 98 Conference Record 1998 IEEE InternationalConference on Vol 2 IEEE June 1998 pp 874ndash879 DOI 101109ICC1998685137 URL httpciteseernjneccomchen98findinghtml

[10] MO Damen H El Gamal and G Caire ldquoOn maximum-likelihood detectionand the search for the closest lattice pointrdquo In Information Theory IEEETransactions on 4910 (2003) pp 2389 ndash2402 ISSN 0018-9448 DOI 101109TIT2003817444

[11] M Failli Digital land mobile radio communications COST 207 Tech repEuropean Commission 1989

[12] Michael R Garey and David S Johnson Computers and Intractability AGuide to the Theory of NP-Completeness Vol 174 New York NY USAWHFreeman amp Co Ltd 1979 ISBN 0716710447

34

[13] M Goguen ldquoPrivate Network-Network Interface Specification Version 10rdquoIn PNNI Specification Working Group ATM forum March 1996

[14] Roche A Gueacuterin and Ariel Orda ldquoQoS routing in networks with inaccurate in-formation theory and algorithmsrdquo In IEEEACM Transactions on Networking(TON) 73 (1999) pp 350ndash364

[15] IEEE Standard for Information technologyndashTelecommunications and informa-tion exchange between systems Local and metropolitan area networksndashSpecificrequirements Part 11 Wireless LAN Medium Access Control (MAC) and Phys-ical Layer (PHY) Specifications IEEE -Org 2012 DOI 101109ieeestd20126178212 URL httpdxdoiorg101109IEEESTD20126178212

[16] ISOIEC standard 7498-11994 - OSIISO 7 layer model httpstandardsisoorgittfPubliclyAvailableStandardss020269_ISO_IEC_7498-1_1994(E)zip

[17] JM Jaffe ldquoAlgorithms for finding paths with multiple constraintsrdquo In Net-works 141 (1984) pp 95ndash116

[18] Luo Junhai Xue Liu and Ye Danxia ldquoResearch on multicast routing protocolsfor mobile ad-hoc networksrdquo In Computer Networks 525 (2008) pp 988ndash997 ISSN 1389-1286 DOI DOI101016jcomnet200711016

[19] Mirosław Kantor Piotr Chołda and Andrzej Jajszczyk ldquoLeast Cost Routing(LCR) solution for inter-domain traffic distributionrdquo In TelecommunicationSystems (2011) 101007s11235-011-9606-1 pp 1ndash13 ISSN 1018-4864 URLhttpdxdoiorg101007s11235-011-9606-1

[20] John Klincewicz James Schmitt and Richard Wong ldquoIncorporating QoSinto IP Enterprise Network Designrdquo In Telecommunication Systems 20 (12002) 101023A1015441400785 pp 81ndash106 ISSN 1018-4864 URL httpdxdoiorg101023A1015441400785

[21] GA Kochenberger F Glover B Alidaee and C Rego ldquoA unified modelingand solution framework for combinatorial optimization problemsrdquo In ORSpectrum 262 (2004) pp 237ndash250

[22] GaryA Kochenberger Fred Glover Bahram Alidaee and Cesar Rego ldquoSolvingCombinatorial Optimization Problems Via Reformulation and Adaptive Mem-ory Metaheuristicsrdquo In Frontiers of Evolutionary Computation Ed by AnilMenon Vol 11 Genetic Algorithms and Evolutionary Computation SpringerUS 2004 pp 103ndash113 ISBN 978-1-4020-7524-7 DOI 1010071-4020-7782-3_5 URL httpdxdoiorg1010071-4020-7782-3_5

[23] Vachaspathi P Kompella Joseph C Pasquale and George C Polyzos ldquoMul-ticast routing for multimedia communicationrdquo In IEEEACM Trans Netw13 (1993) pp 286ndash292 ISSN 1063-6692 DOI httpdxdoiorg10110990234851

35

[24] Vachaspathi Peter Kompella ldquoMulticast routing algorithms for multimediatrafficrdquo PhD thesis La Jolla CA USA University of California at San Diego1993

[25] EG Larsson ldquoMIMO Detection Methods How They Work [Lecture Notes]rdquoIn Signal Processing Magazine IEEE 263 (2009) pp 91 ndash95 ISSN 1053-5888 DOI 101109MSP2009932126

[26] Whay Chiou Lee ldquoSpanning tree method for link state aggregation in large com-munication networksrdquo In INFOCOMrsquo95 Fourteenth Annual Joint Conferenceof the IEEE Computer and Communications Societies Bringing Informationto People Proceedings IEEE IEEE Washington DC USA IEEE ComputerSociety 1995 pp 297ndash302 ISBN 0-8186-6990-X

[27] Jaacutenos Levendovszky Alpaacuter Fancsali Csaba Veacutegso and Gaacutebor Reacutetvaacuteri ldquoQoSRouting with Incomplete Information by Analog Computing Algorithmsrdquo InQuality of Future Internet Services Second COST 263 International WorkshopQofIS 2001 Coimbra Portugal September 24ndash26 2001 Proceedings Ed byMikhail I Smirnov Jon Crowcroft James Roberts and Fernando BoavidaBerlin Heidelberg Springer Berlin Heidelberg 2001 pp 127ndash137 ISBN978-3-540-45412-0 DOI 1010073- 540- 45412- 8_10 URL httpdxdoiorg1010073-540-45412-8_10

[28] Dean H Lorenz and Ariel Orda ldquoQoS routing in networks with uncertainparametersrdquo In IEEEACM Transactions on Networking 66 (Dec 1998)pp 768ndash778 ISSN 1063-6692 DOI 10110990748088 URL citeseeristpsuedulorenz98qoshtml

[29] Z Luo W Ma AMC So Y Ye and S Zhang ldquoSemidefinite relaxation ofquadratic optimization problemsrdquo In Signal Processing Magazine IEEE 273(2010) pp 20ndash34

[30] Nguyen Cong Luong Dinh Thai Hoang Ping Wang Dusit Niyato Dong InKim and Zhu Han ldquoData collection and wireless communication in Internetof Things (IoT) using economic analysis and pricing models A surveyrdquo InIEEE Communications Surveys amp Tutorials 184 (2016) pp 2546ndash2590

[31] Haci Mantar ldquoA scalable QoS routing model for diffserv over MPLS networksrdquoIn Telecommunication Systems 34 (3 2007) 101007s11235-007-9035-3pp 107ndash115 ISSN 1018-4864 URL httpdxdoiorg101007s11235-007-9035-3

[32] Steacutephane Martignoni and Thomas Kuumlhnel ldquoExtension of Classical IP overATM to support QoS at the application levelrdquo In Telecommunication Systems11 (3 1999) 101023A1019161721356 pp 291ndash303 ISSN 1018-4864 URLhttpdxdoiorg101023A1019161721356

[33] MeasurementLab disco dataset blogpost httpswwwmeasurementlabnetblogdisco-datasetnew-disco-switch-telemetry-datasetAccessed 20190514

36

[34] MeasurementLab homepage httpswwwmeasurementlabnet Ac-cessed 20190514

[35] Peter Merz and Kengo Katayama ldquoMemetic algorithms for the unconstrainedbinary quadratic programming problemrdquo In Biosystems 781ndash3 (2004) pp 99ndash118 ISSN 0303-2647 DOI 101016jbiosystems200408002URL http www sciencedirect com science article pii S0303264704001376

[36] JC Picard and HD Ratliff ldquoA graph-theoretic equivalence for integer pro-gramsrdquo In Operations Research (1973) pp 261ndash269

[37] JC Picard and HD Ratliff ldquoMinimum cuts and related problemsrdquo In Net-works 54 (1975) pp 357ndash370

[38] C Pornavalai G Chakraborty and N Shiratori ldquoA neural network approachto multicast routing in real-time communication networksrdquo In ICNPrsquo95Proceedings of the 1995 International Conference on Network Protocols IEEEComputer Society 1995 p 332 ISBN 0-8186-7216-1

[39] Chotipat Pornavalai Goutam Chakraborty and Norio Shiratori ldquoRoutingwith multiple QoS requirements for supporting multimedia applicationsrdquo InTelecommunication Systems 9 (3 1998) 101023A1019160226383 pp 357ndash373 ISSN 1018-4864 URL http dx doi org 10 1023 A 1019160226383

[40] Bhaskar Prasad Rimal Eunmi Choi and Ian Lumb ldquoA taxonomy and surveyof cloud computing systemsrdquo In INC IMS and IDC 2009 NCMrsquo09 FifthInternational Joint Conference on Ieee 2009 pp 44ndash51

[41] Maria Alejandra Rodriguez and Rajkumar Buyya ldquoA taxonomy and surveyon scheduling algorithms for scientific workflows in IaaS cloud computingenvironmentsrdquo In Concurrency and Computation Practice and Experience(2016)

[42] Hussein F Salama Douglas S Reeves and Yannis Viniotis ldquoEvaluation ofmulticast routing algorithms for real-time communication on high-speed net-worksrdquo In IEEE Journal on Selected Areas in Communications 15 (1997)pp 332ndash345

[43] Anees Shaikh Jennifer Rexford and Kang G Shin ldquoDynamics of quality-of-service routing with inaccurate link-state informationrdquo In Ann Arbor1001CSE-TR-350-97 (1997) pp 48109ndash2122 URL citeseeristpsuedushaikh97dynamicshtml

[44] Sukhpal Singh and Inderveer Chana ldquoA survey on resource scheduling incloud computing Issues and challengesrdquo In Journal of Grid Computing 142(2016) pp 217ndash264

[45] Tim Szigeti and Christina Hattingh End-to-end qos network design Ciscopress 2005

37

[46] Daacutevid Tisza Andraacutes Olaacuteh and Jaacutenos Levendovszky ldquoMulti-user detectionusing non-parametric Bayesian estimation by feed forward neural networksrdquoIn Telecommunication Systems (2015) pp 1ndash11

[47] Daacutevid Tisza Andraacutes Olaacuteh and Jaacutenos Levendovszky ldquoNovel algorithms forquadratic programming by using hypergraph representationsrdquo In Wirelesspersonal communications 773 (2014) pp 2305ndash2339

[48] Daacutevid Tisza Peacuteter Vizi Janos Levendovszky and Andraacutes Olaacuteh ldquoMulticastRouting in Wireless Sensor Networks with Incomplete Informationrdquo In Wire-less Conference 2011 - Sustainable Wireless Technologies (European Wireless)11th European 2011 pp 1ndash5

[49] Sergio Verdu Multiuser Detection Cambridge University Press 1998 ISBN0521593735

[50] J Wang ldquoDiscrete Hopfield network combined with estimation of distributionfor unconstrained binary quadratic programming problemrdquo In Expert Systemswith Applications 378 (2010) pp 5758ndash5774

[51] Minxian Xu Wenhong Tian and Rajkumar Buyya ldquoA Survey on Load Bal-ancing Algorithms for VM Placement in Cloud Computingrdquo In arXiv preprintarXiv160706269 (2016)

[52] Changsheng Yu Li Yu Yuan Wu Yanfei He and Qun Lu ldquoUplink Schedulingand Link Adaptation for Narrowband Internet of Things Systemsrdquo In IEEEAccess 5 (2017) pp 1724ndash1734

38

• Technological motivation
• • Applied methodology
  • • New scientific results and theses of the dissertation
    • Summary of the dissertation and closing remarks
    • Acknowledgement