Radar Problem Real-Life Problem Electromagnetic Modeling

1

www.abakus.computing.technology

II EE EE EE AA PP Distinguished Lecturer

LLeevveenntt Gürel UPC, Barcelona, Spain

FFaasstt AAllggoorriitthhmmss aanndd PPaarraalllleell CCoommppuuttiinngg:: SSoolluuttiioonn ooff EExxttrreemmeellyy LLaarrggee RReeaall--LLiiffee PPrroobblleemmss

iinn CCoommppuuttaattiioonnaall EElleeccttrroommaaggnneettiiccss

LLeevveenntt GGüürreell CCEEOO,, AABBAAKKUUSS CCoommppuuttiinngg TTeecchhnnoollooggiieess

PPrrooffeessssoorr EEmmeerriittuuss,,!!BBiillkkeenntt UUnniivveerrssiittyy,, TTuurrkkeeyy FFoouunnddeerr,, CCoommppuuttaattiioonnaall EElleeccttrroommaaggnneettiiccss RReesseeaarrcchh CCeenntteerr ((BBiiLLCCEEMM))

AAddjjuunncctt!!PPrrooffeessssoorr,, DDeepptt.. ooff EECCEE,,!!UUnniivveerrssiittyy ooff IIlllliinnooiiss aatt UUrrbbaannaa--CChhaammppaaiiggnn




!"#$""%&'

(")**+',+*-%.%/*"'

,0*#*"12'3+4&#%5&'

6$#%7%#$+1%5&'

8%)%+'94&#$7&'

:-/2%5''(7%.1".''94&#$7&'

;1*5*.12%5'9#+<2#<+$&'

=8;3>'

?1+$5$&&'3*77<"12%/*"&'

@ABC8%".$'31+2<1#&'

!"#$%&'(")*$"+,-./#01"2+3)4-.5$"/3)6,)7'8"-"1,)9-":;"1+'"3)

PPhhoottoonniiccss NNaannoommaatteerriiaallss BBiiootteecchhnnoollooggyy



LLeevveenntt Gürel UPC, Barcelona, Spain Radar Problem




Real-Life Problem Electromagnetic Modeling of Microcircuits

2




20 GHz

30 GHz

Photonic crystals

!""#$%&'()*+,&-"#./0*12(3()$%/*

Ö. Ergül, T. Malas, and L. Gürel, Analysis of dielectric photonic-crystal problems with MLFMA and Schur- complement preconditioners, J. Lightwave Technol., vol. 29, no. 6, pp. 888–897, Mar. 2011.



LLeevveenntt Gürel UPC, Barcelona, Spain Radiation into Living Organisms




Rx Antenna Rx Antenna

Rx Antenna

Rx Antenna

Tx Antenna

Tx Antenna

Tx Antenna

Tx Antenna

Microwave Imaging




Double-Ridged Horn

Fractal Spiral

Log-Periodic

Vivaldi

Conical-Corrugated Horn

Archimedian

Patch+SRRs

Conical Spiral

Patch

Antennas

3



LLeevveenntt Gürel UPC, Barcelona, Spain Mobile Device Antennas




Satellite Antennas



LLeevveenntt Gürel UPC, Barcelona, Spain Antennas Mounted on Platforms

•! Interaction of multiple antennas •! Characteristics of mounted antennas (different from isolated antennas) •! Optimization of the placement of the antennas



LLeevveenntt Gürel UPC, Barcelona, Spain 4$-5#&'()*+)6$7()-.)3*

4




Maxwell s Equations

!"E(r ,t) =#

$$t

B(r ,t)

!"H(r ,t) =

##t

D(r ,t)+J (r ,t)

!"B(r ,t) = 0

!"D(r ,t) = !(r ,t)




Surface Integral Equations

CFIE = !EFIE + (1! !)MFIE

•! Electric-Field Integral Equation (EFIE):

•! Magnetic-Field Integral Equation (MFIE):

•! Combined-Field Integral Equation (CFIE):

•! Hybrid-Field Integral Equation (HFIE):

HFIE = !(r)EFIE + 1! !(r)"

#$%&'MFIE

J(r)! n̂(r)" d #r J( #r )" #$ g r, #r( )#S% = n̂(r)"H

inc(r)

!t̂(r) " ik d #r I ! $#$

k2

%

&''''

(

)****g r, #r( ) "J( #r )

#S+ =

1!t̂(r) "E inc(r)



LLeevveenntt Gürel UPC, Barcelona, Spain Geometry Discretization

Stealth Antennas

Airborne Mesh Size: !/10 www.abakus.computing.technology


LLeevveenntt Gürel UPC, Barcelona, Spain Discretization

ZmnE = dr tm(r)

Sm

! " d #r G r, #r( ) "bn( #r )Sn

!

ZmnM = dr tm(r)

Sm

! "bn(r)# dr tm(r)Sm

! " n̂ $ d %r bn( %r )$ %& g r, %r( )Sn

!

Z

mn

E,M ,C ,Ha

n=

n=1

N

! vm

E,M ,C ,H , m = 1,2,…N

•! Matrix equations:

•! Matrix elements:

Zmn

C = !ZmnE + (1! !)

ik

ZmnM

Basis functions

Testing functions

J(r) = an bn(r)

n=1

N

!Number of unknowns

Z

mn

H = !mZ

mn

E + (1! !m

)i

kZ

mn

M

5



LLeevveenntt Gürel UPC, Barcelona, Spain Matrix Elements"

"are electromagnetic interactions

ZmnE = dr tm(r)

Sm

! " d #r G r, #r( ) "bn( #r )Sn

!

ZmnE an =

n=1

N

! vmE , m = 1,2,…N

Basis functions Testing functions



LLeevveenntt Gürel UPC, Barcelona, Spain Geometry Discretization

Modeling with millions of triangles.

Mesh Size: !/10



LLeevveenntt Gürel UPC, Barcelona, Spain Matrix Equation

System of Linear Equations

A!x=b

Z !a=vwww.abakus.computing.technology



Two Equations with Two Unknowns

6




Three Equations with Three Unknowns




Two Equations with Two Unknowns

<-#/"-=3)!;$") ))



LLeevveenntt Gürel UPC, Barcelona, Spain Three Equations with Three Unknowns

>?.)*:;#2.13)?',@))

>?.)A1B1.?13)

<-#/"-=3)!;$")




Algorithmic Complexity

!! "#$%%&#'()*&+&'#,&-'.(((!!""#$!! /0#+10(2$*1.((((((((((((((((((((!!"##%$!!""#$!! 3,10#,&41(51,6-7%.((((((((((!!&"'#$!! 8#%,(9*:-0&,6+%.((((((((((((((!!"$()*"#%$!!"#$

Matrix Equation System of Linear Equations

Z !a=v

7




Iterative Solutions

Z !a = v

x

Z !x = y y

M ! z = w

w

z

•! Large matrix equations

a

•! CG and CGS •! BiCG and BiCGStab •! QMR and TFQMR •! GMRES •! LSQR

Parallel Multilevel Fast Multipole Algorithm (MLFMA)

Matrix-Vector Multiplications Iterative Algorithm

•! BDP •! NFP •! Filtered NFP •! ILU and ILUT •! SAI •! Nested PCs

Preconditioner



LLeevveenntt Gürel UPC, Barcelona, Spain Fast Multipole Method

Aggregation

Translation

Disaggregation

Cluster of basis functions

F !G

(k̂) = Fn !G

(k̂) an

n" !G#

F

GT(k̂) = T

L(k,D)

!G "N (G)# F !G

(k̂)

Z

mna

nn=1

N

! =14!

d 2k̂ FmGC (k̂) "F

GT(k̂)#

Cluster of testing functions

Complexity: O(N3/2)



LLeevveenntt Gürel UPC, Barcelona, Spain C;$2$"D"$)9#3,)C;$2E.$")6$0.-',@/




8&9$&3.9**(7*$)%(-$):*;.#9/*

O(N logN )!!*<(-"#.,$3=0*

>7..*/375%357.*8.%57/$6.*%#5/3.7$):*

<#5/3.7/*

?5#'#.6.#*@&/3*?5#'"(#.*!#:(7$32-*

8




Reduced-Complexity Solutions

Astrophysics (Gravitational Force) Acoustics

Electrodynamics (Helmholtz s Equation)

Molecular Dynamics

Quantum Mechanics (Schroedinger s Equation)

Electrostatics (Laplace s Equation)

Fast Multipole Methods

Structural Mechanics Fluid Dynamics

Heat Equation




Acoustics and Elastics

!! ;<(=<(/61'>(?<(/<(/61@>(#'7(A<(B10-$:>(C8#%,(+$*,&D-*1(+1,6-7(#%(#'(1EF&G&1',(%-*410(E-0(HI(1*#%,&G(@#41(%$0E#G1(&',1:0#*(1J$#,&-'%>K(/-+D$,#,&-'#*(51G6#'&G%>(4-*<(HL>(DD<(MNOPOLQ>(RNNS<(

!! 5<(A<(T-':>(?<(/<(/61@>(#'7(5<(U<(?6&,1>(C5$*,&*141*(E#%,(+$*,&D-*1(#*:-0&,6+(E-0(#G-$%,&G(@#41(%G#,,10&':(VW(,0$'G#,17(:0-$'7(@&,6(,01'G61%>K(U-$0'#*(-E(9G-$%,&G(A-G&1,W(-E(9+10&G#>(4-*<(RHX>('-<(O>(DD<(HORXPHOHR>(HLLY<((

!! 5<(A<(T-':(#'7(?<(/<(/61@>(C5$*,&*141*(E#%,(+$*,&D-*1(#*:-0&,6+(E-0(1*#%,&G(@#41(%G#,,10&':(VW(*#0:1(XI(-VZ1G,%>K(U-$0'#*(-E(/-+D$,#,&-'#*([6W%&G%>(4-*<(HHY>('-<(X>(DD<(NHRPNXH>(HLLN<((



LLeevveenntt Gürel UPC, Barcelona, Spain Acoustics

Noise Control This is an exterior acoustic wave radiation problem. Total complex DOFs = 541,152, solved on a desktop PC



LLeevveenntt Gürel UPC, Barcelona, Spain Acoustics

9



LLeevveenntt Gürel UPC, Barcelona, Spain Seismic Wave Propagation

S. Chaillat, J.F. Semblat, M. Bonnet, A preconditioned 3-D multi-region fast multipole solver for seismic wave propagation in complex geometries. Communications in Computational Physics (special issue WAVES 2009), Vol. 11, 594-609, 2012.




+,-./012*3.45$6789:1)2

!! ;)(8<3:.1,$=95:$<8(:1>)(3$<3:-)0$=).$<)03(12*$+,-./012*3.45$3789:1)2$

B6&J&'(B6#->(\#0#W#'(]-44#*&>(?1'V&'(^&'>(/6#':P=-&(96'>(^$&%1(/-$G6+#'>(^#@01'G1(/#0&'(I1D#0,+1',(-E()*1G,0&G#*(#'7(/-+D$,10()':&'110&':>(I$_1(`'&410%&,W>(I$06#+>(\/(HSSLYPLHNR>(`A9a(\#4#*(21%1#0G6(^#V-0#,-0W>(?#%6&':,-'>(I/>(`A9a(AG6--*(-E()*1G,0-'&G()':&'110&':>(`'&410%&,W(-E()*1G,0-'&G(AG&1'G1(#'7(T1G6'-*-:W(-E(/6&'#>(/61':7$>(/6&'#>(U-$0'#*(-E(/-+D$,#,&-'#*([6W%&G%(b3+D#G,(8#G,-0.(H<RMc<(LQdHLLSa(Ie3.(RL<RLRQdZ<ZGD<HLLQ<RR<LLX(



LLeevveenntt Gürel UPC, Barcelona, Spain Molecular Electrostatics

Molecular model of a protein (PDB id:1PPE, 436 atoms). (a) The van der Waals surface of the protein which models the molecule as a union of balls. (b) The variational molecular surface gives a smooth approximation of the van der Waals surface. (c) The variational surface is then triangulated and then decimated to produce a smaller mesh. This decimated mesh contains 1,000 triangles. (d) The algebraic spline molecular surface (ASMS) fits a smooth surface over the triangular mesh. (e) Electrostatic potential computed using the 1,000 patch ASMS. (f) Electrostatic potential using an ASMS with 74,812 patches. The surfaces in (e) and (f) are colored by the electrostatic potential, ranging from !3.8 kbT/ec (red) to +3.8 kbT/ec (blue).

AN EFFICIENT HIGHER-ORDER FAST MULTIPOLE BOUNDARY ELEMENT SOLUTION FOR POISSON-BOLTZMANN BASED MOLECULAR ELECTROSTATICS Chandrajit Bajaj, Shun-Chuan Chen, and Alexander Rand SIAM J Sci Comput. Jan 1, 2011; 33(2): 826–848.



LLeevveenntt Gürel UPC, Barcelona, Spain Biomolecular Electrostatics

Surface charge plot on a lysozyme molecule Validation of the PyGBe code for Poisson-Boltzmann equation with boundary element methods Christopher D. Cooper, Lorena A. Barba George Washington University

10



LLeevveenntt Gürel UPC, Barcelona, Spain Thermal Analysis

Fuel Cells There are 9,000 small side holes in this model! Total DOFs = 530,000, solved on a desktop PC) Dr. Yijun Liu, CAE Research Lab, University of Cincinnati http://urbana.mie.uc.edu/yliu/Software/ Engine Block



LLeevveenntt Gürel UPC, Barcelona, Spain Elasticity

Fiber Composites Up to 16,000 CNT fibers and total DOFs = 28,800,000, solved on a supercomputer at Kyoto University http://urbana.mie.uc.edu/yliu/Images/FMM_BEM.JPG



LLeevveenntt Gürel UPC, Barcelona, Spain Stokes Flow

MEMS This is an exterior Stokes flow problem. Total DOFs = 1,087,986, solved on a desktop PC




Stokes Flow

11



LLeevveenntt Gürel UPC, Barcelona, Spain Books




Parallel Computing

NODES

NETWORK SWITCH

SERVERCONNECTED TO INTERNET

SLAVE HOST

Constructing a Parallel Computer Cluster




Parallel Computers




Parallel Computing

Constructing Parallel Computer Clusters

8 nodes

Per node: Two Intel Xeon 5345 2.33 GHz Quad-Core Processors

Total of 64 cores

Per node: 32 GB DDR2-667 533 MHz RAM

Total of 256 GB of RAM

Network: Infiniband

17 nodes

Per node: Two Intel Xeon 5472 3 GHz Quad-Core Processors

Total of 136 cores

Per node: 32 GB DDR2-667 533 MHz RAM

Total of 544 GB of RAM

Network: Infiniband

136-Core Cluster 64-Core Cluster

Not in www.top500.org!!!

12




What is the Main Source of Efficiency?

N Unknowns

O(N3) Gaussian Elimination

O(N2) Iterative MOM

(MVM)

O(N3/2) Single-Level FMM

O(N logN) Multi-Level FMM

1000 1 s 2 s 4 s 8 s

106 32 years 23 days 35 h 7 h

107 32 K years 6.3 years 46 days 89 h

108 32 M years 630 years 4 years 46 days

109 32 G years 63 K years 127 years 1.5 years

(555 days)




53 Million Unknowns

Sphere with radius of 120! and diameter of 240! November 2007

120!

53,112,384 Unknowns




69 Million Unknowns

Sphere with radius of 150! and diameter of 300! December 2007

150!

69,177,600 Unknowns



LLeevveenntt Gürel UPC, Barcelona, Spain 85 Million Unknowns

Sphere with radius of 150! and diameter of 300!

January 2008

150!

85,148,160 Unknowns

Scattering results are compared to analytical values to demonstrate the accuracy of the numerical solution.

13



LLeevveenntt Gürel UPC, Barcelona, Spain Intel Pamphlet on the World Record

Available at www.cem.bilkent.edu.tr





August 2008

135,164,928 Unknowns


180!





August 2008



190!





September 2008



210!

14





December 2009



260!





December 2009



280!





September 2010

540,659,712 Unknowns Scattering results are compared to analytical values to demonstrate the accuracy of the numerical solution.

340!




How Large are Large Matrix Equations?

Each element of the matrix is a complex number with real and imaginary parts

If we could fit each element of the matrix in a square of 1 cm by 1 cm…

7.654321E+02 + j 1.234567E-03

1 cm

1 cm

15




How Large are Large Matrix Equations?





2013

670 Million Unknowns Scattering results are compared to analytical values to demonstrate the accuracy of the numerical solution.

340!

340!





2014


880!



LLeevveenntt Gürel UPC, Barcelona, Spain 1.1 Billion Unknowns


2014


1000!

16




Nanomaterials: Metamaterials Split-Ring Resonators




SRR Walls dB

dB dB

1 Layer

4 Layer 2 Layer

Pow

er T

rans

mis

sion

(dB

)

Frequency (GHz)

Meta Property: Stop band in frequency!

No Pass

Pass




Special Configurations

Closed-Ring Resonators (CRRs) Thin Wire Array (TWA)

All pass! No pass!




Composite Metamaterial (CMM)

Split-Ring Resonators (SRRs) Thin Wire Array (TWA)

17




CMM Walls dB

dB dB

1 Layer

4 Layer 2 Layer

Pow

er T

rans

mis

sion

(dB

)

Frequency (GHz)

Meta Property: Pass band in frequency!

No Pass

Pass




109/26

Red Blood Cells (RBCs)

Ordinary (Healthy) Red Blood Cell

Macrocyte (Macrocytosis)

Microcyte (Microcytosis)

Spherocyte (Spherocytosis) Sickle Cell

(Sickle Cell Anemia)



LLeevveenntt Gürel UPC, Barcelona, Spain Flow Cytometry

Blood sample

Forward scattering

Side scattering Backscattering

Source: http://nirfriedmanlab.blogspot.in/2010_04_01_archive.html



LLeevveenntt Gürel UPC, Barcelona, Spain Scattering from and Imaging of

Red Blood Cells Electromagnetic fields

18




Forward Scattering

No Detection

Detection of an Extra-Ordinary Cell





Forward Scattering

No Detection



An extra-ordinary cell may have normal

SCS values

An extra-ordinary cell can be detected for some orientations

An extra-ordinary cell can be detected for all orientations




Decision Chart




SOLUTION OF LARGE

COMPUTATIONAL ELECTROMAGNETICS

PROBLEMS

Mathematics Integral Equations Numerical Analysis

Linear Algebra Iterative Solvers Preconditioners

Computer Engineering Parallel Architectures Multicore Processors

Computer Science Fast Algorithms

Parallel Programming

Geometry Modelling 3-D CAD Modelling

Meshing

Physics Electromagnetic Theory Radiation and Scattering

Multi-Disciplinary Approach

Documents

Radar Problem Real-Life Problem Electromagnetic Modeling