CS4234Optimiz(s)ation Algorithms

S1 AY21/22

Welcome!

Optimization Algorithms ('z'/American and 's'/British)– https://www.comp.nus.edu.sg/~stevenha/cs4234.html

Materials updated from 6 AYs ago, has branched out:– https://www.comp.nus.edu.sg/~gilbert/CS4234/

Instructor: Steven Halim– Office: COM2-03-37 (unless WFH)– Office Phone: not used, use email/Discord– Email: stevenhalim@gmail.com– Discord: “the lecturer”

CS4234 Overview (1)

• Read my IOI stuff (IOI 2021 and IOI 2020)or my ICPC stuff (ICPC Jakarta Online 2020)or my achievements (this news, NDA18, ATEA, or this list)

• Social Media:– On Facebook (search my gmail),– On Instagram and Telegram

(steven_halim_7)• Has 1 wife (Grace) and

3 'J' kids (Jane, Joshua, Jemimah)• Wish for CS4234 S1 AY21/22:

– To restore my ≥ 4.3 teaching feedback• It fell to 4.1 due to a very busy IOI 2020 last year

CS4234 Overview (2)

Finally…

Tutor: Me again…– I am unable to secure any suitable TA…

• Predominantly due to my busyness with the IOI 2021pre/actual/post event activities in May/June/July 2021

• Hopefully any of you who are still year 3 now (or younger) nowand scored A- or above in CS4234 this semestercan keep this TA vacancy in mind for next S1 AY 2022/23…

– Slots: All F2F, at SR_LT19 next door (of this LT19)• T1: Mon, 11am-12noon• T2: Mon, 2-3pm• T3: Mon, 5-6pm

– If sick (MC), kena QO (self or immediate family member), etc• Show me a valid proof, you will be excused without penalty,

but read the modal answers carefullyand ask me personally if you have further questions

CS4234 Overview (3)

Yes I am in LT19 today and actually prefer to do F2F(hopefully most if not all of you)

Rough count, about 20/26 onsite today

Yes… but I still prefer online..(hopefully minority or best: 0)

(lectures are recorded)(but onsite tutorial attendance...??)

Rough count, about 6/26 onsite today

I am still overseas/SHN/LOA/others(so canNOT be in LT19 today)

(let me know when you will be onsite?)

Rough count, about 4 on SHN this week

CS4234 Overview – Survey (1)Among those in LT19…, raise your hand (obviously the bottom right cell cannot be filled in now)

Time to do attendance taking • https://inetapps.nus.edu.sg/ctr/HomeYou need to:• Declare your temperature regularly• Do first ART test soon (ignore if you have done so)• Should *not* be on SHN/LOA at the moment

Before I forget (check point 1)

Optimization:Find the minimum/maximum:

Discrete: a collection of items

Combinatorial: a collection of items generated

by counting, combining, and enumerating

Examples:Integers

Graphs and Trees

Similar structures…

Optimization Algorithms

• Searching• Shortest Paths

– 25=32 different (how to count?) (acyclic) simple paths from 010• With a correct algorithm, which is ____, we can solve this in O(V+E)

• Minimum Spanning Tree– 44-2 = 42 = 16 different (how to count?) spanning trees

• With a correct algorithm, which is _________,we can solve this in O(E log V)

Search Space Size != DifficultyA = [5, 4, 2, 7, 3, 6, 1], O(N) to find-min, how?

Sorted-A = [1, 2, 3, 4, 5, 6, 7], O(1), how?

Find the “best” item in a large set of items:Problem Set of items Size DifficultySearching List of integers Linear Easy

Shortest Path All paths in a graph Exponential Easy

Minimum Spanning Tree All spanning trees Exponential Easy

Steiner Tree All steiner trees Exponential Hard

Travelling Salesman All possible tours Exponential Hard

Matching All possible matchings Exponential Easy

Bipartite Vertex Cover All possible covers Exponential Easy

Vertex Cover All possible covers Exponential Hard

Maximum Clique All possible subsets Exponential Hard

Combinatorial Optimization

Find the “best” item in a large set of items:Problem DifficultyMaintain student records Easy

Exam timetable scheduling Hard

Program halting problem Impossible

Data compression Easy

VLSI chip layout Hard

Job assignment problem Easy

Computer deadlock problem Easy

Finding patterns in a database Easy

Combinatorial Optimization

Operations Research:How to make better decisions (e.g., maximize profit)

Project planning / critical path analysisFacility location: where to open stores / plantsFloorplanning: layout of factory or computer chipsSupply chain managementBerth assignment problem (BAP): port managementAssignment problems (e.g., weapon target assignment)Routing / transportation problems: buses, subways, trucking.Airline ticket pricing

Combinatorial Optimization

Optimization:Find the minimum/maximum:

Discrete: a collection of items

Combinatorial: a collection of items generated

by counting, combining, and enumerating

Continuous: given a function f(x), find the

vector x that maximizes f(x)Calculus stuffs, not the focus of CS4234

Optimization Algorithms

Optimization Algorithms

Goals: Algorithmic1. Design (problem solving)

2. Analysis (rigorous, deep understanding)

3. Implementation (able to put it to use)

CS3230

CS4234

“If you need your software to run twice as fast, hire better programmers.

But if you need your software to run more than twice as fast, use a better algorithm.”

-- Software Lead at Microsoft

Now the sixth iteration under Steven– First offered in S1 AY16/17, +ve feedback

– Quite OK also in S1 AY17/18, AY18/19, AY19/20– Poor in S1 AY20/21

• Mostly due to Steven missing for *most* part of September 2020 due to IOI 2020…

– https://ioi2020.sg/schedule/

– Subset of CS5234 (which has also evolved)• CS5234: Combinatorial and Graph Algorithms• More general: all sorts of optimization (not only graphs)• More specific: just optimization

CS4234 : Optimization Algorithms

Target students:– Advanced (3rd or 4th year) undergraduates

• There are (a few) master (5)/local? exchange (7) students– And also, ex CS3233 students (6)

– (Very) interested in algorithms– Interested in tools for solving hard problems

Prerequisites: – CS3230 (Analysis of Algorithms) CS2040 CS1231

– Mathematical fundamentals (currently MA1101R, but good performance in CS1231 is needed too)

CS4234 : Students

I took CS3233/join NUS ICPC before

1. Fidella Widjojo2. Gabriel Goh Kheng Lin3. Ganapathy Sanath Balaji4. Guan Yang Ze5. Lim Li6. Sherman Lim Jun HongYeah that’s all

I took CS2040C with you before(skip if you then also took CS3233)

1. Chen Chao

Do I miss anyone?

I know you from other activity/ies (elaborate)1. Agnes Natasya (from CS2040S)2. Audrey Felicio Anwar (helped with Moscow Workshop 2019)

Do I miss anyone?

CS4234 Overview – Survey (3)Do you know me before?

(just raise your hand and I roughly edit this slide myself)

I am currently in Kattis top 100https://open.kattis.com/ranklist

94. Ganapathy Sanath Balaji99. Guan Yang Ze

I am not in top 100 but top 50 herehttps://open.kattis.com/universities/nus.edu.sg

201. Koh Rayson211. Sherman Lim Jun Hong

I have more than 500 Kattis points actually…396. Fidella Widjojo437. Lim Li813. Audrey Felicio Anwar826. Gabriel Goh Kheng Lin

Do I miss anyone else?

CS4234 Overview – Survey (4)Top Kattis users

(just raise your hand and I roughly edit this slide myself)

Discord channel (I can be anonymous)

Many

Facebook post (no longer used)

From 2021 onwards

Weekly Zoom sessions with Steven (date/time TBC)

TBC… may not be needed

CS4234 Overview – Survey (-)e-Consultation Preference

(give a tick/anonymous is ok on your preferred means for this class)

You must already know these (use https://visualgo.net to review):• Data Structures (with Analysis)

– Linked Lists, Stacks, Queues,

– (Binary) Heaps and Priority Queues,

– Binary Search Trees, Balanced BSTs

• Algorithm Design Paradigms (with Analysis)– Standard Sorting and Searching algorithms,

– Complete Search, Divide-and-Conquer,Greedy Algorithms, Dynamic Programming,

– Graph traversal algorithms: DFS, BFS,

– MST Algorithms and Shortest Path Algorithms,

Pre-requisites (1)

You must already know these (use https://visualgo.net to review):• Analysis of Algorithms

– Expertise with Big-O, , notations

– Summation of series, Master Theorem

– Competent with Algorithmic Analysis:

Quicksort, Heapsort, Divide-and-Conquer algorithms

DFS, BFS, MST & Shortest Paths algorithms

Pre-requisites (2)

• https://nus.kattis.com/sessions/ng4cwg, graded this time (5% :O)– So that I can quickly be alerted on any “Prerequisites” issue involving some of you

– This is an elective level 4 algorithm module after all…• There is 1 CS2040C S2 AY20/21 exam question among the 5,

which is ______– I expect everyone to AC this by the end of Wk2, eventually

– It involves a simple data structure taught in CS2040 level, which is _______

– And a problem solving paradigm taught in CS2040+CS3230 level, which is _______• There are ≥ 1 tasks involving spanning trees (frequently used in CS4234):

– SSSP (also produces a spanning tree), which is/are _______• Use this algorithm _______

– MST (obviously produces a spanning tree), which is _______• Use this algorithm _______

– Graph Traversal (that produces a spanning tree), which is _______• Use this algorithm (or that algorithm) _______

PS1 – Prerequisites (1)

• There is 1 task involving Dynamic Programming (DP), which is ______– You should have learned DP in CS3230 and a prerequisite for the early topics of CS4234

– That task is also can be classified as an NP-hard (decision, not optimization) problem• That’s the last part of CS3230 and the topic of the first few weeks of CS4234

• The first 3 tasks are considered "high school level" tasks– They are ______, ______, and ______

– Please get AC(cepted) for at least these 3 out of 5 tasks

– Strongly reconsider taking CS4234 this sem if you struggled badly just to get these 3 ACs• The other 2 tasks are sufficiently more challenging

– Do not feel bad if they are still not AC after deadline, there will be partial marks!!

– But still of CS2040 and/or CS3230 level

– Not discussed today, but maybe during next Week 02 depending on the situation

PS1 – Prerequisites (2)

Four PS deadlines on Week 02, 04, 06, 09Must register by NOW (12 Aug 2021)! https://nus.kattis.com/courses/CS4234/CS4234_S1_AY2122/registerSecret registration key in the welcome email!

Mini project deadline on Week 13 Mid-term Test @ LT19 F2F :O, Week 08

Thursday, 07 October 2021, using lecture time Final Assessment @ ?? F2F (I asked for it)

Friday, 26 November 2021, 2.30-4.30 PM

CS4234 Important Dates

Grading20% Problem Sets15% Mini Project5% Tutorial Attendance + Participation Level25% Mid-term Test35% Final Assessment

Problem Sets– 4 sets (approximately every 2-3 weeks, 5% each)

– Algorithm design, analysis, with implementation• Using just one Online Judge : Kattis this time

CS4234 Grading

Mini-Project– Small project, triple? (41/3 = 14 groups, TBC)

– Idea: To put together some of the different ideaswe have used in the class

– Time scale: Last 3-4 weeks of the semester

– 15% Tutorial Attendance & Participation

– 5%, bell curve system, i.e.,more active in F2F class = more points

CS4234 Grading

Already ReleasedPS1-Prerequisites:

Released on Week 00 (Thursday, 05 August 2021, 08.00am) but will not due until the end of Week 02 (Saturday, 21 August 2021, 07.59am), so that you can use this information to retain/drop CS4234

As of 11am today, out of 41 students:– 24 have ≥ 3 ACs (considered safe)– 11 have registered but not much attempt (yet)– 6 MIA?

CS4234 Problem Sets

Problem Set GradingSimple scheme (graded with help of Online Judge):

5 : Perfect (coding style is ignored)

4 : Excellent, minor errors

3 : Satisfactory, mostly right

2 : Some mistakes

1 : Many mistakes (won’t be 0 if you submit)

0 : No submission (or caught by plagiarism checker)

CS4234 Problem Sets

What to submit (online judge version):C++ (17), Python (3), or Java (11) code that solves the optimization problem(s) --- I am not familiar with other languages :O

Especially if not AC at the end: Write additional comments inside source code (only the last submitted code will be read) – optional if AC:

1. Summary of the coded algorithm in English

2. Write a "proof of correctness" (hopefully your non AC is just due to coding bug) and "performance analysis" (hopefully your TLE is just due to constant factor weakness)

CS4234 Problem Sets

C++

About ~10 onsite

Python

About ~6 onsiteWarning: may TLE at some problems!

Use C++ if you can

Java

About ~3 onsite

Others (elaborate)

About ~2-3 onsite

CS4234 Overview – Survey (5)Your “mother tongue” programming language is

(just raise your hand, I will only do a rough count)

What to submit (paper version):Concise and precise answers:

Solutions should be rigorous, containing all necessary detail, but no more

Algorithm descriptions consist of:1. Summary of results/claims2. Description of algorithm in English3. Pseudocode, if helpful4. Worked example of algorithm5. Diagram / picture6. Proof of correctness and performance analysis

CS4234 Problem Sets

How to draw (graph) pictures?By hand:

Either submit hardcopy, or scan,Or take a picture with your phone!Or use a tablet / iPad…

Digitally: 1. VisuAlgo (graph) drawing feature2. Excel or Powerpoint (hmmm)3. ???

CS4234 Problem Sets

Policy on plagiarism:Do your work yourself:

Your submission should be unique, unlike anything else submitted, on the web, etc

You can discuss with other students (e.g., Discord), but…: 1. Discuss only general approach and techniques2. Just take general notes during such discussions3. Code the solution on your own4. List all collaborators in your submission

Do not ask for solutions on the web:Use web to learn techniques and to review material from class; students who ask CS4234stuffs in StackOverflow, Quora etc will be punished

CS4234 Problem Sets

Policy on plagiarism:Penalized severely:

First offense: Minimum of one letter grade lost on final grade for class (or referral to SoC disciplinary committee)

Second offense: F for the class and/or referral to SoC disciplinary committee

Do not copy/compare solutions!

PS: Do not store your coded solutions in publicrepository (e.g., GitHub, ideone.com, repl)with public setting

CS4234 Problem Sets

Introduction to Algorithms– Cormen, Leiserson, Rivest, Stein

– Strongly Recommended…

Textbooks

Algorithm Design– Kleinberg and Tardos

– Recommended…

Textbooks

Competitive Programming 4 - esp Book 2– Steven Halim, Felix Halim, and Suhendry Effendy

– Only a subset of these are relevant (especially Book 2 - Chapter 8+9)…• Also get Book 1 if you think your DS/BF/Greedy/DnC/DP/simple Graph concepts are still shaky

– Get from NUS co-op (Central Forum), or order online• https://www.nuscoop.sg/search?keywords=competitive%20programming

Textbooks

Stochastic Local Search: Foundations and Applications– Holger H. Hoos & Thomas Stützle

– For the last part of this course (a hard book to get)

Textbooks

Topics (90% same as last 5 AYs)Introduction to Combinatorial Optimization

Vertex Cover, Set Cover, Steiner Tree, TSP

Linear Programming

LPs, Relaxations, Rounding

Flows and Matching (slightly expanded)Maximum Flow (PR dropped – due to PH on Wk12),

(Bipartite) Matching (now plus weighted)

Local SearchGradient Descent, Meta-heuristics

CS4234 Topics

Time to do attendance taking • https://inetapps.nus.edu.sg/ctr/Home

You need to:• Declare your temperature regularly• Do first ART test soon (ignore if you have done so)• Should *not* be on SHN/LOA at the moment

Before I forget (check point 2)