2
Part I Introduction to Stochastic Recursive Algorithms

[Lecture Notes in Control and Information Sciences] Stochastic Recursive Algorithms for Optimization Volume 434 || || Front_matter

  • Upload
    la

  • View
    213

  • Download
    1

Embed Size (px)

Citation preview

Part IIntroduction to Stochastic Recursive

Algorithms

Stochastic recursive algorithms are one of the most important tools for problemsof stochastic optimization. In recent times, an important class of such algorithmsthat are based on the simultaneous perturbation technique has become popular be-cause of their superior computational time performance in converging to an opti-mum point. This has resulted in a flurry of research activity on stochastic algorithmsthat involve simultaneous perturbation.

This part of the book consists of three chapters. Chapter 1 gives an introduction tostochastic optimization problems and provides a motivation of where such problemsarise and why they are important. It also provides an overview of the remainingchapters.

Chapter 2 discusses some of the well-known deterministic algorithms for opti-mization. Stochastic recursive algorithms turn out to be the stochastic analogs ofthese algorithms.

The basic stochastic recursive algorithm is the Robbins and Monro scheme. It isfound to be applicable in a wide variety of settings, in particular, stochastic optimiza-tion. In Chapter 3, we discuss in detail the Robbins-Monro algorithm and analyzeits convergence. The Robbins-Monro scheme (so named after its inventors, Robbinsand Monro) is normally applicable when the objective function is an expectationof a noisy cost objective. Many times, one is faced with a problem of optimizing along-run average cost objective in order to, say, optimize a steady-state system per-formance. Multi-timescale stochastic approximation plays an important role in suchscenarios. We also present in Chapter 3, a general two-timescale stochastic recursivescheme and present its convergence analysis under general conditions.