Introduction to the New Asynchronous API in the .NET Driver

Introduction to the New Asynchronous API

in the .NET Driver

Robert Stam

MongoDB, Inc.

Intended audience

• Familiar with C# and .NET

• Familiar with MongoDB in general

• Probably familiar with previous versions of the .NET driver

• Maybe haven’t used async programming before

• Interested in using the 2.0 version of the .NET driver

Topics

• Quick review of async programming in C#

• Benefits of async programming

• Sample data import application using the new async API

• Sample web application using the new async API

• A few advanced async related topics

Quick overview of async programming

T result = SomeMethod();

// vs

Task<T> task = SomeMethodAsync();

New async/await keywords

public async Task<T> SomeMethodAsync()

{

T result = await SomeOtherMethodAsync();

return result;

}

Benefits of async programming

• Threads are expensive (and often limited)

• A blocked thread is not doing any work

• A blocked thread is still consuming resources

• We can get more work done with fewer threads

• We can get higher throughput with the same hardware

Async does have some overhead

• The compiler transform for async methods is not free

• Measurable, but usually insignificant

• Takeaway is to keep the granularity of async methods coarse

2.0 .NET driver API is async-only

• There is a trend toward providing async only APIs

• (e.g. HttpClient)

• Microsoft is heavily promoting async programming

Sample applications

• Using flight delay data from the Department of Transportation• http://www.transtats.bts.gov/OT_Delay/OT_DelayCause1.asp

• Console application to import the data

• Web application to query the data

• Source code in github• https://github.com/rstam/SampleMongoDBApplication

Sample data

• Flights collection

• Date, Airline, Flight Number, From, To, Scheduled Times, Delays, etc…

• Airlines collection

• Code, Description

• Airports collection

• Code, Description

Data Importer Console Application

• Using async APIs in a console application

• Dropping collections

• Loading collections

• Creating indexes

Using the async API from a console application

public static void Main(string[] args)

{

try

{

MainAsync(args).GetAwaiter().GetResult();

}

catch (Exception ex)

{

Console.WriteLine("Exception: {0}", ex);

}

}

The top level MainAsync method

private static IMongoClient __client;

private static IMongoDatabase __database;

private static async Task MainAsync(string[] args)

{

var connectionString = "mongodb://localhost";

__client = new MongoClient(connectionString);

__database = __client.GetDatabase("flights");

await DropCollectionsAsync();

await LoadCollectionsAsync();

await CreateIndexesAsync();

}





{







}





{







}

Dropping collections

private static async Task DropCollectionsAsync()

{

await __database.DropCollectionAsync("flights");

await __database.DropCollectionAsync("airlines");

await __database.DropCollectionAsync("airports");

}

Loading collections

private static async Task LoadCollectionsAsync()

{

await LoadFlightsAsync();

await LoadAirlinesAsync();

await LoadAirportsAsync();

}

Loading the airlines collection

private static async Task LoadAirlinesAsync(string csvFilename)

{

var airlines = new List<Airline>();

using (var csvReader = new CsvReader(new StreamReader(csvFilename)))

{

foreach (var airline in csvReader.GetRecords<Airline>())

{

if (__seenAirlineIds.Contains(airline.Code))

{

airlines.Add(airline);

}

}

}

var collection = __database.GetCollection<Airline>("airlines");

await collection.InsertManyAsync(airlines);

}



{



{


{


{


}

}

}



}



{



{


{


{


}

}

}



}

Loading the flights collection

private static async Task LoadFlightsAsync(string csvFilename){

var collection = __database.GetCollection<Flight>("flights");

var batchSize = 1000;using (var csvReader = new CsvReader(new StreamReader(csvFilename))){

var flights = new List<Flight>();

foreach (var flight in csvReader.GetRecords<Flight>()){

PreprocessFlight(flight);flights.Add(flight);

if (flights.Count == batchSize){

await collection.InsertManyAsync(flights);flights.Clear();

}}

if (flights.Count > 0){

await collection.InsertManyAsync(flights);}

}}

Loading the flights collection

private static async Task LoadFlightsAsync(string csvFilename){


var batchSize = 1000;using (var csvReader = new CsvReader(new StreamReader(csvFilename))){

var flights = new List<Flight>();

foreach (var flight in csvReader.GetRecords<Flight>()){

PreprocessFlight(flight);flights.Add(flight);

if (flights.Count == batchSize){

await collection.InsertManyAsync(flights);flights.Clear();

}}

if (flights.Count > 0){

await collection.InsertManyAsync(flights);}

}}

Preprocessing a flight document

private static void PreprocessFlight(Flight flight)

{

if (flight.FL_DATE.HasValue)

{

flight.FL_DATE = DateTime.SpecifyKind(flight.FL_DATE.Value, DateTimeKind.Utc);

}

if (flight.AIRLINE_ID.HasValue)

{

__seenAirlineIds.Add(flight.AIRLINE_ID.Value);

}

if (flight.ORIGIN_AIRPORT_ID.HasValue)

{

__seenAirportIds.Add(flight.ORIGIN_AIRPORT_ID.Value);

}

if (flight.DEST_AIRPORT_ID.HasValue)

{

__seenAirportIds.Add(flight.DEST_AIRPORT_ID.Value);

}

}

Preprocessing a flight document

private static void PreprocessFlight(Flight flight)

{

if (flight.FL_DATE.HasValue)

{

flight.FL_DATE = DateTime.SpecifyKind(flight.FL_DATE.Value, DateTimeKind.Utc);

}

if (flight.AIRLINE_ID.HasValue)

{

__seenAirlineIds.Add(flight.AIRLINE_ID.Value);

}

if (flight.ORIGIN_AIRPORT_ID.HasValue)

{

__seenAirportIds.Add(flight.ORIGIN_AIRPORT_ID.Value);

}

if (flight.DEST_AIRPORT_ID.HasValue)

{

__seenAirportIds.Add(flight.DEST_AIRPORT_ID.Value);

}

}

Creating an index

private static async Task CreateIndexesAsync()

{

var indexKeysBuilder = Builders<Flight>.IndexKeys;

var indexKeys = indexKeysBuilder

.Ascending(f => f.FL_DATE)

.Ascending(f => f.AIRLINE_ID)

.Ascending(f => f.ORIGIN_AIRPORT_ID)

.Ascending(f => f.DEST_AIRPORT_ID);


await collection.Indexes.CreateOneAsync(indexKeys);

}

Creating an index

private static async Task CreateIndexesAsync()

{

var indexKeysBuilder = Builders<Flight>.IndexKeys;

var indexKeys = indexKeysBuilder

.Ascending(f => f.FL_DATE)

.Ascending(f => f.AIRLINE_ID)

.Ascending(f => f.ORIGIN_AIRPORT_ID)

.Ascending(f => f.DEST_AIRPORT_ID);


await collection.Indexes.CreateOneAsync(indexKeys);

}

Sample Web Application

• Two pages

• Search criteria form

• Search results

Search Criteria Form

Search Results Page

MongoClient lifecycle

public class HomeController : Controller

{

private const string __connectionString = "mongodb://localhost";

private static Lazy<MongoClient> __client =

new Lazy<MongoClient>(() => new MongoClient(__connectionString));

// ... rest of HomeController class

}

Creating the Search Criteria Form

public async Task<ActionResult> Index()

{

var viewModel = await CreateIndexViewModelAsync();

return View(viewModel);

}

public class IndexViewModel

{

public IEnumerable<Airline> Airlines { get; set; }

public IEnumerable<Airport> Airports { get; set; }

}

Creating the Search Criteria Form

public async Task<ActionResult> Index()

{

var viewModel = await CreateIndexViewModelAsync();

return View(viewModel);

}

public class IndexViewModel

{

public IEnumerable<Airline> Airlines { get; set; }

public IEnumerable<Airport> Airports { get; set; }

}

Creating the IndexViewModel

private async Task<IndexViewModel> CreateIndexViewModelAsync()

{

// note: the two queries will be run in parallel

var findAirlinesTask = FindAirlinesAsync();

var findAirportsTask = FindAirportsAsync();

await Task.WhenAll(findAirlinesTask, findAirportsTask);

return new IndexViewModel

{

Airlines = findAirlinesTask.Result.OrderBy(a => a.Description),

Airports = findAirportsTask.Result.OrderBy(a => a.Description)

};

}

Creating the IndexViewModel

private async Task<IndexViewModel> CreateIndexViewModelAsync()

{

// note: the two queries will be run in parallel

var findAirlinesTask = FindAirlinesAsync();

var findAirportsTask = FindAirportsAsync();

await Task.WhenAll(findAirlinesTask, findAirportsTask);

return new IndexViewModel

{

Airlines = findAirlinesTask.Result.OrderBy(a => a.Description),

Airports = findAirportsTask.Result.OrderBy(a => a.Description)

};

}

Finding the airlines

private async Task<IEnumerable<Airline>> FindAirlinesAsync()

{

var client = __client.Value;

var database = client.GetDatabase("flights");

var collection = database.GetCollection<Airline>("airlines");

return await collection.Find(new BsonDocument()).ToListAsync();

}

The Index view

@model WebApplication.Models.IndexViewModel

@{Layout = null;ViewBag.Title = "Flight Delays";

}

@using (Html.BeginForm("Search", "Home")){

<div>Search flight delay data:

</div><div>

@Html.Label("From Date:")@Html.TextBox("FromDate")

</div><div>

@Html.Label("To Date:")@Html.TextBox("ToDate")

</div><div>

@Html.Label("Airline:")@Html.DropDownList("AirlineId", Model.Airlines.Select(a => new SelectListItem { Value = a.Id.ToString(), Text = a.Description }), "All")

</div><div>

@Html.Label("Origin:")@Html.DropDownList("OriginId", Model.Airports.Select(a => new SelectListItem { Value = a.Id.ToString(), Text = a.Description }), "All")

</div><div>

@Html.Label("Destination:")@Html.DropDownList("DestinationId", Model.Airports.Select(a => new SelectListItem { Value = a.Id.ToString(), Text = a.Description }), "All")

</div><div>

<input type="submit" value="Search" /></div>

}

Creating the Search Results Page

[HttpPost]

public async Task<ActionResult> Search(SearchCriteriaModel criteriaModel)

{

var viewModel = await CreateSearchResultViewModelAsync(criteriaModel);

return View("SearchResultView", viewModel);

}

public class SearchResultViewModel

{

public int TotalNumberOfFlights { get; set; }

public int TotalNumberOfDelayedFlights { get; set; }

public double AverageDelayInMinutes { get; set; }

}

Creating the Search Results Page

[HttpPost]

public async Task<ActionResult> Search(SearchCriteriaModel criteriaModel)

{

var viewModel = await CreateSearchResultViewModelAsync(criteriaModel);

return View("SearchResultView", viewModel);

}

public class SearchResultViewModel

{

public int TotalNumberOfFlights { get; set; }

public int TotalNumberOfDelayedFlights { get; set; }

public double AverageDelayInMinutes { get; set; }

}

Creating the Search Results View Model

private async Task<SearchResultViewModel> CreateSearchResultViewModelAsync(SearchCriteriaModel criteriaModel)

{var client = __client.Value;var database = client.GetDatabase("flights");var collection = database.GetCollection<Flight>("flights");

var aggregateOfFlight = collection.Aggregate();

var filter = CreateFilter(criteriaModel);if (filter != null){

aggregateOfFlight = aggregateOfFlight.Match(filter);}

var aggregateOfSearchResultViewModel = aggregateOfFlight.Group(f => 1,g => new SearchResultViewModel{

TotalNumberOfFlights = g.Count(),TotalNumberOfDelayedFlights = g.Sum(f => f.ArrivalDelay > 0.0 ? 1 : 0),AverageDelayInMinutes =

(double)g.Average(f => f.ArrivalDelay > 0.0 ? (double?)f.ArrivalDelay : null)});

return await aggregateOfSearchResultViewModel.SingleAsync();}































A sample resulting aggregation pipeline

[

{ $match : {

FL_DATE : {

$gte : ISODate("2014-12-01T00:00:00Z"),

$lte : ISODate("2014-12-31T00:00:00Z") },

AIRLINE_ID : 19790, /* Delta Airlines */

ORIGIN_AIRPORT_ID : 10397, /* Atlanta Hartsfield International */

DEST_AIRPORT_ID : 12953 } /* New York La Guardia */

},

{ $group : {

_id : 1,

TotalNumberOfFlights : { $sum : 1 },

TotalNumberOfDelayedFlights :

{ $sum : { $cond : [{ $gt : ["$ARR_DELAY", 0.0] }, 1, 0] } },

AverageDelayInMinutes :

{ $avg : { $cond : [{ $gt : ["$ARR_DELAY", 0.0] }, "$ARR_DELAY", null] } } }

}

]

Creating the Search Results Filter

private FilterDefinition<Flight> CreateFilter(

SearchCriteriaModel criteriaModel)

{

var filterBuilder = Builders<Flight>.Filter;

var clauses = CreateClauses(criteriaModel, filterBuilder);

if (clauses.Count > 0)

{

return filterBuilder.And(clauses);

}

else

{

return null;

}

}

Creating the clauses

private List<FilterDefinition<Flight>> CreateClauses(SearchCriteriaModel criteriaModel,FilterDefinitionBuilder<Flight> filterBuilder)

{var clauses = new List<FilterDefinition<Flight>>();if (criteriaModel.FromDate != null){

var fromDate = DateTime.SpecifyKind(DateTime.Parse(criteriaModel.FromDate), DateTimeKind.Utc);

var clause = filterBuilder.Gte(f => f.FlightDate, fromDate);clauses.Add(clause);

}if (criteriaModel.ToDate != null){

var toDate = DateTime.SpecifyKind(DateTime.Parse(criteriaModel.ToDate), DateTimeKind.Utc);

var clause = filterBuilder.Lte(f => f.FlightDate, toDate);clauses.Add(clause);

}// code for 3 more clauses (see next slide)return clauses;

}










}










}

Creating the clauses (continued)


{// ... (continued from previous slide)

if (criteriaModel.AirlineId != null){

var clause = filterBuilder.Eq(f => f.AirlineId, criteriaModel.AirlineId.Value);clauses.Add(clause);

}if (criteriaModel.OriginId != null){

var clause = filterBuilder.Eq(f => f.OriginAirportId, criteriaModel.OriginId.Value);clauses.Add(clause);

}if (criteriaModel.DestinationId != null){

var clause = filterBuilder.Eq(f => f.DestinationAirportId, criteriaModel.DestinationId.Value);

clauses.Add(clause);}return clauses;

}

The SearchResult view

@model WebApplication.Models.SearchResultViewModel

@{

Layout = null;

}

<!DOCTYPE html>

<html>

<head>

<meta name="viewport" content="width=device-width" />

<title>SearchView</title>

</head>

<body>

<div>

Search Results:

</div>

<div>

@Model.TotalNumberOfFlights total number of flights

</div>

<div>

@Model.TotalNumberOfDelayedFlights total number of delayed flights

</div>

<div>

@Model.AverageDelayInMinutes average delay (in minutes)

</div>

<div>

@Html.ActionLink("New search", "Index")

</div>

</body>

</html>

Advanced Topics

• IAsyncCursor

• Cancellation

• Timeouts using CancellationToken

• ConfigureAwait(false)

IAsyncCursor

public interface IAsyncCursor<out TDocument> : IDisposable

{

IEnumerable<TDocument> Current { get; }

Task<bool> MoveNextAsync();

}

using (var cursor = await collection.FindAsync(filter))

{

while (await cursor.MoveNextAsync())

{

var batch = cursor.Current;

foreach (var document in batch)

{

// process document

}

}

}

IAsyncCursor

public interface IAsyncCursor<out TDocument> : IDisposable

{

IEnumerable<TDocument> Current { get; }

Task<bool> MoveNextAsync();

}

using (var cursor = await collection.FindAsync(filter))

{

while (await cursor.MoveNextAsync())

{

var batch = cursor.Current;

foreach (var document in batch)

{

// process document

}

}

}

Cancellation

using (var cancellationTokenSource = new CancellationTokenSource())

{

var cancellationToken = cancellationTokenSource.Token;

var task = SomeMethodAsync(cancellationToken);

// ...

cancellationTokenSource.Cancel(); // request cancellation

var result = await task;

}

Cancellation


{



// ...



}

Cancellation


{



// ...



}

Timeouts using CancellationToken

using (var timeoutSource = new CancellationTokenSource(timeout))

{

var cancellationToken = timeoutSource.Token;

var filter = ...;

var cursor = await collection.FindAsync(filter, cancellationToken);

}

// or: search the Web for “WithTimeout” helper method

var cursor = await collection.FindAsync(filter).WithTimeout(timeout);

Timeouts using CancellationToken

using (var timeoutSource = new CancellationTokenSource(timeout))

{

var cancellationToken = timeoutSource.Token;

var filter = ...;

var cursor = await collection.FindAsync(filter, cancellationToken);

}

// or: search the Web for “WithTimeout” helper method

var cursor = await collection.FindAsync(filter).WithTimeout(timeout);

ConfigureAwait(false)

Which thread runs the code after an await?

Depends on the captured SynchronizationContext.

We use ConfigureAwait(false) 100% of the time inside the

driver.

Improves performance. Helps prevent deadlocks.

.NET Users Birds of a Feather

Tuesday 12:20 pm to 1:50 pm

Metropolitan East Ballroom

Conclusion

Thanks for coming!

Questions?

Technology

Introduction to the New Asynchronous API in the .NET Driver