Embed Size (px)
Citation preview
ReadmeHPC Word-Association
Lab version: 1.0.0
Last updated: 5/5/2023
Contents
OVERVIEW................................................................................................................................................. 3
GETTING STARTED................................................................................................................................... 5Task 1 – Creating an SQL Azure Server................................................................................................5
Task 2 – Inspecting the Corpus Loader Project....................................................................................9
Task 3 – Inspect and Configure the Corpus Loader Configuration File.................................................9
Task 4 – Inspecting the WCF Service Projects....................................................................................10
Task 5 – Inspecting the WCF Service Configuration...........................................................................11
Task 6 – Inspecting the Client Configuration......................................................................................12
Task 7 – Inspecting the Client Code for Calling the WordAssociation Service....................................14
DEPLOYMENT.......................................................................................................................................... 16Task 1 – Run the WordAssociation.Tools.CorpusLoader Utility.........................................................16
Task 2 – Copying the WCF Service Configuration to the Head Node.................................................17
Task 3 – Deploying the WCF Service to Windows Azure Nodes.........................................................17
RUNNING THE CLIENT............................................................................................................................ 19Task 1 – Verifying Cluster State.........................................................................................................19
Task 2 – Running the Client Application............................................................................................20
INSPECTING THE SERVICE OUTPUT....................................................................................................22Task 1 – Inspecting the Job’s Trace Files............................................................................................22
SUMMARY................................................................................................................................................ 24
Overview
The HPC word-association service demonstrates how to build, deploy, and execute an HPC SOA application that can run in Windows Azure. This sample includes the following projects:
WordAssociation.Tools.CorpusLoader: This project is designed for the one-time loading of the Brown Corpus into an SQL Azure database.
WordAssociation.Service: This project contains the WCF service that will be deployed to the Windows HPC Server 2008 R2 SP2 cluster as an SOA application.
WordAssociation.Dal: This Entity Framework project is a shared data access infrastructure layer for both the WordAssociation.Tools.CorpusLoader and WordAssociation.Service projects.
WordAssociation.ConsoleClient: This project contains the client application for the WordAssociation service. This client application demonstrates how to use the durable session with the BrokerClient class.
The architecture of the sample and the steps of its execution are depicted in Figure 1:
Figure 1Architecture of the Word Association sample
1. The Corpus Loader client application deploys the Brown Corpus to an SQL Azure database.
2. The client application submits a service job to the HPC Job Scheduler.
3. The head node starts the HPC service host in each of the Windows Azure nodes, communicates with the WCF broker node to get a list of available brokers and queues, and then returns the session information to the client application, including information for connecting to the broker node.
4. The client application sends requests to the word-association service through the WCF broker node, each request containing a list of words to match.
5. The word-association service downloads pieces of the Brown Corpus database from the SQL Azure database and runs the linguistic word-association algorithm.
6. The client application polls the message queue containing the results of the word-association service, aggregates the results, and outputs the summary.
Key Features
This sample demonstrates the following:
Creating and using an SQL Azure database.
Creating a WCF service and configuring it to run in an HPC cluster.
Deploying a WCF service to Windows Azure nodes.
Creating a client HPC SOA application that calls the WCF service with a durable session.
Getting Started
The following prerequisites are required for running the sample:
On the HPC Cluster’s head node:
◦ HPC Pack 2008 R2 SP2
On the Development machine:
◦ Microsoft Visual Studio 2010
◦ Windows Azure SDK and Windows Azure Tools for Microsoft Visual Studio (March 2011)
◦ Microsoft Sync Framework 2.1 Software Development Kit (SDK) (x86 version)
◦ HPC Pack 2008 R2 SP2 Client Utilities
To run this sample, you need to have administrative access to your HPC cluster’s head node and WCF broker node, and you must have a valid Windows Azure account, a Windows Azure worker node template defined in your head node, and several Windows Azure worker nodes in the HPC cluster that are started and online. Follow the Deploying Windows Azure Worker Nodes in Windows HPC Server 2008 R2 Step-by-Step Guide on TechNet for further information.
Note: Although this solution was built using Visual Studio 2010, the WordAssociation.Service project uses .NET Framework 3.5; this is because the HPC cluster currently cannot host WCF services compiled for .NET Framework 4.
Task 1 – Creating an SQL Azure Server
In this task, you will create a new SQL Azure server.
1. Log on to the Windows Azure Platform Management Portal. Select database in the left lower pane, as shown in Figure 2:
Figure 2Selecting the Database section from the bottom-left options pane
2. In the main pane, which is titled Getting Started, select Create a new SQL Azure Server, as shown in Figure 3:
Figure 3Creating a new SQL Azure Server
3. In the Create Server dialog box, select the Windows Azure account you wish to use, as shown in Figure 4, and click next.
Figure 4Selecting a Windows Azure subscription
4. In the next window of the dialog box, select a region for the SQL Azure server, as shown in Figure 5, to match the region used in the Windows Azure node template in the HPC cluster.
Figure 5Selecting a region
5. In the next window of the dialog box, create an administrator login name and password, as shown in Figure 6:
Figure 6Setting up the Administrator login
6. In the final window of the dialog box, configure the firewall rules, as shown in Figure 7. Select Allow other Windows Azure services to access this server, and create firewall rules that allow access to all the on-premises machines from which you intend to access this database.
Figure 7Setting firewall rules
Task 2 – Inspecting the Corpus Loader Project
In this task, you will inspect the corpus loader project used to deploy the “Brown Corpus” to an SQL Azure database.
1. Open Microsoft Visual Studio 2010 from Start | All Programs | Microsoft Visual Studio 2010 | Microsoft Visual Studio 2010.
2. Open the WordAssociation.sln solution file located in the WordAssociation\Source folder. In the Solution Explorer window, expand the WordAssociation.Tools.CorpusLoader project node, as shown in Figure 8:
Figure 8The WordAssociation.Tools.CorpusLoader project
3. Open the Program.cs file and review the contents of the LoadCorpus method. This function has two sections:
a. The first section uses the Corpus.edmx.sql file to create the Corpus database in SQL Azure and the tables that will hold the Brown Corpus data.
b. The second part synchronizes the Corpus.sdf file (a local SQLCE database containing the Brown Corpus) with the SQL Azure Corpus database.
Task 3 – Inspect and Configure the Corpus Loader Configuration File
In this task, you will inspect the Corpus Loader project’s App.config file and configure it to use the SQL Azure database you created.
1. In the WordAssociation.Tools.CorpusLoader project, open the App.config file and look for the <connectionStrings> section.
App.config
<connectionStrings>
<add name="CorpusEntities" connectionString="metadata=res://*/Corpus.csdl|res://*/Corpus.ssdl|res://*/Corpus.msl;provider=System.Data.SqlClient;provider connection string=" data source=[serverName].database.windows.net;initial catalog=Corpus;user id=[LoginForDb]@[serverName];password=[Password];App=EntityFramework"" providerName="System.Data.EntityClient" /> <add name="Master" connectionString="Server=[serverName].database.windows.net;Database=master;User ID=[LoginForDb]@[serverName]; Password=[password]; Trusted_Connection=False; Encrypt=True;"/></connectionStrings>
The following fields need to be configured in each of the connection strings (CorpusEntities and Master) to enable connections to the database server you created in Task 1.
a. [serverName]. This field should be replaced with the name of the server you created.
b. [LoginForDb]. This field should be replaced with the administrator login name you created.
c. [Password]. This field should be replaced with the administrator password you created.
Task 4 – Inspecting the WCF Service Projects
In this task, you will inspect the WCF service project to see the contract and implementation of the service.
1. Open Microsoft Visual Studio 2010 from Start | All Programs | Microsoft Visual Studio 2010 | Microsoft Visual Studio 2010.
2. Open the WordAssociation.sln solution file, which is located in the WordAssociation\Source folder. In the Solution Explorer window, expand the WordAssociation.Service project node, as shown in Figure 9:
Figure 9The WordAssociation.Service project
3. Open the IWordAssociationService.cs file and inspect the service contract. The service contract contains only one method: the GetTextMatches method.
4. Open the WordAssociationService.cs file and inspect the contents of the service. The service implementation is a linguistic word-association algorithm – that is, it looks for the most common words that appear in close proximity to each other in a given text (examples of word-associations from common speech are ‘bread’ and ‘butter’ or ‘life’ and ‘death’). For this example, the Brown University Standard Corpus of Present-Day American English will be used as input.
a. Note the Console.WriteLine method call. This call will output a message, which you will be able to see in the task results window in the HPC 2008 R2 Cluster Manager application.
b. Also note the ServiceContext.Logger.TraceEvent method call (in the MatchInText method). This method call outputs debug information to a local trace file, which can be collected after the job finishes running.
Task 5 – Inspecting the WCF Service Configuration
In this task, you will inspect the contents of the service’s configuration file. The service configuration file allows you to set how the HPC service host locates your service and hosts it. The configuration file also includes information on how to expose the service through the WCF broker node.
1. In the WordAssociation.Service project, open the WordAssociationService.cs file and locate the following constant:
C#
public const string ConnectionString = @"metadata=res://*/Corpus.csdl|res://*/Corpus.ssdl|res://*/Corpus.msl; provider=System.Data.SqlClient; provider connection string=""data source=[serverName].database.windows.net; initial catalog=Corpus;user id=[LoginForDb]@[serverName]; password=[Password];App=EntityFramework""";
The following fields need to be configured to enable connections to the database server you created in Task 1.
a. [serverName]. This field should be replaced with the name of the server you created.
b. [LoginForDb]. This field should be replaced with the administrator login name you created.
c. [Password]. This field should be replaced with the administrator password you created.
2. In the WordAssociation.Service project, open the WordAssociation.config file. Look for the <microsoft.Hpc.Session.ServiceRegistration> XML element:
SquareService.config
<microsoft.Hpc.Session.ServiceRegistration> <service assembly="WordAssociation.Service.dll" contract="WordAssociation.Service.IWordAssociationService" type="WordAssociation.Service.WordAssociationService" includeExceptionDetailInFaults="true" maxConcurrentCalls="0" serviceInitializationTimeout="60000" stdError="" maxMessageSize="65536"> </service></microsoft.Hpc.Session.ServiceRegistration>
3. The following attributes are required to configure the hosting of the service:
a. assembly. Defines the location of the service’s assembly file, which should contain both the service contract and the service implementation. When running WCF services in Windows Azure nodes, only the name of the assembly needs to be set.
b. contract. Defines the fully qualified name of the service contract.
c. type. Defines the fully qualified name of the service’s implementation class.
4. Inside the WordAssociation.config file, locate the <brokerServiceAddresses> XML element under the <microsoft.Hpc.Broker>\<services> element. Each of the <add> XML elements defines a base address that the WCF broker node uses to listen for client service requests.
5. Inside the WordAssociation.config file, locate the <system.serviceModel> XML element and inspect its contents. This XML element contains the configuration for the broker service’s message logging as well as the binding configuration for the endpoints exposed by the broker service.
Task 6 – Inspecting the Client Configuration
In this task, you will explore the client configuration used to create the new service job (also referred to as session).
1. In the Solution Explorer window, expand the WordAssociation.ConsoleClient project node, as shown in Figure 10:
Figure 10The WordAssociation.ConsoleClient project
2. In the WordAssociation.ConsoleClient project, open the app.config file and locate the <appSettings> XML element:
C#
<appSettings> <add key="HeadNodeName" value="MyHeadNode"/> <add key="SoaServiceName" value="WordAssociation"/> <add key="NodeGroup" value="AzureNodes"/></appSettings>
3. Set the HeadNodeName key to match the name of the head node machine. If you intend to test this sample with on-premises nodes in addition to Windows Azure nodes, change the name of the NodeGroup key to that of the desired node group.
Note: There is no need to change the SoaServiceName key, as this attribute is currently set to the name of the tested service, which is WordAssociation.
4. Still in the App.config file, locate the <connectionStrings> section:
App.config
<connectionStrings> <add name="CorpusEntities" connectionString="metadata=res://*/Corpus.csdl|res://*/Corpus.ssdl|res://*/Corpus.msl;provider=System.Data.SqlClient;provider connection string=" data source=[serverName].database.windows.net;initial catalog=Corpus;user id=[LoginForDb]@[serverName];password=[Password];App=EntityFramework"" providerName="System.Data.EntityClient" /></connectionStrings>
The following fields need to be configured to enable connections to the database server you created in Task 1.
a. [serverName]. This field should be replaced with the name of the server you created.
b. [LoginForDb]. This field should be replaced with the administrator login name you created.
c. [Password]. This field should be replaced with the administrator password you created.
Task 7 – Inspecting the Client Code for Calling the WordAssociation Service
In this task, you will examine the client code that creates a durable session, sends requests to the WCF service hosted in the HPC cluster, and polls the WCF broker node for completed service calls.
5. In the WordAssociation.ConsoleClient project’s Program.cs file, locate the AnalyzeFile method and examine its contents.
6. The AnalyzeFile method creates a new Microsoft.Hpc.Scheduler.Session.SessionStartInfo object and then sets its properties, as shown in the following code snippet:
C#
// The name of the head node in the cluster.string schedulerName = ConfigurationManager.AppSettings["HeadNodeName"];
// The name of the called service.string serviceName = ConfigurationManager.AppSettings["SoaServiceName"];
SessionStartInfo info = new SessionStartInfo(schedulerName, serviceName);
info.Secure = false;
info.NodeGroupList.Add( ConfigurationManager.AppSettings["NodeGroup"]);
Note: You can also set other properties of the SessionStartInfo object to configure how the job executes. For instance, you can set the SessionResourceUnitType property to SessionUnitType.Cores, and then set the MinimumUnits and MaximumUnits to define how many cores will be available for each hosted service.
7. After setting the session start information, the AnalyzeFile method uses the Microsoft.Hpc.Scheduler.Session.SessionStartInfo type to define how to start the durable session created by the Microsoft.Hpc.Scheduler.Session.DurableSession.CreateSession method. Once the session is created, the AnalyzeFile method creates a
BrokerClient<IWordAssociationService> object that is used both to send service requests to the WCF Broker node and to poll the broker node for responses with the PollBroker method:
C#
using (var session = DurableSession.CreateSession(info)){ using (var client = new BrokerClient<IWordAssociationService>(session)) { // creating a long running task for polling the WCF broker node PollBroker(result, countdown, client);
foreach (var text in texts) { client.SendRequest<GetTextMatchesRequest>( new GetTextMatchesRequest(text.Id, words)); } client.EndRequests(); countdown.Wait(); } session.Close();}
8. Build the WordAssociation solution.
Deployment
In order to run this sample, you will need to do the following:
Run the WordAssociation.Tools.CorpusLoader utility to load the baseline data to the Azure database you created in Task 1 of the Getting Started section.
Deploy the WCF service assemblies and configuration to your head node and Windows Azure nodes.
To copy applications to each of the Windows Azure nodes, you will use the hpcpack command. This command can be used for packaging applications, uploading them to Windows Azure storage, and deploying packages to every Windows Azure worker node in the cluster, as shown in Figure 11:
Figure 11Using the hpcpack command
Task 1 – Run the WordAssociation.Tools.CorpusLoader Utility
In this task, you will deploy the Brown Corpus data to the SQL Azure database server you previously created, using the WordAssociation.Tools.CorpusLoader utility.
1. Open Windows Explorer, navigate to the WordAssociation lab folder, and from there navigate to the Source\WordAssociation.Tools.CorpusLoader\bin\Debug folder.
2. Run the WordAssociation.Tools.CorpusLoader.exe executable.
Task 2 – Copying the WCF Service Configuration to the Head Node
In this task, you will copy the WCF service configuration file to the head node’s service registration folder. This allows you to use the HPC 2008 R2 Cluster Manager application to diagnose the service deployment and create new service tasks. It also allows clients to create new sessions (interactive and durable) against the head node.
1. Open the command prompt window from Start | All Programs | Microsoft Visual Studio 2010 | Visual Studio Tools | Visual Studio Command Prompt (2010).
2. Navigate to the WordAssociation lab folder, and from there navigate to the Source\WordAssociation.Service\bin\Debug folder.
3. Run the following command to copy the service configuration file to the head node:
CMD
xcopy WordAssociation.config \\MyHeadNode\HpcServiceRegistration\ /Y
Note: Replace the MyHeadNode machine name with the name of your head node machine. The HpcServiceRegistration share is created when the head node is installed.
Task 3 – Deploying the WCF Service to Windows Azure Nodes
In this task, you will deploy the WCF service to your Windows Azure nodes.
1. Open the command prompt window from Start | All Programs | Microsoft Visual Studio 2010 | Visual Studio Tools | Visual Studio Command Prompt (2010).
2. Navigate to the WordAssociation lab folder, and from there navigate to the Source\ WordAssociation.Service folder.
3. Run the following command to create a deployment package for the service storage (this step is illustrated in step 1 of Figure 11):
CMD
hpcpack create WordAssociation.zip bin\debug\
Note: Do not change the name of the package; it must match that of the service configuration file and the service name used by the client application.
4. Run the following command to upload the deployment package to the Windows Azure package storage (this step is illustrated in step 2 of Figure 11):
CMD
hpcpack upload WordAssociation.zip /nodetemplate:"Azure node template"
Note: Change the value of the nodetemplate parameter to the name of your Windows Azure node template.
5. If you already have Windows Azure nodes started in the HPC cluster, you need to copy the new package to them. To sync the Windows Azure nodes with the new packages stored in the application packages blob, run the following command (this step is illustrated in step 3 of Figure 11):
CMD
clusrun /nodegroup:AzureNodes hpcsync
Running the Client
Task 1 – Verifying Cluster State
In this task, you will verify that the nodes in your cluster are online.
1. Open the HPC 2008 R2 Cluster Manager application from Start | All Programs | Microsoft HPC Pack 2008 R2 | HPC Cluster Manager.
2. In the Cluster Manager application, enter the Node Management section and verify that the Windows Azure nodes in the cluster are online, as shown in Figure 12:
Figure 12Verifying the state of the Windows Azure nodes
3. Enter the Configuration section, select the Services option from the configuration list, and run the SOA Service Loading Diagnostics Test on the WordAssociation service, as shown in Figure 13:
Figure 13Running service load diagnostics
4. In the Run Diagnostic Tests window, select the AzureNodes node group from the Nodes in this group drop-down list, check the Go to Test Results to see test progress and results checkbox, and then click Run.
5. In the Diagnostics window, wait for the test to complete and verify its success.
Note: If the service diagnostic fails, view the diagnostic information and perform the required fix. If this involves changing the service configuration or recompiling the service assembly, you will need to redeploy your service before running the diagnostics again.
Task 2 – Running the Client Application
In this task, you will test the word-association service in the HPC cluster by running the client application.
1. Open Microsoft Visual Studio 2010 from Start | All Programs | Microsoft Visual Studio 2010 | Microsoft Visual Studio 2010.
2. Open the WordAssociation.sln solution file located in the WordAssociation\Source folder.
3. In the Solution Explorer window, set the WordAssociation.ConsoleClient project as the startup project.
4. Build the solution and run the WordAssociation.ConsoleClient project. You should see the following output at first:
Figure 14Viewing the ConsoleClient output
5. The client application will submit a new service job to the HPC Job Scheduler, and after a few minutes you should be able to see the results of the first run, as shown in Figure 15:
Figure 15Results for the adventure-related text
6. Examine the results shown in the above figure; they indicate that the genre that has the most word-association matches for the adventure-related text is the Fiction: Adventure genre.
7. After analyzing the first file, the application will continue analyzing the other 11 data files (essays, general fiction, hobbies, etc.), outputting the results for each type of content. Wait until all the jobs are completed.
Inspecting the Service Output
Task 1 – Inspecting the Job’s Trace Files
In this task, you will create the trace output for the finished job and inspect the trace files created for each node.
1. Open the HPC 2008 R2 Cluster Manager application from Start | All Programs | Microsoft HPC Pack 2008 R2 | HPC Cluster Manager.
2. In the Cluster Manager application, enter the Job Management section, locate the jobs named WordAssociation - WCF service, select the last job to finish, and then click Collect Trace… from the Actions section, as shown in Figure 16:
Figure 16Collecting trace Information
3. In the Collect Trace window, specify the target directory to which the trace files will be saved, and click OK. After the operation completes, the target directory will open in a Windows Explorer window.
4. Open each of the trace files by double-clicking it. The trace files will open in the Microsoft Service Trace Viewer application; use the application to look at the trace outputted by the service.
Figure 17Viewing the contents of the trace file
Summary
After running the square service sample you should have learned the following:
How to create a WCF service that can run inside Windows Azure nodes.
How to create a database in SQL Server Azure and access it from applications.
How to create a configuration file for the WCF service and deploy it to the head node.
How to deploy a WCF service assembly to Windows Azure nodes.
How to use a durable session to call the WCF service.
How to diagnose your service and inspect the trace output of a service task.
