Twister4Azure contains the following samples.
- WordCount MapReduce [Quick Start - MapReduce]
- KMeans iterative MapReduce [Quick Start Guide]
- Cap3 sequence assembly (Untested with new SDK)
- Blast sequence alignment (Untested with new SDK)
- Smith Waterman sequence distance calculation (Untested with new SDK)
- PageRank Iterative MapReduce (Under development)
1. Visual Studio 2010
2. Azure SDK (http://www.microsoft.com/windowsazure/sdk/)
All the samples are configured to run using the Azure development storage.
Running/Debugging using Azure local development fabric
Run Visual Studio 2010 as administrator (Required for deploying in the development fabric) and open the sample solution. Run/Debug the sample solution. The web based monitoring console will open in a browser window.
Deploying in Azure
In order to deploy in Azure, first you have to edit the “DataConnectionString” with your storage account information for each role in the solution. This can be performed by,
1. Right clicking on the role and select properties.
2. Go to settings
3. Double click on the “Twister4AzureWorker” Role under “Roles” in the "Twister4AzureCloud" project. Go to “Settings” tab. Click on “DataConnectionString” setting and click on “...”
in the value and select "Enter storage account credentials". Enter your azure storage account credentials. Do the same for "DiagnosticsConnectionString" and "Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString".
Make sure you perform this configuration for both the Worker Role (Twister4AzureWorker) and the Web Role(Twister4AzureUI).
4. Enter your Azure Storage Account information.
Follow these tutorials from MSDN to learn how to deploy Azure Applications from Visual Studio.
This sample counts the number of occurrences of all the words in a given set of text files stored under a Azure storage container similar to the Hadoop WordCount sample. Sample contains a web-role
based client as well as a command line client.
1. Use the data file in “sample_input.zip” file and replicate it as many times you desire (or any other text file you are interested in processing) to a container in the Blob storage. You can use an external program (eg: Cloudberry
explorer) to upload data and to manage the storage account.
To be completed.....