On behalf of the Microsoft High Performance Computing team I am pleased to announce that the HPC Pack 2008 R2, Windows Server 2008 R2 HPC Edition, and Windows HPC Server 2008 R2 Suite have hit their 'RTM' (Release to Manufacturing) milestone!
This third version of our product have been in development for two years, and has many new features and enhancements. Check out the 'What's New' document on TechNet and the official product site's Windows HPC Server 2008 R2 Suite overview page for more information.
New to our v3 release is having several 'editions' of the HPC Pack.
All three of those, as well as the Windows Server 2008 R2 HPC Edition operating system, are available individually. We also have a 'Windows HPC Server 2008 R2 Suite' product purchaseable through VL and OEM channels that contains the 'Enterprise' HPC Pack and the 'HPC Edition' operating system together.
Please check out the two links mentioned before; other R2 documentation is available on that TechNet site, with more on the way.
Head over to our discussion forums if you have any questions, we'll be happy to help you out!
We are pleased to announce that Release Candidate 1 (RC 1) of the Microsoft HPC Pack 2008 R2 & Windows HPC Server 2008 R2 solution is now available.
We've improved the stability and performance in this release, as well as made enhancements to features based on feedback that you guys gave us on our Beta 2. Check out a selection of the improvements by reading the 'What's New' document on TechNet.
This is your last chances to give us feedback on this version, so come and join our beta program to give it a try, and you can give us feedback (positive or negative) on it: http://connect.microsoft.com/HPC/content/content.aspx?ContentID=6923
The second service pack to the HPC Pack 2008 software is now available!
This service pack is a minor bug fixing release, it does not add any major new functionality. It does fix various reliability issues, and is suggested that all users of HPC Pack 2008 install it.
The service pack supercedes (includes the fixes for) the Post-SP1 updates that have been released:
The update installer is available at http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=9893ee85-b3c4-4ad8-ac3e-76c4478dbfa5. This can be used to manually install the service pack on any system running HPC Pack 2008 SP1 (head nodes, compute nodes, broker nodes, client machines). Starting next month the update will be available through Microsoft Update for all systems except failover/HA head node pairs (those need to be manually installed). Note, for either installation method SP1 must already be installed for this service pack to apply.
Please review the 'release notes' at http://go.microsoft.com/fwlink/?LinkId=191953 for tips on installing and a list of known issues.
The SDK has also been updated and is available at http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=13644dcd-4022-4bbc-b18f-1c9e9461d5bd.
The second service pack to the HPC Pack 2008 software is now available!
This service pack is a minor bug fixing release, it does not add any major new functionality. It does fix various reliability issues, and is suggested that all users of HPC Pack 2008 install it.
The service pack supercedes (includes the fixes for) the Post-SP1 updates that have been released:
The update installer is available at http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=9893ee85-b3c4-4ad8-ac3e-76c4478dbfa5. This can be used to manually install the service pack on any system running HPC Pack 2008 SP1 (head nodes, compute nodes, broker nodes, client machines). Starting next month the update will be available through Microsoft Update for all systems except failover/HA head node pairs (those need to be manually installed). Note, for either installation method SP1 must already be installed for this service pack to apply.
Please review the 'release notes' at http://go.microsoft.com/fwlink/?LinkId=191953 for tips on installing and a list of known issues.
The SDK has also been updated and is available at http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=13644dcd-4022-4bbc-b18f-1c9e9461d5bd.
Recently a couple of my fellow technical specialists and I were working on a Top 500 run at a major US university. We needed to create node.XML templates that associated MAC addresses with node names. It would also be nice if it could also list the rack location of the node. We wanted to deploy the cluster by rack to allow for more orderly provisioning. The client gave us a text file that they created from the LINUX configuration of the cluster. This text file contained several pieces of data for each node. There was one node entry per line in the text file. The important data was the node name, MAC address and node location. To help ease creation of the node.XML templates I wrote a c# application that takes as input a text file with per rack data and created a node template for that rack.
So it takes input like the following:
node16 00:14:5E:55:AF:D4 rack01
node17 00:14:5E:55:8E:67 rack01
node18 00:14:5E:55:A4:D9 rack01
and creates a node.xml template that looks like:
<?xml version="1.0" encoding="utf-8"?>
<Nodes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/HpcNodeConfigurationFile/2007/12">
<Node Name="node16" Domain="mshpc">
<Template Name="HPCTemplate" />
<Location Primary="rack01" />
<MacAddress>00145E55AFD4</MacAddress>
</Node>
<Node Name="node17" Domain="mshpc">
<Template Name="HPCTemplate" />
<Location Primary="rack01" />
<MacAddress>00145E558E67</MacAddress>
</Node>
<Node Name="node18" Domain="mshpc">
<Template Name="HPCTemplate" />
<Location Primary="rack01" />
<MacAddress>00145E55A4D9</MacAddress>
</Node>
</Nodes>
The node.xml template has the following key items that are created by the program:
Of course the program is expandable to allow for more data to be written to the node.xml template and I will be updating the code as needed as well as making the source available to the community to allow others to update as necessary. The domain and template name that are listed above are contained in a configuration file that is used by the application. This file app.config needs to be changed by the user to add their domain and template name. Below is what the createnodetemplate.exe.config file looks like:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<add key="domain" value="mshpc"/>
<add key="MAC delimiter" value=":"/>
<add key="Template Name" value="HPCTemplate"/>
</appSettings>
</configuration>
So the user will need to add their domain name replacing "mshpc" with the updated name in double quotes. They will also have to give a template name in double quotes. The MAC delimiter entry is currently not allowed to be changed. The MAC address must be in the 00:00:00:00:00 format with each pair separated by ':'.
The application has the following command line:
createnodetemplate infile outfile
The following command would take input data in rack11.dat and create a node template file named rack11.xml
createnodetemplate rack11.dat rack11.xml
A zip file of the executable and sample data and xml files is attached to this posting for download.
I hope the community finds this tool useful for deploying large cluster from bare metal.
Recently a couple of my fellow technical specialists and I were working on a Top 500 run at a major US university. We needed to create node.XML templates that associated MAC addresses with node names. It would also be nice if it could also list the rack location of the node. We wanted to deploy the cluster by rack to allow for more orderly provisioning. The client gave us a text file that they created from the LINUX configuration of the cluster. This text file contained several pieces of data for each node. There was one node entry per line in the text file. The important data was the node name, MAC address and node location. To help ease creation of the node.XML templates I wrote a c# application that takes as input a text file with per rack data and created a node template for that rack.
So it takes input like the following:
node16 00:14:5E:55:AF:D4 rack01
node17 00:14:5E:55:8E:67 rack01
node18 00:14:5E:55:A4:D9 rack01
and creates a node.xml template that looks like:
<?xml version="1.0" encoding="utf-8"?>
<Nodes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/HpcNodeConfigurationFile/2007/12">
<Node Name="node16" Domain="mshpc">
<Template Name="HPCTemplate" />
<Location Primary="rack01" />
<MacAddress>00145E55AFD4</MacAddress>
</Node>
<Node Name="node17" Domain="mshpc">
<Template Name="HPCTemplate" />
<Location Primary="rack01" />
<MacAddress>00145E558E67</MacAddress>
</Node>
<Node Name="node18" Domain="mshpc">
<Template Name="HPCTemplate" />
<Location Primary="rack01" />
<MacAddress>00145E55A4D9</MacAddress>
</Node>
</Nodes>
The node.xml template has the following key items that are created by the program:
Of course the program is expandable to allow for more data to be written to the node.xml template and I will be updating the code as needed as well as making the source available to the community to allow others to update as necessary. The domain and template name that are listed above are contained in a configuration file that is used by the application. This file app.config needs to be changed by the user to add their domain and template name. Below is what the createnodetemplate.exe.config file looks like:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<add key="domain" value="mshpc"/>
<add key="MAC delimiter" value=":"/>
<add key="Template Name" value="HPCTemplate"/>
</appSettings>
</configuration>
So the user will need to add their domain name replacing "mshpc" with the updated name in double quotes. They will also have to give a template name in double quotes. The MAC delimiter entry is currently not allowed to be changed. The MAC address must be in the 00:00:00:00:00 format with each pair separated by ':'.
The application has the following command line:
createnodetemplate infile outfile
The following command would take input data in rack11.dat and create a node template file named rack11.xml
createnodetemplate rack11.dat rack11.xml
A zip file of the executable and sample data and xml files is attached to this posting for download.
I hope the community finds this tool useful for deploying large cluster from bare metal.
Avoiding Timeouts
In HPC 2008 (V2) SOA, a service call lasts more than 10 minutes will cause the broker closes the connection to the client. This will cause a timeout.
Example:
- Assume call durations may be longer than 10 minutes but shorter than 30 minutes.
- My service dll is SimpleService.dll
- My service name is SOATests.SoaSimpleService
- My service contract is SOATests.ISoaSimpleService
1. Client: add the following to the client code.
client.InnerChannel.OperationTimeout = TimeSpan.FromMinutes(30);
2. Service side.
Add a <serviceAssembly>.dll.config and deploy it together with the <serviceAssembly>.dll to all nodes.
Template:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.serviceModel>
<services>
<service name="<ServiceImplName>">
<endpoint binding="netTcpBinding" bindingConfiguration="ServiceBinding"
name="tcpbinging0" contract="<ServiceContractName>" />
</service>
</services>
<bindings>
<netTcpBinding>
<binding name="ServiceBinding" receiveTimeout="00:30:00" portSharingEnabled="true">
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
</system.serviceModel>
</configuration>
Example: (file is called SimpleService.dll.config)
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="myBinding" receiveTimeout="00:30:00">
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
<services>
<service name="SOATests.SoaSimpleService">
<endpoint address="" binding="netTcpBinding"
bindingConfiguration="myBinding"
contract="SOATests.ISoaSimpleService"/>
</service>
</services>
</system.serviceModel>
</configuration>
Avoiding Exceptions
In HPC 2008 (V2) SOA, a service call that contains a big array (or list, or other type of collections), WCF might throw an exception - Exception:
The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter. The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota.'
Solution:
This is because there are some built-in limits in WCF. This can be configured through app.config file. There are 2 config files that need to be changed.
1. Client side app.config. Find the binding you are using and add attributes like following sample.
<system.serviceModel>
<behaviors>
<endpointBehaviors>
<behavior name="maxitembehavior">
<dataContractSerializer maxItemsInObjectGraph="6553600"/>
</behavior>
</endpointBehaviors>
</behaviors>
<bindings>
<netTcpBinding>
<binding name="netTcpBindingConfiguration"
maxBufferSize="99000000"
maxReceivedMessageSize="99000000">
<readerQuotas maxDepth="900000"
maxStringContentLength="900000"
maxArrayLength="900000"
maxBytesPerRead="900000"
maxNameTableCharCount="900000" />
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint name ="echosvcclient" binding ="netTcpBinding" bindingConfiguration ="netTcpBindingConfiguration" contract ="EchoSvcReference.IEchoSvc" behaviorConfiguration="maxitembehavior">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
</client>
</system.serviceModel>
2. The service assembly app.config (depending on your deployment it could be per Compute node or a file share. It’s called <service_assembly_name>.dll.config). Find the Net.tcp binding you are using and add those attributes. Following is an example:
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="EchoSvc.EchoSvcBehavior">
<dataContractSerializer maxItemsInObjectGraph="6553600" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netTcpBinding>
<binding name="myBinding"
maxBufferSize="99000000"
maxReceivedMessageSize="99000000"
receiveTimeout="00:30:00">
<readerQuotas maxDepth="900000"
maxStringContentLength="900000"
maxArrayLength="900000"
maxBytesPerRead="900000"
maxNameTableCharCount="900000" />
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
<services>
<service name="EchoSvc.EchoSvc" behaviorConfiguration="EchoSvc.EchoSvcBehavior">
<endpoint address="" binding="netTcpBinding"
bindingConfiguration="myBinding"
contract="EchoSvc.IEchoSvc"/>
</service>
</services>
</system.serviceModel>
For both configuration files, the key attribute is maxItemsInObjectGraph. But you also want to make sure the message size settings are adjusted according to the size of the object actually transported as parameter.
And here is how I create the service client from client side to use the client binding configuration and transport a big message:
using (Session session = Session.CreateSession(info))
{
EchoSvcClient client = new EchoSvcClient("echosvcclient", session.EndpointReference);
Dictionary<string, string> a = new Dictionary<string, string>();
for (int i = 0; i < 17000; i++)
a.Add(i.ToString(), "abc");
Dictionary<string, string> b = client.EchoArray(a);
client.Close();
}
Avoiding Timeouts
In HPC 2008 (V2) SOA, a service call lasts more than 10 minutes will cause the broker closes the connection to the client. This will cause a timeout.
Example:
- Assume call durations may be longer than 10 minutes but shorter than 30 minutes.
- My service dll is SimpleService.dll
- My service name is SOATests.SoaSimpleService
- My service contract is SOATests.ISoaSimpleService
1. Client: add the following to the client code.
client.InnerChannel.OperationTimeout = TimeSpan.FromMinutes(30);
2. Service side.
Add a <serviceAssembly>.dll.config and deploy it together with the <serviceAssembly>.dll to all nodes.
Template:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.serviceModel>
<services>
<service name="<ServiceImplName>">
<endpoint binding="netTcpBinding" bindingConfiguration="ServiceBinding"
name="tcpbinging0" contract="<ServiceContractName>" />
</service>
</services>
<bindings>
<netTcpBinding>
<binding name="ServiceBinding" receiveTimeout="00:30:00" portSharingEnabled="true">
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
</system.serviceModel>
</configuration>
Example: (file is called SimpleService.dll.config)
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="myBinding" receiveTimeout="00:30:00">
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
<services>
<service name="SOATests.SoaSimpleService">
<endpoint address="" binding="netTcpBinding"
bindingConfiguration="myBinding"
contract="SOATests.ISoaSimpleService"/>
</service>
</services>
</system.serviceModel>
</configuration>
Avoiding Exceptions
In HPC 2008 (V2) SOA, a service call that contains a big array (or list, or other type of collections), WCF might throw an exception - Exception:
The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter. The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota.'
Solution:
This is because there are some built-in limits in WCF. This can be configured through app.config file. There are 2 config files that need to be changed.
1. Client side app.config. Find the binding you are using and add attributes like following sample.
<system.serviceModel>
<behaviors>
<endpointBehaviors>
<behavior name="maxitembehavior">
<dataContractSerializer maxItemsInObjectGraph="6553600"/>
</behavior>
</endpointBehaviors>
</behaviors>
<bindings>
<netTcpBinding>
<binding name="netTcpBindingConfiguration"
maxBufferSize="99000000"
maxReceivedMessageSize="99000000">
<readerQuotas maxDepth="900000"
maxStringContentLength="900000"
maxArrayLength="900000"
maxBytesPerRead="900000"
maxNameTableCharCount="900000" />
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint name ="echosvcclient" binding ="netTcpBinding" bindingConfiguration ="netTcpBindingConfiguration" contract ="EchoSvcReference.IEchoSvc" behaviorConfiguration="maxitembehavior">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
</client>
</system.serviceModel>
2. The service assembly app.config (depending on your deployment it could be per Compute node or a file share. It’s called <service_assembly_name>.dll.config). Find the Net.tcp binding you are using and add those attributes. Following is an example:
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="EchoSvc.EchoSvcBehavior">
<dataContractSerializer maxItemsInObjectGraph="6553600" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netTcpBinding>
<binding name="myBinding"
maxBufferSize="99000000"
maxReceivedMessageSize="99000000"
receiveTimeout="00:30:00">
<readerQuotas maxDepth="900000"
maxStringContentLength="900000"
maxArrayLength="900000"
maxBytesPerRead="900000"
maxNameTableCharCount="900000" />
<security mode="Transport" />
</binding>
</netTcpBinding>
</bindings>
<services>
<service name="EchoSvc.EchoSvc" behaviorConfiguration="EchoSvc.EchoSvcBehavior">
<endpoint address="" binding="netTcpBinding"
bindingConfiguration="myBinding"
contract="EchoSvc.IEchoSvc"/>
</service>
</services>
</system.serviceModel>
For both configuration files, the key attribute is maxItemsInObjectGraph. But you also want to make sure the message size settings are adjusted according to the size of the object actually transported as parameter.
And here is how I create the service client from client side to use the client binding configuration and transport a big message:
using (Session session = Session.CreateSession(info))
{
EchoSvcClient client = new EchoSvcClient("echosvcclient", session.EndpointReference);
Dictionary<string, string> a = new Dictionary<string, string>();
for (int i = 0; i < 17000; i++)
a.Add(i.ToString(), "abc");
Dictionary<string, string> b = client.EchoArray(a);
client.Close();
}
About 4 months ago we released the first major pre-release build of the Microsoft HPC Pack 2008 R2 & Windows HPC Server 2008 R2 solution. Now we announce our second, Beta 2.
There are a number of enhancements in this release, a couple of them are:
This is one of your last chances to give us feedback on what we will ship later this year, so come and join our beta program to give it a try, and you can give us feedback (positive or negative) on it: http://connect.microsoft.com/HPC/content/content.aspx?ContentID=6923
About 4 months ago we released the first major pre-release build of the Microsoft HPC Pack 2008 R2 & Windows HPC Server 2008 R2 solution. Now we announce our second, Beta 2.
There are a number of enhancements in this release, a couple of them are:
This is one of your last chances to give us feedback on what we will ship later this year, so come and join our beta program to give it a try, and you can give us feedback (positive or negative) on it: http://connect.microsoft.com/HPC/content/content.aspx?ContentID=6923
Windows HPC Server เป็นอีกขั้นของระบบปฏิบัติการเซิร์ฟเวอร์ของไมโครซอฟท์ Windows HPC Server นั้นต่างจาก Windows Server รุ่นปกติ (เช่น Windows Server 2008) ตรงที่รุ่น HPC นั้นออกแบบมาสำหรับรองรับเซิร์ฟเวอร์ฟาร์มขนาดใหญ่ มีหน่วยประมวลผลนับพัน และมีสเปกของฮาร์ดแวร์ที่แตกต่างกันอย่างสิ้นเชิง อย่างเช่นระบบกริดและคลัสเตอร์ต่างๆ เป็นต้น
