Software Quality

October 15, 2011

How we sped up our Microsoft CRM data conversion using the Microsoft Task Parallel Library

Filed under: Case Study, Microsoft CRM, Parallel Programming — David Allen @ 8:49 am

In an earlier post, I summarized the business benefit and overall performance improvement we achieved in a data conversion program that loaded data into a new Microsoft CRM system.  I have received several questions asking for more detail on the problem and the techniques used. This post provides some additional detail. (more…)

May 31, 2011

How to speed up a data conversion program that loads Microsoft CRM 4.0

Filed under: Case Study, Microsoft CRM, Parallel Programming — David Allen @ 9:43 pm

Overview

Our organization recently adopted Microsoft CRM 4.0. We developed a data conversion program to convert the data from the source systems and insert it into the new CRM system. Our initial effort was unsatisfactory because it ran way too slowly. We eventually solved our performance problems, and experienced a six-fold improvement in performance over our initial design! This report summarizes the challenges we faced, and our solutions, so that others who face performance issues with data conversion programs might benefit from the lessons we learned. Some of the lessons are specific to Microsoft CRM 4.0. But other lessons are of general interest. (more…)

How to avoid socket exhaustion on web servers by disabling HTTP Keep Alive

Filed under: Microsoft CRM — David Allen @ 8:48 pm

Problem:

Given

a web server, running IIS,

Hosting the web services for Microsoft CRM 4.0,

and a console application that runs a conversion program, which tries to update thousands of records per second through the CRM web service,

When

we run the conversion application at its fastest possible speed,

We observe

After only 26 seconds, we see failures in the web services calls of type System.Net.Sockets.SocketException.

If we put a deliberate delay of 100ms in the loop, we are able to process without errors, but at a reduced speed. (more…)

May 25, 2011

Parallel Programming with the Task Parallel Library from Microsoft

Filed under: Microsoft CRM, Parallel Programming, Presentations — David Allen @ 9:03 pm

Last week I attended a VERY cool talk on some new ways to do parallel programming, using the Task Parallel Library (TPL) introduced in .NET 4.0.   Microsoft has made it so easy, that even a frozen caveman programmer like me can do it! If you are trying to decide what to learn next month, put this near the top of your list. (more…)

Blog at WordPress.com.