Andri Yadi

A geeky technopreneur, trying to do something big with his startup

April 2010 - Posts

  • A Simple, Working (Not Elegant) Solution for Multi-Threading Issue in VSTO App

    I talked about how to bind data to Excel worksheet in this post, by comparing three methods.

    Now, you should have known which method you’ll use. However, you still experience poor performance and long UI blocking when trying to load data by calling remote WCF service. Not because of the data binding, but simply because there’s needed time and resources needed for calling the service itself.

    Then, you think, why don’t I create a separate thread for calling WCF service and hopefully UI thread will not be blocked. It seems like a good solution. Turn out, it’s not as easy as it seems.

    There are several discussions on the net about this multi-threading issue in VSTO application. There’s post from Alvin Bruney in this discussion (http://www.dotnetmonster.com/Uwe/Forum.aspx/vs-net-office/1123/Multithreading-vsto-2003 ), that said “Excel is not built to support multiple concurrent users. As such, there is no graceful means to handle concurrency except from exception triggers…”.

    There’s also post from Geoff Darst – MSFT here (http://social.msdn.microsoft.com/Forums/en/vsto/thread/f38edd9b-34dd-4cc0-bad2-71d42b667477), commenting about how to do multi-threading in Excel. He said,
    It sounds like you might benefit from reviewing the COM threading documentation which can be found here: http://windowssdk.msdn.microsoft.com/en-us/library/ms693344.aspx.  Hopefully, that will fill in any of the gaps in your reading of other threads.

    It is possible to do multi-threading in VSTO solutions regardless of whether your threads interact with the Office OM.  However, in the latter case, you need to implement IMessageFilter (the COM version, not System.Windows.Forms version) and you need to understand that all calls into the Office OM will be serialized so while one thread is calling into Office, the remaining threads would be blocked from doing so.

    All of this is because the Office object model isn't thread safe.  Since COM can't control the fact that a client might try to call a non-thread safe server with multiple threads, it does the next best thing which is to serialize the calls.  On the other hand, if you are calling into a component that is thread-safe, then COM doesn't need to get in the way and concurrency is allowed.

    As I said, it is always possible to use multiple threads in VSTO.  The issue that you need to understand is how much time your threads will spend being blocked and whether that will invalidate the performance gain you are hoping for.  If your tasks are truly self-contained (or call into Free Threaded components) then you won't blocked by COM if you run them on background threads.

    Turn out, it’s not that easy to create multi-threaded VSTO-based Excel application. And then do we give up using Excel and VSTO just because of that? Well, there’s a will, there’s a way.

    I’ve found a solution for that issue. Even it’s not an elegant solution, probably not correct one, it works for me. From previous post, I’ve create an app that has this architecture:

    image

    Then, I change the architecture to something like this.

    image

    Hmm…there’s a block called Loader. What the heck is that?

    Loader

    Loader is our guy that calls WCF service to get data, and then bind the data to Excel worksheet. It can be console app or better be a Windows service. Here’s inside of Loader.

     

    image

    Loader actually just hosts a WCF service called DataLoaderService. DataLoaderService exposes a service called LoadToExcel, that can be called by Excel to initiate WCF call and data binding. Here’s the DataLoaderService.cs code:

       1: using System;
       2: using System.Linq;
       3: using Excel = Microsoft.Office.Interop.Excel;
       4: using System.Threading.Tasks;
       5: using ILoaderService;
       6: using System.Runtime.InteropServices;
       7: using System.Diagnostics;
       8: namespace Loader
       9: {
      10:     public class DataLoaderService : IDataLoaderService
      11:     {
      12:         public void LoadToExcel(String filename)
      13:         {
      14:             var taskA = new Task(() =>
      15:             {
      16:                 Stopwatch sw = new Stopwatch();
      17:                 sw.Start();
      18:  
      19:                 Excel.Application excelApp = null;
      20:                 try
      21:                 {
      22:                     //Get the currently running Excel application by BindToMoniker the workbook file. 
      23:                     //This is useful to ancipate if there're more than one instances of Excel app
      24:                     Excel.Workbook wb = (Excel.Workbook)Marshal.BindToMoniker(filename);
      25:                     excelApp = wb.Application;
      26:                 }
      27:                 catch
      28:                 {
      29:                     //If BindToMoniker failed, then just get any instance of Excel app
      30:                     excelApp = (Excel.Application)System.Runtime.InteropServices.Marshal.GetActiveObject("Excel.Application");
      31:                 }
      32:  
      33:                 if (excelApp == null)
      34:                     return;
      35:  
      36:                 Excel.Workbook workBook = excelApp.ActiveWorkbook;
      37:                 Excel.Worksheet activeSheet = (Excel.Worksheet)workBook.ActiveSheet;
      38:                 activeSheet.Range["A1", "A1"].Value2 = "Loading....";
      39:                 
      40:                 //Call the WCF data service
      41:                 Services.NorthwindEntities ctx = new Services.NorthwindEntities(new Uri("http://localhost:3270/Northwind.svc/"));
      42:                 //Take a nap a moment
      43:                 System.Threading.Thread.Sleep(500);
      44:  
      45:                 Console.WriteLine("Elapsed: {0} second", sw.ElapsedMilliseconds / 1000);
      46:  
      47:                 var count = ctx.Orders.Count();
      48:                 Object[,] data = new Object[count, 11];
      49:  
      50:                 var orders = from o in ctx.Orders select o;
      51:                 int i = 0;
      52:                 foreach (var o in orders)
      53:                 {
      54:                     data[i, 0] = o.OrderID;
      55:                     data[i, 1] = o.OrderDate.HasValue ? o.OrderDate.Value.ToString() : "";
      56:                     data[i, 2] = o.CustomerID;
      57:                     data[i, 3] = o.ShipName;
      58:                     data[i, 4] = o.ShippedDate.HasValue ? o.ShippedDate.Value.ToString() : "";
      59:                     data[i, 5] = o.ShipAddress;
      60:                     data[i, 6] = o.ShipCity;
      61:                     data[i, 7] = o.ShipCountry;
      62:                     data[i, 8] = o.ShipRegion;
      63:                     data[i, 9] = o.ShipPostalCode;
      64:                     data[i, 10] = o.ShipVia;
      65:                     i++;
      66:                 }
      67:  
      68:                 Console.WriteLine("Elapsed: {0} second", sw.ElapsedMilliseconds / 1000);
      69:  
      70:                 String[] columnNames = new String[] {"Order ID", "Order Date", "Customer", "Ship Name", "Ship Date",
      71:                    "Ship Address", "Ship City", "Ship Country", "Ship Region",  "Ship Postal",  "Ship Via"};
      72:  
      73:                 activeSheet.Range["A1"].get_Resize(1, 11).Value = columnNames;
      74:                 activeSheet.Range["A2"].get_Resize(RowSize: count, ColumnSize: 11).Value = data;
      75:                 
      76:                 activeSheet.Range["A1", "K" + count].Columns.AutoFit();
      77:  
      78:                 sw.Stop();
      79:                 Console.WriteLine("Elapsed: {0} second", sw.ElapsedMilliseconds / 1000);
      80:  
      81:                 Marshal.ReleaseComObject(excelApp);
      82:             });
      83:             // Start the task.
      84:             taskA.Start();
      85:         }
      86:     }
      87: }

     

    In above code, I use Task Parallel Library to simplify multi-threading code. It’s new in .NET 4.0.

    You see that DataLoaderService implement an interface IDataLoaderService. It is just an interface that defines WCF contract.

       1: using System;
       2: using System.ServiceModel;
       3:  
       4: namespace ILoaderService
       5: {
       6:     [ServiceContract(Namespace = "http://www.dycode.com/DataLoaderService")]
       7:     public interface IDataLoaderService
       8:     {
       9:         [OperationContract]
      10:         void LoadToExcel(String filename);
      11:     }
      12: }

    Program.cs of Loader.exe is like this:

       1: using System;
       2: using System.ServiceModel;
       3:  
       4: namespace Loader
       5: {
       6:     class Program
       7:     {
       8:         static void Main(string[] args)
       9:         {
      10:             using (ServiceHost host = new ServiceHost(typeof(DataLoaderService)))
      11:             {
      12:                 NetTcpBinding binding = new NetTcpBinding();
      13:                 host.AddServiceEndpoint(typeof(ILoaderService.IDataLoaderService),
      14:                                         binding,
      15:                                         "net.tcp://localhost:9000/DataLoaderService");
      16:  
      17:                 host.Open();
      18:                 Console.WriteLine("Press any key to terminate service");
      19:                 Console.ReadLine();
      20:             }
      21:         }
      22:     }
      23: }

    Program.cs is just a host for WCF service that listen at port 9000.

    Excel

    On Excel side, I create a method in ThisWorkbook.cs. That method will call LoadToExcel() method of DataLoaderService (that is hosted by Loader.exe), by supplying parameter of currently opened Workbook’s name. The code is:

       1: public void LoadDataFromLoader()
       2: {
       3:     EndpointAddress addr = new EndpointAddress("net.tcp://localhost:9000/DataLoaderService");
       4:     NetTcpBinding binding = new NetTcpBinding();
       5:     IChannelFactory<IDataLoaderService> channelFactory = new ChannelFactory<IDataLoaderService>(binding);
       6:     IDataLoaderService proxy = channelFactory.CreateChannel(addr);
       7:  
       8:     String fileName = this.FullName; //get workbook full path
       9:     proxy.LoadToExcel(fileName);
      10: }

    Method LoadDataFromLoader() will be called by a button in my custom Ribbon.

    To test this scenario:

    1. Run the Loader.exe, so that the service is up

    2. Run Excel application. Click the button in Ribbon (that will call LoadDataFromLoader() method) and walah!, the UI is not blocking. After waiting 2 seconds, the data will be loaded to Excel worksheet. It works.

    This video proves my solution.

    A Simple, Working (Not Elegant) Solution for Multi-Threading Issue in VSTO App from Andri on Vimeo.

    Come on…a little applause may be?

    Just download the code and play around with it. Don’t expect it’s perfect, I code it during my free time, and just for fun. Here's the file:

     

    Share this post: | | | |
    Posted Apr 30 2010, 03:16 PM by andriyadi with no comments
    Filed under:
  • Improve VSTO/Excel App Performance

    There’s a thread in INDC mailing list, entitled “Menyimpan Data Langsung ke Database MS SQL dari MS EXCEL” (starting here: http://dotnet2.netindonesia.net/?0::55374). The thread’s started by a quite simple question, is it possible to store data from SQL Server to Excel. Well, the quite simple answer is YES, it’s possible. Then, there’s a post from my friend, Z, he experienced poor performance when calling WCF service from VSTO-based Excel app. OK, that’s a problem and it challenges me :)

    OK, let’s say you develop a VSTO-based Excel app that calls WCF service or WCF Data Service, and returns a plenty amount of data. Then you need to bind those data to Excel worksheet. Before you blame VSTO poor performance, due to too many layers (PIA, VSTOR, CLR, Native), and other craps, please check your code that actually do data binding.

    I give you an example. Below is an architecture of the system, where the VSTO code get the data from WCF Data Service (WCF Data Service will get the data from actual tables in database via Entity Framework), and bind the data to Excel worksheet.

    image

    Let’s say I want to load all Orders records (> 800 records) from Northwind database. Here’re what I do:

    1. Create three projects:

    • Excel Workbook project, for displaying data
    • Northwind.Model class library project, for creating ADO.NET Entity Data Model
    • Northwind.Service web application project, for creating WCF Data Service

    2. Add reference to WCF Data Service, and set the namespace as “Services”.

    3. Add following code at Excel’s ThisWorkbook_Startup method. This code will create NorthwindEntities (WCF data service context) instance, then retrieve orders collection from WCF Data Service. After that, it will bind the orders collection to Sheet1 worksheet by looping through it and set each order’s property to individual cell.

       1: Services.NorthwindEntities svc = new Services.NorthwindEntities(
       2:                 new Uri("http://localhost:3270/Northwind.svc/"));
       3:  
       4: this.Application.ScreenUpdating = false;
       5: Stopwatch sw = new Stopwatch();
       6: sw.Start();
       7:  
       8: var orders = from o in svc.Orders select o;
       9: int i = 2;
      10:  
      11: foreach (var o in orders)
      12: {
      13:     Globals.Sheet1.Range["A1"].Cells[i, 1].Value2 = o.OrderID;
      14:     Globals.Sheet1.Range["B1"].Cells[i, 1].Value2 = o.OrderDate.HasValue ? o.OrderDate.Value.ToString() : "";
      15:     Globals.Sheet1.Range["C1"].Cells[i, 1].Value2 = o.CustomerID;
      16:     Globals.Sheet1.Range["D1"].Cells[i, 1].Value2 = o.ShipName;
      17:     Globals.Sheet1.Range["E1"].Cells[i, 1].Value2 = o.ShippedDate.HasValue ? o.ShippedDate.Value.ToString() : "";
      18:     Globals.Sheet1.Range["F1"].Cells[i, 1].Value2 = o.ShipAddress;
      19:     Globals.Sheet1.Range["G1"].Cells[i, 1].Value2 = o.ShipCity;
      20:     Globals.Sheet1.Range["H1"].Cells[i, 1].Value2 = o.ShipCountry;
      21:     Globals.Sheet1.Range["I1"].Cells[i, 1].Value2 = o.ShipRegion;
      22:     Globals.Sheet1.Range["J1"].Cells[i, 1].Value2 = o.ShipPostalCode;
      23:     Globals.Sheet1.Range["K1"].Cells[i, 1].Value2 = o.ShipVia;
      24:     i++;
      25: }
      26:  
      27: Globals.Sheet1.Range["A1", "K" + orders.Count()].Columns.AutoFit();
      28:  
      29: this.Application.ScreenUpdating = true;
      30: sw.Stop();
      31:  
      32: MessageBox.Show("Elapsed: " + sw.ElapsedMilliseconds / 1000 + " seconds");

     

    For 830 rows and 11 columns, it takes 7 seconds to complete, starting from WCF Data Service call until data binding. And off course, Excel UI is blocked during that 7 seconds.

    image

    Let’s change the way to bind the orders collection to worksheet. Something like this:

       1: Services.NorthwindEntities svc = new Services.NorthwindEntities(new Uri("http://localhost:3270/Northwind.svc/"));
       2:  
       3: this.Application.ScreenUpdating = false;
       4: Stopwatch sw = new Stopwatch();
       5: sw.Start();
       6:  
       7: var count = svc.Orders.Count();
       8: Object[,] data = new Object[count, 11];
       9: var orders = from o in svc.Orders select o;
      10: int i = 0;
      11: foreach (var o in orders)
      12: {
      13:     data[i, 0] = o.OrderID;
      14:     data[i, 1] = o.OrderDate.HasValue ? o.OrderDate.Value.ToString() : "";
      15:     data[i, 2] = o.CustomerID;
      16:     data[i, 3] = o.ShipName;
      17:     data[i, 4] = o.ShippedDate.HasValue ? o.ShippedDate.Value.ToString() : "";
      18:     data[i, 5] = o.ShipAddress;
      19:     data[i, 6] = o.ShipCity;
      20:     data[i, 7] = o.ShipCountry;
      21:     data[i, 8] = o.ShipRegion;
      22:     data[i, 9] = o.ShipPostalCode;
      23:     data[i, 10] = o.ShipVia;
      24:     i++;
      25: }
      26:  
      27: Excel.Range dataRange = (Excel.Range)Globals.Sheet1.Range["A2"].get_Resize(RowSize: count, ColumnSize: 11);
      28: dataRange.Value = data;
      29: dataRange.Columns.AutoFit();
      30:  
      31:  
      32: this.Application.ScreenUpdating = true;
      33: sw.Stop();
      34:  
      35: MessageBox.Show("Elapsed: " + sw.ElapsedMilliseconds / 1000 + " seconds");

    Walah!, 1 second. Isn’t that great? In above code, I use two dimension array to temporarily store order collections and then resize range “A2” to display the array elements.

    image

    Binding using ListObject and BindingSource

    To be fair, let’s try binding using ListObject and BindingSource. It turns out the speed is very good. Only 2 seconds! And you also get table formatting for free :)

    image

     

    So as a recap, in term of data binding, and for the sake of table formatting, binding using ListObject and BindingSource is a good choice.

    On the next post, I’ll talk about how to improve performance on calling WCF service from Excel app.

     

    Share this post: | | | |
    Posted Apr 30 2010, 03:13 PM by andriyadi with 601 comment(s)
    Filed under:
  • A SQL Query Joke

    Waktu nonton screencast tentang what’s new in Entity Framework 4, sambil nunggu coding selesai salah satu pembicaranya nge-joke tentang SQL Query:

    A SQL query walks into a bar and sees two tables. He walks up to them and says 'Can I join you?'

    Ngerti kan? :)) Why so serious lah bro?

    Disclaimer: don’t try to tell that joke to your girlfriend, wife, or your child, only geeks will understand :)

    Share this post: | | | |
    Posted Apr 25 2010, 04:58 AM by andriyadi with 1 comment(s)
    Filed under:
  • Disambut Oleh Pengumuman iPhone OS 4

    [UPDATED]
    It's official. iPhone OS 4 preview is officially announced today, Apr 8, 2010 10.00AM Pacific Daylight Time (PDT). That time is equal to April 9, 2010 00:00AM West Indonesia Time (check here). So, I can say that my birthday is celebrated with the announcement of iPhone OS 4. Thanks Apple!
    Today, as iPhone OS reaches fourth version, I'm officially 28 years old. I wish Visual Studio 2010 is released at the same day, bakal tambah rame!. Tapi sepertinya VS2010 akan di-release someday this month, gak bisa cerita banyak, NDA :)

    More story about my birthday later. All I can tell now are the present and birthday cake :)


    The cute girl presents the cake...delicious.


    Berlebihaaaan...Thanks ya baby...

    Setelah dikejutkan dengan birthday surprise tepat jam 12.00, 9 April, siangnya gw dikejutkan lagi dengan tumpengan di kantor. Thanks buat DyCoders yang arrange acara tumpengannya dan semua yang hadir bareng keluarga. Here's the tumpeng:

    Looks and tastes delicious

    Cerita selengkapnya bisa dinikmati di sini: http://dycode.com/blogs/corporatenews/archive/2010/04/09/happy-birthday-our-ceo.aspx

    Thanks a huge for all B'day sayings, wishes, and prays. It's gonna be a great year, hopefully. Hopefully the even number of my age will bring more success and prosperity, and I can accomplish what I've started.

    Share this post: | | | |