Thursday, October 27, 2005

1374.aspx

Windows Embedded conference November 2-3

I am going to the Windows Embedded conference next week (November 2-3) in Milano



See you there?

1371.aspx

How to replace multiple strings in XSL

It has been a while since I have done some serious xsl coding so I had some fun today when I was asked how to replace multiple strings in XSL. This is a piece of cake if you use a Microsoft XML parser (MSXML or .NET version) as you can call JavaScript using the proprietary msxml:script tag in xsl. Just write a simple function like this:


<?xml version="1.0" encoding="ISO-8859-1"?>


<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:egilh="http://egilh.com/" version="1.0">


 <msxsl:script language="JScript" implements-prefix="egilh">


 function replaceString(source, find, replaceWith) {


  var result = source;


  while (result.indexOf(find) >= 0){


   result = result.replace(find, replaceWith);


  }


  return result;


 }


 </msxsl:script>


 


 <xsl:template match="/">


   Test: <xsl:value-of select="egilh:replaceString('abcd', 'b', ' CoolOrWhat ')"/>


 </xsl:template>


</xsl:stylesheet>


 


 


Xpath 2.0 solves the problem as you use the replace function but in this case they were using a Java XSL parser...


I have tried to implement a generic and configurable solution. It uses an external XML file that contains the search and replace strings. You only have to include the replace.xsl in your stylesheet and call the template to replace as many strings as you want.


Step 1: Include the xsl
Add the following line after the <xsl:stylesheet> declaration but before the first <xsl:template>:


    <xsl:include href="replace.xsl"/>


Note: You must put the replace.xsl file in the same directory as your stylesheet or modify the href


Step 2: call the "replace  routine"
Call the replace routine which is implemented as a template. You have to pass it the following arguments:



  • the text to search in

  • the XML nodes with find/replace elements

Example:
<!-- Replace strings found in replace.xml --> 
<xsl:call-template name="replaceStrings">
  <xsl:with-param name="inputText" select="."/>
  <xsl:with-param name="search" select="document('replace.xml')/SearchAndReplace/search"/>
</xsl:call-template>


The example above workes on the current node (.). It gets the list of items to search and replace from the external file 'replace.xml'. You must put the replace.xml file in the same directory as your stylesheet or modify the path.


Step 3: configure the text to search and replace
The replace.xml file defines what to search for and the text to replace it with. It has the following format


<SearchAndReplace>


 <search>


  <find>Test1</find>


  <replace>Did you say 1?</replace> 


 </search>


</SearchAndReplace>



How it works
Replace.xsl uses two recursive templates to do its work:



  • replaceStrings (inputText, search):


    • Loops on the search nodes. It calls the replaceString() for the first element in the list, then it calls itself with the remaining elements in the list

  • replaceString(sourceText, findText, replaceText):


    • Loops on itself as long as findText is found in the sourceText.

    • It replaces the first occurrence it finds using substring, then it calls itself again. 

    • NB!: You can get never ending loops that bomb with a out of stack space errors if the findText is contained in the replaceText. Replacing "a" with "abc" for example will forever replace the first element with "abc". It is possible to avoid this by passing an offset to replaceString.


Replace.xsl


<?xml version="1.0" encoding="ISO-8859-1"?>


<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fo="http://www.w3.org/1999/XSL/Format">  


           


      <xsl:template name="replaceStrings">


        <xsl:param name="inputText" />


        <xsl:param name="search" />


       


      <!-- Debug -->


      <!--


      [inputText: <xsl:value-of select="$inputText"/>] 


      [find: <xsl:value-of select="$search[1]/find"/>] 


      [replace: <xsl:value-of select="$search[1]/replace"/>]  


      -->


        <!-- Replace the FIRST string in the replace list -->


        <xsl:variable name="replaced_text">


            <xsl:call-template name="replaceString">


              <xsl:with-param name="sourceText" select="$inputText" />


              <xsl:with-param name="findText" select="$search[1]/find" />


              <xsl:with-param name="replaceText" select="$search[1]/replace" />


            </xsl:call-template>


        </xsl:variable>


        <xsl:choose>


            <!-- If there is (at least) one more element -->


            <xsl:when test="$search[2]">


              <!-- Call ourself with the remaining list of elements


                    (removing the first one -->


              <xsl:call-template name="replaceStrings">


                  <xsl:with-param name="inputText" select="$replaced_text" />


                  <xsl:with-param name="search" select="$search[position() > 1]" />


              </xsl:call-template>


     


            </xsl:when>


            <xsl:otherwise>


              <xsl:value-of select="$replaced_text" />


            </xsl:otherwise>


        </xsl:choose>


      </xsl:template>


     


     


      <xsl:template name="replaceString">


        <xsl:param name="sourceText"/>


        <xsl:param name="findText"/>


        <xsl:param name="replaceText"/>


        <!-- Debug statements -->


        <!--     


          [sourceText: <xsl:value-of select="$sourceText"/>]


            [findText: <xsl:value-of select="$findText"/>]


            [replaceText: <xsl:value-of select="$replaceText"/>]


         -->


        <xsl:choose>         


            <!-- If the output contains the text we are looking for -->


            <xsl:when test="contains($sourceText,$findText)"> 


              <!-- Replace using substring before + new text + substring after -->


              <xsl:value-of select="concat(substring-before($sourceText,$findText),$replaceText)"/>


              <!-- Apply template again to replace the next occurence -->


              <xsl:call-template name="replaceString">


                  <xsl:with-param name="sourceText" select="substring-after($sourceText,$findText)"/>


                  <xsl:with-param name="findText" select="$findText"/>


                  <xsl:with-param name="replaceText" select="$replaceText"/>


              </xsl:call-template>


            </xsl:when>


            <xsl:otherwise>


              <xsl:value-of select="$sourceText"/>


            </xsl:otherwise>


        </xsl:choose>


      </xsl:template>


</xsl:stylesheet>



Example Replace.xml


<SearchAndReplace>


 <search>


  <find>Test1</find>


  <replace>Did you say 1?</replace> 


 </search>


 <search>


  <find>Test2</find>


  <replace>OK!</replace>


 </search>


</SearchAndReplace>



I based my implementation on the hints I found in the following articles:


Wednesday, October 26, 2005

1365.aspx

How to track down performance problems in COM+ applications

Someone asked me yesterday if the COM+ Call Time Tracker can be used dig deeper into the component to see which function is causing the high call times. It is an interesting subject, so I am posting the reply on the blog.


The answer is no, COM+ only gives call information on an object level. It does not give information about internal information like function call times. This means that the COM+ objects must be split into smaller COM+ objects you want to track down the problem with COM+ call time tracking. It is an ugly solution, and only works on a "macro" level as it is not feasible to make all the functions separate COM+ objects. There are better alternatives which I cover below.


First of all; keep in mind that call time is not the same a CPU time. The call time shows how long COM+ has been waiting for a the object to return. The object may be busy:



  • calculating something (using a lot of CPU),

  • writing something to disk

  • waiting for an external resource like a Web Service or a query to a DB

  • waiting for a lock on a shared resource

- Task Manager -
This is the first think I have a look at if I have problems with high call times in COM+. It is already installed on the machine and it gives an idea what is going on.


Things to check in the "Performance Tab":



  • CPU Usage. It should be lower than 50% on average, never over 75% for long periods of time

  • Commit Charge must be lower than Physical Memory, otherwise the machine does not have enough memory and will swap. Everything slows down when the machine swaps so you will get high call times in all components..

Then I look at the process running the COM+ Application. The COM+ Call Time Tracker already gives you the process ID in /applications/application/processID but you can also get it from Component Services. I usually check the following information in Task Manger for the Process ID; CPU, Mem Usage, Peak Mem Usage, VM Size, Handles


Some quick rules of thumb:



  • High CPU usage is pretty obvious. Improve your code or get more/faster CPUs

  • VM Size > Mem Usage means a memory leak. The machine starts swapping out memory and everything slows down

  • Handles:  Should be stable. An every increasing number means a handle leak in the component (usually references to external objects/files that are not closed)

  • If Peak Mem Usage is a lot larger than "Mem Usage", there are periods when the component uses a lot of memory


- Performance Monitor -
Performance Monitor (perfmon.exe) is a very powerful tool if you know how to interpret the results. It can be used to monitor most resources on the machine, often on a process by process basis. It allows you to find the overall bottle neck: cpu, disk, memory, network, web worker processes etc.


- Process Explorer -
The free Process Explorer by Sysinternals is a fantastic tool. Consider it Task Manager on steroids enhanced with Performance Monitor data. The latest version tells you everything you ever wanted to know about the processes running on your PC; how many threads they have, how much CPU each thread is using, which DLLs it is using etc.


One thing to check for .NET applications: The “% time in GC” counter below shows how much of the applications CPU time is spent on garbage collection. It must be less than 4-5% on average. Higher values means .NET spends a lot of time garbage collecting objects that have not been closed or disposed properly. I have seen cases of >70% cpu usage for garbage collection due to bugs in the code.


 


- Logging -
A good log is crucial when troubleshooting any problem, including performance. I usually use log4net but any other logging framework that works in a multi threaded environment  is OK. Use the different log levels to control how much is logged as detailed logs will slow the system. You can use a trace/info level to log function entry/exit points, so you can see which function grabs the time, but log input/ouput and more verbose data at the more detailed debug log level.



- Code Profiler -
Code Profilers monitor the execution of a process and tells you afterwards how many times the different functions were called, how long it took etc. Code Profilers usually run on a development or test PC so you have to simulate the calls you make in a production environment using a dedicated tester.


There are several professional tools on the market like ANTS Profiler by Red Gate and DevPartner Studio by Compuware


So far I have never had to use one as I have managed to track down bugs with a combination of performance counters, logs and code reviews.

Tuesday, October 25, 2005

1363.aspx

KPL: Kids Programming Language

The free Kids Programming Language development environment comes with a many fun programs, mostly games, that the kids can play around with and modify as they wish.  I learned programming by playing around with the examples that came with Turbo Pascal and I really believe in Learning by doing.  The working games with neat graphics and sounds is fun to play around with and tweak for kids of all ages.




Kid’s Programming Language (KPL) was designed and developed based on the principle that programming is fun - and that learning is best achieved when the learning material is fun, intuitive and interactive. KPL is composed of:



  • a readable and modular programming language

  • a kid-usable but functionally complete integrated development environment

  • a growing collection of entertaining and educational sample programs

  • an experience that enables and encourages kids' imagination, curiosity and creativity

We developed Kid’s Programming Language (KPL) as a way to teach our own kids how fun it is to write code, and KPL turned out so well that we are working hard to figure out how to also share it with as many other kids as we can. 



The language itself borrows a lot of the syntax from VB.NET with some elements from C#. The GUI looks like a simplified version of Visual Studio.NET (which makes it simpler for dad to give support :-)


Microsoft will promote the new programming language by Morrison Schwartz  to replace basic as the first language students learn.

Friday, October 21, 2005

1358.aspx

Windows Bluetooth scanner

The BlueScanner, by Network Chemistry, is a Bluetooth discovery tool for Windows. It finds all Bluetooth devices in range and the the services they offer. I had some problems on my Compaq nx7010 but you may have better luck on your PC


Thursday, October 20, 2005

1352.aspx

DVD Jon lands a job for MP3Tunes

Fellow countryman Jon Lech Johansen (aka "DVD Jon")  recently started working with Michael  Robertson of MP3.com, SIPphone and Lindows Linspire fame.

Wall Street Journal ran a front-page story about Jon Lech relocating so he got loads of new job offers, but he seems happy where he is:



 I [Michale Robertson] talked to him over lunch today and asked if he wanted to work at those other companies. "Not really," he replied in a typical minimalist Scandinavian-style reply, forcing me to ask why not. "I want to work on open systems, which is why I came to you."


The best of luck to Jon in his new adventure!


Via [Michael's Minute]

Wednesday, October 12, 2005

1341.aspx

How to add a Subscribe with Google Reader link to your blog

I have played around with Google Reader lately. I wanted to add a "subscribe with Google Reader" link to my blogs so I created a basic Google Reader subscription generator. Just enter the complete URL of you RSS feed and it generates the Google Reader subscription url:




I generated the "Sub | Google" button using the Button Maker by Adam Kalsey


Add the following to /blog/Skins/your skin/Controls/MyLinks.ascx if you have .TEXT



<asp:HyperLink


    ImageUrl="~/images/SubGoogle.png"


    Runat="server"


    NavigateUrl="Google Reader Subscription URL"


    ID="GoogleReader">


    Google Reader


</asp:HyperLink>


Here is the complete source code for the Google Reader subscription generator:


<% Option Explicit


Dim blogURL


dim googleURL


%>


<html>


      <body>


      <h1>Google Reader subscription generator</h1>


      <%


      blogURL = Request("BlogUrl")


      if blogURL <> "" then


            googleURL = "http://www.google.com/reader/preview/" & _


                        Server.URLEncode(blogURL) & "/feed/" & blogURL


           


            %>


            The <a href="http://reader.google.com/">Google Reader</a>


            subscription url for <b><%=blogURL%> is:</b><br/>


            <textarea cols="75" rows="2"><%=googleURL%></textarea>


            <br/><br/>


            Example: <a href="<%=googleURL%>">


            <img src="http://text2blogger.appspot.com/static/egilh/Images/subGoogle.png"></a>


            <%


      else


      %>


      <p>


      Enter your entire RSS url in the field below to calculate the


<a href="http://reader.google.com/">Google Reader</a>


subscription url.<br/>


      Example: http://www.egilh.com/blog/Rss.aspx


      </p>


      <form action="SubGoogleReader.asp">


      Blog RSS Feed <input type="text" name="BlogUrl" size="50"/><br/>


      <input type="submit" value="Calculate URL"/>


      </form>


      <%


      end if


      %>


      </body>


</html>

1335.aspx

Quake 3 Arena source code released

The full source code for Quake III Arena has been released under GPL. The 24MB package (when uncompressed)  comes with everything you need:



  • Quake III Arena source code ( renderer, game code, OS layer etc. )

  • Bot routes compiler source code

  • Retargetable C compiler ( produces assembly to be turned into qvm bytecode by q3asm )

  • Assembly to qvm bytecode compiler

  • Map compiler ( .map -> .bsp ) - this is the version that comes with Q3Radiant 200f

  • Q3Radiant map editor build 200f ( common/ and libs/ are support dirs for radiant )

Check the internet for spinof projects that tues the build environment and code for the various supported platforms: Windows, GNU/Linux and Mac.


I built it without problems using Visual Studio .NET 2003. The world of first person shooters has certainly moved on since I had a look at the graphics behind Caste Wolfenstein.


The source code for the older Quake versions can be downloaded from the idsoftware technology downloads site

1334.aspx

Linus on development: a spec is close to useless

Interesting point of view by Linus;



"A 'spec' is close to useless. I have _never_ seen a spec that was both big enough to be useful _and_ accurate.


And I have seen _lots_ of total crap work that was based on specs. It's _the_ single worst way to write software, because it by definition means that the software was written to match theory, not reality.


Looks like Linux is heading for total anarchy as well as for extinction...

Tuesday, October 11, 2005

1331.aspx

How to recycle com+ applications with high call times using .NET

Windows 2003 introduces a new RecycleApplicationInstances method in the COM+ Admin API. It allows transparent recycling of COM+ applications as existing requests are allowed to finish while new requests are routed to a new application instance. A lot better than the ShutDownApplication in Win2k which shuts down the process directly and returns an error to the calling client.


Win2k3 offers several recycling policies in the "Pooling & Recycling" tab of COM+ applications:



  • Lifetime limit: The application runs max X minutes before it is recycled

  • Memory Limit: The application is recycled if it uses more than X Kilo Bytes of memory. A life saver if you have third party C++ components that leak  memory

  • Call Limit: Max number of calls before the application is recycled

  • Activation Limit: Max number of object activations before the application is recycled

But a very useful limit is missing: max call time. There is no way to tell COM+ to recycle the application if an object blocks for some reason and gives very high call times.Using the egilh.comtracker you an roll your own max call time manager using .NET.


The code below loads the XML returned by egilh.comtracker in a XmlDom and checks the responseTimes. It uses the RecycleApplicationInstances COM+ API to shut down the application if the call time is higher than the configured MAX_CALL_TIME and there are objects in call.


XML returned by egilh.comtracker


<applications>


      <application>


      <guid>{98CDBB6E-3CAF-46F2-ABE6-EABECD6AA4EB}</guid>


      <ID>10</ID>


      <processID>1792</processID>


      <statistics>


            <callsPerSecond>0</callsPerSecond>


            <totalCalls>0</totalCalls>


            <totalClasses>1</totalClasses>


            <totalInstances>1</totalInstances>


      </statistics>


      <classes>


            <class>


                  <progID>Test.Sleep.1</progID>


                  <bound>1</bound>


                  <inCall>0</inCall>


                  <pooled>-1</pooled>


                  <references>1</references>


                  <responseTime>120</responseTime>


                  <callsCompleted>0</callsCompleted>


                  <callsFailed>0</callsFailed>


            </class>


      </classes>


      </application>


</applications>



Code for recycling applications with high call times


/// <summary>


/// Check the COM+ Call times, recycling applications with


/// components that have high call times


/// </summary>


private void checkComCallTimes()


{


      const int MAX_CALL_TIME = 10000; // Max call time in miliseconds


      const int RECYCLE_REASON = 408;  // The reason for the recyle. Any number is OK


 


      // Get the call statistics


      EGILHCOMTRACKERLib.ComTrackerClass comTracker = new EGILHCOMTRACKERLib.ComTrackerClass();


      string statistics = comTracker.getStatistics();


 


      // Load the statistics in a DOM and loop on the call times


      XmlDocument xmlDoc = new XmlDocument();


      xmlDoc.LoadXml(statistics);


      foreach (XmlNode node in xmlDoc.SelectNodes("//class"))


      {


            // Get class info


            string className = node.SelectSingleNode("progID").InnerText.ToLower();


            int inCall = System.Convert.ToInt32(node.SelectSingleNode("inCall").InnerText);


            long responseTime = System.Convert.ToInt64(node.SelectSingleNode("responseTime").InnerText);


 


            // Recycle if objects are in call and the call time is too high


            if (inCall > 0 && responseTime > MAX_CALL_TIME)


            {                                                                      


                  // get application PID from ../../


                  XmlNode appNode = node.ParentNode.ParentNode;


                  recycleApplication(System.Convert.ToInt32(appNode.SelectSingleNode("processID").InnerText),


                                                RECYCLE_REASON);


           


            }


      }


 


}


 


/// <summary>


/// Recycle a COM+ application instance given the process ID


/// </summary>


/// <param name="processID">Application instance process ID</param>


/// <remarks>Only works on Win2k3 and WinXP. You have to catch


/// various com+ exceptions like "The application has already been recycled"


/// and "The specified application is not longer running"</remarks>


void recycleApplication(int processID, int reason)


{


      COMAdmin.COMAdminCatalogClass adminCatalog = new COMAdmin.COMAdminCatalogClass();


 


      // Get instance guid from process id


      object instanceGUID = adminCatalog.GetApplicationInstanceIDFromProcessID(processID);


 


      // Wrap it in a single element array


      object [] applications = new object[1];                    


      applications[0] = instanceGUID;


      object recycleObject = (object) applications;


 


      // Pass reference with list of objects to recycle to RecylceApplicationInstances


      adminCatalog.RecycleApplicationInstances(ref recycleObject, reason);                                       


}


 
The API is difficult to use from VBScript as it requires a Variant pointer to a Variant array that contains the list of application instances to use. The code above only works on Windows 2003. Use the VB Script in How to shut down COM+ applications with high call times with VB and VBScript if you use Windows 2000.