Alexey Solofnenko blog

With my fascination with old forgotten roads I could not skip articles about Old Big Oak Flat Road that my Google Reader's Explore found for me. Yesterday we went to Tuolumne Grove on the old road. The road is supposed to go down into Yosemite Valley to somewhere around El Capitan. But where? The road was destroyed by many rock slides and there is nothing on the modern maps and on Yosemite trail maps. Fortunately I remembered that at the time before Google Earth there was Nasa's WordWind that showed USGS maps with many old roads. Many years and operating systems came after that time and now I tried to run it on my new computer with Windows 7. To my surprise there were no USGS maps after I installed. Searching in Google returned that I might be missing images.xml. Instead on my machine there was @Images.xml with a reference to http://worldwind25.arc.nasa.gov/layerConfig/earthimages.xml. So I exited WordWind, downloaded it and saved in "C:\Program Files (x86)\NASA\World Wind 1.4\Config\Earth", and started WordWind again... I got it working! I can see old Big Oak Flat Road and Old Tioga Road. Surprisingly the more detailed maps have altitude in feet and ones before them are in meters (altitude differences are important for hiking trails). Now I have to buy an approved bear container and I am set.

Well, almost it did not. I had to reconnect a monitor and keyboard - first it started into "recovery menu". Second it did not want to boot because of my backup USB drive. Google says that my USB cable is most likely at fault. Fortunately it is a standard mini-USB cable. I have few in the car just in case.

I have finished setting up my photo server. My server computer can connect only two SATA hard drives and the photos were on the smaller disk that I disconnected. Fortunately I was prepared and received Sabrent's SATA/IDE USB adapter, used it to connect the old drive, and copy the photos back onto a new hard drive. The only strange thing happened is that my other backup USB drive is not recognized any more...

After some time my server platform became too old. There are new Linux, b2evolution, and other releases and it was too dangerous to make any changes on the system because they can break anything. After my update backlog became too big I decided it is time to switch to virtual machines with their snapshots to make backup before making any big changes. I used VMWare before, but now I am a big fan of VirtualBox. It is an easy to use virtual machine solution with niceties like automatic changing VM screen resolution when the console window is resized. At this moment I only moved my blog to a new system (see http://manual.b2evolution.net/Move_my_installation for instructions, how to do it). My photo library is too big to be hosted on a VMWare partition. Instead it will be saved on a main hard drive. I will need to research what is the best way to mount phisical drive in a VirtualBox guest system. I guess I will have to use read only NFS.

Recently we have been hit with a new error in our multi platform builds with files stored on NAS - different build commands could fail on Unix with "not owner" error message. At first it was Solaris linker: "ild: Not owner: chmod() failed for file ...", then we switched to another machine and got:

java.io.IOException: java.io.IOException: Not owner
  at java.io.UnixFileSystem.createFileExclusively(Native Method)
  at java.io.File.createNewFile(File.java:883)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)

And there were no obvious changes in the build that could cause it. So we set together with IT and tried to diagnose the problem. What was even more surprising, the same commands could sometimes succeed when executed manually. After few attempts (the whole process took few hours) we found that we cannot change a permission on just created folders. Even "truss" was showing EPERM error code. When waited for about few (<30) seconds the same chmod would succeed. We tried Solaris 10 and Linux with the same result. At least we got something easily reproducible.

So the working theory became that NAS assigns some different permissions to just created folders and then changes them to correct permissions later. To remove NIS server out of the picture we tried a local account and got just created directory owned by root(!). Of cause after that chmod failed with the same "Not owner" error. We went back and forth thinking why it happened and decided that the test did not show what we needed - it is possible that NAS did not know the account and use the account we use for mount.

Google suggested changing directory permissions to 777 and it, of cause, worked. Unfortunately, sticky bits and umask did not seem working on NAS, so all new folders were created with 755 permissions. After few more experiments and cups of coffee we noticed that the build root directory is owned by root. That was odd.

And we changed its ownership to the correct account and group. We also changed its permission to 777. And it helped. It looks like our NAS uses the parent folder for initial ownership settings and then changes it to the correct once and uses parent directory permissions as-is. Now initial owner became correct (or 777 helped) and the builds are running correctly. I will try to set the builds root folder permissions to 755 to actually see what change fixed the build, but not now - now we need the builds working.

I have had a problem that Google did not have an answer for - how to search for an attribute value, if the attribute name is not case sensitive? Usually the situation arises when the original file is not case sensitive (for example, HTML files are not). In my situation the XML was processed by another code that treated it in a case-insensitive way. Surprisingly the only good answer in Google I got that XML is case sensitive and live with it. After reading that I was about to replace one search with two searches using two different cases that can actually happen in my files, but decided to look further, if it is a possible to write a condition with a node name. Of cause there was - name() function returns the current node name and it can be tested. Surprise, surprise, the query did not work at first - I used '=='instead of '=', but eventually I got it right. This is it:

/Path/To/element/@*[translate(name(), 'abcdefghijklmnopqrstuvwxyz', 'ABCDEFGHIJKLMNOPQRSTUVWXYZ')='UPPERCASE_ATTR_NAME']

While our installer developer was busy on another project, I created a simple patch installer with WIX that just places few assemblies with their policies into GAC. It required some tweaking. My first approach of reading instructions only after all other possibilities failed did not really work. There was one positive outcome - Wix 2.0 was unusable (what is up with 8.3 names!?). I wonder how other people are using it. So I quickly downloaded Wix 3.0 beta and decided to use VisualStudio integration to create my installer. VisualStudio helped a little, but soon I was back editing the file in UltraEdit. After few minutes the installer was ready and it was installing the patch without any user interaction. That was the problem - I needed a question "do you really want to install this thing or you clicked on it by mistake?". Fortunately there were pre-built UIs and I selected WixUI_Minimal. Again it worked great, but what is up with EULA? I did not need it for a patch installer. Searching on internet did not return anything useful besides putting a readme instead of a licence. There were comments about changing standard UIs, so I downloaded WIX sources and started tweaking. Simple replacing WelcomeEulaDlg with WilcomeDlg did not work - there were messages about "next" button left without any action, but what is the "install" action? After few tries I broke my perfectly working installer into something unusable. Fortunately I foresaw that problem and put the script into ClearCase to be able to track my progress. I restored the last working revision and started gradually copying pre-built UI out of WixUI_Minimal into my script. First the UI itself, then the WelcomeEulaDlg. After each iteration I checked that the installer was still working. The last step was to update the dialog with content similar to WelcomeDlg dialog. The whole process actually was a trivial task. By the end the installer worked correctly without any funny EULA business.

This is what I have right now. I guess it can be cleaned up a little, but it should be done with care, each time checking that the installer is still working, so I am not going to do it now:

<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
  <Product Id="...." Name="... HotFix 3" Language="1033" Version="3.1.0.3" Manufacturer="..." UpgradeCode="...">
    <Package InstallerVersion="200" Compressed="yes" />

    <Media Id="1" Cabinet="hotfix3.cab" EmbedCab="yes" />

    <Directory Id="TARGETDIR" Name="SourceDir">
      <Directory Id="ProgramFilesFolder">
        <Directory Id="INSTALLLOCATION" Name="hotfix3">
          <Component Id="....dll" Guid="..." DiskId="1" Feature="ProductFeature">
            ....
          </Component>
          ...
        </Directory>
      </Directory>
    </Directory>

    <Feature Id="ProductFeature" Title="Hotfix 3" Level="1">
    </Feature>
    <Property Id="WIXUI_INSTALLDIR" Value="TARGETDIR" />
    <UI>
        <TextStyle Id="WixUI_Font_Normal" FaceName="Tahoma" Size="8" />
        <TextStyle Id="WixUI_Font_Bigger" FaceName="Tahoma" Size="12" />
        <TextStyle Id="WixUI_Font_Title" FaceName="Tahoma" Size="9" Bold="yes" />

        <Property Id="DefaultUIFont" Value="WixUI_Font_Normal" />
        <Property Id="WixUI_Mode" Value="Minimal" />

        <DialogRef Id="ErrorDlg" />
        <DialogRef Id="FatalError" />
        <DialogRef Id="FilesInUse" />
        <DialogRef Id="MsiRMFilesInUse" />
        <DialogRef Id="PrepareDlg" />
        <DialogRef Id="ProgressDlg" />
        <DialogRef Id="ResumeDlg" />
        <DialogRef Id="UserExit" />

        <Dialog Id="WelcomeEulaDlg2" Width="370" Height="270" Title="!(loc.WelcomeEulaDlg_Title)">
            <Control Id="Bitmap" Type="Bitmap" X="0" Y="0" Width="370" Height="234" TabSkip="no" Text="!(loc.WelcomeEulaDlgBitmap)" />
            <Control Id="Description" Type="Text" X="135" Y="80" Width="220" Height="60" Transparent="yes" NoPrefix="yes" Text="!(loc.WelcomeDlgDescription)" />
            <Control Id="Back" Type="PushButton" X="180" Y="243" Width="56" Height="17" Disabled="yes" Text="!(loc.WixUIBack)" />
            <Control Id="Install" Type="PushButton" ElevationShield="yes" X="236" Y="243" Width="56" Height="17" Default="yes" Text="!(loc.WelcomeEulaDlgInstall)">
                <Publish Event="SpawnWaitDialog" Value="WaitForCostingDlg">CostingComplete = 1</Publish>
                <Publish Event="EndDialog" Value="Return"><![CDATA[OutOfDiskSpace <> 1]]></Publish>
                <Publish Event="SpawnDialog" Value="OutOfRbDiskDlg">OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 0 AND (PROMPTROLLBACKCOST="P" OR NOT PROMPTROLLBACKCOST)</Publish>
                <Publish Event="EndDialog" Value="Return">OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 0 AND PROMPTROLLBACKCOST="D"</Publish>
                <Publish Event="EnableRollback" Value="False">OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 0 AND PROMPTROLLBACKCOST="D"</Publish>
                <Publish Event="SpawnDialog" Value="OutOfDiskDlg">(OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 1) OR (OutOfDiskSpace = 1 AND PROMPTROLLBACKCOST="F")</Publish>
            </Control>
            <Control Id="InstallNoShield" Type="PushButton" ElevationShield="no" X="236" Y="243" Width="56" Height="17" Default="yes" Text="!(loc.WelcomeEulaDlgInstall)" Hidden="yes">
                <Publish Event="SpawnWaitDialog" Value="WaitForCostingDlg">CostingComplete = 1</Publish>
                <Publish Event="EndDialog" Value="Return"><![CDATA[OutOfDiskSpace <> 1]]></Publish>
                <Publish Event="SpawnDialog" Value="OutOfRbDiskDlg">OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 0 AND (PROMPTROLLBACKCOST="P" OR NOT PROMPTROLLBACKCOST)</Publish>
                <Publish Event="EndDialog" Value="Return">OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 0 AND PROMPTROLLBACKCOST="D"</Publish>
                <Publish Event="EnableRollback" Value="False">OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 0 AND PROMPTROLLBACKCOST="D"</Publish>
                <Publish Event="SpawnDialog" Value="OutOfDiskDlg">(OutOfDiskSpace = 1 AND OutOfNoRbDiskSpace = 1) OR (OutOfDiskSpace = 1 AND PROMPTROLLBACKCOST="F")</Publish>
            </Control>
            <Control Id="Cancel" Type="PushButton" X="304" Y="243" Width="56" Height="17" Cancel="yes" Text="!(loc.WixUICancel)">
                <Publish Event="SpawnDialog" Value="CancelDlg">1</Publish>
            </Control>
            <Control Id="Title" Type="Text" X="135" Y="20" Width="220" Height="60" Transparent="yes" NoPrefix="yes" Text="!(loc.WelcomeDlgTitle)" />
            <Control Id="BottomLine" Type="Line" X="0" Y="234" Width="370" Height="0" />
        </Dialog>

        <Publish Dialog="ExitDialog" Control="Finish" Event="EndDialog" Value="Return" Order="999">1</Publish>

        <Publish Dialog="VerifyReadyDlg" Control="Back" Event="NewDialog" Value="MaintenanceTypeDlg">1</Publish>

        <Publish Dialog="MaintenanceWelcomeDlg" Control="Next" Event="NewDialog" Value="MaintenanceTypeDlg">1</Publish>

        <Publish Dialog="MaintenanceTypeDlg" Control="RepairButton" Event="NewDialog" Value="VerifyReadyDlg">1</Publish>
        <Publish Dialog="MaintenanceTypeDlg" Control="RemoveButton" Event="NewDialog" Value="VerifyReadyDlg">1</Publish>
        <Publish Dialog="MaintenanceTypeDlg" Control="Back" Event="NewDialog" Value="MaintenanceWelcomeDlg">1</Publish>

        <Property Id="ARPNOMODIFY" Value="1" />
    </UI>
        <InstallUISequence>
            <Show Dialog="WelcomeEulaDlg2" Before="ProgressDlg">NOT Installed</Show>
        </InstallUISequence>

    <UIRef Id="WixUI_Common" />
  </Product>
</Wix>

My primary GMail account gets very little SPAM and 99% of it is caught. I guess the spammers know that it does not make sense to spam it. So I decided to make an experiment and forwarded all my home emails to an another GMail account. At first GMail decided that most of newsletters I get is a spam (I guess many people are marking them as such instead of unsubscribing), but after writing few filters with disabled SPAM checks the things worked well. Google keeps spam messages for 30 days and now, after a month, I have the number - total 40K spam messages.

We were already running SpamAssassin on our server, so effectively I use GMail to weed extra 100 messages per day that SpamAssassin does not catch. In addition Thunderbird rules were automatically moving expected emails into subfolders, so I just had to delete [almost] all messages from inbox. I was not in a bad shape before, but now it is much better. However I wonder, how "normal" people live with it...

Unlike our previous trip there, this Saturday we were prepared. I would say even over-prepared. Last time we were driving FWD Honda CRV and had to turn back few miles east from Bowman Lake. This time we were driving real 4x4 SUV, had GPS, during daylight, together with a friend on another real SUV, and, just in case, had a strong rope...

In the morning I found a relatively inexpensive gas station in Sacramento 73 miles from home. I always like to have an almost full tank before going into woods. Car's computer also told me that there is enough gasoline for 73 miles. That was not risky - for some reason the computer believes that the tank is 2 gallons less than it is actually and we already drove several times past "0 miles" when even analog gauge showed negative gasoline level. By the time I got to the gas station, the computer still reported about 20 miles left in the tank and we pumped only 20 gallons. I wonder what my car thinks about our usual way of driving... We met with Eugene and his son Gena and continued on the path.

Driving up on highway 80 was fun, except for few slow vehicles. I was surprised to see a Prius driving together with us. We turned on highway 20 and then took Bowman Lake Road. We stopped at Jordan Creek and jumped on the rocks. There was a lot of people swimming, but we did not take our swim suits. Regardless Katya swam in the creek far from the people.

At first Bowman Lake Road was fine, then it became a 1.5 lane paved road and then the pavement ended. It looked worse than the previous time we were there, but maybe I was driving slower on CRV. It was interesting to drive on a single lane road with a cliff on one side and a possibility that another car can be driving towards you. A recent fire was still smoking near the road. There is a shortage of water in California this year and Bowman Lake, being actually a reservoir, was about 15 meters lower than the maximum level. The weather was about 22 C and the water was not comfortable. It did not stop Katya from diving. Being a young man, Gene had to swim too. Then that my wife went swimming. After that me and Eugene had to go swimming too...

After a small picnic, we prepared to go further. However the road was fixed and the drive was uneventful. We went to Truckee. Eugene and Gene went home. We stayed, ate a pizza and decided to take Donner Pass Road back. We stopped on the vista. There was so much smoke in the air, Donner Lake was almost invisible. We looked at artificial railroad tunnel - a train track covered in concrete and noticed cars driving there. Obviously the tacks were removed there and we decided to go there too. We drove into two first tunnels, but they were narrow for just a single can to pass. The big tunnel was wider, but we decided not to go there yet. Maybe next time.

Ever wondered if your policy assemblies work?

.Net assemblies remember what versions of referenced assemblies they were built with. In general it is a good thing, but it creates its own problems when building a product that consists of many separate parts. In the interest of QA we often build mismatched package builds with some of the application products built with earlier versions of core libraries. Fortunately our build scripts (directly from VisualStudio) generate all necessary policy assemblies for previous and the current releases and installer puts them into GAC.

At first there were rumors that one of applications (developed in a different office and built on its own schedule) does not start in mismatched builds. In the beginning the warnings were dismissed because the APIs were not stable, but later it became apparent that something is wrong. And finally the problem came to me...

After enabling assembly loader log, it showed that policy assembly was recognized and the redirect actually happened, but the target assembly was not found even it was right there in the GAC. However there was a message that the platform was locked to the policy assembly platform. This particular assembly had native DLLs attached to it, so its platform was not a generic "MSIL", but "x86" and this is why it could not be found.

To test this idea I regenerated policy file with matching platform, but how to install it into GAC? Removing old policy was not possible - it was locked by installer. Fortunately Cygwin`s Midnight Commander sees real file system and ignores all Microsoft crap and I was able to remove the assembly directly from hard drive and install a new one into GAC. And it helped - the application started and worked without problems.

The policy generation (and some other "black magic" staff, including support for linked resources) is done via an include in our project files and is a part of another utility project. After that project rebuild, our automatic build picked it up and certified all dependent projects to pass unit tests. Now all our policy assembly platform match their assembly's platform and everything quiet in our kingdom.