blissful abstraction

Using PowerShell and Reflection to Update CanToggleHidden

Paul Horsfall — Mon, 12 Nov 2007 20:12:21 GMT

If you have SharePoint list that contains a hidden field with the CanToggleHidden field set to True, there's no way for you make the field visible through either the UI or the object model. No way that is, except for to use reflection to reach into the SPField class and have a non-public method do the work for you. This is the situation I found myself in and Colin's solution was perfect, but rather than creating ConsoleApplication617 I thought it would be nice to use PowerShell to do this more interactively. It went a little like this:

   1:  [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

   2:  $site = new-object "Microsoft.SharePoint.SPSite" "http://localhost/applications/ms"

   3:  $web = $site.OpenWeb()

   4:  $list = $web.Lists["Categorizations"]

   5:  $field = $list.Fields["Sequence"]

   6:  $type = $field.GetType()

   7:  $mi = $type.GetMethod("SetFieldBoolValue", [Reflection.BindingFlags]"Instance,NonPublic")

   8:  $mi.Invoke($field, ("CanToggleHidden", $true))

   9:  $field.Hidden = $false

  10:  $field.Update()

(PowerShell's enum syntax is interesting in that it will coerce a comma-separated list of values into OR'd flags as on line 7.)

Error - This page has been modified since you opened it. You must open the page again.

Paul Horsfall — Thu, 27 Sep 2007 22:52:53 GMT

We were seeing an intermittent problem on our MOSS farm whereby we'd receive the following error when saving changes to list items and web pages etc.

"This page has been modified since you opened it. You must open the page again."

There is an article on TechNet talking about a similar issue which affects SharePoint Portal Server 2003 when you're using Windows NLB and have affinity set to 'none' as we had on our MOSS farm. It suggests making sessions sticky (as Joel Oleson does) which we've done, and up to this point it looks to have resolved the issue with no side effects.

SharePoint Bookmarklets

Paul Horsfall — Mon, 13 Aug 2007 18:26:02 GMT

To avoid littering my bookmarks toolbar with a zillion buttons, I've created these bookmarklets which help get you to useful places in SharePoint relative to the site you're currently viewing.

To add them to your browser, right-click on each of the links above and select "Bookmark this link" or "Add to Favorites". For convenience you'll probably want to add them to either the "Bookmarks toolbar" or "Links" folder, making sure the relevant toolbar is visible in your browser. I've added a few these to a MOSS sub-folder so that they appear as a pop-up menu. And before you say anything, I know I could add these to the Site Actions menu but I think these are quicker to use and easier to set-up.

Some of these are MOSS specific but it should be fairly obvious which will work with WSS. They all work in the same way, using the 'L_Menu_BaseUrl' JavaScript variable to build up the URL of a page usually in the '_layouts' folder. If I've not included a bookmarklet for that page you're always hunting for, it should be easy enough for you to put it together yourself.

Hope you find them useful.

Importing Existing HTML Content into MOSS

Paul Horsfall — Sun, 29 Jul 2007 17:29:03 GMT

I had a requirement to import about 1800 pages from our existing Intranet into a new MOSS based site. Here are a few resources I've used which might be useful if you're doing something similar.

Programmatically Adding Pages to a MOSS Publishing Site
I decided to put together a tool that would walk an XML file exported from the existing site describing the structure of the content, creating MOSS sites (from custom site definitions) and importing pages as it went. At its core is code based on Andrew Connell's Programmatically adding pages to a MOSS Publishing site post.
Html Agility Pack - A .NET Html Parser
To ensure that links within our content continue to function it was necessary to parse each Html page and pull out the href attribute from each A tag so that the Url could be rewritten. Html Agility Pack is a wonderful .NET library that makes this simple; you load Html from files, streams or strings and query for Html nodes using XPATH. A simple href rewriting sample would look something like this:
```
   1:  HtmlDocument doc = new HtmlDocument();
```
```
   2:  doc.Load("file.htm");
```
```
   3:  foreach(HtmlNode link in doc.DocumentElement.SelectNodes("//a[@href]")
```
```
   4:  {
```
```
   5:    HtmlAttribute href = link.Attributes["href]";
```
```
   6:    href.Value = FixLink(att);
```
```
   7:  }
```
```
   8:  doc.Save("file.htm");
```
I've ended up using this for a lot more than rewriting links as this proved the perfect opportunity to modify some ids that are better applied as classes, set certain links to open in new windows and replace tokens used by the existing CMS.
Automatically Publishing All Items in a Publishing Site
If you're doing anything like this you'll invariably find yourself with a few hundred pages to check-in, publish or approve at some point. Well, Mr. Connell has saved us some work again, his extensions to stsadm.exe make publishing all of your pages as simple as:
```
stsadm.exe -o publishallitems -url http://localhost -list Pages -includesubsites
```
Note: There's a bug in the version I downloaded which results in an infinite loop when using the 'includesubsites' option; the solution is documented in the comments of Andrew's post.
3rd Party Tools
Depending on your specific situation it might be appropriate to look at 3rd party tools and although I've never used it, Metalogix Migration Manager looks to be a comprehensive solution. (Their site is down at the time of posting. In the meantime, you can read more in this post by Stefan Gossner.) If there are any other tools I should mention leave a comment and I'll add them here.

Unable to Configure MOSS Property Import Mappings

Paul Horsfall — Fri, 13 Jul 2007 20:43:23 GMT

Today I spent some time troubleshooting a problem whereby it wasn't possible to add or edit user profile property import mappings because the 'Data source field to map' drop down and the 'Enter field to map' text box were missing from the 'User Profile Property' page in shared services administration. Instead of seeing this:

I had the following, with the note 'The selection of directory service properties is disabled because the portal is in an untrusted domain or no directory service import is configured yet'.

Additionally, the 'Manage Connections' > 'Edit Connection' page was missing the options to select the active directory source from which to import profiles. Any attempt to change the settings on this page resulted in a popup with the following error. 'This directory service setting has not been validated because the current user is not authenticated to the directory. It may be a valid setting. Do you want to save this directory service connection setting?'

Although not entirely clear, the current user to which this error refers is the account used for the shared service provider application pool. I had it configured to use a local service account which would successfully import profiles from AD (using the default content access account) but this apparently isn't enough to allow you to work with property import mappings. Once I'd switched the SSP application pool to use a domain account everything behaved as expected.

Passing the Trunk Test with SharePoint

Paul Horsfall — Wed, 27 Jun 2007 00:04:05 GMT

Having just got back from Las Vegas I've spent the last couple of days getting back up to speed on the SharePoint project I'm working on. (Actually, I've been back a little more than a week now, but never mind.) I haven't yet posted anything about the work we're doing (although I hope to change that in the not too distant future) but in essence two developers, a designer and I are working on migrating an Intranet to SharePoint. Although I certainly consider myself a developer rather than a designer the work we are doing has piqued my interest in web usability enough that I packed a copy of Don't Make Me Think for the trip and killed the majority of time I was awake on the flight out reading it cover to cover. If you're new to user experience or you’re looking for some help taking a fresh look at an existing site (which you may well have stopped looking at objectively a long time ago), I think you'd likely find it worth your time spending a couple of hours exploring the principals of simplicity and convention extolled in Steve Krug's excellent book. Ok, so this book is short, but not so short as to make me distilling it down to two principles fair; read this recent review for a more detailed appraisal. (It's worth pointing out that this book is specifically about web sites, including e-commerce sites, rather than web applications.)

About halfway through the book Krug presents his "Trunk Test", the idea being that there are six questions you should be able to answer "without hesitation" (emphasis mine) if you were "dumped on a page somewhere deep in the bowels of a Web site". This got me thinking about how we're tackling each of these on our project and more generally about the tools that SharePoint, and in particular the MOSS web content management features give you to help make sure you pass the test.

What site is this? (Site ID)
An easy starter for 10 then, master pages are your friend. (Although you might not be on speaking terms once you've finished...) Use them to give all you pages a consistent look and feel.
What page am I on? (Page Title)
Again, simple on the surface, as long as you don't lose your mind and leave the page name out altogether. The trick is in the placement, and the Title field control within each of your page layouts correctly encourages you to have the title frame the content, rather than the content and the navigation for example.
What are the major sections of this site?
You'll probably handle this using the Global Navigation. Although it can be used as a drop down menu we prefer to configure it to display simple tabs by setting MaximumDynamicDisplayLevels to 0 on the AspMenu control within the master page. To have your root site ('Home' in our case) show up as a tab, set StaticDisplayLevels to 2 and set StartFromCurrentNode to true on PortalSiteMapDataSource. You have considerable control over the look and feel via CSS, with the ability to specify a class to be used on each tab plus two further classes to be used for the hover and selected states.
What are my options at this level?
Things start getting interesting when you look at SharePoint's Current Navigation. There are a lot of options available both in the GUI and as properties of the menu control that are worth experimenting with for a while. While the GUI options are set per site, the properties of the menu control are set per master page, so our approach has been to create a master page and an accompanying site definition for each level of our navigation. Each site definition sets the correct master page and navigation options for a given level, and ensures that sub-sites can only be created using an appropriate site definition. Be sure to use the PortalSiteMapDataSource site map data source here as it contains enhancements that make it more suitable than the default site map data source for use on a MOSS publishing site. If you've not already done so, read Part 1 and Part 2 of the MOSS Navigation Deep-dive to get an overview of how all of this hangs together, then read the comments to get an idea of some of the challenges.
Where am I in the scheme of things?
As well as the breadcrumb (implemented using ASP.NET's SiteMapPath control within PlaceHolderTitleBreadcrumb) consider the more subtle cues from the currently selected elements of the Global (tabs) and Current Navigation.
How can I search?
Every master page includes a search box by default and unless you have a good reason to do otherwise you should leave it there. On a publishing site use the SearchBoxEx control which is more configurable than the default SearchBox although not well documented. Have a look at the out of the box publishing master pages (BlueBand.master etc.) and the class documentation on MSDN as a starting point. (Tip: If you want your search box to respect Site Collection Administration > Search Settings, be sure to set the UseSiteDefaults property to true.)

This isn't meant to be a comprehensive list of the usability related publishing features in MOSS by any means, and don't get me wrong, this stuff can be difficult and it isn't made any easier by the sporadic documentation. However, once up the learning curve there is much at your disposal to tackle the questions Krug poses, and I'm happy with the answers we've come up with so far.

How are you using SharePoint to pass the Trunk Test?

Drag and Drop, Today and Tomorrow

Paul Horsfall — Thu, 31 May 2007 00:34:43 GMT

Once or twice we've talked at work about how we'd like the ability to reorder programs (buttons) on the taskbar, and for a while now I've been running a small utility at home called Taskbar Shuffle that allows you to do just that. While it might not be quite ready to challenge Jeff Han's multi-touch work or Microsoft Surface, it does fill one of those "I'm surprised it's not in Windows" holes and is worth a look. (It works great on my XP box and although Vista isn't officially supported, some folk report that it runs under the 32-bit version albeit with some graphical glitches.)

P.S. If you download and get use from a piece of free software, don't forget to either register it or make a donation to help your fellow developer.

Stsadm.exe and iisreset Slow Behind Proxy on SharePoint

Paul Horsfall — Sun, 27 May 2007 17:09:28 GMT

While putting together my last post I ran across a strange problem where the first page access after an iisreset was taking more than 30 seconds to load. At its best, this always takes about twice as long as you'd like but this was unworkable. Worse still the same delay happened each time I used stsadm.exe or any other application that loads the SharePoint DLL.

After a reasonable amount of time troubleshooting this I was fast running out of ideas, and desperate enough to reach for Ethereal. Thankfully, it was there I found the answer to my problem. Each delay was accompanied by outbound requests to http://crl.microsoft.com or http://crl.verisign.com that from behind our proxy were going nowhere fast and taking a long while to timeout. (In this case, CRL stands for Client Revocation List, and I assume it is been checked to ensure the validity of the digitally signed SharePoint assemblies as they are loaded into the IIS process.)

The solution for my situation then, was as simple as specifying the correct proxy using proxycfg.exe. From a command prompt, you'd do something like this:

proxycfg -p 192.168.0.1:8080

Update 27/06/2007: You need to modify this command to exclude local addresses to prevent MOSS crawls failing, as they too respect this setting. You can exclude all local addresses (those that don't contain a dot) as follows:

proxycfg -p 192.168.0.1:8080 "<local>"

Alternatively, you can exclude specific URLs:

proxycfg -p 192.168.0.1:8080 "http://moss01"

See the proxycfg.exe documentation for further details.

This workaround is good enough to get me going again. However, I'm no expert on certificate infrastructure so leave me a comment if I've missed something or got something wrong.

Hope this helps.

SharePoint, Sysprep and SQL

Paul Horsfall — Mon, 21 May 2007 18:31:28 GMT

A while back, I posted about creating a cloneable SharePoint development machine. Although the method I described works well enough, you're somewhat limited by the fact that it requires you to install SharePoint in Standalone mode. This is workable for some scenarios, but the performance of SQL Server Embedded Edition (which Standalone mode uses) doesn't seem all that great, and by not running a full version of SQL you're missing out on the profiler and Reporting Services integration etc. Additionally, you're stuck running everything on one box with nowhere to go if you want to add additional virtual servers to test other topologies.

For these reasons, I've spent some time making Sysprep work with a Complete, rather than Standalone install of MOSS. If you've not done so already, now would probably be a good time to go back and read my previous post as I'll just point out the differences here to avoid duplication.

If you're updating an existing Standalone install you need to uninstall SharePoint. You'll also want to manually uninstall SSEE.
Install a full version of SQL Server 2005 with SP2. I'm using the developer edition configured to use Windows authentication and run as Local System. (I'm happy with this, although you might want to follow the principal of least privilege and use a less powerful account.)
(Re)Install SharePoint, and from the Advanced screen select Web Front End if you're using WSS, or Complete if you're using MOSS. As before, do not run the SharePoint Products and Technologies Configuration Wizard after the install.
Create a local account for use as a SharePoint service account. You don't need to assign any specific permissions to the account at this point, this will happen when we call psconfig.exe from our configuration script. Update: If you intend to set-up custom profile import mappings read this follow-up post.
From the zip file linked at the bottom of this post, extract mossconfig2.bat into c:\Scripts. (You will still need the files from the previous post.) You also need to modify c:\Sysprep\sysprep.inf to call this new script, which now takes 4 parameters:
```
mossconfig2.bat <SiteOwner> <SiteOwnerEmail> <ServiceAccount> <ServiceAccountPassword>
```
Since this script runs after the first logon following Sysprep we can assume that the current user is the intended site owner and build the first two parameters from environment variables. The last two parameters are the username and password of the service account we created earlier. The script requires the service account username to be prefixed with the computer name, which is again achieved using environment variables. See sample.inf included in the download for an example of how to call this from Sysprep.

Once you've made these changes you can take a snapshot of the machine and run Sysprep to reseal to machine.

The trick to making this work is the significantly updated configuration script that automates the work which normally happens when you run the SharePoint Products and Technologies Configuration Wizard. It looks like this:

   1:  @echo off

2:

   3:  psconfig -cmd configdb -create -server %COMPUTERNAME% -database SharePoint_Config -user %3 -password %4

   4:  if not errorlevel 0 goto errhnd

   5:  psconfig -cmd helpcollections -installall

   6:  if not errorlevel 0 goto errhnd

   7:  psconfig -cmd secureresources

   8:  if not errorlevel 0 goto errhnd

   9:  psconfig -cmd services -install

  10:  if not errorlevel 0 goto errhnd

  11:  psconfig -cmd installfeatures

  12:  if not errorlevel 0 goto errhnd

  13:  psconfig -cmd adminvs -provision -port 8000 -windowsauthprovider onlyusentlm

  14:  if not errorlevel 0 goto errhnd

  15:  psconfig -cmd applicationcontent -install

16:

  17:  :SetupServices

  18:  REM *** Moss Only ***

  19:  REM echo Starting search

  20:  stsadm -o osearch -action start -role indexquery -farmcontactemail %2 -farmserviceaccount %3 -farmservicepassword %4 -defaultindexlocation "C:\Program Files\Microsoft Office Servers\12.0\Data\Applications"

  21:  if not errorlevel 0 goto errhnd

22:

  23:  :CreateSSP

  24:  REM *** Moss Only ***

  25:  REM echo Creating webapps for SSP

  26:  stsadm -o extendvs -url http://%COMPUTERNAME%:8010 -ownerlogin %1 -owneremail %2 -exclusivelyusentlm -apidname "SSP - 8010" -apcreatenew -apidtype configurableid -apidlogin %3 -apidpwd %4

  27:  if not errorlevel 0 goto errhnd

  28:  stsadm -o extendvs -url http://%COMPUTERNAME%:8020 -ownerlogin %1 -owneremail %2 -exclusivelyusentlm -apidname "My Sites - 8020" -apcreatenew -apidtype configurableid -apidlogin %3 -apidpwd %4

  29:  if not errorlevel 0 goto errhnd

30:

  31:  REM *** Moss Only ***

  32:  REM echo Creating SSP

  33:  stsadm -o createssp -title "Shared Service Provider" -url http://%COMPUTERNAME%:8010 -mysiteurl http://%COMPUTERNAME%:8020 -indexserver %COMPUTERNAME% -indexlocation "%Programfiles%\Microsoft Office Servers\12.0\Data\Applications" -ssplogin %3 -ssppassword %4

  34:  if not errorlevel 0 goto errhnd

35:

  36:  :CreateSiteCollection

  37:  REM For WSS select a suitable site template e.g. STS

  38:  REM echo Creating site collection

  39:  stsadm -o extendvs -url http://%COMPUTERNAME%:80 -ownerlogin %1 -owneremail %2 -exclusivelyusentlm -apidname "SharePoint - 80" -sitetemplate BLANKINTERNETCONTAINER -apidtype configurableid -apidlogin %3 -apidpwd %4

  40:  if not errorlevel 0 goto errhnd

41:

  42:  iisreset

  43:  "C:\Scripts\wssdebug.exe" http://localhost /enable

  44:  goto end

45:

  46:  :usage

  47:  REM echo Usage

  48:  REM echo mossconfig2.bat OwnerLogin OwnerEmail ServiceAccount ServiceAccountPassword

  49:  REM echo\

  50:  REM goto end

51:

  52:  :errhnd

  53:  :end

  54:  pause

This is heavily based on Jukka Paajanen's How to create a web application from the command line article, with some modifications for running in this context. Once it's finished, you'll have the following setup:

A site collection built with the Publishing Portal template that ships with MOSS running on port 80.
Central Administration running on port 8000.
A Shared Service Provider running on port 8010.
A separate web application for hosting My Sites running on port 8020.

If you're using WSS you'll need to make a couple of modifications. You don't have the Shared Service Provider, My Sites or search to configure so comment out lines 21, 27, 29 and 34. In addition, change the site template parameter supplied to stsadm.exe (line 40) from BLANKINTERNETCONTAINER to STS to have the site collection use the Team Site template. Have a look in C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\1033\XML at WEBTEMP.xml (and also at WEBTEMPSPS.xml if you're using MOSS) to see all of the available templates and their names for use with this parameter.

It's worth pointing out that there is nothing in the script to perform any configuration on SQL 2005. Despite the machine having its SID replaced and its name changed SQL just keeps on working, although I don't know if this would be the case if you were to use SQL 2000. Having said that, SQL does store the machine's name internally, and the steps you need to follow to update it could easily be automated if required. Incidentally, if you need to rename a machine that is already running SharePoint check out the new 'renameserver' stsadm.exe operation.

It's less than ideal having this spread over two posts, but hopefully it's clear enough for you to get what you came here for. If not, feel free to let me know in the comments where I'll try and answer any questions.

WssSysprep2.zip

Counting Mouse Clicks With Clickr

Paul Horsfall — Tue, 08 May 2007 23:05:50 GMT

I recently found myself wondering how many times a day I click the mouse. A quick Google turned up a few promising tools, but none of them were really what I had in mind. As a big fan of keyboard shortcuts, I wanted to know not only how many times I click the mouse but also how many times I click the mouse per application, the idea being that I could use this to figure out which application's shortcuts might be worth learning. With this in mind, I present Clickr. Here's what it does:

Counts the total number of left, right and middle (wheel) mouse clicks.
For each click logs either the description of the executable (default) or window title of the current foreground window. (Logging the description makes the most sense I think. Run two instances of Clickr and log both if you like.)
Displays the total clicks, clicks per window and the date and time logging began on a Windows Form.
Minimizes to the notification area (system tray).

Most of this was put together using standard .NET 2.0 Windows Forms stuff. The key to making it work was the fact that you can use .NET to set a low level hook to monitor mouse events, and from there it only takes a little more Win32 work to get a handle to the foreground window and ultimately the window title or executable description. Once I had this working, I added a couple of optimizations to firstly cache the descriptions (which are too expensive to fetch for every click) and secondly ensure that I'm not trying to update the form when it's minimized, as this is also fairly expensive and clearly not very sensible.

I'm not sure anyone else will find this particularly useful but I had fun putting it together and so I thought I'd share it here. There's a link to the executable at the bottom of this post, and although it's not exactly production quality I'll put the source code up as well. It should run on Vista and XP (with the .NET 2.0 Framework) but I've not tested it beyond running it on my machine for a few days, so you mileage may very well vary.

Clickr_1_0.zip
Clickr_Source_1_0.zip