Error Logging in SharePoint

FRUB938D3AJG
As identified in my post Exception Handling in SharePoint it is important to determine the most appropriate form of logging for your situation and use it. My original intention for this post was to create more of a how-to for each possibility but in my research discovered that the number of quality resources on this topic is extensive. This being the case, the focus for this post will instead discuss the different options available, point out some of the best resources I found on the subject if appropriate and summarise what I think is the best overall option to use in most instances.

The forms of logging I identified and will discuss below include logging to the SharePoint ULS, to the Event Log, via email, a database, CRM or a SharePoint list.

SharePoint ULS

Logging to the SharePoint ULS is arguably the most logical option when it comes to logging exceptions in your custom code. It is most appropriate for developers or support staff if they have access to the files and are able to monitor them consistently. Ideally, the ULS should be monitored regardless to ensure the ongoing health of the site but under some scenarios the files may not be made available and hence logging here would go unnoticed – in this situation an alternative logging option should be considered.

The difficulty with logging to the ULS surrounds the different options available depending on what version of SharePoint you’re using. SharePoint 2010 makes this task a breeze with its SPDiagnosticsService functionality which is available in both the free (SPF) and licenced (SPS) versions of the product. The best articles I found on this topic were Tobias Zimmergren’s article on SP 2010: Developing for performance Part 4 – Logging and Waldek Mastykarz’s extension of this in Logging to ULS in SharePoint 2010.

The task is more difficult in SharePoint 2007. MOSS at least provides some functionality to write to the logs via Microsoft.Office.Server.Diagnostics.PortalLog.LogString (briefly documented in Clayton James’ post Log errors to SharePoint Log file) but that is not available in WSS. There are some examples listed including Eli Robillard’s SharePoint Trace Logs and the Unified Logging Service (ULS) however a number of links in that article are broken now. Gunnar Peipman’s SharePoint: Writing messages to ULS (Unified Logging System) is another quality post however a reasonably complicated solution. There is also a Codeplex SharePoint Logging Library by Ayman El-Hattab that may be able to be harnessed. As you’re probably appreciating now, logging to the ULS in WSS was not made easy.

There is however one saving grace for any version of SharePoint when it comes to logging to the ULS and that is the
patterns & practices SharePoint Guidance Library which i’ll discuss in more detail later.

The other possible qualm about logging to the SharePoint ULS is the difficulty in reading them. This is made easier in SharePoint 2010 by Microsoft’s ULS Viewer or Stefan Gordon’s SharePoint ULS Log Viewer on Codeplex. Tools also exist for SharePoint 2007 including Florin Muntean’s SPTraceView and Stuart Starrs’ WSS / MOSS Log File Reader – both on Codeplex.

Event Logs

The other logical place to log exceptions is the Event Log. It is most suitable for system administrators or developers and support staff who have access to the server (hopefully this isn’t the case in production environments!). Much like the ULS it is reliant on the logs being monitored on a consistent basis and also on the system administrators reporting any issues back to the development or support teams.

Again SharePoint 2010 steps up to the plate with it’s SPDiagnosticsService class, also documented in Tobias’ post outlined previously. SharePoint 2007 has to rely on core functionality in the System.Diagnostics namespace called EventLog.WriteEntry briefly documented in SharePoint King’s Event Log Access Denied : Sharepoint /.NET. As that article points out (to a degree), you need to ensure that access to write to the Event Log is granted to the relevant accounts.

One thing to point out here is, like the ULS logs above, the patterns & practices SharePoint Guidance Library provides the ability to write to the Event Logs as well.

Email

Logging exceptions via email is an interesting one. In my opinion, email should only be used hand in hand with another logging option primarily for the purposes of notification. It is mostly required if the primary log store isn’t checked on a regular basis. There are 2 options available here; either use the in-built SharePoint emailing capability of SPUtility.SendEmail or using .NETs SmtpClient class – both documented in Mohd Sukri At’s post Send Email in SharePoint via .NET SmtpClient Class and SharePoint SPUtility Class.

Database

I’m not a huge advocate of using a database logging mechanism in a SharePoint context however it may be appropriate if there is a need to aggregate exceptions from multiple (and non-SharePoint) applications across an organisation. If database logging is desired then a library should be used such as log4net or the Enterprise Library Application Block. There is a good article on Configuring Enterprise Library 4.0 for Exception handling and Logging by Suleman Ibrahim and some potential issues in Cory Roth’s How to get Enterprise Library working with SharePoint if you choose to go down this path. While on the topic of the Enterprise Library it may be worth pointing out that it is also possible to use it to log to the ULS logs demonstrated in Madhur Ahuja’s post Implementation of Logging and Instrumentation Application Blocks in MOSS 2007.

CRM

My opinions on logging to CRM are similar to that of a database. The most appropriate reason for this however would be in a tightly integrated SharePoint and CRM application where interaction mostly takes place on the CRM side. I’ve actually been involved in projects like this where logging exceptions to CRM was deemed more appropriate than SharePoint as it would aggregate issues from both systems and be more closely monitored. At the risk of exposing my lack of CRM knowledge and in lieu of providing a link to an example, the premise would be to have a log entity in CRM and have a library code-side which creates a dynamic entity of that type, populates the relevant fields, creates a TargetCreateDynamic object binding the dynamic entity and a request binding the target, then executing the request via the CrmService. Sounds easy, is relatively easy, but will leave this to the CRM experts.

SharePoint list

I find it interesting that I have yet to see an implementation of exception logging to a SharePoint list. It’s a highly visible and accessible location to monitor issues within a SharePoint site. Another plus to logging to a SharePoint list is that it is sandbox-friendly and hence a viable option for SharePoint 365. Timmy Gilissen identifies this in his post SharePoint 365 logging and the example is not restricted to just sandboxed solutions (note that if you’re going to take the code from that page you may also want to read Rob Garrett’s post Efficient way to add a new item to a SharePoint list).

On the topic of sandboxed solutions it is something worth pointing out in terms of the other logging mechanisms. It essentially makes them far more difficult to implement, for instance logging to the ULS or Event Logs in a sandbox requires a full-trust proxy due to the Microsoft.SharePoint.Administration namespace not being available in a sandboxed environment. The patterns & practices SharePoint Guidance Library however includes a full trust proxy which enables the use of it’s SharePoint Logger within the sandbox.

The patterns & practices SharePoint Guidance Library

I’ve been a little coy throughout this post regarding the patterns & practices SharePoint Guidance Library solely for the reason that I think it’s worthy of it’s own section in this post. The library is amazing on a number of fronts and this is no different when it comes to Logging. In fact, it is my most recommended option for logging exceptions assuming logging to the ULS and/or the Event Logs is suitable for your circumstances. I highly recommend you visit the page and read what the library offers both from a SharePoint 2007 and 2010 perspective. Another post worth reading in regards to the library is Alex Angas’ Intro to SharePoint 2010 patterns & practices – Logging.

As you can see there are a number of options when it comes to logging exceptions in SharePoint and depending on your circumstances, different ones may be the most suitable. The important part is that you use some form of exception logging and do it in a best practice manner.

Exception Handling in SharePoint

I feel like i’m going to be opening a can of worms with this one. I’ve given it a fair bit of thought over time and for this post specifically. I’ve read a lot of guidance around the topic. For the most part, best practice exception handling is reasonably well covered in the ASP.NET space but starts to become a bit murky when you delve into the SharePoint sphere. Some of my own personal preferences clash slightly with other advice that exists and like anything to do with SharePoint, often circumstances may dictate the path you need to take.

As such, I was slightly hesitant to write this post. The reason I have is because exception handling seems to be one of the most poorly implemented (or at least forgotten/ignored) tasks in SharePoint development. It’s also an important task when you factor in user experience, performance and ability to monitor your deployed applications. I’ve seen some shockers; from potential exception-throwing lines of code going unmanaged in web parts which could  bring down the entire page, to swallowed exceptions that you’ll never even know are there. I’ve seen catch (Exception ex) blocks scattered everywhere and try-catch blocks used to control execution flow or used in place of pre-exception checks.

There is a lot of good information out there. MSDN has version-specific editions regarding Handling and Throwing Exceptions. There are some good articles written for .NET in general including Scott Mitchell’s Exception Handling Advice for ASP.NET Web Applications, Daniel Turini’s Exception Handling Best Practices in .NET and Russell Allen’s C# .net Exception Handling Best Practice – As Easy as 1, 2, 3?.

The general consensus tends to boil down to a few rules. Generally, the base exception classes (Exception, ApplicationException) shouldn’t be caught or thrown. Exceptions should only be handled if you can add value to how they’re processed, otherwise they should be allowed to bubble up the chain. Finally, exceptions should be reported so corrective action can be taken if nececssary – preferably using an existing framework such as the Exception Management Application Block.

The question is how these rules and surrounding information can be tweaked to be suitable for SharePoint projects. I’ll structure my comments around this by identifying some general rules and outlining where they’re applicable in a SharePoint context.

1. Never, ever swallow an exception.

Pretty obvious, but the following is a definite no-no.

try
{
    // code
}
catch { }

At the very least the error should be reported using error logging functionality, if not handled sufficiently. I have seen an instance whereby the exception was expected and superfluous to report – I wish I could find it now or even remember why to explain – but if this unlikely case ever occurs it should be properly documented in the code.

2. If you’re going to throw an exception, use throw;

A common piece of code I see goes something along the lines of:

try
{
    // code
}
catch (Exception ex)
{
    throw ex;
}

This is actually quite bad practice. The reasons why were highlighted in some of the links I mentioned earlier, but essentially by doing this you clear the stack trace which is very useful when debugging the exception down the track. You’re better off using the following:

try
{
    // code
}
catch
{
    throw;
}

3. Wherever possible handle the specific exception.

The most common form of exception handling I see is something along the lines of:

try
{
    // code
}
catch (Exception ex)
{
    // handle exception
}

The problem is this goes against the general philosophy behind exception handling. Lee Dumond’s post Friends Don’t Let Friends catch (Exception) explains this nicely.

If you know the object model, or at least know where to go to find the information, you’ll be able to find out which exception/s you should be handling. Take the GetList function in SPWeb. That page identifies clearly the exceptions that may be thrown so you should handle them appropriately.

try
{
    spList = spWeb.GetList("/Path/List");
}
catch (FileNotFoundException ex)
{
    // handle exception
}
catch (ArgumentException ex)
{
    // handle exception
}

An important point to note is that your catch blocks must go in order of most specific to least specific.

4. If you can check for a condition before it throws an exception, do so.

A common block of code you may see is something along the lines of:

try
{
    myString = listItem["MyField"].ToString();
}
catch (Exception ex)
{
    // handle exception
}

Sure, if MyField was never populated in the list item it would throw an exception when you attempt to convert it to a string, but why not check to see if it was populated in the first place?

if (!string.IsNullOrEmpty((string)listItem["MyField"]))
{
    myString = listItem["MyField"].ToString();
}

5. You got to know when to hold ’em, know when to fold ’em.

Ok I may have stretched the poetic licence a bit there. What i’m talking about somewhat contradicts the rule regarding letting exceptions bubble up the chain to be handled at the application level. Consider you have a web part on a page, relatively non-critical to the page or application as a whole, a weather web part for instance. Do you really want the whole page to error out because it generated an exception? I wouldn’t begrudge you a global catch (Exception) in that instance.

On the flip side, imagine an event receiver that is processing a list item that is currently being added. The processing is necessary to complete the step of adding the list item, but the exception is caught, reported and held. The list item will exist, but the process did not complete, and depending on the level of reporting the user may never have a clue.

The point is you really need to use your best judgement when it comes to error handling in SharePoint. It’s hard to state a catch-all rule that can be applied across the board, often it will depend on context.

6. Determine the most appropriate form of logging and use it.

I would have preferred to state how errors should be logged in a SharePoint application much more definitively than this. The most important takeaway is to ensure that the errors are logged. The 2 most logical locations to log the errors would be the ULS or Event Log, however this may not always be appropriate. If these locations aren’t regularly monitored or accessible by the people who need to know about the errors, then it is essentially no better than swallowing the errors in the first place. Email, CRM, a database or even a SharePoint list are other potential locations that could be more accessible. Logging to a file on the file system makes little sense seeing if that is an accessible option you could just use the ULS. I write on this topic in more detail and explain my preference for logging to the ULS/Event Log using the SharePoint Guidance Library in my post Error Logging in SharePoint.

7. Use a custom error page.

Nothing looks worse on a public facing internet site or even an intranet application than when you get the ‘yellow screen of death’. Wherever possible you should display a user friendly message rather than an ugly generic error page. I explain the options available around this topic in my post Custom Error Pages in SharePoint.

That about covers it. Just thinking about this post made my head spin around in circles so it’s no wonder why a vast number of developers take the shortcut route and use global try-catch blocks with generic Exceptions. Hopefully this post sheds some light however on why doing so is a bad idea and that there are better options available to be leveraged. If anyone wishes to comment on the topic I’d be very interested in hearing opposing views and ideas – I’d still consider myself learning the best practice approach to exception handling at this point.

Application Settings in SharePoint

One of the most commonly neglected features in SharePoint I find across a number of different projects is the manner in which application settings are managed. Far too often it is left to manually modifying the <AppSettings> section of the web.config file for the relevant web application. Perhaps this is due to legacy behaviour of ASP.NET developers moving into the SharePoint sphere, perhaps its due to environments being small and the risks of manually modifying the file not being exposed. It may be due to organisational process or procedure, or (most likely) it may be because on face value it appears to be the most simple, quick and easy option to maintain some configuration values.

It’s ironic that in projects laced with features and solutions to deploy resources ‘the right way’ that configuration is overlooked and remains a manual process. Aside from wanting to avoid any manual modification of files directly on the server by principle, dangers exist in terms of ensuring the web.config files are the same on all web front end servers. Maintaining application settings this way also requires access to the web.config file (and hence server) itself which wouldn’t be considered best practice, and the act of changing a variable in the web.config directly would cause the application pool to refresh and cause the site to load slowly on the next hit.

I write this post with a tinge of regret – I too have been guilty on multiple occassions of maintaining the status quo and leaving (and even adding to) configuration values maintained manually in a web.config file. So not only should this post serve as a source of information regarding the best practice methods to maintain application settings, but it should also serve as a motivator to myself and others to practice what I/we preach on all future greenfield projects.

Anyone who has studied for or taken the application development exam (70-576) would have noticed that 4 main options consistently get presented for application configuration: web.config, lists, property bags and the hierarchical object store. I’ll discuss each option briefly and provide some situations where the method would be appropriate.

Web.config

Now it may seem like i’m beginning to contradict myself here – I previously mentioned that manually maintaining application settings in the web.config was a BAD idea. It is. The beauty of SharePoint though is that it provides a mechanism to maintain application settings in the web.config file automatically via the SPWebConfigModification class. If you’re wanting to store them in the web.config, then a feature should be created which on activation uses the SPWebConfigModification class to modify the web.config for you. This ensures that the same values are written to the web.config files on all front-end web servers and even ensures they’re consistent amongst new servers added to the farm.

You’re likely to want to use this option if you just can’t pull yourself away from the web.config for variable configuration. At least its a safer method. There are some valid reasons however – it’s reasonably simple to incorporate this functionality and then easy to program with seeing you’re probably used to it already. It ensures values are consistent across all front-end web servers and are available and consistent across the entire web application. It’s also fairly useful for ‘set and forget’ the values – if you’re unlikely to need to change them in the future, then you won’t run into the application pool refresh issue I mentioned previously and will get the performance benefits of the cached settings.

Before running off and using this method however, be warned that the web is littered with issues and problems with using SPWebConfigModification. I’d probably want to avoid making any ‘unnecessary’ changes to the web.config especially for the purposes of setting configuration values considering there are other options available.

SharePoint lists

It almost seems too easy and obvious. SharePoint provides us with these useful things called lists. Most of us would be familiar with or have used database-driven variable configuration mechanisms before. Lists, on face value, are reasonably similar to database tables. Why don’t we use lists for variable configuration? It’s not actually that silly a proposition and is in fact one of the more reasonable options. One of the beauties of using lists is that you get all that SharePoint listy goodness. Versioning, security and an out-of-the-box UI for example. You’re likely already familiar with list-based programming, so that isn’t too much of an inconvenience.

Using SharePoint lists is definitely one of the preferred methods. It’s probably easier to outline some of the reasons why you may not want to use it; a site with lax security restrictions would probably be wise to avoid this method – you wouldn’t want any user being able to modify or delete configuration items or the list itself. If you wanted the application settings to be available to multiple site collections/web applications then you’d potentially end up duplicating the list values across these sites.

There’s another huge advantage to choosing this method. An MVP named Chris O’Brien has written a Config store framework available on Codeplex which makes the option even more attractive and erases some of the drawbacks from choosing it. I’d definitely recommend reading a bit about it in his post A better Config Store for SharePoint sites.

Property bags

This option is perhaps a little less obvious but similarly powerful. A number of SharePoint objects contain a Properties property which stores a collection of key/value pairs. These obviously lend themselves towards being used for storing application settings.

This option is possibly the best one to harness when scope is an important consideration for configuration. It lends itself to being able to maintain numerous variables with different values on a per-scope basis (per web, per web application and so forth).

Like the SharePoint list option, this also benefits from addons which improve the usability and effectiveness of the option. The SharePoint Guidance Library provides the Application Setting Manager which is an API to read and write application settings. Codeplex also has the SharePoint Property Bag Settings project which provides a GUI in Central Administration to manage the settings.

Hierarchical Object Store

This is a farm-scoped store similar to a property bag. It requires custom code to create and is far more complex than the other options, however has the benefit of being able to store more complex types if that’s neccessary. Generally for the purpose of variable configuration though, in my opinion i’d go for one of the other options.

You’d possibly choose this option if you were looking for a global storage option for values or required complex values to be stored. If the values are simple key/value pairs however a farm-scoped property bag would probably be easier to use and do the trick.

WebPart properties

I know I said before there were 4 main options, but I wanted to briefly cover this one too. In the event that your custom web parts require configuration values specific to the web part then it will often make sense to include custom properties editable in the web part itself – they are then able to be set by modifying the settings of the web part.

My initial concept for this post was to include an array of advantages, disadvantages and code samples on each option to more completely highlight where each could be used and how they’re used, however during my research for this post came across an MSDN article written by David Mann titled Managing Custom Configuration Options for a SharePoint Application – it essentially covers in reasonable detail those points and I would strongly recommend reading.

Overall I’d have to say I prefer the SharePoint list method the best. I’m somewhat surprised, I remember thinking a while ago that using lists for application settings would be a bit primitive and that surely the other options which have been provided (obviously for the purpose of variable configuration?) like SPWebConfigModification, SPPersistedObject and the Properties collections would be the best practice method to use. I guess sometimes the easy option actually is the best, and it’s sandbox-friendly to boot.