A co-worker was working on a new MVC project using role based authorization when he ran into a concern where a view was nearly identical between two different roles. The only difference between the views was one role had the ability to “Delete” and included a delete button, the other role didn’t have the delete capability.

One solution to the problem would have been to add the following code into the view:

if (User.IsInRole("Administrator")) {
  <button>delete</button>
}

Using the above wouldn’t be the end of the word, since the “Delete” action on the controller would be limited to only the “Administrator” role. However, in the future as the abilities of the two roles diverge, we’re likely to continue sprinkling more authorization code into our view creating a bundle of mud. I recommended the developer to create a dedicated view for each role, using partials where able to minimize code duplication.

By doing so, we can have the controller determine the correct view to render.

public class HomeController: Controller
{
  [AcceptVerbs(HttpVerbs.Get)]
  public virtual ActionResult Index()
  {
    if (User.IsInRole("Administrator")) {
        return (View("AdministratorIndex"));
    }
    return (View());
  }
}

That will work and is better than having the the authorization logic inside of the view. We could even make it better by moving the custom views into a subfolder based on the role to help keep our files organized.

- Views
  - Home
    - Administrator
      index.cshtml
    index.cshtml

That’ll help keep our role custom views organized.

But something still “smells” here. Our controller / action is now dealing with authorization concerns. Can we somehow move that authorization logic outside of our controller? Essentially our authorization logic is merely selecting what view to render. Let’s see if we can override the view selection logic…

ViewEngine (RazorViewEngine) To The Rescue

Looking at the ASP.NET MVC stack, we can see that there are ViewEngine classes, (RazorViewEngine, WebFormViewEngine) which will apply a set of rules to locating the appropriate view.

To implement this, we’ll create an RoleBasedRazorViewEngine class with the following contents:

/// <summary>
/// A razor based view engine that locates views based on their role.
/// </summary>
public class RoleBasedRazorViewEngine: RazorViewEngine
{
  private readonly IEnumerable<string> _roles;

  /// <summary>
  /// Creates an instance of the RoleBasedRazorViewEngine class.
  /// </summary>
  /// <param name="roles">The list of roles in priority order supported by the application.</param>
  public RoleBasedRazorViewEngine(IEnumerable<string> roles): this(roles, null)
  {
  }

  /// <summary>
  /// Creates an instance of the RoleBasedRazorViewEngine class.
  /// </summary>
  /// <param name="roles">The list of roles in priority order supported by the application.</param>
  /// <param name="viewPageActivator">The ViewPageActivator to use for page dependency resolution.</param>
  public RoleBasedRazorViewEngine(IEnumerable<string> roles, IViewPageActivator viewPageActivator): base(viewPageActivator)
  {
    _roles = roles ?? new String[0];

    AreaViewLocationFormats = new [] {
      "~/Areas/{2}/Views/{1}/{{0}}/{0}.cshtml",
      "~/Areas/{2}/Views/{1}/{{0}}/{0}.vbhtml",
      "~/Areas/{2}/Views/Shared/{{0}}/{0}.cshtml",
      "~/Areas/{2}/Views/Shared/{{0}}/{0}.vbhtml"
    }.Concat(base.AreaViewLocationFormats).ToArray();
    AreaMasterLocationFormats = new[] {
      "~/Areas/{2}/Views/{1}/{{0}}/{0}.cshtml",
      "~/Areas/{2}/Views/{1}/{{0}}/{0}.vbhtml",
      "~/Areas/{2}/Views/Shared/{{0}}/{0}.cshtml",
      "~/Areas/{2}/Views/Shared/{{0}}/{0}.vbhtml"
    }.Concat(base.AreaMasterLocationFormats).ToArray();
    AreaPartialViewLocationFormats = new[] {
      "~/Areas/{2}/Views/{1}/{{0}}/{0}.cshtml",
      "~/Areas/{2}/Views/{1}/{{0}}/{0}.vbhtml",
      "~/Areas/{2}/Views/Shared/{{0}}/{0}.cshtml",
      "~/Areas/{2}/Views/Shared/{{0}}/{0}.vbhtml"
    }.Concat(base.AreaPartialViewLocationFormats).ToArray();

    ViewLocationFormats = new[] {
      "~/Views/{1}/{{0}}/{0}.cshtml",
      "~/Views/{1}/{{0}}/{0}.vbhtml",
      "~/Views/Shared/{{0}}/{0}.cshtml",
      "~/Views/Shared/{{0}}/{0}.vbhtml"
    }.Concat(base.ViewLocationFormats).ToArray();
    MasterLocationFormats = new[] {
      "~/Views/{1}/{{0}}/{0}.cshtml",
      "~/Views/{1}/{{0}}/{0}.vbhtml",
      "~/Views/Shared/{{0}}/{0}.cshtml",
      "~/Views/Shared/{{0}}/{0}.vbhtml"
    }.Concat(base.MasterLocationFormats).ToArray();
    PartialViewLocationFormats = new[] {
      "~/Views/{1}/{{0}}/{0}.cshtml",
      "~/Views/{1}/{{0}}/{0}.vbhtml",
      "~/Views/Shared/{{0}}/{0}.cshtml",
      "~/Views/Shared/{{0}}/{0}.vbhtml"
    }.Concat(base.PartialViewLocationFormats).ToArray();
  }

  /// <summary>
  /// Creates a partial view using the specified controller context and partial path.
  /// </summary>
  /// <returns>
  /// The partial view.
  /// </returns>
  /// <param name="controllerContext">The controller context.</param><param name="partialPath">The path to the partial view.</param>
  protected override IView CreatePartialView(ControllerContext controllerContext, string partialPath)
  {
    return base.CreatePartialView(controllerContext, GetRoleBasedPath(controllerContext, partialPath));
  }

  /// <summary>
  /// Creates a view by using the specified controller context and the paths of the view and master view.
  /// </summary>
  /// <returns>
  /// The view.
  /// </returns>
  /// <param name="controllerContext">The controller context.</param>
  /// <param name="viewPath">The path to the view.</param>
  /// <param name="masterPath">The path to the master view.</param>
  protected override IView CreateView(ControllerContext controllerContext, string viewPath, string masterPath)
  {
    return base.CreateView(controllerContext, 
                           GetRoleBasedPath(controllerContext, viewPath), 
                           GetRoleBasedPath(controllerContext, masterPath));
  }

  /// <summary>
  /// Resolves the path based on the role information.
  /// </summary>
  /// <param name="controllerContext">The controller context.</param>
  /// <param name="viewPath">The path to the view.</param>
  /// <returns>The resolved view path.</returns>
  private string GetRoleBasedPath(ControllerContext controllerContext, string viewPath)
  {
    if ((! String.IsNullOrEmpty(viewPath)) && 
        (controllerContext.HttpContext.User != null)) {
      IPrincipal principal = controllerContext.HttpContext.User;
      foreach (string role in _roles.Where(role => principal.IsInRole(role))) {
        string resolvedViewPath = String.Format(CultureInfo.InvariantCulture, viewPath, role);
        if (base.FileExists(controllerContext, resolvedViewPath)) {
          return (resolvedViewPath);
        }
      }
    }
    return (viewPath);
  }

  /// <summary>
  /// Gets a value that indicates whether a file exists in the specified virtual file system (path).
  /// </summary>
  /// <returns>
  /// true if the file exists in the virtual file system; otherwise, false.
  /// </returns>
  /// <param name="controllerContext">The controller context.</param><param name="virtualPath">The virtual path.</param>
  protected override bool FileExists(ControllerContext controllerContext, string virtualPath)
  {
    if (controllerContext.HttpContext.User != null) {
      IPrincipal principal = controllerContext.HttpContext.User;
      if (_roles.Where(role => principal.IsInRole(role))
                .Any(role => base.FileExists(controllerContext, String.Format(CultureInfo.InvariantCulture, virtualPath, role)))) {
        return (true);
      }
    }
    return(base.FileExists(controllerContext, virtualPath));
  }

  /// <summary>
  /// Finds the specified partial view by using the specified controller context.
  /// </summary>
  /// <returns>
  /// The partial view.
  /// </returns>
  /// <param name="controllerContext">The controller context.</param>
  /// <param name="partialViewName">The name of the partial view.</param>
  /// <param name="useCache">true to use the cached partial view.</param>
  /// <exception cref="T:System.ArgumentNullException">The <paramref name="controllerContext"/> parameter is null (Nothing in Visual Basic).</exception>
  /// <exception cref="T:System.ArgumentException">The <paramref name="partialViewName"/> parameter is null or empty.</exception>
  public override ViewEngineResult FindPartialView(ControllerContext controllerContext, string partialViewName, bool useCache)
  {
    return(base.FindPartialView(controllerContext, partialViewName, false));
  }

  /// <summary>
  /// Finds the specified view by using the specified controller context and master view name.
  /// </summary>
  /// <returns>
  /// The page view.
  /// </returns>
  /// <param name="controllerContext">The controller context.</param>
  /// <param name="viewName">The name of the view.</param>
  /// <param name="masterName">The name of the master view.</param>
  /// <param name="useCache">true to use the cached view.</param>
  /// <exception cref="T:System.ArgumentNullException">The <paramref name="controllerContext"/> parameter is null (Nothing in Visual Basic).</exception>
  /// <exception cref="T:System.ArgumentException">The <paramref name="viewName"/> parameter is null or empty.</exception>
  public override ViewEngineResult FindView(ControllerContext controllerContext, string viewName, string masterName, bool useCache)
  {
    return (base.FindView(controllerContext, viewName, masterName, false));
  }
}

A Walkthrough

The logic is pretty simple.

All of the available application roles are injected into the RoleBasedRazorViewEngine in priority order. Basically if someone is a member of multiple roles, the first role with a matching view will be returned.

We prepend custom views paths to include a file path based on roles. The defaults paths use the .NET string formatting place holders. The default .NET placeholders for MVC paths are:

  • {0} – The name of the action.
  • {1} – The name of the controller.
  • {2} – The name of the area.

When dealing with .NET place holder values, if you actually want to output the value {0} in a string that is being formatted, you wrap it with double braces like {{0}}. In our custom paths, we want the default .NET MVC string substitutions to occur. After that parsing has occurred, we’ll do a second string format / replacement specifying each of the user’s roles.

  ViewLocationFormats = new[] {
    "~/Views/{1}/{{0}}/{0}.cshtml",
    "~/Views/{1}/{{0}}/{0}.vbhtml",
    "~/Views/Shared/{{0}}/{0}.cshtml",
    "~/Views/Shared/{{0}}/{0}.vbhtml"
  }.Concat(base.ViewLocationFormats).ToArray();

Given the previous example, for the Index action on the Home controller, our custom search path of view locations would be:

  "~/Views/Home/{0}/index.cshtml",
  "~/Views/Home/{0}/index.vbhtml",
  "~/Views/Shared/{0}/index.cshtml",
  "~/Views/Shared/{0}/index.vbhtml",
  "~/Views/Home/index.cshtml",
  "~/Views/Home/index.vbhtml",
  "~/Views/Shared/index.cshtml",
  "~/Views/Shared/index.vbhtml"

Or our second pass, we’ll dynamically fill in what {0} should be with one of the user’s roles.

In the FileExists method, we check to see if the user belongs to any of the predefined roles. At this point, we don’t actually return the name of the file, we just indicate if we can resolve the requested path.

  protected override bool FileExists(ControllerContext controllerContext, string virtualPath)
  {
    if (controllerContext.HttpContext.User != null) {
      IPrincipal principal = controllerContext.HttpContext.User;
      if (_roles.Where(role => principal.IsInRole(role))
                .Any(role => base.FileExists(controllerContext, String.Format(CultureInfo.InvariantCulture, virtualPath, role)))) {
        return (true);
      }
    }
    return(base.FileExists(controllerContext, virtualPath));
  }

If we had the following defined roles for our application: “Administrator”, “Operator”, “User”, the following file names would be searched in order.

  "~/Views/Home/Administrator/index.cshtml",
  "~/Views/Home/Administrator/index.vbhtml",
  "~/Views/Shared/Administrator/index.cshtml",
  "~/Views/Shared/Administrator/index.vbhtml",
  "~/Views/Home/Operator/index.cshtml",
  "~/Views/Home/Operator/index.vbhtml",
  "~/Views/Shared/Operator/index.cshtml",
  "~/Views/Shared/Operator/index.vbhtml",
  "~/Views/Home/User/index.cshtml",
  "~/Views/Home/User/index.vbhtml",
  "~/Views/Shared/User/index.cshtml",
  "~/Views/Shared/User/index.vbhtml",
  "~/Views/Home/index.cshtml",
  "~/Views/Home/index.vbhtml",
  "~/Views/Shared/index.cshtml",
  "~/Views/Shared/index.vbhtml"

If one of those paths match, the CreateView or CreatePartialView methods will dynamically expand the path using a helper function to locate the correct view based on the role.

  private string GetRoleBasedPath(ControllerContext controllerContext, string viewPath)
  {
    if ((! String.IsNullOrEmpty(viewPath)) && 
        (controllerContext.HttpContext.User != null)) {
      IPrincipal principal = controllerContext.HttpContext.User;
      foreach (string role in _roles.Where(role => principal.IsInRole(role))) {
        string resolvedViewPath = String.Format(CultureInfo.InvariantCulture, viewPath, role);
        if (base.FileExists(controllerContext, resolvedViewPath)) {
          return (resolvedViewPath);
        }
      }
    }
    return (viewPath);
  }

To use this new custom ViewEngine, we simply need to register it in our Global.asax.cs file.

    /// <summary>
    /// Occurs when the first resource is requested from the web server and the web application starts.
    /// </summary>
    protected void Application_Start()
    {
      ViewEngines.Engines.Clear();
      ViewEngines.Engines.Add(new RoleBasedRazorViewEngine(new[] { "Administrator", "Operator", "User" }));
    }

With the new view engine registered, we can go back and remove our authorization or view selection logic from our controller.

public class HomeController: Controller
{
  [AcceptVerbs(HttpVerbs.Get)]
  public virtual ActionResult Index()
  {
    return (View());
  }
}

Nice, simple, clean code with a separation of concerns! We let the view engine do it’s job and pick the right view for our controller action.

As everyone knows, mocking the HttpContext and associated classes is a nightmare and should just be avoided. I recently joined a different team at work where they were still running a lot of .NET 1.0 code. Most of the code was poorly designed and highly coupled having been written primarily by developers without proper object oriented design training. Calling this code “spaghetti code” would have been an insult to spaghetti code.

How bad? Most methods are over 1000 lines of code, filled with nested if/else statements and copy/pasted code all over. Extracting the business logic from one method and class resulted in 15 new classes. Here is an quick example of the code quality.

if (_username.ToLower().PadRight(12, ' ').Substring(0, 7).Equals("demo123")) {
}

Lots of useless string parsing and casting to wade through… But anyway, that isn’t the point of this post. Long story short is that as I’m modularizing this code I’ve run into the dreaded HttpContext.Current integrated throughout the code. Before I make extensive changes to the code, I wanted to have some unit tests to ensure that I wasn’t breaking the existing functionality as I modified the code. So the first thing I did was to inject the HttpContext as a dependency into the class. Although far from ideal, it allows me to at least run the code outside of IIS.

Here is my test helper to get the HttpContext:

/// <summary>
/// Retreives an HttpContext for testing.
/// </summary>
/// <returns>An HttpContext for testing.</returns>
internal HttpContext GetHttpContext(string url = "http://127.0.0.1/")
{
  var request = new HttpRequest(String.Empty, url, String.Empty);
  var response = new HttpResponse(new StringWriter());
  var context = new HttpContext(request, response);
  return(context);
}

Unfortunately, the code has numereous references to Request.ServerVariables. If you try to add to this NameValueCollection you’ll find that it is a read only collection. Here is the decompiled code:

public sealed class HttpRequest
{
  private HttpServerVarsCollection _serverVariables;
  public NameValueCollection ServerVariables
  {
    get
    {
      if (HttpRuntime.HasAspNetHostingPermission(AspNetHostingPermissionLevel.Low)) {
        return this.GetServerVars();
      }
      return this.GetServerVarsWithDemand();
    }
  }
  private NameValueCollection GetServerVars()
  {
    if (this._serverVariables == null) {
      this._serverVariables = new HttpServerVarsCollection(this._wr, this);
      if (!(this._wr is IIS7WorkerRequest)) {
        this._serverVariables.MakeReadOnly();
      }
    }
    return this._serverVariables;
  }
}

We can’t override ServerVariables since it’s not virtual and there is no setter. Digging deeper finds an internal HttpServerVarsCollection class which has the following Add signature:

public override void Add(string name, string value)
{
  throw new NotSupportedException();
}

The rabbit hole keeps getting deeper. Fortunately we find the AddStatic method which gives us some hope:

internal void AddStatic(string name, string value)
{
  if (value == null) {
    value = string.Empty;
  }
  base.InvalidateCachedArrays();
  base.BaseAdd(name, new HttpServerVarsCollectionEntry(name, value));
}

That looks promising. So let’s try making this work using reflection.

  var field = request.GetType()
                     .GetField("_serverVariables", BindingFlags.Instance | BindingFlags.NonPublic);
  if (field != null) {
    var variables = field.GetValue(request);
    var type = field.FieldType;
    if (variables == null) {
      var constructor = type.GetConstructor(BindingFlags.Instance | BindingFlags.NonPublic, null,
                                            new[] { typeof(HttpWorkerRequest), typeof(HttpRequest) }, null);
      variables = constructor.Invoke(new[] { null, request });
    }
    type.GetProperty("IsReadOnly", BindingFlags.Instance | BindingFlags.NonPublic)
        .SetValue(variables, false, null);
    var addStatic = type.GetMethod("AddStatic", BindingFlags.Instance | BindingFlags.NonPublic);
    addStatic.Invoke(variables, new[] { "REMOTE_ADDR", "127.0.0.1" });
    addStatic.Invoke(variables, new[] { "HTTP_USER_AGENT", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36" });
  }

SUCCESS! But we can do even better. How about we make this code extend the HttpRequest object and clean things up a little.

/// <summary>
/// Extension methods for the HttpRequest class.
/// </summary>
public static class HttpRequestExtensions
{
  /// <summary>
  /// Adds the name/value pair to the ServerVariables for the HttpRequest.
  /// </summary>
  /// <param name="request">The request to append the variables to.</param>
  /// <param name="name">The name of the variable.</param>
  /// <param name="value">The value of the variable.</param>
  public static void AddServerVariable(this HttpRequest request, string name, string value)
  {
    if (request == null) return;

    AddServerVariables(request, new Dictionary<string, string>() {
      { name, value }
    });
  }

  /// <summary>
  /// Adds the name/value pairs to the ServerVariables for the HttpRequest.
  /// </summary>
  /// <param name="request">The request to append the variables to.</param>
  /// <param name="collection">The collection of name/value pairs to add.</param>
  public static void AddServerVariables(this HttpRequest request, NameValueCollection collection)
  {
    if (request == null) return;
    if (collection == null) return;

    AddServerVariables(request, collection.AllKeys
                                          .ToDictionary(k => k, k => collection[k]));
  }

  /// <summary>
  /// Adds the name/value pairs to the ServerVariables for the HttpRequest.
  /// </summary>
  /// <param name="request">The request to append the variables to.</param>
  /// <param name="dictionary">The dictionary containing the pairs to add.</param>
  public static void AddServerVariables(this HttpRequest request, IDictionary<string,string> dictionary)
  {
    if (request == null) return;
    if (dictionary == null) return;

    var field = request.GetType()
                       .GetField("_serverVariables", BindingFlags.Instance | BindingFlags.NonPublic);
    if (field != null) {
      var type = field.FieldType;

      var serverVariables = field.GetValue(request);
      if (serverVariables == null) {
        var constructor = type.GetConstructor(BindingFlags.Instance | BindingFlags.NonPublic, null,
                                              new[] { typeof(HttpWorkerRequest), typeof(HttpRequest) }, null);
        serverVariables = constructor.Invoke(new[] { null, request });
        field.SetValue(request, serverVariables);
      }
      var addStatic = type.GetMethod("AddStatic", BindingFlags.Instance | BindingFlags.NonPublic);

      ((NameValueCollection) serverVariables).MakeWriteable();
      foreach (var item in dictionary) {
        addStatic.Invoke(serverVariables, new[] { item.Key, item.Value });
      }
      ((NameValueCollection)serverVariables).MakeReadOnly();
    }
  }
}

You might have noticed, that I also created a NameValueCollection extension to modify the IsReadOnly property. Of course, use this with care… “with great power comes great responsibility“. The creator of the NameValueCollection you’re consuming likely set the IsReadOnly property for a reason…

/// <summary>
/// Extension methods for the NameValueCollection class.
/// </summary>
public static class NameValueCollectionExtensions
{
  /// <summary>
  /// Retreives the IsReadOnly property from the NameValueCollection
  /// </summary>
  /// <param name="collection">The collection to retrieve the propertyInfo from.</param>
  /// <param name="bindingFlags">The optional BindingFlags to use. If not specified defautls to Instance|NonPublic.</param>
  /// <returns>The PropertyInfo for the IsReadOnly property.</returns>
  private static PropertyInfo GetIsReadOnlyProperty(this NameValueCollection collection, BindingFlags bindingFlags = BindingFlags.Instance | BindingFlags.NonPublic)
  {
    if (collection == null) return (null);
    return(collection.GetType().GetProperty("IsReadOnly", bindingFlags));
  }

  /// <summary>
  /// Sets the IsReadOnly property to the specified value.
  /// </summary>
  /// <param name="collection">The collection to modify.</param>
  /// <param name="isReadOnly">The value to set.</param>
  private static void SetIsReadOnly(this NameValueCollection collection, bool isReadOnly)
  {
    if (collection == null) return;

    var property = GetIsReadOnlyProperty(collection);
    if (property != null) {
      property.SetValue(collection, isReadOnly, null);
    }
  }

  /// <summary>
  /// Makes the specified collection writable via reflection.
  /// </summary>
  /// <param name="collection">The collection to make writable.</param>
  public static void MakeWriteable(this NameValueCollection collection)
  {
    SetIsReadOnly(collection, false);
  }

  /// <summary>
  /// Makes the specified collection readonly via reflection.
  /// </summary>
  /// <param name="collection">The collection to make readonly.</param>
  public static void MakeReadOnly(this NameValueCollection collection)
  {
    SetIsReadOnly(collection, true);
  }
}

And there you have it. A way to add ServerVariables. Keep in mind that this code is extremely fragile because it’s using reflection to access the internal workings of code that we don’t have control over. Below are examples of using the extension method.

public class Example
{
  public void Test() 
  {
    string url = "http://127.0.0.1";
    var request = new HttpRequest(String.Empty, url, String.Empty);
    request.AddServerVariable("REMOTE_ADDR", "127.0.0.1");

    // or
    
    request.AddServerVariables(new Dictionary<string, string>() {
      { "REMOTE_ADDR", "127.0.0.1" },
      { "HTTP_USER_AGENT", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36" }
    });
  }
}

I hope you find this useful and it can save you some time.

Disclaimer: I am neither an expert, nor fan of SharePoint. Simple fact is I abhore SharePoint.

Unfortunately, I was brought into a project that required a SharePoint list to be updated via an external web service. There are two ways to accomplish this:

  • Use the SharePoint Client Object Model (client.svc) interface and get the rich client interface similar to writing applications that are hosted within SharePoint. This requires that the Microsoft.SharePoint.Client libraries are included or available to your application.
  • Use the WCF DataServices or REST interfaces (listdata.svc) to interact with the Lists on the SharePoint site. Unlike the Client Object Model, you’ll be limited to only working with the list data doing CRUD (Create, Read, Update, Delete) operations. Note: If the list columns change, or new lists added to the site, the service reference will need to be updated since the generated code is type-safe.

For this project, we opted to go with the WCF DataServices approach since we only needed to populate and update existing list data. In Visual Studio it’s easy to add the WCF web service reference which will create the proxy for you. Basically reference your site url and prefix ‘_vti_bin/listdata.svc’ to the end of the url.

When creating a new column in SharePoint, you have the following options for the ‘type’:

  • Single line of text
  • Multiple lines of text
  • Choice (menu to choose from)
  • Number (1, 1.0, 100)
  • Currency ($, ¥, €)
  • Date and Time
  • Lookup (information already on this site)
  • Yes/No (check box)
  • Person or Group
  • Hyperlink or Picture
  • Calculated (calculation based on other columns)
  • External Data
  • Managed Metadata

When using the WCF proxy, the bold items identified above require special treatment for both reading and writing the values. Let’s look at reading these values via the WCF proxy.

For this example setup, we’re going to have a ‘Products’ list with the following columns:

  • Shipping: Choice (Multi-value)
  • Color: Choice (single value)
  • Category: Lookup (Multi-value)
  • Manufacturer: Lookup (single value)

Two additional lookup lists were created called ‘Categories’ and ‘Manufacturers’. Nothing special about these lookup lists, just used the default ‘Id’ and ‘Title’ columns to store our data.

For the purpose of this demo, the SharePoint site name was ‘WCF Test’. In our Visual Studio project, the web service reference was named ‘StoreSite’.

Reading Lists

First we’ll create a little utility function to get our DataContext.

/// <summary>
/// Get the WCF data context proxy.
/// </summary>
/// <param name="url">The optional Url for the sharepoint site listdata.svc endpoint.</param>
/// <returns>The DataContext to operate on.</returns>
static WCFTestDataContext GetDataContext(string url = null)
{
  if (url == null) url = ConfigurationManager.AppSettings["SharePointSiteURL"];
      
  var context = new StoreSite.WCFTestDataContext(new Uri(ConfigurationManager.AppSettings["SharePointSiteURL"]));
  context.Credentials = CredentialCache.DefaultNetworkCredentials;
  return (context);
}

The first step is reading the results from the ‘Products’ list. Go ahead and create some sample data manually via the SharePoint site.

/// <summary>
/// Displays the products to the console.
/// </summary>
static void DisplayProducts()
{
  var context = GetDataContext();
  var products = context.Products;
  foreach (var product in products) {
    Console.WriteLine("[{0}] {1} @ {2}", product.Id, product.Title, product.Created);
    Console.WriteLine("  Manufacturer: [{0}] {1}", product.Manufacturer.Id, product.Manufacturer.Title);
    Console.WriteLine("  Color:        {0}", product.Color.Value);
    Console.WriteLine("  Category:     {0}", String.Join(",", product.Category.Select(c => String.Format("[{0}] {1}", c.Id, c.Title))));
    Console.WriteLine("  Shipping:     {0}", String.Join(",", product.Shipping.Select(s => s.Value)));
  }
}

If you run this, you’ll get a System.NullReferenceException. If you trace through the debugger, you’ll find that all of the choice and lookup columns are null and/or don’t contain any items. Essentially, SharePoint is not doing the joins on that data to avoid unnecessary data transfer and optimal query performance. We have to explicitly tell SharePoint that we want that data included in our results. At the same time, we’ll make our output code a little more robust with some null checking.

Replace the previous function with the following:

/// <summary>
/// Displays the products to the console.
/// </summary>
static void DisplayProducts()
{
  var context = GetDataContext();
  var products = context.Products.Expand(p => p.Manufacturer)
                                 .Expand(p => p.Category)
                                 .Expand(p => p.Shipping)
                                 .Expand(p => p.Color);
  foreach (var product in products) {
    if (product.Manufacturer != null) {
      Console.WriteLine("  Manufacturer: [{0}] {1}", product.Manufacturer.Id, product.Manufacturer.Title);
    }
    if (product.Color != null) {
      Console.WriteLine("  Color:        {0}", product.Color.Value);
    }
    if (product.Category != null) {
      Console.WriteLine("  Category:     {0}", String.Join(",", product.Category.Select(c => String.Format("[{0}] {1}", c.Id, c.Title))));
    }
    if (product.Shipping != null) {
      Console.WriteLine("  Shipping:     {0}", String.Join(",", product.Shipping.Select(s => s.Value)));
    }
  }
}

The key in the above code is the .Expand() function.

  var products = context.Products.Expand(p => p.Manufacturer)
                                 .Expand(p => p.Category)
                                 .Expand(p => p.Shipping)
                                 .Expand(p => p.Color);

It basically instructs SharePoint to also return that auxiliary data in the results. With that change, we should now get our expected results. For each choice and lookup field that you want included or populated, you need to include a corresponding .Expand() statement with a lambda selector.

[1] Bosch 10-in Table Saw @ 7/1/2014 11:51:35 AM
  Manufacturer: [6] Bosch
  Color:        Green
  Category:     [8] Tools,[9] Saws
  Shipping:     FedEx,UPS

Creation

For creating a new item, we’ll need to create a new ProductsItem. Once you’re ProductsItem is created, you’ll need to add it to your context which is keeping track of all changes.

var context = GetDataContext();
var product = new ProductsItem() {
  Title = "Bosch 10-in Table Saw"
};
context.AddToProducts(product);

// TODO: Assign addition properties / fields

context.saveChanges();

The above code snippet will be used for each specialized example. Note: The new product MUST be added to the context (tracked) before you can attach or link the choice and lookup columns.

Modifying Single Choice Columns

Modifying a single choice column is pretty simple. In our case, our ‘Color’ field in a single choice. Essentially we just need to set the value using the *Value attribute. Using the Visual Studio intellisense, you’ll see that ProductsItem has a ‘Color’ as well as a ‘ColorValue’ property. We can simply set the ‘ColorValue’ property.

// Color: Choice (Single)
product.ColorValue = "Green";

You can get the list of all the available choices with the following:

foreach (var color in context.ProductsColor) {
  Console.WriteLine("{0}", color.Value);
}

Note: You can set the ‘ColorValue’ string to anything. It doesn’t have to exist in the list although the native SharePoint tools and editor will likely not be happy and lose the custom value on a subsequent edit.

Modifying Multiple Choice Columns

For a multiple choice column, the ‘ProductItem’ class contains a ‘Shipping’ field of type DataServiceCollection<>. Included with this method are convenient .Add() methods. You might think that you only need to do the following:

var product = new ProductItem();
var ups = ProductsShippingValue.CreateProductsShippingValue("UPS");
product.Shipping.Add(ups);

Unfortunately, the above won’t generate an error, but neither will your data be saved. Go ahead and try it.

To save this item, we must get an existing ProductsShippingValue which is already being tracked by the context or create a new one and manually attach it to the context.

Use / Retrieve Existing Tracked Context

The following code shows how to query the list of available choices and add it to the multi choice column.

// Shipping: Choice (Multiple)
var ups = context.ProductsShipping.Where(s => s.Value == "UPS").FirstOrDefault();
var fedex = context.ProductsShipping.Where(s => s.Value == "FedEx").FirstOrDefault();
product.Shipping.Add(ups);
product.Shipping.Add(fedex);
context.AddLink(product, "Shipping", ups);
context.AddLink(product, "Shipping", fedex);

Essentially we lookup one of the available choice values which is being tracked, add it to multi-choice column and then notify the DataContext that the values are “linked”. Please note, that the above code should be made more robust by checking for null values, etc.

Create New Entity and Track It

The above example has the overhead of running a remote query. Since we’re just matching on a predetermined or known string we can instead manually create our ProductsShippingValue and accomplish the same things. The only difference is that we need to make our context start tracking our new item. We accomplish this by “attaching” it. Otherwise the code is nearly identical.

// Manufacturer: Lookup (Single)
var ups = ProductsShippingValue.CreateProductsShippingValue("UPS");
var fedex = ProductsShippingValue.CreateProductsShippingValue("FedEx");
context.AttachTo("ProductsShipping", ups);
context.AttachTo("ProductsShipping", fedex);
product.Shipping.Add(ups);
product.Shipping.Add(fedex);
context.AddLink(product, "Shipping", ups);
context.AddLink(product, "Shipping", fedex);

Either option works although I would argue that the later option, although more code, makes more sense since you’re matching and selecting based predetermined strings. An enumeration would probably be ideal for this and could be streamlined with some extension method overloads.

Modifying Single Lookup Columns

For modifying a lookup field with a single value, we simply set the appropriate ‘*Id’ value that corresponds to our lookup value. You can also dynamically lookup this value which the following example demonstrates:

// Manufacturer: Lookup (Single)
var manufacturer = context.Manufacturers.Where(m => m.Title == "Bosch").FirstOrDefault();
product.ManufacturerId = manufacturer.Id;

For brevity, null checks and other exceptions were omitted and should be included in your production code.

Modifying Multiple Lookup Columns

Setting a multiple lookup column is very similar to setting a multi-choice column value. We can either query the existing lookup value which will already be “tracked” by the DataContext or we can manually create our items.

Use / Retrieve Existing Tracked Context

We query the existing lookup value although we’re only interested and need to set the Id value.

// Category: Lookup (Multiple)
var tools = context.Categories.Where(m => m.Title == "Tools").FirstOrDefault();
var saws = context.Categories.Where(m => m.Title == "Saws").FirstOrDefault();
product.Category.Add(tools);
product.Category.Add(saws);
context.AddLink(product, "Category", tools);
context.AddLink(product, "Category", saws);

Create New Entity and Track It

We can also create our objects manually, assuming we know their Id fields. When creating the CategoriesItem, the only thing we need to set is the Id field.

// Category: Lookup (Multiple)
var tools = new CategoriesItem() { Id = 8 };
var saws = new CategoriesItem() { Id = 9 };
context.AttachTo("Categories", tools);
context.AttachTo("Categories", saws);
product.Category.Add(tools);
product.Category.Add(saws);
context.AddLink(product, "Category", tools);
context.AddLink(product, "Category", saws);

Remarks

Hopefully this helps save you time. At the time I did this, it took my several days of digging, searching and experimenting before I found the right references and ordering. Special thanks to the following post on the MSDN forums which really helped to get things going in the right direction.

And yes, SharePoint sucks…


Below is the complete example of adding a product with basic error checking:

class Program
{
  /// <summary>
  /// Get the WCF data context proxy.
  /// </summary>
  /// <param name="url">The optional Url for the sharepoint site listdata.svc endpoint.</param>
  /// <returns>The WCFDataContext to operate on.</returns>
  static WCFTestDataContext GetDataContext(string url = null)
  {
    if (url == null) url = ConfigurationManager.AppSettings["SharePointSiteURL"];
    
    var context = new StoreSite.WCFTestDataContext(new Uri(ConfigurationManager.AppSettings["SharePointSiteURL"]));
    context.Credentials = CredentialCache.DefaultNetworkCredentials;
    return (context);
  }

  /// <summary>
  /// Displays the products to the console.
  /// </summary>
  static void DisplayProducts()
  {
    var context = GetDataContext();
    var products = context.Products.Expand(p => p.Manufacturer)
                                   .Expand(p => p.Category)
                                   .Expand(p => p.Shipping)
                                   .Expand(p => p.Color);
    foreach (var product in products) {
      Console.WriteLine("[{0}] {1} @ {2}", product.Id, product.Title, product.Created);
      if (product.Manufacturer != null) {
        Console.WriteLine("  Manufacturer: [{0}] {1}", product.Manufacturer.Id, product.Manufacturer.Title);
      }
      if (product.Color != null) {
        Console.WriteLine("  Color:        {0}", product.Color.Value);
      }
      if (product.Category != null) {
        Console.WriteLine("  Category:     {0}", String.Join(",", product.Category.Select(c => String.Format("[{0}] {1}", c.Id, c.Title))));
      }
      if (product.Shipping != null) {
        Console.WriteLine("  Shipping:     {0}", String.Join(",", product.Shipping.Select(s => s.Value)));
      }
    }
  }

  private static void Main(string[] args)
  {
    var context = GetDataContext();
    var product = new ProductsItem() {
      Title = "Bosch 10-in Table Saw"
    };
    context.AddToProducts(product);

    // Color: Choice (Single)
    product.ColorValue = "Teale";
    foreach (var color in context.ProductsColor) {
      Console.WriteLine("{0}", color.Value);
    }

    // Shipping: Choice (Multiple)
    var ups = context.ProductsShipping.Where(s => s.Value == "UPS").FirstOrDefault();
    var fedex = context.ProductsShipping.Where(s => s.Value == "FedEx").FirstOrDefault();
    //var ups = ProductsShippingValue.CreateProductsShippingValue("UPS");
    //var fedex = ProductsShippingValue.CreateProductsShippingValue("FedEx");
    //context.AttachTo("ProductsShipping", ups);
    //context.AttachTo("ProductsShipping", fedex);
    product.Shipping.Add(ups);
    product.Shipping.Add(fedex);
    context.AddLink(product, "Shipping", ups);
    context.AddLink(product, "Shipping", fedex);

    // Manufacturer: Lookup (Single)
    var manufacturer = context.Manufacturers.Where(m => m.Title == "Bosch").FirstOrDefault();
    if (manufacturer != null) {
      product.ManufacturerId = manufacturer.Id;
    }

    // Category: Lookup (Multiple)
    var tools = new CategoriesItem() { Id = 8 };
    var saws = new CategoriesItem() { Id = 9 };
    context.AttachTo("Categories", tools);
    context.AttachTo("Categories", saws);
    //var tools = context.Categories.Where(m => m.Title == "Tools").FirstOrDefault();
    //var saws = context.Categories.Where(m => m.Title == "Saws").FirstOrDefault();
    if (tools != null) {
      product.Category.Add(tools);
      context.AddLink(product, "Category", tools);
    }
    if (saws != null) {
      product.Category.Add(saws);
      context.AddLink(product, "Category", saws);
    }

    Console.WriteLine("Adding new product '{0}'...", product.Title);
    context.SaveChanges();

    DisplayProducts();
  }
}

Who doesn’t love LINQ? Who doesn’t love extension methods in .NET?

Unfortunately, Microsoft could have made things easier for developers by handling nulls gracefully. What do I mean? Take the following code as an example:

IEnumerable<int> numbers = null;
if (numbers.Any()) {
}

Obviously, as a programmer you would know that you should get a NullReferenceException from the above code. Below is the correct way to write that function:

IEnumerable<int> numbers = null;
if ((numbers != null) && (numbers.Any())) {
}

Simple and logical fix, but it’s just “noisy”. When dealing with an extension method, I disagree with how null references were handled in the LINQ libraries. Any extension function should detect null and return early (when possible). I believe that Microsoft’s view and defense is that an IEnumerable should never be “null” but instead be an empty collection.

Below is the decompiled implementation of the .Any() LINQ extension method (courtesy of .Net Reflector v6).

[__DynamicallyInvokable]
public static bool Any<TSource>(this IEnumerable<TSource> source)
{
  if (source == null) {
    throw Error.ArgumentNull("source");
  }
  using (IEnumerator<TSource> enumerator = source.GetEnumerator()) {
    if (enumerator.MoveNext()) {
      return true;
    }
  }
  return false;
}

The following simple change to the above method would remove all of that extra “noise” and make our code easier to read.

[__DynamicallyInvokable]
public static bool Any<TSource>(this IEnumerable<TSource> source)
{
  if (source == null) return(false);
  using (IEnumerator<TSource> enumerator = source.GetEnumerator()) {
    if (enumerator.MoveNext()) {
      return true;
    }
  }
  return false;
}

Of course, the other solution (and recommended best practice) is to not return null from a function that returns an IEnumerable, but to instead return an empty collection. Unfortunately, when dealing with other people’s code or libraries you may not have that luxury. Below is a simple and efficient example of how to NOT return null for an IEnumerable result. Please note that the example is contrived and a String.Split function already exists.

public IEnumerable<string> Split(string input, string value) {
  if (input == null) return(new string[0]);
  ...
}

So when you’re writing you own extension methods in .NET, do the world a favor and handle null much better.