Disclaimer: I am neither an expert, nor fan of SharePoint. Simple fact is I abhore SharePoint.

Unfortunately, I was brought into a project that required a SharePoint list to be updated via an external web service. There are two ways to accomplish this:

  • Use the SharePoint Client Object Model (client.svc) interface and get the rich client interface similar to writing applications that are hosted within SharePoint. This requires that the Microsoft.SharePoint.Client libraries are included or available to your application.
  • Use the WCF DataServices or REST interfaces (listdata.svc) to interact with the Lists on the SharePoint site. Unlike the Client Object Model, you’ll be limited to only working with the list data doing CRUD (Create, Read, Update, Delete) operations. Note: If the list columns change, or new lists added to the site, the service reference will need to be updated since the generated code is type-safe.

For this project, we opted to go with the WCF DataServices approach since we only needed to populate and update existing list data. In Visual Studio it’s easy to add the WCF web service reference which will create the proxy for you. Basically reference your site url and prefix ‘_vti_bin/listdata.svc’ to the end of the url.

When creating a new column in SharePoint, you have the following options for the ‘type’:

  • Single line of text
  • Multiple lines of text
  • Choice (menu to choose from)
  • Number (1, 1.0, 100)
  • Currency ($, ¥, €)
  • Date and Time
  • Lookup (information already on this site)
  • Yes/No (check box)
  • Person or Group
  • Hyperlink or Picture
  • Calculated (calculation based on other columns)
  • External Data
  • Managed Metadata

When using the WCF proxy, the bold items identified above require special treatment for both reading and writing the values. Let’s look at reading these values via the WCF proxy.

For this example setup, we’re going to have a ‘Products’ list with the following columns:

  • Shipping: Choice (Multi-value)
  • Color: Choice (single value)
  • Category: Lookup (Multi-value)
  • Manufacturer: Lookup (single value)

Two additional lookup lists were created called ‘Categories’ and ‘Manufacturers’. Nothing special about these lookup lists, just used the default ‘Id’ and ‘Title’ columns to store our data.

For the purpose of this demo, the SharePoint site name was ‘WCF Test’. In our Visual Studio project, the web service reference was named ‘StoreSite’.

Reading Lists

First we’ll create a little utility function to get our DataContext.

/// <summary>
/// Get the WCF data context proxy.
/// </summary>
/// <param name="url">The optional Url for the sharepoint site listdata.svc endpoint.</param>
/// <returns>The DataContext to operate on.</returns>
static WCFTestDataContext GetDataContext(string url = null)
{
  if (url == null) url = ConfigurationManager.AppSettings["SharePointSiteURL"];
      
  var context = new StoreSite.WCFTestDataContext(new Uri(ConfigurationManager.AppSettings["SharePointSiteURL"]));
  context.Credentials = CredentialCache.DefaultNetworkCredentials;
  return (context);
}

The first step is reading the results from the ‘Products’ list. Go ahead and create some sample data manually via the SharePoint site.

/// <summary>
/// Displays the products to the console.
/// </summary>
static void DisplayProducts()
{
  var context = GetDataContext();
  var products = context.Products;
  foreach (var product in products) {
    Console.WriteLine("[{0}] {1} @ {2}", product.Id, product.Title, product.Created);
    Console.WriteLine("  Manufacturer: [{0}] {1}", product.Manufacturer.Id, product.Manufacturer.Title);
    Console.WriteLine("  Color:        {0}", product.Color.Value);
    Console.WriteLine("  Category:     {0}", String.Join(",", product.Category.Select(c => String.Format("[{0}] {1}", c.Id, c.Title))));
    Console.WriteLine("  Shipping:     {0}", String.Join(",", product.Shipping.Select(s => s.Value)));
  }
}

If you run this, you’ll get a System.NullReferenceException. If you trace through the debugger, you’ll find that all of the choice and lookup columns are null and/or don’t contain any items. Essentially, SharePoint is not doing the joins on that data to avoid unnecessary data transfer and optimal query performance. We have to explicitly tell SharePoint that we want that data included in our results. At the same time, we’ll make our output code a little more robust with some null checking.

Replace the previous function with the following:

/// <summary>
/// Displays the products to the console.
/// </summary>
static void DisplayProducts()
{
  var context = GetDataContext();
  var products = context.Products.Expand(p => p.Manufacturer)
                                 .Expand(p => p.Category)
                                 .Expand(p => p.Shipping)
                                 .Expand(p => p.Color);
  foreach (var product in products) {
    if (product.Manufacturer != null) {
      Console.WriteLine("  Manufacturer: [{0}] {1}", product.Manufacturer.Id, product.Manufacturer.Title);
    }
    if (product.Color != null) {
      Console.WriteLine("  Color:        {0}", product.Color.Value);
    }
    if (product.Category != null) {
      Console.WriteLine("  Category:     {0}", String.Join(",", product.Category.Select(c => String.Format("[{0}] {1}", c.Id, c.Title))));
    }
    if (product.Shipping != null) {
      Console.WriteLine("  Shipping:     {0}", String.Join(",", product.Shipping.Select(s => s.Value)));
    }
  }
}

The key in the above code is the .Expand() function.

  var products = context.Products.Expand(p => p.Manufacturer)
                                 .Expand(p => p.Category)
                                 .Expand(p => p.Shipping)
                                 .Expand(p => p.Color);

It basically instructs SharePoint to also return that auxiliary data in the results. With that change, we should now get our expected results. For each choice and lookup field that you want included or populated, you need to include a corresponding .Expand() statement with a lambda selector.

[1] Bosch 10-in Table Saw @ 7/1/2014 11:51:35 AM
  Manufacturer: [6] Bosch
  Color:        Green
  Category:     [8] Tools,[9] Saws
  Shipping:     FedEx,UPS

Creation

For creating a new item, we’ll need to create a new ProductsItem. Once you’re ProductsItem is created, you’ll need to add it to your context which is keeping track of all changes.

var context = GetDataContext();
var product = new ProductsItem() {
  Title = "Bosch 10-in Table Saw"
};
context.AddToProducts(product);

// TODO: Assign addition properties / fields

context.saveChanges();

The above code snippet will be used for each specialized example. Note: The new product MUST be added to the context (tracked) before you can attach or link the choice and lookup columns.

Modifying Single Choice Columns

Modifying a single choice column is pretty simple. In our case, our ‘Color’ field in a single choice. Essentially we just need to set the value using the *Value attribute. Using the Visual Studio intellisense, you’ll see that ProductsItem has a ‘Color’ as well as a ‘ColorValue’ property. We can simply set the ‘ColorValue’ property.

// Color: Choice (Single)
product.ColorValue = "Green";

You can get the list of all the available choices with the following:

foreach (var color in context.ProductsColor) {
  Console.WriteLine("{0}", color.Value);
}

Note: You can set the ‘ColorValue’ string to anything. It doesn’t have to exist in the list although the native SharePoint tools and editor will likely not be happy and lose the custom value on a subsequent edit.

Modifying Multiple Choice Columns

For a multiple choice column, the ‘ProductItem’ class contains a ‘Shipping’ field of type DataServiceCollection<>. Included with this method are convenient .Add() methods. You might think that you only need to do the following:

var product = new ProductItem();
var ups = ProductsShippingValue.CreateProductsShippingValue("UPS");
product.Shipping.Add(ups);

Unfortunately, the above won’t generate an error, but neither will your data be saved. Go ahead and try it.

To save this item, we must get an existing ProductsShippingValue which is already being tracked by the context or create a new one and manually attach it to the context.

Use / Retrieve Existing Tracked Context

The following code shows how to query the list of available choices and add it to the multi choice column.

// Shipping: Choice (Multiple)
var ups = context.ProductsShipping.Where(s => s.Value == "UPS").FirstOrDefault();
var fedex = context.ProductsShipping.Where(s => s.Value == "FedEx").FirstOrDefault();
product.Shipping.Add(ups);
product.Shipping.Add(fedex);
context.AddLink(product, "Shipping", ups);
context.AddLink(product, "Shipping", fedex);

Essentially we lookup one of the available choice values which is being tracked, add it to multi-choice column and then notify the DataContext that the values are “linked”. Please note, that the above code should be made more robust by checking for null values, etc.

Create New Entity and Track It

The above example has the overhead of running a remote query. Since we’re just matching on a predetermined or known string we can instead manually create our ProductsShippingValue and accomplish the same things. The only difference is that we need to make our context start tracking our new item. We accomplish this by “attaching” it. Otherwise the code is nearly identical.

// Manufacturer: Lookup (Single)
var ups = ProductsShippingValue.CreateProductsShippingValue("UPS");
var fedex = ProductsShippingValue.CreateProductsShippingValue("FedEx");
context.AttachTo("ProductsShipping", ups);
context.AttachTo("ProductsShipping", fedex);
product.Shipping.Add(ups);
product.Shipping.Add(fedex);
context.AddLink(product, "Shipping", ups);
context.AddLink(product, "Shipping", fedex);

Either option works although I would argue that the later option, although more code, makes more sense since you’re matching and selecting based predetermined strings. An enumeration would probably be ideal for this and could be streamlined with some extension method overloads.

Modifying Single Lookup Columns

For modifying a lookup field with a single value, we simply set the appropriate ‘*Id’ value that corresponds to our lookup value. You can also dynamically lookup this value which the following example demonstrates:

// Manufacturer: Lookup (Single)
var manufacturer = context.Manufacturers.Where(m => m.Title == "Bosch").FirstOrDefault();
product.ManufacturerId = manufacturer.Id;

For brevity, null checks and other exceptions were omitted and should be included in your production code.

Modifying Multiple Lookup Columns

Setting a multiple lookup column is very similar to setting a multi-choice column value. We can either query the existing lookup value which will already be “tracked” by the DataContext or we can manually create our items.

Use / Retrieve Existing Tracked Context

We query the existing lookup value although we’re only interested and need to set the Id value.

// Category: Lookup (Multiple)
var tools = context.Categories.Where(m => m.Title == "Tools").FirstOrDefault();
var saws = context.Categories.Where(m => m.Title == "Saws").FirstOrDefault();
product.Category.Add(tools);
product.Category.Add(saws);
context.AddLink(product, "Category", tools);
context.AddLink(product, "Category", saws);

Create New Entity and Track It

We can also create our objects manually, assuming we know their Id fields. When creating the CategoriesItem, the only thing we need to set is the Id field.

// Category: Lookup (Multiple)
var tools = new CategoriesItem() { Id = 8 };
var saws = new CategoriesItem() { Id = 9 };
context.AttachTo("Categories", tools);
context.AttachTo("Categories", saws);
product.Category.Add(tools);
product.Category.Add(saws);
context.AddLink(product, "Category", tools);
context.AddLink(product, "Category", saws);

Remarks

Hopefully this helps save you time. At the time I did this, it took my several days of digging, searching and experimenting before I found the right references and ordering. Special thanks to the following post on the MSDN forums which really helped to get things going in the right direction.

And yes, SharePoint sucks…


Below is the complete example of adding a product with basic error checking:

class Program
{
  /// <summary>
  /// Get the WCF data context proxy.
  /// </summary>
  /// <param name="url">The optional Url for the sharepoint site listdata.svc endpoint.</param>
  /// <returns>The WCFDataContext to operate on.</returns>
  static WCFTestDataContext GetDataContext(string url = null)
  {
    if (url == null) url = ConfigurationManager.AppSettings["SharePointSiteURL"];
    
    var context = new StoreSite.WCFTestDataContext(new Uri(ConfigurationManager.AppSettings["SharePointSiteURL"]));
    context.Credentials = CredentialCache.DefaultNetworkCredentials;
    return (context);
  }

  /// <summary>
  /// Displays the products to the console.
  /// </summary>
  static void DisplayProducts()
  {
    var context = GetDataContext();
    var products = context.Products.Expand(p => p.Manufacturer)
                                   .Expand(p => p.Category)
                                   .Expand(p => p.Shipping)
                                   .Expand(p => p.Color);
    foreach (var product in products) {
      Console.WriteLine("[{0}] {1} @ {2}", product.Id, product.Title, product.Created);
      if (product.Manufacturer != null) {
        Console.WriteLine("  Manufacturer: [{0}] {1}", product.Manufacturer.Id, product.Manufacturer.Title);
      }
      if (product.Color != null) {
        Console.WriteLine("  Color:        {0}", product.Color.Value);
      }
      if (product.Category != null) {
        Console.WriteLine("  Category:     {0}", String.Join(",", product.Category.Select(c => String.Format("[{0}] {1}", c.Id, c.Title))));
      }
      if (product.Shipping != null) {
        Console.WriteLine("  Shipping:     {0}", String.Join(",", product.Shipping.Select(s => s.Value)));
      }
    }
  }

  private static void Main(string[] args)
  {
    var context = GetDataContext();
    var product = new ProductsItem() {
      Title = "Bosch 10-in Table Saw"
    };
    context.AddToProducts(product);

    // Color: Choice (Single)
    product.ColorValue = "Teale";
    foreach (var color in context.ProductsColor) {
      Console.WriteLine("{0}", color.Value);
    }

    // Shipping: Choice (Multiple)
    var ups = context.ProductsShipping.Where(s => s.Value == "UPS").FirstOrDefault();
    var fedex = context.ProductsShipping.Where(s => s.Value == "FedEx").FirstOrDefault();
    //var ups = ProductsShippingValue.CreateProductsShippingValue("UPS");
    //var fedex = ProductsShippingValue.CreateProductsShippingValue("FedEx");
    //context.AttachTo("ProductsShipping", ups);
    //context.AttachTo("ProductsShipping", fedex);
    product.Shipping.Add(ups);
    product.Shipping.Add(fedex);
    context.AddLink(product, "Shipping", ups);
    context.AddLink(product, "Shipping", fedex);

    // Manufacturer: Lookup (Single)
    var manufacturer = context.Manufacturers.Where(m => m.Title == "Bosch").FirstOrDefault();
    if (manufacturer != null) {
      product.ManufacturerId = manufacturer.Id;
    }

    // Category: Lookup (Multiple)
    var tools = new CategoriesItem() { Id = 8 };
    var saws = new CategoriesItem() { Id = 9 };
    context.AttachTo("Categories", tools);
    context.AttachTo("Categories", saws);
    //var tools = context.Categories.Where(m => m.Title == "Tools").FirstOrDefault();
    //var saws = context.Categories.Where(m => m.Title == "Saws").FirstOrDefault();
    if (tools != null) {
      product.Category.Add(tools);
      context.AddLink(product, "Category", tools);
    }
    if (saws != null) {
      product.Category.Add(saws);
      context.AddLink(product, "Category", saws);
    }

    Console.WriteLine("Adding new product '{0}'...", product.Title);
    context.SaveChanges();

    DisplayProducts();
  }
}

If you deal with SSL certificates long enough, eventually you’ll run into a trust issue or error regardless of what programming language you are using. Dealing with SSL certificates can be confusing at first, but hopefully this article can simplify a few things or at least get you past the error.

Recently, a user reported the following error when using my node-activedirectory plugin. The error was:

throw er; // Unhandled 'error' event
^
Error: CERT_UNTRUSTED
at SecurePair. (tls.js:1370:32)
at SecurePair.EventEmitter.emit (events.js:92:17)
at SecurePair.maybeInitFinished (tls.js:982:10)
at CleartextStream.read as _read
at CleartextStream.Readable.read (_stream_readable.js:320:10)
at EncryptedStream.write as _write
at doWrite (_stream_writable.js:226:10)
at writeOrBuffer (_stream_writable.js:216:5)
at EncryptedStream.Writable.write (_stream_writable.js:183:11)
at write (_stream_readable.js:583:24)

Well, what does that mean? At first you might think something is wrong with your code or perhaps a problem with the library you are using. No, that’s not the problem. The basic problem is that the SSL certificate the remote server is sending you is not “trusted” by your computer or potentially it has been tampered with (i.e. man-in-the middle attack).

What do you mean by “trusted”? I just want my connection encrypted!

Default List of Trusted Certificates
The “trust” is a key part of what makes SSL work. With SSL, there are a number of certificate authorities (CA) which your computer has been preconfigured to “trust”. When someone needs an SSL certificate, they will typically send their request to a certificate authority to “sign” their certificate which creates a “chain of trust”. With SSL, if you trust a certificate, you trust that certificate and all of the certificates below it or certificates which that certificate has “signed” or verified for integrity. Although your computer was preconfigured with a bunch of trusted certificate authorities, you can add or remove from that list as needed. A certificate can also be signed by itself, this is referred to as a self-signed certificate and is common for quick testing and securing internal resources when a public key infrastructure (PKI) doesn’t exist. There is also a method for revoking bad or hacked certificates which is beyond the scope of this article; just know that this is typically referred to as a “certificate revocation list” or CRL for short.

That brief introduction was only 300,000 foot view and barely does the topic justice but should be sufficient background to understand the error. To learn even more visit the following links:

TL;DR – How Do I Fix It?

To fix the problem, you typically have three options:

  1. Don’t use SSL for the connection. But that’s probably not a good idea…
  2. Disable certification trust verification in your application framework. Another bad idea since it makes a man-in-the middle attack possible. But sometimes for quick testing and proof of concept this is acceptable. But NEVER do this in production!!!
  3. Verify, import and trust the certificate authority or individual certificate.

Of those options, let’s do it the right way and get our trust properly established.

Getting the certificate

The first step is getting the public certificate (or preferably the certificate authority for that certificate) that you want to trust. You don’t need the private key that goes with that certificate and no admin that knew what they were doing would give it to you anyway. There are a number of different ways to get this information depending on what type of access you have to the server that contains the SSL certificate.

Access To The Server

If you have direct access to the server, you can export the certificate to a file. To get started, follow these steps:

Export Certificate via Microsoft Management Console (mmc).
  1. On the server that has the certificate you want to export, start the Microsoft Management Console. Press Windows-R on your keyboard to bring up the “Run…” command line. Then enter ‘mmc’ and press enter.
  2. File -> Add/Remove snap in…
  3. windows-ssl-export-01Choose ‘Certificate’ from the list and click on the ‘Add >’ button.
  4. windows-ssl-export-02Choose ‘Computer Account’ and click on ‘Next >’.
  5. windows-ssl-export-03Choose ‘Local computer: (the computer this console is running on)’. Click on ‘Finish’.
  6. Find the certificate in the list that you want to export. Most likely located in the ‘Personal > Certificates’ or ‘Trusted Root Certification Authorities > Certificates’ locations. Right click on the certificate you want to export, then choose ‘All Tasks’ -> ‘Export…’.
  7. windows-ssl-export-04This starts the Certificate Export Wizard. Click ‘Next >’.
  8. windows-ssl-export-05If the certificate you have also had a corresponding private key, choose ‘No, do not export the private key’. As discussed previously, you do NOT want to export the private key or give it to anyone else. Click on ‘Next >’.
  9. windows-ssl-export-06Choose ‘Base-64 encoded X.509 (.CER)’ and click on ‘Next >’. Note: You could choose any of the other formats, depending on where you will be using the certificate. I personally find the Base-64 encoded certificate to be more compatible across platforms.
  10. windows-ssl-export-07Choose a location and filename to save the file to. Make sure it ends with either ‘*.cer’ or ‘*.crt’ so that the Windows operating system will recognize the type of file. Click on ‘Next >’.
  11. windows-ssl-export-08Click on ‘Next >’.
  12. windows-ssl-export-08You should receive a confirmation that the certificate was exported successfully. Click on ‘OK’.
No Access to the server

If you don’t have direct access to the server, you can still get the public certificate that is presented to you. If you’re trying to get a certificate for an HTTP/HTTPS server, you can easily view the certificate and save it to file. However, if you’re trying to get the SSL certificate from LDAPS or perhaps IMAPS, we’ll need to use the OpenSSL utilities to view the certificates. OpenSSL isn’t installed by default on Windows machines, however if you have access to a Linux server, typically the openssl tools will be available. For Windows, we can download the OpenSSL binaries or use Cygwin to install GNU / POSIX compatible binaries.

OpenSSL Tools

Once you have the openssl binaries available, we can use the following command to view the SSL certificates for any type of connection. The following example would view the certificates for an LDAPS (LDAP via SSL) connection on the default port. Please note that Active Directory and LDAP are basically the same. Active Directory is Microsoft’s implementation of LDAP.

openssl s_client -showcerts -host remoteserver.domain.name -port 636

Running that command, you should see something similar to the following:

CONNECTED(00000003)
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
   i:/C=US/O=Google Inc/CN=Google Internet Authority G2
-----BEGIN CERTIFICATE-----
MIIEdjCCA16gAwIBAgIICNMg30SopiMwDQYJKoZIhvcNAQEFBQAwSTELMAkGA1UE
BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
cm5ldCBBdXRob3JpdHkgRzIwHhcNMTQwNTIyMTEyNTU4WhcNMTQwODIwMDAwMDAw
WjBoMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEXMBUGA1UEAwwOd3d3
Lmdvb2dsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCNJSwk
PPpMgw/J/diF5cAqGbmNe/Bih1rLVsfBwJDS3zunxMI1IAhoudccuQd0h4OWYcGc
z1Y8aNvpPz+3qY0GUvQcVGLh8JydQJI8eBlXL9v8J/uK2GBT/37Bkcga94DqpOLG
9n5Fvsd6F87+jpuCyDXW1hv6aNr4uiyFwa7I3HlTSr6BauM+aS0PXUTJSBi0BG73
gJbpTB/MgFlILp3x5bYSpn+3eSdME4EKEq42uy/oVHFrXsgZA6/lmWMiM/Is530x
FJfu0Bz7OgPRYsAiGiGjhPyPUs4oTOQERq2j9cIM4OXHVtZqehESE6noDvlNhptA
R+6lpPoDgQp2O5BjAgMBAAGjggFBMIIBPTAdBgNVHSUEFjAUBggrBgEFBQcDAQYI
KwYBBQUHAwIwGQYDVR0RBBIwEIIOd3d3Lmdvb2dsZS5jb20waAYIKwYBBQUHAQEE
XDBaMCsGCCsGAQUFBzAChh9odHRwOi8vcGtpLmdvb2dsZS5jb20vR0lBRzIuY3J0
MCsGCCsGAQUFBzABhh9odHRwOi8vY2xpZW50czEuZ29vZ2xlLmNvbS9vY3NwMB0G
A1UdDgQWBBQAC/6CV31piwAaimR5f1OK/QdilTAMBgNVHRMBAf8EAjAAMB8GA1Ud
IwQYMBaAFErdBhYbvPZotXb1gba7Yhq6WoEvMBcGA1UdIAQQMA4wDAYKKwYBBAHW
eQIFATAwBgNVHR8EKTAnMCWgI6Ahhh9odHRwOi8vcGtpLmdvb2dsZS5jb20vR0lB
RzIuY3JsMA0GCSqGSIb3DQEBBQUAA4IBAQB3JODw4UTUX3xJyr55rbd5EIeMQjcs
sKRvH/oJmEcIl1hrOaiNnpEbQis+2N5YR2PMMU825iO30L66hIswPOxfFiBeb1ZM
TWPJG5WXx7fctFPDXbJ3Zjq3cANIaX8Vlu4nSayNEhKnNuZog1YVSrg3Mu3E+8Ln
bzt+pZa9iTH821hu/il3TmRzXndT3dEz+n1XkrT3F9NBL3ZyYceDU5uB9fo7x25H
pLP/8pxIqu3+AcoGqNmJpxSfWlaqKXqd3TZ++edHtgTO5t8KV65GgCQ9+Wl0amtG
odT0vGI0eRPRl6s+Nnk6Aguz4bkRPsYTuEVJdEd3F+f9kxHrVWI0c+J4
-----END CERTIFICATE-----
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
-----BEGIN CERTIFICATE-----
MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
MRYwFAYDVQQKEw1HZW9UcnVzdCBJbmMuMRswGQYDVQQDExJHZW9UcnVzdCBHbG9i
YWwgQ0EwHhcNMTMwNDA1MTUxNTU1WhcNMTUwNDA0MTUxNTU1WjBJMQswCQYDVQQG
EwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzElMCMGA1UEAxMcR29vZ2xlIEludGVy
bmV0IEF1dGhvcml0eSBHMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AJwqBHdc2FCROgajguDYUEi8iT/xGXAaiEZ+4I/F8YnOIe5a/mENtzJEiaB0C1NP
VaTOgmKV7utZX8bhBYASxF6UP7xbSDj0U/ck5vuR6RXEz/RTDfRK/J9U3n2+oGtv
h8DQUB8oMANA2ghzUWx//zo8pzcGjr1LEQTrfSTe5vn8MXH7lNVg8y5Kr0LSy+rE
ahqyzFPdFUuLH8gZYR/Nnag+YyuENWllhMgZxUYi+FOVvuOAShDGKuy6lyARxzmZ
EASg8GF6lSWMTlJ14rbtCMoU/M4iarNOz0YDl5cDfsCx3nuvRTPPuj5xt970JSXC
DTWJnZ37DhF5iR43xa+OcmkCAwEAAaOB+zCB+DAfBgNVHSMEGDAWgBTAephojYn7
qwVkDBF9qn1luMrMTjAdBgNVHQ4EFgQUSt0GFhu89mi1dvWBtrtiGrpagS8wEgYD
VR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAQYwOgYDVR0fBDMwMTAvoC2g
K4YpaHR0cDovL2NybC5nZW90cnVzdC5jb20vY3Jscy9ndGdsb2JhbC5jcmwwPQYI
KwYBBQUHAQEEMTAvMC0GCCsGAQUFBzABhiFodHRwOi8vZ3RnbG9iYWwtb2NzcC5n
ZW90cnVzdC5jb20wFwYDVR0gBBAwDjAMBgorBgEEAdZ5AgUBMA0GCSqGSIb3DQEB
BQUAA4IBAQA21waAESetKhSbOHezI6B1WLuxfoNCunLaHtiONgaX4PCVOzf9G0JY
/iLIa704XtE7JW4S615ndkZAkNoUyHgN7ZVm2o6Gb4ChulYylYbc3GrKBIxbf/a/
zG+FA1jDaFETzf3I93k9mTXwVqO94FntT0QJo544evZG0R0SnU++0ED8Vf4GXjza
HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto
WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6
yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx
-----END CERTIFICATE-----
 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
-----BEGIN CERTIFICATE-----
MIIDfTCCAuagAwIBAgIDErvmMA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT
MRAwDgYDVQQKEwdFcXVpZmF4MS0wKwYDVQQLEyRFcXVpZmF4IFNlY3VyZSBDZXJ0
aWZpY2F0ZSBBdXRob3JpdHkwHhcNMDIwNTIxMDQwMDAwWhcNMTgwODIxMDQwMDAw
WjBCMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNR2VvVHJ1c3QgSW5jLjEbMBkGA1UE
AxMSR2VvVHJ1c3QgR2xvYmFsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEA2swYYzD99BcjGlZ+W988bDjkcbd4kdS8odhM+KhDtgPpTSEHCIjaWC9m
OSm9BXiLnTjoBbdqfnGk5sRgprDvgOSJKA+eJdbtg/OtppHHmMlCGDUUna2YRpIu
T8rxh0PBFpVXLVDviS2Aelet8u5fa9IAjbkU+BQVNdnARqN7csiRv8lVK83Qlz6c
JmTM386DGXHKTubU1XupGc1V3sjs0l44U+VcT4wt/lAjNvxm5suOpDkZALeVAjmR
Cw7+OC7RHQWa9k0+bw8HHa8sHo9gOeL6NlMTOdReJivbPagUvTLrGAMoUgRx5asz
PeE4uwc2hGKceeoWMPRfwCvocWvk+QIDAQABo4HwMIHtMB8GA1UdIwQYMBaAFEjm
aPkr0rKV10fYIyAQTzOYkJ/UMB0GA1UdDgQWBBTAephojYn7qwVkDBF9qn1luMrM
TjAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjA6BgNVHR8EMzAxMC+g
LaArhilodHRwOi8vY3JsLmdlb3RydXN0LmNvbS9jcmxzL3NlY3VyZWNhLmNybDBO
BgNVHSAERzBFMEMGBFUdIAAwOzA5BggrBgEFBQcCARYtaHR0cHM6Ly93d3cuZ2Vv
dHJ1c3QuY29tL3Jlc291cmNlcy9yZXBvc2l0b3J5MA0GCSqGSIb3DQEBBQUAA4GB
AHbhEm5OSxYShjAGsoEIz/AIx8dxfmbuwu3UOx//8PDITtZDOLC5MH0Y0FWDomrL
NhGc6Ehmo21/uBPUR/6LWlxz/K7ZGzIZOKuXNBSqltLroxwUCEm2u+WR74M26x1W
b8ravHNjkOR/ez4iyz0H7V84dJzjA1BOoa+Y7mHyhD8S
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
issuer=/C=US/O=Google Inc/CN=Google Internet Authority G2
---
No client certificate CA names sent
---
SSL handshake has read 3231 bytes and written 432 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-SHA
    Session-ID: D915A1902D9384B5E11F80BF76037C61048F24C763802E9C5CE9F56684F713B2
    Session-ID-ctx: 
    Master-Key: DB66D3BC903435A25D3C436D39E4ED2E513731ED7DF3F6BB6AA8C7245ED70433B61546CB0BA3F1465D8B87A978CB5715
    Key-Arg   : None
    Start Time: 1401542147
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---

Note: This example was run against a pubic Google WWW / HTTPS server.

In the example output above, we see a full certificate authority chain. Typically when working with SSL, you want to trust the certificate authority that signed the actual certificate. You could trust each certificate individually, however this doesn’t scale beyond a handful of servers.Each BEGIN CERTIFICATE and END CERTIFICATE block contains the base-64 encoded version of that SSL certificate. Once you find the certificate you want, copy the BEGIN / END block inclusively and save to a file with a *.crt or *.cer extension.

-----BEGIN CERTIFICATE-----
MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
MRYwFAYDVQQKEw1HZW9UcnVzdCBJbmMuMRswGQYDVQQDExJHZW9UcnVzdCBHbG9i
YWwgQ0EwHhcNMTMwNDA1MTUxNTU1WhcNMTUwNDA0MTUxNTU1WjBJMQswCQYDVQQG
EwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzElMCMGA1UEAxMcR29vZ2xlIEludGVy
bmV0IEF1dGhvcml0eSBHMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AJwqBHdc2FCROgajguDYUEi8iT/xGXAaiEZ+4I/F8YnOIe5a/mENtzJEiaB0C1NP
VaTOgmKV7utZX8bhBYASxF6UP7xbSDj0U/ck5vuR6RXEz/RTDfRK/J9U3n2+oGtv
h8DQUB8oMANA2ghzUWx//zo8pzcGjr1LEQTrfSTe5vn8MXH7lNVg8y5Kr0LSy+rE
ahqyzFPdFUuLH8gZYR/Nnag+YyuENWllhMgZxUYi+FOVvuOAShDGKuy6lyARxzmZ
EASg8GF6lSWMTlJ14rbtCMoU/M4iarNOz0YDl5cDfsCx3nuvRTPPuj5xt970JSXC
DTWJnZ37DhF5iR43xa+OcmkCAwEAAaOB+zCB+DAfBgNVHSMEGDAWgBTAephojYn7
qwVkDBF9qn1luMrMTjAdBgNVHQ4EFgQUSt0GFhu89mi1dvWBtrtiGrpagS8wEgYD
VR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAQYwOgYDVR0fBDMwMTAvoC2g
K4YpaHR0cDovL2NybC5nZW90cnVzdC5jb20vY3Jscy9ndGdsb2JhbC5jcmwwPQYI
KwYBBQUHAQEEMTAvMC0GCCsGAQUFBzABhiFodHRwOi8vZ3RnbG9iYWwtb2NzcC5n
ZW90cnVzdC5jb20wFwYDVR0gBBAwDjAMBgorBgEEAdZ5AgUBMA0GCSqGSIb3DQEB
BQUAA4IBAQA21waAESetKhSbOHezI6B1WLuxfoNCunLaHtiONgaX4PCVOzf9G0JY
/iLIa704XtE7JW4S615ndkZAkNoUyHgN7ZVm2o6Gb4ChulYylYbc3GrKBIxbf/a/
zG+FA1jDaFETzf3I93k9mTXwVqO94FntT0QJo544evZG0R0SnU++0ED8Vf4GXjza
HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto
WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6
yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx
-----END CERTIFICATE-----

You now have the public certificate of the remote server or certificate authority. Our next step will be to import that certificate into our Trusted Root Certificate Authorities.

Trusting / Importing a Certificate

Once you have your certificate, we can go ahead and import that certificate and trust it. Follow these steps:

  1. windows-ssl-import-01 In Windows, double click or open the certificate that you extracted earlier. Click on the ‘Install Certificate…’ button.
  2. windows-ssl-import-02 In the Certificate Import Wizard, click on ‘Next >’.
  3. windows-ssl-import-03 Choose ‘Place all certificates in the following store’. Then click on the ‘Browse…’ button.
  4. windows-ssl-import-04 Click on the ‘Trusted Root Certificate Authorities’ and then click on the ‘OK’ button.
  5. windows-ssl-import-05 Click on ‘Next >’.
  6. windows-ssl-import-06 Click on ‘Finish’.
  7. windows-ssl-import-07 Depending on the certificate, you may receive the following warning. Basically this is warning you to make sure that you have the right certificate. Click on ‘Yes’ to continue.
  8. windows-ssl-import-08 Click on ‘OK’.

The certificate should now be installed and trusted on your local computer.

Note, the above steps would import the certificate for your account. To add it for the computer (and all users), you would need to use the Microsoft Management Console (MMC). And use the ‘All Tasks -> Import…’ option.

Hopefully this has helped. Please note, that if you’re using a Java application or doing application development with Java, it’s trusted keystore information is stored separately from the operating system and is managed through the ‘keytool’ utility. We’ll save that topic for another day…

If you’ve just deployed a .svc file to a server (or your local IIS server) and you get an error that it doesn’t recognize the .svc mime type (or the type was blocked), they you need to do the following:

  1. Ensure that the .NET 3.5.1 Framework is installed.
  2. Ensure that “Windows Communication Foundation HTTP Activation is enabled and installed. This can be access via the Program Features in the Control panel under “Turn Windows Features on or off”.
    Enable WCF HTTP Activation.

    Ensure that Windows Communication Foundation HTTP Activation is installed.

  3. Execute the following command with elevated privileges (as administrator):
    "%WINDIR%\Microsoft.Net\Framework\v3.0\Windows Communication Foundation\ServiceModelReg.exe" -i
         

    When you run the WCF service model registration, your machine web.config file will be updated. Below is output from the above command:

    Microsoft(R) Windows Communication Foundation Installation Utility
    [Microsoft (R) Windows (R) Communication Foundation, Version 3.0.4506.5420]
    Copyright (c) Microsoft Corporation.  All rights reserved.
    
    Installing: Machine.config Section Groups and Handlers (WOW64)
    Installing: Machine.config Section Groups and Handlers
    Installing: System.Web Build Provider (WOW64)
    Installing: System.Web Compilation Assemblies (WOW64)
    Installing: HTTP Handlers (WOW64)
    Installing: HTTP Modules (WOW64)
    Installing: System.Web Build Provider
    Installing: System.Web Compilation Assemblies
    Installing: HTTP Handlers
    Installing: HTTP Modules
    Installing: Protocol node for protocol net.tcp (WOW64)
    Installing: TransportConfiguration node for protocol net.tcp (WOW64)
    Installing: ListenerAdapter node for protocol net.tcp
    Installing: Protocol node for protocol net.tcp
    Installing: TransportConfiguration node for protocol net.tcp
    Installing: Protocol node for protocol net.pipe (WOW64)
    Installing: TransportConfiguration node for protocol net.pipe (WOW64)
    Installing: ListenerAdapter node for protocol net.pipe
    Installing: Protocol node for protocol net.pipe
    Installing: TransportConfiguration node for protocol net.pipe
    Installing: Protocol node for protocol net.msmq (WOW64)
    Installing: TransportConfiguration node for protocol net.msmq (WOW64)
    Installing: ListenerAdapter node for protocol net.msmq
    Installing: Protocol node for protocol net.msmq
    Installing: TransportConfiguration node for protocol net.msmq
    Installing: Protocol node for protocol msmq.formatname (WOW64)
    Installing: TransportConfiguration node for protocol msmq.formatname (WOW64)
    Installing: ListenerAdapter node for protocol msmq.formatname
    Installing: Protocol node for protocol msmq.formatname
    Installing: TransportConfiguration node for protocol msmq.formatname
    Installing: HTTP Modules (WAS)
    Installing: HTTP Handlers (WAS)
    

In the process of working through this problem, I first attempt to add the WCF HTTP Activation, which unfortunately corrupted my machine web.config file which prevented access to the IIS manager and associated application pools due to the error:

The configuration section 'system.serviceModel' cannot be read because it is missing a section declaration  

It broke everything .NET related… GREAT!!!

Basically, for some reason my machine web.config was missing the configuration section DLL registrations for the WCF serviceModel. By using the ServiceModelReg.exe tool, it ensured that those base registrations were entered correctly. Once that was updated, the add new features option was able to complete successfully.

If after those changes, you are receiving the following error:

Could not load type ‘System.ServiceModel.Activation.HttpModule’ from assembly ‘System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

This error can occur when there are multiple versions of the .NET Framework on the computer that is running IIS, and IIS was installed after .NET Framework 4.0 or before the Service Model in Windows Communication Foundation was registered. [MSDN]

To fix this error, start a command prompt as an administrator. And run the following commands:

cd %WINDIR%\Microsoft.NET\Framework64\v4.0.30319
aspnet_regiis.exe -iru
iisreset

On my ActiveDirectory (AD) node.js plugin, a user requested support for LDAP referrals and chasing (#8). Unfortunately, I didn’t have access to a partitioned AD installation to test this.

The user wasn’t very helpful with giving me additional information or test scenarios to work with it so I took it upon myself to install a partitioned AD environment… as well as some heavy reading about how Microsoft implemented their referrals and partitioning. I probably still don’t understand it…

The one interesting thing is that by default they create two application partitions, one for DNS entries (Forest & Domain) and one for configuration which allows for these areas to be replicated. Those example referrals look like the following, although replace domain.com with your context.

  • ldap://ForestDnsZones.domain.com/dc=domain,dc=com
  • ldap://DomainDnsZones.domain.com/dc=domain,dc=com
  • ldap://dc.domain.com/CN=Configuration,dc=domain,dc=com

With referral chasing, you end up basically sending a request to each of the referrals with the same original query in order to get “everything”. So that one LDAP query can quickly lead to an “N+1” select problem, overhead on the network and slower responses. To get around the problem, I just created a couple of regular expressions to exclude and ignore those referrals.

var defaultReferrals = {
  enabled: false,
  // Active directory returns the following partitions as default referrals which we don't want to follow
  exclude: [
    'ldaps?://ForestDnsZones\\..*/.*',
    'ldaps?://DomainDnsZones\\..*/.*',
    'ldaps?://.*/CN=Configuration,.*'
  ]
};

The other way around this problem is to use the Global Catalog (GC) instead of direct LDAP queries. Essentially the GC just listens on a different port (3268) but any LDAP search requests will be for the entire “forest”.

Who doesn’t love LINQ? Who doesn’t love extension methods in .NET?

Unfortunately, Microsoft could have made things easier for developers by handling nulls gracefully. What do I mean? Take the following code as an example:

IEnumerable<int> numbers = null;
if (numbers.Any()) {
}

Obviously, as a programmer you would know that you should get a NullReferenceException from the above code. Below is the correct way to write that function:

IEnumerable<int> numbers = null;
if ((numbers != null) && (numbers.Any())) {
}

Simple and logical fix, but it’s just “noisy”. When dealing with an extension method, I disagree with how null references were handled in the LINQ libraries. Any extension function should detect null and return early (when possible). I believe that Microsoft’s view and defense is that an IEnumerable should never be “null” but instead be an empty collection.

Below is the decompiled implementation of the .Any() LINQ extension method (courtesy of .Net Reflector v6).

[__DynamicallyInvokable]
public static bool Any<TSource>(this IEnumerable<TSource> source)
{
  if (source == null) {
    throw Error.ArgumentNull("source");
  }
  using (IEnumerator<TSource> enumerator = source.GetEnumerator()) {
    if (enumerator.MoveNext()) {
      return true;
    }
  }
  return false;
}

The following simple change to the above method would remove all of that extra “noise” and make our code easier to read.

[__DynamicallyInvokable]
public static bool Any<TSource>(this IEnumerable<TSource> source)
{
  if (source == null) return(false);
  using (IEnumerator<TSource> enumerator = source.GetEnumerator()) {
    if (enumerator.MoveNext()) {
      return true;
    }
  }
  return false;
}

Of course, the other solution (and recommended best practice) is to not return null from a function that returns an IEnumerable, but to instead return an empty collection. Unfortunately, when dealing with other people’s code or libraries you may not have that luxury. Below is a simple and efficient example of how to NOT return null for an IEnumerable result. Please note that the example is contrived and a String.Split function already exists.

public IEnumerable<string> Split(string input, string value) {
  if (input == null) return(new string[0]);
  ...
}

So when you’re writing you own extension methods in .NET, do the world a favor and handle null much better.

HTML5

If you’re lucky enough to be able to switch to HTML5, you’ll be able to use the new built-in date picker controls.

<input type="date" id="mydate" value="2014-03-14" />

If you’re viewing this page with an HTML5 capable browser, the value will render as follows:

No special jQuery plugins or custom calendar controls necessary! With HTML5, there are a total of three different date and date/time picker controls:

  • date – The date (no timezone).
  • datetime – The date/time (with time zone).
  • datetime-local – The date/time (no timezone).

JavaScript Date/Time Parsing

If you’ve ever worked clientside with JavaScript, you’ll know that any new Date() function call will return a Date object that is localized based on the timezone set for the computer. You can also create a new date from an existing date or by parsing the string as follows:

  var date = new Date("Tue Mar 04 2014 16:00:00 GMT-0800");
  console.log(date);

With the above example, assuming you are located in CST (or a different timezone than PST), you’ll find the following output:

Tue Mar 04 2014 18:00:00 GMT-0600 (Central Standard Time)

Notice that the time has automatically corrected itself to the local timezone. OK, that’s good to know, but how does that affect me?

Try the following time:

  var date = new Date("Tue Mar 04 2014 00:00:00 GMT-0000");
  console.log(date);

Assuming you’re in the western hemisphere and you don’t live near the prime meridian, you should see something like the following:

Mon Mar 03 2014 18:00:00 GMT-0600 (Central Standard Time)

Did you catch that? The date has moved back to March 03 instead of March 04 because of the timezone conversion. OK, you get it, enough with the geography and math lessons… why should you care?

HTML5 Date Input

When using the <input type=”date” /> control, the value that is posted and returned from the input control is formatted as: YYYY-MM-DD. Since we have a date, we can assume that we can just let the JavaScript Date object parse it for us.

  var date = new Date('2014-03-04');
  // Mon Mar 03 2014 18:00:00 GMT-0600 (Central Standard Time)

What?!?! Why is it March 03 again? Lets try playing around with the Date object a little bit and see what happens.

  var date = new Date(2014, 2, 4); // Recall that month is 0 based, where 0=January
  // Tue Mar 04 2014 00:00:00 GMT-0600 (Central Standard Time)
  var date = new Date('03/04/2014');
  // Tue Mar 04 2014 00:00:00 GMT-0600 (Central Standard Time)

OK. Now that is what I’m expecting!!! So basically if we provide the string as YYYY-MM-DD, the Date object will parse that date automatically as a GMT time and then localize. However, if we pass it in as MM/DD/YYYY it will already be localized.

So how do we handle this for our HTML5 input control to get the date we expect? The following naive JavaScript will attempt to parse the input received and create a localized Date object. It will also attempt to parse a standard date in the MM/DD/YYYY format if HTML5 isn’t supported.

/**
 * Parses the date from the string input.
 * @param {Number|Date|string} date Teh value to be parsed.
 * @param {Function|Date} [defaultValue] The default value to use if the date cannot be parsed.
 * @returns {Date} The parsed date value. If the date is invalid or can't be parsed, then the defaultValue is returned.
 */
function parseDate(date, defaultValue) {
  if (! date) return(getDefaultValue());
  if (typeof(date) === 'date') return(date);
  if (typeof(date) === 'number') return(new Date(date));

  /**
   * Gets the default value.
   * @returns {Date}
   */
  function getDefaultValue() {
    return((typeof(defaultValue) === 'function') ? defaultValue(name) : defaultValue);
  }

  var results;
  // YYYY-MM-DD
  if ((results = /(\d{4})[-\/\\](\d{1,2})[-\/\\](\d{1,2})/.exec(date))) {
    return(new Date(results[1], parseInt(results[2], 10)-1, results[3]) || new Date(date) || getDefaultValue());
  }
  // MM/DD/YYYY
  if ((results = /(\d{1,2})[-\/\\](\d{1,2})[-\/\\](\d{4})/.exec(date))) {
    date = new Date(results[3], parseInt(results[1], 10)-1, results[2]) || new Date(date) || getDefaultValue();
  }
  return(new Date(date) || getDefaultValue());
}

So the next time you find yourself struggling with dates and times in JavaScript, be wary of the input parsing or the format of your input and / or timezone.

I’ve published my Active Directory authentication (AuthN) and authorization (AuthZ) module for node.js. This module supports large active directory installation where over 1000 entries may be returned from a query via range specifiers. In addition, the module will recursively enumerate and expand all nested users and groups.

You can view or checkout the code online on my github account:

Installation is easy with npm:

npm install activedirectory

Usage is pretty simple:

var ad = new ActiveDirectory('ldap://yourdomain.com', 'dc=yourdomain,dc=com', 'authuser@domain.com', 'authpassword');
var username = 'bob@domain.com';
ad.findUser(username, function(err, user) {
  if (err) {
    console.log('ERROR: ' +JSON.stringify(err));
    return;
  }

  if (! user) console.log('User: ' + username + ' not found.');
  else console.log(JSON.stringify(user));
});

Hope you find it useful!