TFS – HEERES Online

TFS / Nuget: There are duplicate packages

March 4, 2016 by George Heeres·0 Comments

With a continuous integration and release pipeline configured and working, I was going back through projects and creating the necessary build and release tasks. One build job however was failing with the following error:

2016-03-04T13:04:26.3021168Z Set workingFolder to default: D:\TFS\tasks\NuGetInstaller\0.1.16
2016-03-04T13:04:26.3391205Z Executing the powershell script: D:\TFS\tasks\NuGetInstaller\0.1.16\NuGetInstaller.ps1
2016-03-04T13:04:26.6061472Z D:\TFS\agent\worker\tools\NuGet.exe restore "d:\TFS\_work\1\s\src\MySolution.sln"  -NonInteractive
2016-03-04T13:04:26.9761842Z MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
2016-03-04T13:04:27.0951961Z ##[error]There are duplicate packages: log4net.1.2.10
2016-03-04T13:04:27.1041970Z ##[error]Unexpected exit code 1 returned from tool NuGet.exe

The error causing the job to fail is:

There are duplicate packages

Checking The Build

So I checked out the project and compiled with Visual Studio, no problems. Then I tried using the nuget command from the command line.

D:\Projects\MySolution\src>nuget restore
ll packages listed in packages.config are already installed.

The packages folder already exists so we have to clear it first. Since I have Cygwin installed (and I prefer the Unix tools), I just deleted the folder from the command line:

rm -rf Packages

You can obviously do this from the Windows explorer shell or use the native DOS commands.

So running the command again, all of the packages restore without issue. Hmmm…

Replicating The Error

So next, I run the identical command that was logged in the log output for the build job:

D:\TFS\_work\1\s\src>D:\TFS\agent\worker\tools\NuGet.exe restore "d:\TFS\_work\1\s\src\MySolution.sln"  -NonInteractive
MSBuild auto-detection: using msbuild version '12.0' from 'C:\Program Files (x86)\MSBuild\12.0\bin'.
There are duplicate packages: log4net.1.2.10

And now we finally get the error to duplicate.

But why?

Let’s check the versions of our NuGet utility.

On the build server, we have:

D:\TFS\_work\1\s\src>D:\TFS\agent\worker\tools\NuGet.exe
nuget Version: 3.2.1.10581

On the workstation with Visual Studio (with NuGet specified in the path), we have:

D:\Projects\MySolution\src>nuget
NuGet Version: 2.8.50926.602

Different NuGet Versions

So the build server is using the 3.x branch of NuGet which doesn’t handle XML errors as gracefully (or is more strict) as the prior version. So now we know the cause of the problem, let see if we can find the duplicate.

And sure enough, inside of a subproject we have the following inside of the packages.config file:

  <package id="log4net" version="1.2.10" targetFramework="net45" />
  <package id="log4net" version="1.2.10" targetFramework="net45" />

Remove the dupliate line, recommit and push the code.

Problem SOLVED!

TFS / Release Management: Enable IISWebAppDeployment Task (or deploy your own custom tasks)

March 2, 2016 by George Heeres·1 Comment

For my employer, I’ve been working on setting up a continuous integration and release pipeline using Team Foundation Server (TFS) on-premise. Earlier this year, I had started experimenting with the standalone Release Management 2015 tool. Although I made some progress, the XAML based workflows and client tooling left a lot to be desired and I ended up reverting to using msbuild deployment options directly from the build itself. Not ideal, but better than nothing.

I was ecstatic to learn that a web-based version of Release Management had been integrated into Visual Studio Team Services (VSTS), formerly Visual Studio Online, as part of a preview release. But much to my dismay, the new Release Management (in preview), wasn’t scheduled to be available for on-premise TFS until later in 2016. VSTS typically get’s new updates every three weeks, with those updates rolled to TFS on-premise on a quarterly cycle. It was clear that the Release Management 2015 client and server tool were going away so it didn’t pay time to continue exploring the solution.

An Early Birthday Present!

After resigning myself to the fact that I’d have to wait to get my hands on the new Release Management tools, the release candidate for TFS 2015 Update 2 was announced. Much to my surprise and excitement TFS 2015 Update 2 RC1 included the new web-based version of Release Management.

Although I typically avoid release candidates, this couldn’t wait…

Dude, Where’s My IIS Deployment?

With the new bits installed on the server, I finally had access to the new web-based Release Management UI and could start following along with some of the online tutorials I was seeing for VSTS. In one of those tutorials, I saw reference to an “IIS Web Application Deployment” task. Just the task I needed to deploy my application to IIS! Unfortunately, this task wasn’t available on my on-premise installation?

After a little bit of searching, I found the open source vso-agent-task Github repo for all of the tasks included in VSTS:

https://github.com/Microsoft/vso-agent-tasks/

Included in that repo is the IISWebAppDeployment task which is missing for my installation. I assumed that the IISWebAppDeployment task wasn’t available simply because it hadn’t been included in the TFS 2015 Update 2 RC1 release. Should be simple enough to fix…

Use The Source

With the source in hand, the next step was figuring out how to get it deployed. I found an article that discussed how to deploy custom tasks… not a task for the faint of heart. As I prepared myself for the task, I learned that since that post was originally made, Microsoft had released the TFS Cross Platform Command Line Interface which not only simplifies the process but is the supported and recommended method for installing extensions.

TFS Cross Platform Command Line Interface

Insallation

Installation of the tool is easy with node.js and npm.

npm -g install tfx-cli

Getting Authenticated

The first step is to get logged into the TFS server. By default, the logon process is going to look for a VSTS personal authentication code which doesn’t apply (yet) to TFS on-premise. Thankfully basic authentication is supported, but you’ll need to enable it on your TFS server. While you’re enabling basic authentication, I strongly encourage you to configure your TFS / IIS server to force SSL. Sending a basic username and password without an encrypted connection is ALWAYS a bad idea.

Run the following command to interactively login:

C:\Users\gheeres\vso-agent-tasks>tfx login --auth-type basic
TFS Cross Platform Command Line Interface v0.3.15
Copyright Microsoft Corporation
> Service URL: https://tfs.heeresonline.com/MyCollection
> Username: tfs-admin@heeresonline.com
> Password:
Logged in successfully

One logged in, you can run additional commands to view the list of installed tasks.

C:\Users\gheeres\vso-agent-tasks>tfx build tasks list
TFS Cross Platform Command Line Interface v0.3.15
Copyright Microsoft Corporation

id            : 7c6a6b71-4355-4afc-a274-480eab5678e9
name          : DecryptFile
friendly name : Decrypt File (OpenSSL)
visibility    : Build,Release
description   : A thin utility task for file decryption using OpenSSL.
version       : 1.0.5

...

If I recall, when I grep‘ed the output for the IISWebAppDeployment task nothing was found. This should be easy. Let get that deployed.

C:\Users\gheeres\vso-agent-tasks>tfx build tasks upload --task-path Tasks\IISWebAppDeployment
TFS Cross Platform Command Line Interface v0.3.15
Copyright Microsoft Corporation
Error: Failed Request: Conflict(409) - A task definition with id 89a3a82d-4b3e-4a09-8d40-a793849dc94f and version 1.0.9 has already
been uploaded. To replace the existing definition, re-register the task definition before uploading the package.

$&!#? Already Been Uploaded? Seriously?

The error tells us that the task, has already been uploaded. But when we search our Task list in the build and release UIs, the “IIS Web Application Deployment” task is still not seen. Hmmm…

If we look at the task.json file, we notice a “visibility” attribute with the value of “Preview”.

  "visibility": [
    "Preview",
    "Build",
    "Release"
  ]

Although I couldn’t find documentation, this “Preview” tag must prevent the UI for seeing the task. So lets get that changed. Edit the Tasks/IISWebAppDeployment/task.json file to remove the “Preview” option, or apply the following patch.

diff --git a/Tasks/IISWebAppDeployment/task.json b/Tasks/IISWebAppDeployment/task.json
index fc1d9b4..2bd1535 100644
--- a/Tasks/IISWebAppDeployment/task.json
+++ b/Tasks/IISWebAppDeployment/task.json
@@ -6,7 +6,6 @@
   "helpMarkDown": "[More Information](http://aka.ms/iiswebappdeployreadme)",
   "category": "Deploy",
   "visibility": [
-    "Preview",
     "Build",
     "Release"
   ],

Overwrite

Now if we try to deploy it again, we’ll get the same error that the task still exists. One solution is to increment the version information. However if you do that, it could break future updates. So instead, we’ll use the –overwrite command. You can view the list of available options by using the –help command. The output is included below:

C:\Users\gheeres\vso-agent-tasks>tfx build tasks upload --help
TFS Cross Platform Command Line Interface v0.3.15
Copyright Microsoft Corporation

fTfs
fSSSSSSSs
fSSSSSSSSSS
TSSf fSSSSSSSSSSSS
SSSSSF fSSSSSSST SSSSS
SSfSSSSSsfSSSSSSSt SSSSS
SS tSSSSSSSSSs SSSSS
SS fSSSSSSST SSSSS
SS fSSSSSFSSSSSSf SSSSS
SSSSSST FSSSSSSFt SSSSS
SSSSt FSSSSSSSSSSSS
FSSSSSSSSSS
FSSSSSSs
FSFs (TM)

Syntax:
tfx build tasks upload --arg1 arg1val1 arg1val2[...] --arg2 arg2val1 arg2val2[...]

Command: upload
Upload a Build Task.

Arguments:
--task-path Local path to a Build Task.
--overwrite Overwrite existing Build Task.

Global server command arguments:
--auth-type Method of authentication ('pat' or 'basic').
--username Username to use for basic authentication.
--password Password to use for basic authentication.
--token Personal access token.
--service-url URL to the service you will connect to, e.g. https://youraccount.visualstudio.com/DefaultCollection.
--fiddler Set up the fiddler proxy for HTTP requests (for debugging purposes).
--proxy Use the specified proxy server for HTTP traffic.

Global arguments:
--help Get help for any command.
--save Save arguments for the next time a command in this command group is run.
--no-prompt Do not prompt the user for input (instead, raise an error).
--output Method to use for output. Options: friendly, json, clipboard.
--json Alias for --output json.

To see more commands, type tfx build tasks --help

So let’s try deploying that again with the –overwrite option.

C:\Users\gheeres\vso-agent-tasks>tfx build tasks upload --task-path Tasks\IISWebAppDeployment --overwrite
TFS Cross Platform Command Line Interface v0.3.15
Copyright Microsoft Corporation

Task at C:\Users\gheeres\vso-agent-tasks\Tasks\IISWebAppDeployment uploaded successfully!

SUCCESS!!!

IISWebAppDeployment uploaded successfully!

Checking the list of available tasks, we see that it is now available. In a future post, I walk through our release configuration.

IIS: Using UrlRewrite to Force SSL Connection

February 1, 2016 by George Heeres·0 Comments

When setting up web applications, typically we want to use SSL by default. Instead of writing custom application code to redirect, we can configure IIS to automatically do the redirection for us.

Installing Url Rewrite

To get started, the first thing you need to do is to use the Web Platform Installer to install the Url Rewrite module. Simply do a search for “rewrite” and then click on Install.

Once installed, you should have a new option available for the machine as well as the site and each application. Setting defined in Url Rewrite can be overridden successively where an application web.config will override the site web.config which will override the server web.config.

Rule To Redirect HTTP to HTTPS

Creating a rewrite for SSL is pretty simple. With the URL Rewrite applet open, follow these steps:

Open the “URL Rewrite” applet by double clicking on it.
Click on the “Add Rule(s)…” action at the right.
Choose “Blank rule” from the “Inbound rules” and click on the OK button.
Give the rule a name. In this case, let’s call it HTTP to HTTPS redirect
Set the regular expression pattern of the url you want to match or apply this filter to.
Typically you can use (.*) which means to match everything.

Use the “Test pattern…” to ensure that your regular expression is working correctly. Be careful with trailing slash matches.
Scroll down and expand the conditions grouping and click on the “Add…” button. We’re going to be adding two conditions.
- Add a match for the HTTPS environment variable checking for the value of OFF.
  
  Condition Input:
  
  {HTTPS}
  
  Pattern:
  
  ^OFF$
- Lastly we’ll add a match for the server name stored in the HTTP_HOST environment variable. You can omit this if you only have one site configured for your IIS installation.
  
  Condition Input:
  
  {HTTP_HOST}
  
  Pattern:
  
  ^(servername\d?)(?:\.domain\.com)?$
  
  Tip:Assuming you append a number to the end of a server name for each web server in the farm or cluster (i.e. servername1.domain.com, servername2.domain.com, etc.), the previous regular expression will match all of server names with a single digit, including servername.domain.com since the “?” match will match on 0 or 1 numbers.
  
  Adjust the regular expression as necessary for your environment. Make sure to use the “Test Pattern…” button to ensure you have your regular expression correct.
- When you’re done, you should have two conditions set to Match All.
Scroll down to the action grouping. Set the following values:

Action type:

Redirect

Redirect URL:

https://{HTTP_HOST}/{R:1}

Append query string:

checked

Redirect type:

Permanent (301)
To finish, click on the Apply button in the Actions pain at the right.

How It Works

The Url Rewrite applet works by adding configuration sections to the web.config file for the specified location. Instead of using the URL Rewrite, you can manually edit or add the necessary settings with careful XML editing. To do so, add a System.webServer section, if it doesn’t already exist, to your web.config. Then inside of the System.webServer XML element add the rewrite element to the existing element.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <rewrite>
      <rules>
        <clear />
        <rule name="HTTP to HTTPS redirect" enabled="true" stopProcessing="true">
          <match url="(.*)" />
          <conditions>
            <add input="{HTTPS}" pattern="^OFF$" />
            <add input="{HTTP_HOST}" pattern="^(servername\d?)(?:\.domain\.com)?$" />
          </conditions>
          <action type="Redirect" url="https://{HTTP_HOST}/{R:1}" />
        </rule>
      </rules>
    </rewrite>
  </system.webServer>
</configuration>

Note: If you manage an on-premise Team Foundation Server (TFS) server, I encourage you to do this by default to avoid your developers accidentally sending their credentials or confidential information in the clear of the network. Please note that each time you apply and update or upgrade to TFS, you’ll need to reapply the settings in the default web.config file. You can find this file at: C:\Program Files\Microsoft Team Foundation Server 14.0\Application Tier\Web Services\web.config

An engineer's opinionated opinion...

Category: TFS