A Runscope Primer – Tools of the Trade

Eric Bowden has over 19 years of software development experience around enterprise and departmental business productivity applications.


Runscope has recently become a must have tool for me when building integrations with cloud applications such as Office 365 and Salesforce. This past week I used Runscope to debug an issue with a remote event receiver I’d written for Project Online. This is such an awesome and easy to use tool, I wanted to be sure all the cloud devs are aware and know how to make use of it.


I built a WCF service which serves as the endpoint for an event receiver in Project Online, hosted in Office 365. The WCF service is hosted on Windows server, on a dev box virtual machine in Azure. In testing, it appeared as though Project Online was not calling my WCF service. Since this was a dev environment, I attached the Visual Studio debugger and enabled WCF logging. Neither of these options indicated that the call from Project Online was making it through to my WCF service.

In an ideal world, I could view the ULS log in Project Online for easy access to errors generated as a result of the failed event receiver, but this is not an ideal word; it’s the multi-tenant cloud world of Office 365.


Runscope serves as a middle tier proxy which can be inserted between Project Online in Office 365 and my WCF service hosted in Azure. Once configured, Runscope will record all requests from Project Online to my WCF service. I can view the results, edit, and replay requests! Sounds complicated, but it’s not.


Navigate to and sign up for an account. Once you are logged, create a new bucket and name it anything you like.

Click the link for Traffic once the bucket has been created.

The start page for the new bucket includes a form you can use to create the runscope url. In the example below, I entered the url path to my WCF service at Just below, Runscope provides an endpoint which can be used to log requests.

In this example, I originally configured my Project Online event receiver to call into the WCF service at Using Runscope, I configured Project Online to call the event receiver at https://cloud– As a quick test, simply place the url in your browser. You should see an entry appear in the runscope log.

Runscope Logging

Watch as Runscope logs server to server activity, errors and more.

From the traffic log (shown above) you can view the full detail of the request and response. And, you can easily edit and resend any request!

As for my issue, Runscope revealed that the request was making it to my server, and the server replied with a 404. This helped by confirming that Project Online was attempting to make the call and basic connectivity between Project Online and my WCF service was successful. The 404 reply led me to find that my WCF bindings were incorrect for a service hosted over SSL.


Runscope is a must have in my toolbox for cloud service integrations such those hosted in Office 365 and Salesforce. Of course, there are other ways to debug cloud integration issues such as the one I encountered, but this is so easy, I recommend it as your first step.

Do you have any favorite tools for debugging cloud app dev projects? Have you used other features in Runscope that are a must have for app dev in the cloud? I’d love to discuss in the comments section below.

read more
Eric BowdenA Runscope Primer – Tools of the Trade

Using ADFS Auth with SharePoint

Lane is a Senior Software Engineer for ThreeWill. He is a strong technology expert with a focus on programming, network and hardware design, and requirements and capacity planning. He has an exceptional combination of technical and communication skills.

We’ve cherry-picked internal Yammer conversations that might be helpful to the community – this one comes from Lane Goolsby about using ADFS Auth with SharePoint.

[Original Yammer Post – Lane Goolsby – June 6 at 10:27am]

I learned something recently that frankly I am surprised I have never ran into before. Since we are starting to do more Single Page Apps (SPA) I figured I would share as it may have impacts to design and architecture, and certainly impacts implementation with SPA.

If you are using ADFS auth with SharePoint, when a user is authenticated they are provided the Fed Auth cookie as we all have seen one time or another. However, what isn’t readily apparent is that the fed auth cookie only lasts for an hour – period. Now, the user’s session with ADFS is good for up to 8 hours with ADFS (by default) but every 60 minutes (again, by default) the user’s browser will get a HTTP 302 redirect from SP back to ADFS to refresh the auth token.

The 60 minute timeout for the session token is not a sliding session. Meaning, if the user is clicking around and ‘doing stuff’ in SP the token is not updated to [Now + 60]. So in theory a user could click around for 59 minutes then try to upload a file at 60:01 and will have to get a new auth token from ADFS. This doesn’t appear to be too bad if the user is working from within the browser since the browser handles the redirects gracefully and ADFS has some tricks in place to handle form POST data on the return trip to SP.

However, if the action taken at 60:01 is a GET or POST made from $http or $.ajax, then things go all crab. This is because the redirect response from SP ends up throwing a monkey wrench into the equation.

[Kirk Liemohn – June 6 at 11:19am]

Thanks for raising this issue. @Brandon Holloway, you may want to include a test case for this.

[Lane Goolsby – June 9 at 6:11pm]

I am still researching ways to address this but as of right now there are only two viable ‘fixes’ I can find. The first is to enable sliding sessions in SP via PoSh (there are few examples if you Google for it). The second fix is probably the ideal fix but is also the most complicated (I suspect). In short, make the JavaScript code smart enough to chase the redirects. I have found anecdotal evidence that browsers should actually handle all of this automagically (at least for $http) but so far this does not actually appear to be the case.

After spending most of the past couple days looking at this I think I have the root cause clearly understood. The issue is when the AJAX call is made to one of the SP REST endpoints but the token has reached its 60 minute threshold, SP issues a HTTP redirect response code back to ADFS. The problem is this triggers the CORS routines within the browser. IF ADFS used IIS instead of http.sys this wouldn’t be a big deal and we could just add the CORS headers to ADFS to allow the preflighting to pass. But alas. I am still trying to find a viable solution, but so far all the ideas we have thought of either don’t work or are sub optimal.

[Lane Goolsby – June 13 at 3:28pm]

So after dealing with this for what seems like an eternity I have two viable solutions identified.

Option 1: Enable sliding as I mentioned before. There are security risks with this approach so highly advised against if traffic is going over the internet or content is sensitive.

Option 2: Use a reverse proxy between ADFS and the browsers to inject the Access-Control-Allow-Origin header into all responses. I used IIS as reverse proxy using the URL Rewrite module and added the HTTP headers in IIS. This is probably not a viable production configuration, but you should be able to lock it down to make it ready for prime time.

Links you will likely need:

read more
Lane GoolsbyUsing ADFS Auth with SharePoint

How to Fix SqlClient.SqlException (0x80131904) for Very Large SharePoint Farms

Caroline Sosebee is a Software Engineer at ThreeWill. She comes to us with 20+ years of software development experience and a broad scope of general IT support skills.

Did You Get SqlClient.SqlException (0x80131904)?

We recently ran across a very interesting and NOT (or barely) documented problem on one of our client’s on-premises SharePoint 2013 farms which was a doozy to solve.

The site with the problem is in a very large SharePoint farm being used as a central repository for reports that are accessed by employees throughout the company. Many of the reports are published automatically via Business Objects while many others are created and uploaded manually. The net of it is that there are thousands and thousands of files being added/manipulated daily (or weekly or monthly) within the document libraries in this site. All this activity creates a lot of churn in the databases and ultimately caused SharePoint to hit what would seem to be an impossible to reach value.

The problem first manifested itself as an inability to upload a new document or to edit the properties of an existing document. When searching the ULS logs, we found that it was triggering the below SQL error:

Error: SqlClient.SqlException (0x80131904): Arithmetic overflow error converting IDENTITY to data type int.

Definition: “An arithmetic overflow is caused by a calculated column value that exceeds the column’s specified size. When this error occurs, the calculation stops and the remainder of the results pane is not filled.”

Finding the Culprit

So the next step was to figure out which SQL table was throwing this arithmetic overflow error. We didn’t have direct access to the SharePoint SQL database, so we had someone on site who did have access to run a set of queries for us. The first one was used to determine which tables had an ‘ID’ column that could be throwing this error:












,c.COLUMN_NAME,'IsIdentity') = 1 AND

t.TABLE_TYPE = 'Base Table' AND



t.TABLE_NAME NOT LIKE 'syncobj_%'



We then ran a query on each of the tables in order to check their current values:

as SiteQuota from

as Versions from

as AllFileFragments from

SolutionResourceUsageDaily from

as EventCache from

as SolutionResourceUsageWindowed from

as SolutionResourceUsageLog from

as SiteDeletion from

as SiteVersions from

As you can see from the results, it was pretty obvious which table was probably causing the error. What wasn’t so obvious is what this table did and what we could do about it.

IDENTITY columns are of type ‘int’ which allows for a maximum value of 2,147,483,647 (2 billion +) – a really huge number. Even though the current max value of the identity column on the SiteQuota table wasn’t this exact number, it definitely showed us that this one was the likely culprit. We confirmed by running the below query, which is used to return the last identity value generated for the table:

Select ident_current('SiteQuota') 

Returned value was 2,147,483,647

Do No Harm

Unfortunately, we had no idea what this table did so weren’t sure if we could reseed it or not. We only knew that the ‘SiteQuota’ table was a SharePoint system table that Microsoft does not publicly document (that we could find).

By running a series of tests on a local system though, we could tell that it was used every time a SharePoint item was touched (added / removed / updated) and that the IDENTITY column was being incremented by 2 each time one of these actions happened. We could also see that this was apparently a temporary ‘working’ table, meaning that items are added to it for the duration of the transaction then deleted.

The Solution

Given this, we were able to successfully reseed the IDENTITY column so that the system could get back online. We ran the following commands to do this and confirm success:

dbcc checkident ('sitequota', reseed, 0) 

– this reset the identity column back to 0

select ident_current('sitequota') 

– this verified that it was reset correctly

After doing this, all was now right in this particular world!

For a typical SharePoint site, we think this limit should never even come close to being reached, which is probably why we had such a hard time finding any information about it out there. But this is such a huge SharePoint farm that this particular client will need to continue to monitor this value on an ongoing basis in order to keep the system from grinding to a halt again.

I hope this helped you – if it did, please leave a comment below and, as always, feel free to ask me questions.

read more
Caroline SosebeeHow to Fix SqlClient.SqlException (0x80131904) for Very Large SharePoint Farms

VSeWSS 1.3 Deploy Issue

Pete is a Director of Technology at ThreeWill. Pete’s primary role is driving the overall technology strategy and roadmap for ThreeWill. Pete also serves as ThreeWill’s Hiring Manager and is constantly looking for new talent to join the ThreeWill family.

The Error:
No language-neutral solution exists in the store.

The Theory:
Somehow the feature and or solution guids become orphaned in the config/content dbs

Simple Fix: (worked the first two times I saw this issue)
Run “setup -u” from the bin\debug\ directory of the project(s) which builds a WSP. This uninstalls the Feature defined in the setup.bat file using the GUID generated by VSeWSS during the packaging.

Drastic Fix: (worked when Simple Fix did not)
Run the MissingFeature.exe utility and clean the SharePoint. The utility can be found here

Via Reflector: The MissingFeature.exe utility checks both the features in the Content DB and the local file system features GUIDs and notes missing file system GUIDs. This then uses “delete from dbo.Features where FeatureId = @FeatureId” to remove the orphaned items, so use this with caution.

For some details of the issues, here is a good description of the issue(s):

Hopefully, this helps someone else!

VSeWSS the SharePoint developer tools for Visual Studio 2008. These tools are suitable for use with Windows SharePoint Services 3.0 or Microsoft Office SharePoint Server 2007. It includes project and item templates for common SharePoint artifacts, it includes build, packaging and deployment for SharePoint solutions and it includes the SharePoint Solution Generator which creates a new Visual Studio 2008 project from a SharePoint site.

The tools provide item templates for List Definition, List Definition from Content Type, Content Type, Field Control, Web Part, Module, Root File, Template, Event Receiver, and List Instance. It provides project templates for Team Site Definition, Blank Site Definition, List Definition, Web Part and Empty. It works with Visual Basic .NET and C# languages and a comprehensive user guide is included. It does not include the SharePoint Workflow templates as they are built in with Visual Studio 2008.

New features in version 1.3 include:
Can be installed on x64 Server OS machines running SharePoint x64. Previously only x86 Server OS could be used
Separate build commands for package, deploy and retract are added as Visual Studio menu items
WSP View improvements for consistency of deleting feature elements, merging features and adding event receivers to features
Command line build, package and retract commands are included enabling continuous integration and build servers. Previously command line build of SharePoint projects was very difficult
Refactoring support for renaming of Web Parts. Previously renaming a web part required changes in several files in the project
Solution Generator can now generate solutions from publishing sites. Previously only regular sites could be generated
Allowing partial trust BIN deployments of web parts
New project item template for SharePoint RootFiles items
Deployment will now optionally remove conflicting existing features on the development server prior to redeployment. Previously any feature name conflicts would result in an error
Ancillary assemblies such as for business logic can now be added to the SharePoint Solution WSP
Hidden features related to Site Definition projects are now shown in WSP View. They are no longer hidden
For advanced users a fast deploy is included to update only the compiled assembly on the SharePoint development installation
Deployment step logging is included
The List Definition from Content Type template now allows for the creation of a List Definition Event Receiver
The User Guide is now installed with the extensions instead of being a separate download

read more
Pete SkellyVSeWSS 1.3 Deploy Issue

Content Type Dangers

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

Content Type Dangers

Content Types in SharePoint are hierarchical and can even be called object oriented. Each has a parent content type such that a content type can have not only its own behaviors (e.g., workflows, information management policies) and metadata (fields), but it also inherits its parent’s behaviors and metadata.

What you may assume is that you can take the current content type and look at it’s parents recursively until the parent is null, like so:

Don’t try this at home!


public void DoStuff(SPContentType contentType)
// Do some stuff with the content type
// ...
if (contentType.Parent != null)
// Recurse and do some more stuff with the parent content type

Unfortunately, if you try the above within SharePoint server-side code you will likely cause your application pool to be terminated. Then, if you are like me, you will spend several minutes trying to figure out why SharePoint doesn’t respond .

You see, the root content type is “System” and its parent is itself (seems like this is breaking some universal law of biology). So, you will go into an infinite loop if you try the code above. Instead, try the following:

This is OK to try at home

public void DoStuff(SPContentType contentType)
// Do some stuff with the content type
// ...
if (contentType.Parent != null && contentType.Parent.Id != contentType.Id)
// Recurse and do some more stuff with the parent content type

Now we simply make sure that we aren’t going to continue if the parent points to ourself.

read more
Kirk LiemohnContent Type Dangers

DataFormWebPart Video Love

Eric Bowden has over 19 years of software development experience around enterprise and departmental business productivity applications.

Check out the video player/review page below.

For this simple, but effective solution, I combined SharePoint Designer, the DataFormWebPart, and WSS 3.0. Granted, I’ve used this same concept now on several projects, so I’ve got the process down pretty good, but, nonetheless, this took about an hour to put together. This type of solution is a great example of how developers can get big bang functionality out of SharePoint without cracking open Visual Studio.

This same concept can be used to support a “review” pattern for many different types of content. A common usage for me on projects has been to allow users to view PDF documents from within the browser window; I’ve also incorporated an iframe to bring in related page content from external applications or other related content in SharePoint.

So, here’s how you do it….

First create a list. My example is a Video list. The important column is the Video URL. Insert at least one row in the list.

Next, you need to open SharePoint Designer and create a copy of the existing EditForm for the video list.

Now, let’s open the page in SharePoint Designer. The page by default will include an instance of the ListFormWebPart. The ListFormWebPart is responsible for displaying the default edit presentation of the list. We are going to leave the ListFormWebPart in place, but we are going to add the DataFormWebPart to the left side of the page for our video player. So, add a new <td></td> before the existing td.

Now that there is a landing spot for the DataFormWebPart, place your cursor within the new <td></td> and select Show Data from the Videos list in the Data Source Library pane in SharePoint Designer.

Next, you will see the Data Source Details pane. Select the URL field and choose Insert Selected Fields as…Single Item View. This will insert a DataFormWebPart which displays the URL.

Now, we have the basics of what we are after. But, it’s not so great yet huh? The snap below shows the simple video URL is being displayed. We’re getting there!

Next, we need to set a filter on the DataViewWebPart so that it will display the correct row from the list. The item edit form, by default, includes a query string parameter ‘ID’ which we can use in our DataFormWebPart to select the correct list item.

Now, let’s format the display to present the URL as a video player. Scroll down to the .rowview section of the XSL used by the DataFormWebPart. This is where the item of the view is formatted. See the before/after screen snaps below for how I changed the default format to build in the object/embed code needed for windows media player. Brush up on your XSL



Last step. We need to modify the Video List to use our new Edit Form. To do this, right click on the list in SharePoint Designer and select Properties. Navigate to the Supporting Files tab and change the Edit item form to point to the new form you have been working on.

Now, invoke the item edit form in SharePoint, and voila! You have your video review page.

read more
Eric BowdenDataFormWebPart Video Love

SharePoint Content Recovery

Eric Bowden has over 19 years of software development experience around enterprise and departmental business productivity applications.

If you were there when I lost access to my Picture Library in my at-home SharePoint site last weekend, you might have redirected me to this blog post listing out code which can help you retrieve data directly from a SharePoint content database.

See, my son is “student of the week” this week, and part of this honor is that he brings in his favorite pictures to post on the classroom bulletin board. And since we rarely actually “print” our photos, I needed to gather these together and submit to our favorite but rarely used online printer (Wolf Camera) for printing and then pickup that same day.

But, as luck would have it, I had accidentally shut down that server while an update was being applied. The server was running ‘headless’ (no monitor) and I didn’t realize it was installing updates when I killed the power.

Hey, its a home sandbox and takes a lot of abuse!

The end result was that the Windows SharePoint Services Web Application service would not start. I tried a handful of repair steps but it seemed to be getting worse instead of better, and I was running out of time on a Sunday afternoon.

Anyways, I’ve attached the source code which includes the most recently compiled exe which saved the the day.

To use:

restorefiles.exe [connectionString] restorefiles.exe Server=[ServerName];Database=[DatabaseName];Trusted_Connection=True restorefiles.exe Server=\\.\pipe\MSSQL$MICROSOFT##SSEE\sql\query;Database=wss;Trusted_Connection=True;

The utility will extract all documents from all document libraries in the specified content database. It will place these in folders, if you used a folder structure in your document libraries (which I did). Files are placed in the current working directory, where the exe is fired off, so be sure that you have space available. Be sure to test this utility on a small, simple database before going after a larger more important piece of work.

One curve ball was figuring out the correct [ServerName] to use for the SQL Server Express instance I’m using. To remedy, I installed SQL Server Management Studio Express (v9.00.2047) and used the SQL Server Configuration Manager to first enable Named Pipes and also to retrieve the exact name of the …named pipe (see snap below).


Hope that is helpful!

read more
Eric BowdenSharePoint Content Recovery

Recover App Pool Password

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

You can run this from the PowerShell command line or bundle up as part of a PowerShell Script to recover the App pool password:

  • First load your handy dandy SharePoint Assembly:


  • Next, create an object of the site that you want the application pool identity of:

$site = New-Object Microsoft.SharePoint.SPSite(“”)

  • Now create a variable to hold the web application information:

$webapp= $site.WebApplication

  • One more variable to get the Application pool information:

$AppPool= $webapp.ApplicationPool | Out-String (I use the Out-String so I can see the information if I decide to use a script)

  • Lastly, let’s take a look at the application pool information:

write-host $AppPool

  • It should look something like this:

CurrentIdentityType : SpecificUser

Username : DomanName\SPAccount

Password : YourPassword

SecurePassword : System.Security.SecureString

IsCredentialUpdateEnabled : True

IsCredentialDeploymentEnabled : True

Name : SharePoint – 2222

TypeName : Microsoft.SharePoint.Administration.SPApplicationPool

DisplayName : SharePoint – 2222

Id : a808448d-bbee-417c-9936-12bfac9738de

Status : Online

Parent : SPWebService Parent=SPFarm Name=SharePoint_Config

Version : 9013

Properties : {}

Farm : SPFarm Name=SharePoint_Config

UpgradedPersistedProperties : {}

SharePoint is a web application platform in the Microsoft Office server suite. Launched in 2001, SharePoint combines various functions which are traditionally separate applications: intranet, extranet, content management, document management, personal cloud, enterprise social networking, enterprise search, business intelligence, workflow management, web content management, and an enterprise application store. SharePoint servers have traditionally been deployed for internal use in mid-size businesses and large departments alongside Microsoft Exchange, Skype for Business, and Office Web Apps; but Microsoft’s ‘Office 365’ software as a service offering (which includes a version of SharePoint) has led to increased usage of SharePoint in smaller organizations.

While Office 365 provides SharePoint as a service, installing SharePoint on premises typically requires multiple virtual machines, at least two separate physical servers, and is a somewhat significant installation and configuration effort. The software is based on an n-tier service oriented architecture. Enterprise application software (for example, email servers, ERP, BI and CRM products) often either requires or integrates with elements of SharePoint. As an application platform, SharePoint provides central management, governance, and security controls. The SharePoint platform manages Internet Information Services (IIS) via form-based management tooling.

Since the release of SharePoint 2013, Microsoft’s primary channel for distribution of SharePoint has been Office 365, where the product is continuously being upgraded. New versions are released every few years, and represent a supported snapshot of the cloud software. Microsoft currently has three tiers of pricing for SharePoint 2013, including a free version (whose future is currently uncertain). SharePoint 2013 is also resold through a cloud model by many third-party vendors. The next on-premises release is SharePoint 2016, expected to have increased hybrid cloud integration.

Office 365 is the brand name used by Microsoft for a group of software plus services subscriptions that provides productivity software and related services to its subscribers. For consumers, the service allows the use of Microsoft Office apps on Windows and OS X, provides storage space on Microsoft’s cloud storage service OneDrive, and grants 60 Skype minutes per month. For business and enterprise users, Office 365 offers plans including e-mail and social networking services through hosted versions of Exchange Server, Skype for Business Server, SharePoint and Office Online, integration with Yammer, as well as access to the Office software.

After a beta test that began in October 2010, Office 365 was launched on June 28, 2011, as a successor to Microsoft Business Productivity Online Suite (MSBPOS), originally aimed at corporate users. With the release of Microsoft Office 2013, Office 365 was expanded to include new plans aimed at different types of businesses, along with new plans aimed at general consumers wanting to use the Office desktop software on a subscription basis—with an emphasis on the rolling release model.

read more
Kirk LiemohnRecover App Pool Password