Interview with Sean Coleman from BA Insight (Enterprise Search)

Danny serves as Vice President of Business Development at ThreeWill. His primary responsibilities are to make sure that we are building partnerships with the right clients and getting out the message about how we can help clients.


Danny:                  Hello, and welcome to the AppExchange podcast, this is your host Danny Ryan. Today we have Sean Coleman from BA Insight here with us. Thanks for joining us Sean.

Sean:                     No, thank you Danny.

Danny:                  You betcha. Today, we’re focusing in on, is it search for Salesforce? That’s the particular app that we’re going to show off today.

Sean:                     Yes, that’s right, our search for Salesforce App certainly.

Danny:                  Awesome. This will be very interesting to me, and for some of the listeners who know our backgrounds around Microsoft technologies, and learning more about the integration between Salesforce and Microsoft. It’s always an interesting topic to go into. I know a lot of customers are very interested in this topic. Hit it at a 10,000-foot level. Just give us an overview of what the app does for customers.

Sean:                     Yeah, absolutely. I think where the idea for the app came was really with our usage, right? We are a Salesforce user. We use it for our customer support. We build products for SharePoint, so really for SharePoint search and portals and things like that. What we realized was all of the trouble that our customers was having was described to us in Salesforce. All of the solutions to the trouble they were having were stored in our SharePoint [crosstalk 00:01:22] our documents. That was where we use TSS for our bug tracking. We’re a software company.

We realized all of the content that we needed to solve the problems, our documentation and everything, just wasn’t in Salesforce. I don’t think that’s uncommon, I think that people run into that everywhere, document management systems and things like that. We said, you know you could either try to jam all that stuff in Salesforce, but you run into the problem with a lot of organizations, not everyone in the organization has access to Salesforce.

Danny:                  You’re absolutely correct.

Sean:                     They are contributing in other ways. While SharePoint’s a great portal, SharePoint has a great search engine. What if we would integrate that search engine inside of Salesforce, so the Salesforce users don’t have to leave to be able to access content that’s stored in SharePoint? Then through the B-inside technology that we already have, we can extend SharePoint to go to your document management system., we have got like 50 odd connectors, we can go to a bunch of backend systems, document them, and back to Salesforce to integrate it and provide this single view of all your organizational data available to either your support reps or your sales reps to go out and solve whatever problem they’re having. Are they putting together a proposal, and they need some data to pull together? Or they are solving a customers problem or case and they need to send them a document, or they want to point them somewhere. Give them access to that information in line contextually with what they’re doing. That’s really where it came from.

Danny:                  That’s great, so it sounds like initially the concept of putting the app together was your just trying to scratch your own back, since you guys are running into this issue internally and figured customers would be looking for something similar right?

Sean:                     Yeah absolutely. We see Salesforce, we use it extensively, it’s got a lot of penetration in the cerium market, obviously Microsoft and Salesforce are starting to work together, and we realize that there is a good symbiotic relationship there. You don’t typically run into people that say, we don’t have SharePoint, we have Salesforce. It’s not that, they have them both, there’s likely SharePoint has the larger content footprint, right? Because everyone in the organization has access to SharePoint , and not everyone has access to Salesforce. Our philosophy is go to where the content is and bring the content to the people. That’s what we were trying to do.

Danny:                  Very nice. I’m looking here on the APP exchange on your listing. So this is, so you can initiate the search from Salesforce? Or what’s typically the way folks interact with the APP?

Sean:                     Yeah, so it’s actually one better than that. What you’re looking at right there is our main configuration screen.

Danny:                  Okay.

Sean:                     What that does is that it ties together … You kind of say  “Here’s my SharePoint portal address”. The neat part about that is that the next thing they do is a field mapping, because it’s not just “can you search the stuff” it’s actually a little bit of a predictive search. Meaning what you can do is you can embed it on a case object. Embed it on the case object page, when someone launches that case, what it’s going to do is it’s going to pull data off of the case itself and feed it to the SharePoint search and return a result.

Danny:                  Oh, nice.

Sean:                     You could say, send across the product names, send across the product version, send across the description and title that they’re looking for, and on the SharePoint side, it’s going to feed all that stuff in and do a search index. They’re going to start their case already with results based on what Salesforce knows about the case that was filed to them, instantaneously. They load the case up, right there is likely information that’s contextual for what they’re looking at.

Danny:                  I could see you using this for accounts as well, where you’re searching for the account name, for all the documents that are associated to that account … Definitely could see things like that, with the opportunities … There’s a lot of different Salesforce  objects where you could tie in. It’s almost like we’re … We’ve seen folks build search-based applications, its almost like it feels like an APP that you’re using. You’re just tying search in there to build out that app. Very cool stuff. Very nice.

Sean:                     We see a lot of search-based applications and things like that, having the access to the data. No one really … Whenever you run a search, it’s because you have problem to solve, right? So you have to go do a search, or you have an issue and you go to Google and you try and solve it. When you’re looking at a case, you’re trying to solve that. So bring the data that search gives you … Almost make the fact that you’re doing a search secondary to the data. That’s what you see in this screen shot right here. Here’s a test case where someone’s like, “I’ve got an install problem,” right? This is the reps view, and what he sees already is, “here’s some PDFs that are install documents.” With refiners and things to drill down, and you can configure, that this is highly configurable. So it can configure to whatever properties someone uses in their Sales Force to whatever their backend data is, we can do that mapping, and really they can find the information they’re looking for right there in line.

Danny:                  Very nice. So you can go over here, select a certain version and that would show up over here and refine it down to what in particular you’re looking for.

Sean:                     Yeah, absolutely.

Danny:                  That’s great, what a neat idea. I agree with you 100%, there’s so many organizations who … I think I’ve yet to run into a large organization that at least doesn’t have SharePoint somewhere.

Sean:                     Right.

Danny:                  Whether it being used in only particular departments but SharePoint’s all over the place. With Salesforce it’s starting off in the sales organization and maybe making its way over to marketing. They need to be able to get to the appropriate asset at the right time, so this would really help them do that.

Sean:                     Yeah, absolutely. Another thing we see is … One of the problems that we saw and we leveraged in the Salesforce … The way Salesforce search is, right so if you go and do a search in Salesforce, well here’s contact, and here’s accounts, here’s opportunity, right here’s cases that are against what you asked for. So if you know that object you are looking for, you’re in pretty good shape. If you don’t, if you’re like, “I just want to learn about this account” or “I just want to go learn about this case we just had” or something like that, it’s not what we call combined and relevancy ranked which is big from the search side.

Give me what I am interested in, but give it to me in the proper order that the system should give it to me in, rather than giving me a chunk of this, and a chunk of that. We actually have ability to take … We’re obviously exposing the SharePoint search index here and delivering it back, but inside of that search index can include all of Salesforce data too, so you can inline see documents from SharePoint, account records, contact records, case records, documents from your documents management system,, documents in other CRM’s … all inline relevancy ranked. So you actually get the full “enterprise picture” of what you’re looking for rather than the [inaudible 00:08:50] system picture.

Danny:                  Nice.

Sean:                     So Salesforce has a bit of a picture, SharePoint has a bit of a picture, Box has a bit of a picture. So we say, no, you don’t want to have to go to all of these three systems and assemble your puzzle. Put it all in one, and get the full cooperate enterprise picture of what’s going on in that one result. You’re going to get data faster, you are going to be able to make decisions faster, you’re going to make your customers happier, hopefully you’re going to close more deals. That’s what we’re after.

Danny:                  And its not even a federated search where you have to jump to different screens to go see the different results at, it’s all in one result set.

Sean:                     Exactly. All inline, relevancy rank, one result set.

Danny:                  That’s nice. That’s sweet! Very cool. With seeing how your customers are using this, any surprising usages that you’ve seen through the years?

Sean:                     I mean we’ve talking to a number of customers about the way they use it. I think the more surprising things you start to learn are that people use Salesforce in a lot of different ways. They’ve done custom objects, they’ve done things like that, and they’re always delighted to find out that data works just as fine in here as it does anything else.

I can embed this on my custom object page just like I can on the case page, and leverage searches that are relative to that. They always say what we find is …  A lot of apps we find, they work great with the core Salesforce objects. What we do, this custom thing over here, and the way that we built this is that you can embed our visual force pages and you can do that integration, and you can scope your search. You can say, in this use case, I only really want to find this particular bits of data. It’s very refined, they want to go for that. We can configure it so that it only looks at certain data sources when it does that search. A financial application, someone who’s running their leases, or anything through Salesforce.

We do all of our invoicing through sale source, great. I can give you a invoice specific history search that just pulls invoices, gives you documents previews of them, so that they can do that faster. I always say this to financing, whenever you’re doing one invoice, how often do you go to look at the last one you did? Or do a [inaudible 00:10:55] you did, so you can go look at the last one you did to compare it. Well I can give you that capability inline. Right there, so that it’s already found them based on you you’re looking at and based on their passed history was. Then you can mouse over them and get more data on it. They were excited to find that kind of stuff out.

Danny:                  80% of the sales people out there will … Their next contract they write will look somewhat like their last contract or their last statement they wrote. For some reason, I wonder why.

You are absolutely correct. This is great stuff! I’m going to go ahead and switch over to you. I’d love to see a demo from you if that’s okay.

Sean:                     Yeah, that sounds great.

Danny:                  All right, I’ll give you control here. You have the conch.

Sean:                     Okay, let me know when that shows up.

Danny:                  Absolutely. So while we’re switching over here, are you out of Boston? Or where are you based out of?

Sean:                     I’m out of Raleigh, North Carolina.

Danny:                  Oh, nice.

Sean:                     But I work kind of remotely down there. But our corporate office is in Boston, so we have a nice office up there. Right near Boston Commons.

Danny:                  That’s great.

Sean:                     It’s gorgeous this time of year.

Danny:                  I can see your screen.

Sean:                     Okay, excellent. So what we’re looking at here is a story of a case worker. Case worker that’s gotten a case in someone that’s looking for help in their network configuration. This is just an example of … I load this right up, and inline of what I here are the search results already. So this is very similar to the screen shot we looked at earlier. But here are some of the things, here’s the connector framer work release guide, connector configuration. If its certain documents types that I was looking for, I could say, you know what I DO think release notes is what I’m after, so let me filter that down. I really want things that are in this specific version, so 432, I know the customer has that. And it gets down to what I’m looking for. Now some of the cool things that we didn’t show previously is that we have the ability to now do an action with it. So I found it, this is exactly what the customer needs.

Now what I have is one example of an integration, so what I do now is I click the sales force button. It’s going to tell me, “hey, we created a solution for you.” Okay, excellent. What does that mean? That means that we took that information and we actually authored if he happened to be using the solution capability inside the sales force. We offer this solution as a draft, right here and attach it to the case. If you have workflow set up, that would email the customer about that, it would send them the release notes. You can actually go in there and edit it, make it … Maybe it’s not for this specific customer, maybe it’s for other customers, but it’s taken that data you had and put it into solution. I could just as easily configure this to say send an email, or attach this document. Or I could click an email that would take this link and attach it, put it in the email, and I could send it. OR I could attach it. Through our configuration we are actually able to extend that, and do any number of actions that someone would want.

Now that I’ve found the data, what do I do with it? If that makes sense.

So this example is one type of integration. We’ve integrated it also in the service cloud console. Just wanted to show that, in case your sale reps are console users. That have seen this nice interface here. Instead of having it inline on the left, you can have it open on the right. Gives you a little more screen real estate in here. Lets do some other stuff. If you’re someone very much into screen real estate, you can split that up and put more data over here. Same things apply, you can configure what data will feed that connector.

And I wanted to show that. So if I go back to my other app, and go to my configurations, this is where we can figure what URLs we bring in, and what objects we show and bring over. So what we’ve done here in [inaudible 00:14:54] for the case object, if someone has chosen what web browser they want, we’re going to map that over to a search and put client browser. That’s a filter that I have in my share point search, so we’re going to pass that in. If it’s got a … That’s who created it. Any number of these fields we have full access to them. So you count objects, here’s all the different fields, and you can do that mapping. What that does is it makes the initial search that much more intelligent. So it pulls the data off the of case, or off of the account and feeds into the search, so you’re going to get that much more relevant data right to the user when they get there.

Another cool thing to say is that maybe that compact search is great, and you say, “hey Sean, you’ve told me all about all the ways that I can do all my general searching in here. Do I need a larger interface for that?” And you certainly do, so what we’ve done is a larger embedding where we can actually embed … This is our full clamp portal but we can go right to our search center. If we did that same documentation, the results I am able to get here.

Danny:                  You’re probably going to show this, but does this mean I can search Sales Source from SharePoint? Does that make sense? I guess is that what you’re showing?

Sean:                     Yeah that’s right. So basically anything that’s in here … Right now we’re searching documentation. We’ve got some of our [inaudible 00:16:27] with bigger screen real estate we can do things like our document preview, inline preview, we don’t have to do it … Hit highlighting, search highlighting in here. If I click on this … My search key word, I was looking for documentation, but I’m interested in federator, it’ll show me were that’s at. Oh, it’s on page nine. So take me to page nine directly. We’ve got some more of these enhanced things that we can do in there. But also, you can query … you can index Salesforce.

So, just as easy as this document, this could be an account record, this could be an email object, it could be an attachment. Anything in there, a case itself, all could be integrated in and through filtering and things. You can come in and decide I’m only looking for sales [inaudible 00:17:18] objects or things like that.

So full enterprise search capability integrated. Obviously this is our styling, but you can style this and make it look however you want. We actually have a new lightening based interface that’s coming which we’re going to roll out soon, which will take this search center and kind of make it look … And [inaudible 00:17:39] new lightening interface, people are checking that out.

Danny:                  Very cool.

And is this … We’ve been talking in general about SharePoint 2013. Is this work with Office 365 yet?

Sean:                     It does. If you have a share point online search center, you can absolute integrate it, much the same way to do share point 2013. The single sign-on stuff exists there, so yeah! SharePoint online or SharePoint 2013.

Danny:                  Well this was a great [inaudible 00:18:08] Sean, thank you for taking the time to do that.

Sean:                     No problem at all, happy to do it.

Danny:                  So for folks who are really looking to get more out of … Who are using SharePoint for managing a lot of their documents, which is most organizations … People who are using sales force really want to get more out of backend documentation, all that content that you might be storing on SharePoint. This just looks like a great way for you to make up … Hold the relevant content up to the right places inside a sales source. It just looks like a great tool for those folks.

Sean:                     We hope so! That’s what we built it for.

Danny:                  That’s great! Well, thank you for doing this, I am going to … Let’s close this out here. For folks that want to see the transcript for this, and for other episodes, you can go to Thanks Sean again for taking the time to do this, and the great job the BAN insight has done with this product. Congratulations.

Sean:                     Awesome, thanks so much Danny.

Danny:                  You betcha. Every body have a great day, and thank you so much for listening. Bye bye.


read more
Danny RyanInterview with Sean Coleman from BA Insight (Enterprise Search)

Top 5 Reasons to Manage Customer Experience in SharePoint

Matthew Chestnut is a Senior Consultant at ThreeWill. He has over 20 years of software development experience around enterprise and departmental business productivity applications. He has a proven track record of quality software development, on-budget project management and management of successful software development teams.

Customer Experience in SharePoint

Customers interact with a business in numerous ways, whether browsing through the store or online catalog, asking questions in person or through email, purchasing product at point-of-sale or online and returning or exchanging merchandise after the sale. At each of these touch points is an opportunity to capture information about the experience your customer has with the process or people. Whether the customer experience is entered directly by the customer or via an astute employee, here are the top 5 reasons Microsoft SharePoint can assist with capturing this information:

  1. Lists provide a versatile repository for structured data

    SharePoint lists are the place to record the “who, what, when and where” of the customer interaction. As needs change, the list data can be enhanced, in place, to capture additional information about the customer experience without the need for data conversion or migration. Files attached to the list can help document the experience, whether they are photographs, email correspondence, or other content. SharePoint lists contain the “database” of customer information.

  2. Search for similar customer experiences

    Different customers often have similar experiences. SharePoint full-text search can be used to identify other related interactions to consolidate or remove duplicate information. Search can be executed interactively or programmatically depending on how it best fits the processing workflow. Results ranking can be used to allow the most pertinent results appear at the top of the results list. User experience can be greatly enhanced by integrating search results directly into the data entry screens, enabling search-as-you-type or search for duplicates at the initial point of data entry.

  3. Workflows allow for custom routing, review and approval

    SharePoint workflows are a method to automate, augment and enhance business processes. These workflows can send email notifications of newly added items, trigger approval steps as list items are reviewed and data is changed, and perform a myriad of other tasks, many without the need for custom programming, to support the business processes.

  4. Retention policies can trigger reminder emails or even archive older content

    SharePoint retention policies are triggers on list data or documents that occur when an item is deemed to have “expired.” Typical uses for retention policies include moving content to an archive repository when the content reaches a certain age and to send email reminders based on a number of days since the last interaction or the number of days an item has remained in a particular state or queue. Retention policies run once a week by default, but can easily be configured to run once a day.

  5. Integration with Microsoft Office

    A standard feature of SharePoint lists is the ability to export of data into Microsoft Excel. Additionally, Microsoft Access can read, query, process and report on SharePoint data, in real time, without the need to export and process the data locally. Microsoft Word can process Excel data, allowing for robust mail merge capability. Via the OpenXML SDK for Microsoft Office (software development kit) programmatic creation of Excel, Word and PowerPoint slides provide unlimited possibilities for presentation of customer experience data.


Microsoft SharePoint provides a rich out-of-the-box infrastructure for data related to customer interactions and experiences (as well as for many any other data-related needs.) The combination of list data, full-text search, workflows, retention policies and integration with Microsoft Office can cover a wide range of customer experience use cases, empowering your organization to streamline your business data collection and processing.

ThreeWill has the ability and experience to transform standard SharePoint lists and forms into enterprise level applications. Contact us to let us know how we can help you.

read more
Matthew ChestnutTop 5 Reasons to Manage Customer Experience in SharePoint

SharePoint 2013 Hybrid Environments and Search

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

Looking for a New Hybrid to Drive?

Hybrid automobiles are all the rage these days. By combining a familiar technology with newer technologies, we see a new kind of vehicle that provides better mileage and less exhaust emissions.

Office 365 provides us with a similar opportunity – to bring together the familiarity of on-premises servers such as Exchange, Lync, and SharePoint with their cloud-based counterparts in a “hybrid” environment that draws on the best of both worlds.

This article focuses on a hybrid environment with regards to SharePoint and in particular as it relates to the search experience for users. We’ll look at some opportunities as well as limitations, and we’ll discuss approaches on how to configure an environment to improve the search experience.

Taking a Look Under the Hood

Hybrid O365 environments come in 3 flavors called “topologies”:

  • One-way outbound: On-premises SharePoint 2013 farm(s) can reach SharePoint Online, but 
SharePoint Online cannot reach on-premises SP 2013 farm(s).
  • One-way inbound:  SharePoint Online can reach on-premises SharePoint 2013 farm(s), but 
on-premises SP 2013 farm(s) cannot reach SharePoint Online.
  • Two-way: On-premises SharePoint 2013 farm(s) can reach SharePoint Online and visa-versa.

For many enterprises, the only option is one-way outbound because it is not feasible or desirable to open up the firewall and allow Office 365 to communicate directly to the on-premises SharePoint servers. Because we believe many enterprises will have the same restriction and because our experience is around this topology, the rest of this blog will assume a one-way outbound technology. To learn more about the various topologies, check out this guide.

Kicking the Tires

There are currently 3 integration points with SharePoint Online:

  • Search: allows search results from SharePoint Online to show up in SharePoint on-premises search results (and visa-versa) depending on the topology. This can work in any of the topologies listed above.
  • Business Connectivity Services (BCS): allows on-premises data sources to be accessible to SharePoint Online via BCS. This only works with inbound and two-way topologies.
  • Duet Enterprise Online: allows on-premises SAP content to be available to SharePoint Online. As with BCS, this only works with inbound and two-way topologies.

While BCS and Duet are interesting and useful in their own right, our focus for now is on Search.

SharePoint 2013 Hybrid Environments – Checking Over the List of Available Options

Given our stated focus on a one-way outbound topology, we know the following with regards to search:

  • Our on-premises search center(s) can show results from on-premises SharePoint as well as SharePoint Online.
  • Our SharePoint Online search center(s) can only show results from SharePoint Online.

We’ll be using out-of-the-box SharePoint Federated Search for getting results from both of our SharePoint environments and showing them in our on-premises search center. It is possible to use third-party solutions to provide different alternatives on indexing and federating, but that topic is beyond the scope of our discussion.

Here are the prerequisites for configuration:

  • A SharePoint Online tenant
  • A SharePoint 2013 on-premises farm
  • Proper configuration of server-to-server authentication

Point number three is seriously important: If server-to-server authentication isn’t setup correctly then we won’t get any results, even if everything else is setup properly. While attempting to set this up is beyond the scope of this article, this guide and this article will give us what we need to know.

Once we’ve met the prerequisites we’ll configure our SharePoint 2013 on-premises server to crawl any content we wish to show in the search results. While this is typically the local farm, it could also include other SharePoint 2013 farms, SharePoint 2010 farms, and local BCS data sources (such as line-of-business applications). Result Sources for each of these sources of data are typically created in the Search Service Application (the steps for doing this are well-known and well-documented, so they are omitted here for brevity).

After we’ve configured all of the desired local content sources, we’ve got one more left: the one for our SharePoint Online data. The configuration details are specified below and shown in the accompanying screen shot:

  • Result Source Name: SharePoint Online Results (or whatever you like)
  • Protocol: Remote SharePoint
  • Remote Service URL: <your SharePoint Online URL> (such as
  • Type: SharePoint Search Results
  • Query Transform: {searchTerms}
  • Credentials Information: Default Authentication

Next, we must answer the following question: which data is more important in terms of presentation – the on-premises data or that from SharePoint Online? This option is pretty significant because it will drive how search results are presented to users. While the answer to this question will vary by circumstances, the key here is to specify the “default” data source in order to indicate which one will take the preeminent place on the results page (as we shall see, this also affects refiners and other capabilities). The screen shot below illustrates how to set the SharePoint Online Results as the default, but for the remainder of this blog we will assume that the Local SharePoint Results are the default.

The way we’ll identify our secondary source is by creating a Query Rule in the Search Service Application (for this exercise, we’re assuming the secondary source is SharePoint Online):

  • Name: “All SharePoint Online Results”
  • Context: All sources, categories, and segments (we are staying that we want the query rule to always run)
  • Query Conditions: none (remove the default condition provided)
  • Publishing: Accept default value
  • Actions: add a Result Block
    • Block Title: SharePoint Online results
for “{subjectTerms}”
    • Configure Query: {subjectTerms}
    • Search this Source: SharePoint Online Results (Service)
    • More Link (substitute the link for your SharePoint Online instance):
    • Block Location: This block is always shown above core results
    • Routing: leave blank

Note that with SharePoint 2013, much of this configuration can be done within a site collection and doesn’t have to be done at the farm or tenant level.  We’re just configuring at the farm level to make it the default for all site collections.  These configuration changes can be overridden at a site collection as we’ll see in a later blog post.

Let’s Take This Cream-puff for a Spin

Issuing a query in the search center should show results as follows (assuming we’ve entered search terms that get matches in both environments):

Search Results

As we exercise the results, it’s worth noting that the default Result Source sets the agenda for refiners; they are based on the default result source, not secondary. However, if we click one of the refiners, both the default and secondary sources refresh based on the refiner selected.

In addition, even though we can page through results from the default result source, we only see the block of results from the secondary result source on the top of the first page of results.  Incidentally, the Result Block  can be configured to show “within core results”.  In this case the secondary results will still be in one block, but may not be at the top of the page or even on the first page based on relevance.

What if You Run in the Ditch?

Here are some things to check if the results don’t match expectations:

  • Misconfiguration of Query Rule or Result Source can cause missing or incorrect results
  • The hybrid server-to-server has to be configured properly – some good troubleshoot tips are documented here.

More Options to Close the Deal

What about People Search? Videos? Conversations? How do we get search results initiated from SharePoint Online to show us integrated results containing both on-premises and online content? We’ll consider these in future installments. For now, get out there and drive the wheels off that new “hybrid”.

read more
Kirk LiemohnSharePoint 2013 Hybrid Environments and Search

SharePoint Document Generation Webinar

John Underwood is a Technical Evangelist at ThreeWill. He has nearly thirty years of software development experience. He is an experienced technical instructor with superior presentation skills and is proficient in delivering standard curriculum as well as developing and delivering custom curriculum.

Well, good afternoon or morning or evening to all of you, depending on where you’re at. I’m John Underwood and I’m going to be your tour guide today as we go through the SharePoint Document Generation webinar. I’ve had the good fortune of having some of you attend my webinars and classes in the past and if you’ve come back, I thank you very much.

For those of you that are new, just a little bit about me. I’m lucky enough to be working at ThreeWill. My job there is Technical Evangelist and, really, that involves two things. I get to do some engineering chores where I write code and then also get the privilege of doing a lot of presentations, be they webinars or videos or even face-to-face training classes. That’s something that I really enjoy doing.

If you need to contact me for any reason, you see a couple of touch points there. My email, [email protected] Also, if you like, you can follow me on Twitter @jbutweets. You may want to scribble that email address down really for two reasons. Number one, if we were to have any technical problems during the presentation, you could use that as an alternate way to reach me in a moment of desperation. Beyond that, you may have some questions, some follow-up, et cetera, after the webinar. Always welcome your questions. Always have to give the disclaimer that I can’t act as your off-site help desk, but generally if a customer follows up with a specific question about a topic I’ve taught, I always wind up learning something myself. If you have some questions on today’s material, I would welcome those emails in the future.

Now, some of you may have heard that we were giving away an Xbox today. My little bruised ego, I’m not going to think that you showed up just for that, but on the other hand, if it did motivate you, that’s a good thing. Just letting you know that we’ll give you the details of the giveaway and how that’s all going to happen at the end of the presentation. Obviously, we’d like for you to stay engaged until the very end so that we can share some useful information to you.

Who am I going to be speaking to today? First of all, I’m expecting that some of you are in the role of developer and you may be a developer that’s experienced with SharePoint or maybe you’re kind of new to it. Regardless, I’m going to show you some code today. We’re going to do some things in Visual Studio, but even more than that, I’m going to show you some concepts that will let you go back and help your SharePoint users get the most out of SharePoint.

The second audience that I hope to address today are those of you that just showed up and you’re just SharePoint users and you want to learn some cool new stuff. I think what you’re going to see today is really two things. Number one, some tips and techniques that you can actually go use this afternoon after the webinar is over, and then beyond that, just kind of a knowledge of what’s possible. Sometimes that’s really helpful even if you’re not the person that’s going to write this little application that does this wonderful thing. Just knowing what it possible will then allow you to go back and have a conversation with your developers about what you want to accomplish.

Then, finally, some of you are just kind of more in a leadership or a business decision-maker role and more than anything, through the code and the concepts and the tips and the techniques and the knowing what’s possible, my goal is to help you get the most out of your investment in SharePoint. I think SharePoint has some really compelling capabilities, but I think it’s also kind of a shame that a lot of enterprises under-utilize that and that’s part of what we’re hoping to do in this webinar series is not only raise some awareness about ThreeWill, but also some awareness about some of the cool things that you can do with SharePoint inside your company.

With that, what are we going to talk about today? Well, SharePoint is all about collaboration. That’s one of the principle reasons that it exists. We’re going to talk about SharePoint in light of another tool that we use for collaboration and that is email. Whether it’s right or wrong and whatever the reasons for it, I would make the argument that this is the way that most of us collaborate in business. We put an attachment on an email and we ship it off and the process begins. What I’m going to try to do today is convince you that that’s not the right answer in all circumstances and that if you have access to SharePoint, you have a much better mechanism at your disposal for doing that kind of collaboration.

Then from there, we’re going to build on that and talk about what we call around here “a single source of truth”. If we’re going to collaborate on something, then we all kind of have to agree upon what the truth is, what’s the actual data, and what state that data is in. If you’re not already aware of it, I think you’re going to see that SharePoint gives us a really, really easy way at arriving at that. Easy enough, in fact, that we don’t even necessarily need to involve a developer for this part. If we can get our SharePoint users to the point where they understand and can implement this part of the story, then the next part gets pretty interesting and that is the producing of documents.

What I’m going to do is talk you through a couple of application scenarios where we can take data that’s been stored in SharePoint and then use that to produce PDFs or spreadsheets or whatever it is we’re trying to create, and in doing so, automate a repeatable business process. That’s kind of a cornerstone of what I’m talking to you about today. These are not things that are kind of one-offs, but they’re things that have to happen over and over and over. If we have to do a report on our quarterly sales numbers, we have to do that every quarter. If we can make that process simple and easy to consume, that’s just going to make life better for both those that produce the data and those that consume the data.

That’s really the over-arching story of this entire webinar. Even though we may talk about technical details or end-user details. Ultimately, it is about a repeatable process where we can automate the generation of documents that we use in the running of our businesses.

Then, with that, let’s talk a little bit about the collaborating via email model. I’m not trying to hide my disdain for it here, at least in certain circumstances. The reason for that, if we go back through the history of email, originally it was a text medium. It was just a way to send a message to someone and get a response. Really, I can remember in my career I first started using email, the thing that I thought was so awesome about it was that it was in asynchronous form of communication where I could send off a message and when this person got time, they could respond. I didn’t have to interrupt them by calling them on the phone, nor did I have to sit and wait on the phone while they tried to formulate an answer. In some respect, it’s a very, very efficient way for human beings to communicate with each other.

As email evolved, it got some extensions. Most of you are familiar with this acronym of multipurpose internet mail extensions or MIME. Now, several really good things came out of that. The ability to have different character sets. The ability to have some more sophisticated capabilities in mail. Depending on its intended use, one possible unfortunate side effect is the advent of attachments.

I know you guys love attachments. You use them a lot and I use them a lot, too, but I say “unfortunate” in a couple of respects. Really the problem with attachments is that they don’t require a lot of forethought. They don’t require us to put anything in place in order to carry out the process. We can just at the drop of a hat say, “Oh, you need that document? Boom, I’m going to send you an attachment.”

In a general sense, there are three, maybe more, kind of issues with this that I want to talk about. I’m going to focus on the first one. The versioning or the accuracy of the data. That is really hard to do when you’re working with attachments. Give Microsoft credit. Over the years they’ve tried to put some capabilities into their office products where they can track changes and be aware of the fact that it’s being circulated by email, but in the end, the problem here is just this wasn’t meant to be a collaborative authoring tool.

Even if you can get past that, another problem with attachments, and I have to confess I do this all the time, I know a coworker sent me something and I know it’s attached to an email, but gosh, which folder did I put it in or did I accidentally delete it? This whole accountability of being able to locate a piece of important data quickly. If I don’t religiously get those things out of the email and into some well-known place, it’s very, very easy for me to lose track of them. At that point, I’m either at the mercy of the organizational system in my inbox or I’m at the mercy of the search facility in my inbox in order to locate something that’s really important.

Another one that’s a challenge with email attachments, and I’ve experienced this firsthand, you’ve got someone that’s kind of responsible for some important data, they have it in their inbox in a series of attachments, and they depart the company. Before we really get a chance to contemplate what it might mean, their email gets decommissioned. Their inbox gets blown away. This data that might really be important to us is kind of lost there.

My encouragement to you in a very general sense is before you start attaching business documents and shipping them off, think a little bit. Think about maybe there’s a better mechanism that we could use to accomplish that kind of collaboration. Now, the flip side of that, if we’re not really going to be changing the data, if I’m just sending a map of directions to my friends so they can come to my party, attachments are awesome for that. I think it’s just another case of let’s make sure we’re using the right tool for the right job.

Then let’s take a typical scenario and I can say typical not only because of I’ve experienced this, but because literally we’ve written applications for several of our customers that follow this exact pattern. We start out with a series of sales managers. We’ll say that we have a manager of each one of our regions and each one of these managers has got to contribute to a spreadsheet of quarterly numbers. Ultimately, the consumer of this spreadsheet will be people that are high in the company, executives. It’s really important that the information be correct and up-to-date.

How we usually start this, well, one of the managers will step up and say, “Okay, I’ll start the spreadsheet.” He or she goes in and puts in their numbers and then they do an email and they send it to all of the other users at once. Then they all take a solemn oath that they’re going to make sure that they keep track of who’s making what changes so that we can keep everything up-to-date. The reality is that rarely happens. The original author forgot they made a change and they went back in and did it and now two other people don’t know about it and so in a more specific sense, we have multiple copies, multiple changes, but no single, authoritative copy.

It’s funny. I’ve seen people go to just great lengths to make this work. They’ll start attaching version numbers to the end of the documents or they’ll start attaching their initials and a version number. Again, that’s just all treating the symptom instead of treating the actual problem here. Then put yourself in the role of the VP of Sales. This person has now got to try to figure out which one of those spreadsheets has the most correct data and then they’ve got to use that and forward it to the executive team. If we’re willing to do a small amount of work up front, what we can do is make it so that this information can be more reliably entered, more reliably modified, and then more reliably reported to those that are interested in it.

Let’s look at the problem a little bit differently. We said that the goal here was to produce a spreadsheet that had quarterly sales numbers that were accurate and usable, but part of the problem is that we started with the end. Let’s don’t start with the spreadsheet. Instead, let’s start at a higher level. We have a scenario where we’re trying to author or create some content and we have many contributors to that. Whenever you think about problems in business and how you’re going to solve them with technology, that’s something you always want to consider. Who are my authors and how many of them am I going to have? In this case, if we have regions, we’re going to have five authors. If we have thirty regions, we’re going to have thirty authors. We need a mechanism that works as well for two as it does for twenty-two. Something that will scale in the human sense.

Another way of looking at this problem is not only do we have many data contributors, but we have a relatively-small number of reviewers or approvers. In fact, in this case we have exactly one. The Vice President of Sales. Not only do we need to have a mechanism that will allow many people to contribute, we also have a mechanism that will allow those who are responsible for doing so to be able to review, approve, and reject content without a lot of botheration.

Another thing that’s really important here, if we’re going to make this work, we’ve got to have one location for the data and there has to be a mechanism for managing the changes in that data. This is also a really important one. We want to do this in such a way that those who would consume the data can’t see it until it is approved. That way we don’t get illegitimate numbers out in the wild and then we have to play this game of, “Oh my goodness, who all has this and how do I chase them down and tell them how to fix the numbers?”

Only then, once we’ve gone through the process of authoring, approving, and making the data fit to be used, do we worry about the producing of the documents. Then what you’re going to see in the little demonstrations that I’m going to do for you is that in a very basic sense, we can use SharePoint itself to produce a document or we can write some applications that would give us a great deal of power and flexibility on how we produce those documents.

Then let’s contemplate for a moment how we’re going to use SharePoint to drive us toward this single source of truth so that we can all agree upon the data that we’re acting on. Well, in my scenario what we’re going to do is, instead of entering those numbers directly into a spreadsheet, instead we’re going to enter them into a SharePoint. As you’re going to see in the demonstration in just a moment, this is a really, really simple SharePoint list that any user can go in and create on their own. I think that’s one of the most cool things about SharePoint and it’s something that you really need to be tapped in if you’re an end-user and that is you can get this process rolling into a usable state without having to rely upon a developer. Then once you do get it to a usable state, we can bring the developers in and even automate it further.

The user’s going to create the list and then once the list is created, now we have an easy path for all of those that are submitting the data. Quite literally, we can circulate a URL and say, “From now on, when you need to submit quarterly numbers, this is the place you go.” Not only could we give them the URL of the SharePoint site, we can even give them the URL to the specific list. It really doesn’t get much easier than that in terms of them knowing where to go in order to submit the data.

As you’re going to see in the demonstration, SharePoint has some built-in approval capabilities that you may or may not be aware of. They really take two levels. At a very, very industrial-strength level we have this notion of publishing and publishing sites, but at the list and library level, we also have this concept of approval. It’s really a breeze to set up. It just says, “I’m going to make it so that list items are only visible to the public at large or to the consuming public once we’ve approved them.” By default, when the data is submitted, it’s in a state where it’s not going to be seen by anyone other than the contributor and those that are authorized to approve it.

Let me go in and just show you a little bit about how this is going to work. Now, a couple of disclaimers as I go through my demonstrations here. Disclaimer number one is that a lot of what I’m going to be showing you today, I’m actually running on a local VM on my system. Because I’m running it on a laptop, sometimes it might not be the fastest. I wouldn’t draw any negative conclusions about SharePoint on that. It’s just the limitations of running in a VM.

Another thing that I want to point out is as I go through my various demos, I’m actually running some of them against my local SharePoint instance and some of them against Office 365. I think the important point there is just whatever you’re going to learn today, it would actually apply equally to both of those environments. Whether you’re cloud-based or not, it’s not really going to matter.

Maybe a third quick point to make about my demos. These are all very-much focused on SharePoint 2013, however, many of the concepts and much of the code that I’m going to demonstrate could also be applied to a 2010 environment. Some of the application bits that I’m going to do would not be quite as cool or quite as integrated in a SharePoint 2010 environment, but the basic concepts would still exist.

I’m going to start with a very simple demo where I go into the contents for my site and I add a custom list. I’ll start the process by doing Add an App and then choose Custom List as my template. Then, for those of you who haven’t done this before, really all I have to do is give it a name and start customizing it. I’ll just call this, “Sales Numbers List,” and create it. Once I’ve created that list, I can navigate to it. I can click on the List tab and then go to List Settings. Now, obviously the presumption here is that as an end-user, you’re authorized to do this. If you’re not, then that’s something that you wouldn’t want to talk to your administrator about. You don’t have to be a site owner or an administrator to be able to change the capabilities of a list.

There’s two things that I would potentially do here in order to implement this. The first thing I would do is go down and create some columns. Obviously, I would want to go in, as example, and create a column for my gross sales with currency and my net sales and the year and the quarter and all those other things that I want to be able to track. Then once I’ve created the necessary columns, the other thing that I’ve got to do is set up my approval mechanism. Again, this just couldn’t be easier because literally I’m going to go into the Versioning Settings and right here, I’m going to turn on Require Content Approval for Submitted Items. That’s literally all that’s involved in getting this thing up and going.

Now, kind of like they do on the cooking shows here, I’ve shown you the basics of how I would do this. Let me know go over to a site where I’ve actually got it set up and I can show you this list in action. This particular one is named, “Quarterly Numbers,” and you can see that all of my sales managers have done a really, really good job of getting in their numbers for the third quarter. Now we find ourselves approaching the end of the fourth quarter and I’m going to put myself in the role of a sales manager and I’m going to go in and add some data. I’ll just simply go up and add a new list item and then notice the warning that the user gets here. It basically says, “This is a list that’s using the approval mechanism, so when you submit your data, it’s only going to be seen by you and those that are eligible to do approvals.” I’ll talk about that a little more in just a moment, those that are eligible to do approvals.

I’ll go ahead and put in my fourth quarter numbers here and we’ll choose a particular number value for each one of these. Again, I don’t pretend these are accurate numbers. I’m just trying to put in something for demonstration purposes. Now, notice if you will, when I save this, it immediately goes into a pending state. I can see it as an author and then anyone else that has approval permissions would be able to see it.

Now, if I switch roles real quickly, now I’m in the VP of Sales role and I’ve got to go in and make a decision about whether this thing is legit or not. I’ve done a little research and what I’ve discovered is that this number is actually incorrect based on some other things that I’ve seen. I’ll select the item, I’ll go to the Items tab on the ribbon, and then I’ll choose “Approve and Reject”. What I can do is set it to a rejected state and then put in some details about what’s wrong with this and what needs to be fixed. It’s a very simple process. Those that are contributors can easily see whether their data has been approved or not. In fact, they can even set up an alert on this list to let them know when data has changed that they care about. Later on, if they do realize their mistake and they come in and fix it, it’s a pretty simple matter. All they’ve got to do is go in and edit the item and recognize that they put one too many zeros on that and save it. Now it’s gone back to a pending state and someone could go in and approve it.

Those of you out there that are more on the end-user scale of things, that’s a ton of functionality and, again, it’s maybe ten minutes’ worth of work to set it up. If you’re not already familiar with those capabilities of using a list that way, I want to strongly encourage you to dig a little deeper on that and give it a try. I think what you’ll see is that you can go a long way toward accomplishing what it is that you’re trying to accomplish.

Now, the last thing I wanted to show you is how we get permissions for these. I know that some of you would be the ones that would set up permissions. Others of you, you’re not going to be doing this, but you just need to know a little about it so that you can ask the right questions. If I go to my Site Settings page and then I select Site Permissions, what I want to show you is just real simply how you might set this up. What I can do is I can set up a group for people that should be allowed to approve the content. I can name that whatever I wanted to name and I could put in whatever individuals I wanted to put in.

The key part, though, is right down here. Notice the various permission levels. Obviously, we don’t want to give them full control, but in order for someone to be approver, they have to have at least design-level permissions. Obviously, my VP of Sales, he or she is going to be in the role of approving this content. They would need to have this effective permission in order to do that. Most likely, those that are contributing data are just going to have contribute capabilities. Those that might be responsible for configuring lists would need edit capabilities. All of it’s there and it’s out of the box, you just got to know how to stack things up correctly in order to take advantage of it.

Now that we’ve reached a point where we can pretty easily capture that data and feel pretty confident that it’s going to be accurately disseminated, let’s now talk a little bit about how we can get at the data. If you’re lucky enough to have a developer at your disposal, we’re going to talk in a few minutes about some of the amazing things that you could do with that. Even if you don’t have a developer at hand, once you get the list looking the way that you want it to look, it’s a pretty simple thing to get it into Excel. Literally, I’m going to go over into Quarterly Numbers and I’m going to go to the List tab, and then I’m going to choose Export to Excel.

The cool thing about this approach is that it’s not actually downloading the data, rather it’s downloading a query that pulls that data from SharePoint. The idea here is I can now slice and dice this data and accomplish whatever I want. What I would tell you is that in the real world, this is an approach that’s really cool and probably okay for someone that’s a bit of a SharePoint jockey, but for someone that’s more maybe in an executive role and they just want to push a button and have the stuff at their disposal, this probably draws them a little farther into the SharePoint experience than they want to go. For that audience, that’s the case at which we’d probably need to think about writing some applications. Let’s go have a look at that.

How is it that we’re going to produce our documents? Well, in a very general sense, what we’re going to do is write an application that uses SharePoint as its source of data. No doubt many of you have either written or worked on applications where someone wrote a “front end” that knows how to read out a SQL server or some line of business application. We’re essentially doing the same thing. In this case, instead of reading the data and putting it on screen in a data entry form, these are apps that are going to read the SharePoint data and produce some kind of document from it.

I’m going to show you two approaches that you can use to accomplish this in SharePoint 2013. The first of those is a SharePoint 2013 app. The advantage to this one is really kind of multi-fold. Number one, it integrates directly with the SharePoint site. The same destination that someone might go to to enter data, they can go to that same destination and with a couple of mouse clicks extract the data that they need.

Another thing that’s nice about the SharePoint 2013 app approach is that because it has some knowledge of SharePoint, not only do we have the ability to programmatically read data out of the SharePoint list to produce the document, but then we can turn around and take the produced document and store it in the document library. The person responsible can generate that document, but then any other party that’s interested in it can know that if they go to that library, they’re going to see an approved and released version of that document.

The other approach, and this one is completely new as well, something called an Office 2013 app. I have to confess, even though these have been out for a little while, it’s only now that I’ve gotten an opportunity to start working with these. I have to admit I’m actually pretty excited about some of the things that you can do here. The idea with this is instead of having some kind of app that integrates with the SharePoint site directly, instead I’m going to write an app that actually integrates with the desktop version of Excel or PowerPoint or whatever app I’m writing it for. The really cool part of this, as you’ll see in my demonstration, is that the consumer doesn’t have to visit SharePoint. They don’t have any concept that SharePoint is even coming to the party. All they know is that they can launch their Office app and make a few mouse clicks and wind up with the data that they want. I think you’re going to like what we can do with those.

I know some of you have been to prior webinars or maybe you just read up on your own enough that you already know about this, but on the chance that a few of you kind of don’t know the SharePoint 2013 app story, let me just take one slide here and talk about that briefly. First of all, when we say “app,” we kind of joke around the office of App with a capital “A” versus app with a lower case “a”. This is an app with a capital “A”. It’s a particular kind of programming model that’s new for SharePoint 2013. Apps are created by developers. This is not a power user thing, but rather a developer thing that is created on behalf of users. When we write SharePoint 2013 apps, we’re writing code that runs out-of-process or, perhaps betters said, runs outside of SharePoint. There’s a lot of reasons why that’s a really good thing, not the least of which is it kind of insulates our SharePoint server from being slowed down or brought to its knees by bad code.

From a developer perspective, when I say “out-of-process,” what does that mean? Well, it really means your code is going to be running in one of a couple of places. It’s either going to be JavaScript that’s running in the browser or it may be code that’s running on another server. Depending on the kind of server it is, it might likely be .NET codes such as C#. It’s also possible that you could have a non-Microsoft server that’s integrated into this process. Either way, you could just think about this model as being, “I’m going to write my code and then that code is going to remotely call into SharePoint and get whatever data it needs.”

How do you users get these apps? There’s really two models. First of all, you would have something within your enterprise called the App Catalog. Essentially, the App Catalog is like a giant library where all authorized users can find apps and install them to their sites. There’s also a global SharePoint Store. Most companies probably wouldn’t open up to their users for fear of what they might download, but the cool thing about the store is that as a user, I can actually visit there, find an app that I like, and then make a request for it in my company’s app catalog. At that point, an administrator would see that request, see that I want that app, they could go review it, and if it makes sense, they could actually purchase it and put it in the app catalog.

There’s a lot of exciting things going on there. For users, your developers are going to be able to build you some really cool things that are very easy for you to consume. For developers, what you just got is a much better mechanism on rolling out and updating your apps and having a central place where people can actually discover those apps.

For the specifics of the app that I’ve written for you that I’m going to demonstrate here in just a moment, we’ll start with the list that I showed you just a moment ago. I’ve got a SharePoint list and it contains sales numbers for a variety of quarters and years. Then from there, I’m going to write what we would call a cloud-hosted app. That is to say, I’m going to have some server code that runs in another server. It’s going to call in the SharePoint and then it’s going to read the sales numbers from that list for a particular year and for a particular quarter. Then those of you that are developers, what I’m using here is the C# client-side object model. I’m just making a remote call into SharePoint.

Now once I’ve pulled down the data that I need from SharePoint, the next step is to produce the PDF in this case. I’m going to do that via a third-party PDF library. Once I’ve actually produced that document, I’m going to turn right around and have my application write it to a document library and in that same SharePoint site. Then from there, the users will have two approaches to getting at that data. For the user that runs my application, they’ll actually be presented with a hyperlink that will easily let them read that document. In addition to that, anybody on the team that has access to that particular document library can navigate directly to it and see all of the contents within.

All right. Let me take a moment now and show you how this works. Now, those of you that are developers, you’re probably very excited to see that I’m in Visual Studio now. Those of you who aren’t developers, don’t let your eyes glaze over. I’m not going to stay in here very long. I’m just using this as a point to kind of help our developer friends understand what I’m doing here.

For the sake of our demonstration, I want to just focus on this page right here, default.aspx. This is going to run in my remote web and it has a very simple UI that’s just going to allow the user to pick a year, pick a quarter, and then generate the information that they’re interested in. Now, if we go and look at the code behind that is associated with this, even if you haven’t done any SharePoint 2013 programming just yet, if you’ve done some client-side object model in SharePoint 2010, it’s essentially the same here. The ultimate goal is for me to create a client context object. This is what will allow me to call into SharePoint and request the data that I’m interested in.

In this case, the data that I’m interested in, I’m going to extract that via a CAML query. Even if you have no idea what a CAML query is, basically we’re saying, “I want a particular year and I want a particular quarter.” Those are going to come from whatever the user happened to choose on the screen. Once I’ve asked for my data and received it back, then I’m just going to write a simple loop and in that loop I’m going to take that data and render it into a PDF. I’m going to take that PDF and write it to a memory stream. Then finally, I’m going to upload that file. Literally, I’m going to use the client-side object model to take a document that’s buffered in memory and upload it.

There should be two reactions to that. If you’re a developer, that’s pretty cool. I like the fact that I can do that. Then if you’re an end-user and you’re a little bewildered, let me kind of set your mind at ease. The reason you care about this as an end-user is that you didn’t have to think about where you stored the app. You didn’t have to worry about if you stored it in the right place or the right location on your hard drive. The app was responsible for putting it in the correct place and your job was just to ask for what you want and then get it. It doesn’t get much better than that.

All right, with that, let me go ahead and run this. Again, obviously those of you in the user role, you would not be running it from Visual Studio. This app would already be installed on your SharePoint site and you’d be able to navigate to it. Now, if I attempt to generate data for which I don’t really have any data, it’s just going to give me a little message saying that we don’t have any numbers for that particular time unit. Instead, let’s go get the third quarter of 2013 and I’ll click on “Generate”. Notice our users get two things here. They get a little indication that the PDF has been generated and then they also have this click-to-open link. As I click on that, you can see that I literally generated a PDF that just gave them a formatted list of the available numbers.

Really important here. Don’t get hung up on the format of the document. I wasn’t trying to make an artistic statement here so much as I was just trying to show you that you can programmatically create these things and upload them to SharePoint fairly easily. Obviously, we have a particular look and feel that we wanted to achieve and we can certainly do that on behalf of our users.

Now the other thing to think about is if I returned to the site on which this app is running, at any future point, if I want to see this document, I can literally go into the document’s library and there’s the PDF that I just generated. Couple of things that I think are kind of cool about this, not only did I generate the document and it knows that it was me that generated it, there’s another little cool part here and that is, notice that last column, “App Modified By”. Those of you that are developers, you probably want to make a mental or written note of this. When you start writing apps, SharePoint not only has the concept of the user’s identity, but also the app’s identity. There are some really, really clever things that you can do with that. That’s a capability that I think is pretty exciting and holds out some interesting ways to do things on behalf of the user.

All right, so there’s the process. Going forward, how would our users consume this? They’d consume it in one of two ways. They would either go to their SharePoint site, go to the Site Contents area, and click on the app in question, or we could even simply provide them with a link. They use the link and it takes them directly to the app.

I’m going to keep plugging ahead here. Just an FYI, I noticed maybe one or two of you have posted some questions out there. That’s awesome. Please continue to do that. I’m going to save those until the very end just so that I make sure that I get through all of my material. Please don’t think I’m ignoring you. I’m just holding those until the end.

All right, so the SharePoint app is cool enough, but let’s face it. There are some people out there, right or wrong, that are just a little biased against SharePoint. Maybe they had a bad experience. Maybe they just don’t like the way it looks. Doesn’t matter. The point is we can implement this same model that we just saw, a very convenient way for a consumer to get at the data that they want without them ever having to visit SharePoint.

That’s really what the Office 2013 apps model is going to be about here. Conceptually, it’s similar to SharePoint. You’re writing an app. It’s going to enhance this product or customize this product that Microsoft is providing to you. The difference here is instead of integrating through the web browser onto a SharePoint site, it’s actually going to integrate directly with Word or Excel or PowerPoint or whatever you target this to.

When you write these apps, what you’re really doing is you’re writing a web page. Developers, everything you already know about server code and HTML and JavaScript, those would all still apply. The difference here is that the web page that you’re authoring is actually going to be visible or hosted inside of Excel or inside of Word. As you’ll see in the code that I’m going to demonstrate for you in just a moment, you’ll actually have a JavaScript library that will allow your page to interact with the hosting application. I can put together logic that would allow me to inject data into a spreadsheet. I could put together logic that would let me read data out of a Word document and then do some kind of action based on that.

What is the integration point here? Well, because we’re using standard-based web tools here, we can take these pages that we’re going to add to Word or Excel and we can integrate them with a variety of back-end data sources, most notably for this webinar, SharePoint. The idea here is that the user’s going to be completely sheltered from SharePoint. They’re not even going to know it’s a part of the party, but we’ll be able to go out to SharePoint on their behalf, pull down whatever data they need, and present it in an easy-to-use format.

Now there are essentially three different kinds of applications that you can write for Office 2013. The first of those is called a task pane app. That’s the kind of app that I’m going to be building for you. As you can see, the app renders in a Window to the right of your main document and then, as I mentioned a moment ago, this app is going to have both the capabilities of putting things into the document as well as reading things out of the document and taking some kind of answer on them. Then this back-end data source, you can see in this little simulated example, someone’s actually written a task pane app that will take text, call Bing, and translate it. Pretty much any back-end service that’s a breachable service by a REST call or a web service call, we’re going to be able to do those kinds of integrations from within Office.

A second approach is one where we right what’s called a content app. In this case, instead of being in a separate pane, this is actually an application that embeds directly in the document itself. In this example, we’ve actually embedded this app that someone has authored that will read data about states and sales and then actually give us a graphical map that shows where those sales are taking place. Maybe another way for you to think about these if you’re a developer, anytime you’ve got a user that’s trying to write a formula that’s really, really complicated in Excel or they’re trying to create a chart that has some highly-customized capabilities, it could be that you could author a content app on their behalf that would do all of those things and for them, it’s literally a couple of clicks to put it in place and use it.

Then, ironically enough, I kind of ragged on mail and attachments earlier in the presentation, but there are some cases where we’d want to be able to do some clever things with email integration. That’s what a mail app is about. A classic example here. As a user, I could highlight addresses and then I could have a mail-out that would actually go out and do a Bing look-up and show the user the map for that. Anything that you’re doing that is legitimately built around the email process, we can do some very clever things with the mail app.

Once again, for those of you that do better with pictures as opposed to words, we’re starting in the same place here. We’re starting with a SharePoint list that contains the sales numbers that you saw at the beginning of my presentation. Now in this case, instead of the user navigating to a particular place, they’re going to launch Excel. Then their Excel instance will be configured in such a way that they actually have access to this task pane app that I’ve written on their behalf. For them, it’s as simple as picking the right options, clicking a button, the task pane app then will go out and read the SharePoint data on their behalf, pull it out, and then populate it into a spreadsheet or whatever kind of office document that we’d like to put it in.

Let me give you a couple of demonstrations here. I’m going to show you how you can use this process to generate an Excel spreadsheet. I’m also going to show you how you can generate PowerPoint data. I can tell you again, literally, we have written apps on behalf of customers that do both of these things. The story here is just now that we’ve got these Office 2013 apps, the process gets really, really simple and really, really easy.

All right, so I’m going to go back in now and I’m going to open up the first of my two demo apps. Then, for the people in the room that are more on the developer track, you’ll notice that when you create an office app, structurally it looks very similar to a cloud-hosted SharePoint app. That’s a pretty decent analogy. To put that in another way, most of the code that you’re going to be writing is going to be down in this app web and then really the top part is just declaratively describing some information about your app. Among the things that it would tell you, what products can it work with. The fact that this says “workbook” here, that tells me that this is for Excel and that it’s not for PowerPoint. They don’t have to be mutually exclusive. You could actually write an app that would integrate with Word and Excel and PowerPoint all at the same time.

Then the other part of this, the actual logic here for creating the data and pushing it into the document. Again, I don’t want to belabor this too much because I know I have a mixed audience here. Those of you that looked at the code earlier, here’s what you need to take away. I’m using the exact same client-side object model code to fetch the data. Same query and everything. The only difference here is that once I’ve got the data, I’m going to format it in adjacent array format and then send that back as a string. Then I’ll actually have some JavaScript code that can take that, decompose it into an array, and then insert it into the document.

How is it that things get into the document? For those of you that are going to be writing this kind of code, this is probably the most important sentence of all in this demo and that is everything that allows my app to interact with Office is encapsulated in this office.js script. Literally it’s providing me with an object model that will allow me to read and write data from JavaScript down into the document or vice versa.

In this example, I’ve set my data back in a container named “Payload”. I’m going to take that, parse it into an array, and then notice what I’m doing in this example. Because it’s Excel, Excel has this concept of a matrix. To put that in simple terms, you can create a two-dimensional array, put data in it, hand that over to Excel, and Excel knows automatically how to format that data correctly. The method that I’m using here is called “Set Selected Data ASYNC”. Those of you that write a lot of code, you know that when you see that word “ASYNC,” you can optionally provide a callback that would let you do a success or a failure message. For this example, I’m leaving off the callback and I’m just putting the data in directly.

All right. Let’s give this a try here. I’m going to run it now. Again, those of you that are in the user role, you’re not going to run this from Visual Studio. You would just have Excel already configured to have availability to this app. Notice what I get here. When I run it, I’m running Excel. Then the markup you saw just a moment ago has now actually rendered in this window. If I try to get data for a year-end quarter that doesn’t have any associated data, I get the same message that I got before. On the other hand, if I go in and pick some valid settings here, notice that it retrieves the data and immediately puts it into the spreadsheet.

Now if I’m in the role here of the Sales Vice President, watch what I’m going to do. I’m going to go right here and select some data and I’m going to insert a recommended chart and maybe change a title or two and I’m done. Save my work and publish this information to whoever needs to get it. Then when I’ve got to do this again next quarter, I fire up Excel, I create a new spreadsheet, I click the mouse the same three times like I did before, and I’m done. To me, the exciting part of this is just that we’re getting the collaborative capabilities of SharePoint to manage the data, but for those who want to consume it, we consume it very close to the source. They don’t have to know that much about where it came from and how it got constructed.

The model for how we would do this on PowerPoint is almost identical. The one little bit of code that’s going to be different here is that PowerPoint really doesn’t understand the concept of anything other than text data. You’ll remember a moment ago when I wrote my data, I had a coercion type of matrix. In this case, I’m just writing string data. Literally what I’m doing here is just writing a loop that will construct the data and then I’m going to inject that into a PowerPoint document.

Let’s suppose that you’re the VP of Sales and you have to produce a PowerPoint every quarter where you report on how your divisions are doing and then one of the slides has to have your numbers. Well, you’re in PowerPoint. You’re authoring the different parts of the PowerPoint and then once you get to the point in time where you want to add that particular slide, you just simply come over here and click “New Slide,” and then you click in the body of the slide. You choose the year that you’re interested in and the quarter and you click “Get Data”. Notice what it did. It actually read that data from SharePoint, formatted it in a way that is appropriate for my PowerPoint, and so now all I have to do is put on a title and then continue the process. Again, I think that is a really compelling story on how you can generate these things very easily.

All right, let’s hit a few wrap-up points here. We’re getting near the top of the hour. Kind of recapping what we talked about. First of all, if you know you’re going to be collaborating on something where there’s going to be more than one author, don’t use email. Spin the ten or fifteen minutes that it would take to set up a SharePoint list or a library and keep the data there. You’re just going to be way ahead of the curve even if you don’t have the luxury of writing applications. That single source of truth. Really in this one, I focused on lists because the data was more appropriate for a list. If you truly have to author a spreadsheet and that’s the only way you know to do it is just to author the spreadsheet, you still ought to be keeping it in a library as well. Then you can use the check-in and check-out mechanism just to make sure that you’re not trampling each other’s changes. These are going to be much better alternatives than trying to track it by email.

Another thing, take advantage of that SharePoint approval mechanism. Literally one click and you’ve got yourself in a circumstance where people can’t see the data until it’s been legitimized by some reviewing party. Then from there, for those of you that are developers, I hope I kind of whetted your appetite a little bit there in terms of writing SharePoint apps and Office apps as a way to generate these documents. Then, if you’re more on the user side of the house, while you’re not going to be authoring these applications, you now kind of know what’s possible and this is something that you can take back to your developers and kind of blaze a trail on getting these things done. Obviously, part of the point of these webinars is just the fact that ThreeWill does this. We’ve done this for several our customers and we’ve got a really good track record in making it so that people can get these documents produced repeatably and reliably without having to exert a lot of effort.

I hope enough of you stayed awake to this part of the presentation to kind of hear a little bit about our giveaway. Again, in order for you to be eligible for the giveaway, not only did you have to attend the webinar, but you’ve got to properly complete the end-of-webinar survey. You’ll get prompted for that when you exit the webinar. If for some reason you miss that or you lose that link, you can email me and I can send you an alternate link. I encourage you to go ahead and do it. Notice it says, “properly complete.” There is actually a question in there where you have to say, yes, you’d like to be registered for the drawing. The reason that we’re doing that is that we occasionally have customers that work for companies where they’re not allowed to receive gifts over a certain amount. If you’re at a company where you’re not allowed to do that, you may want to forego the contest, but for the rest of you, that’s how you’re going to opt-in.

A couple of details. We’ll perform the drawing tomorrow, so that will give you the rest of today and the evening to get the survey filled out. It shouldn’t take long at all. It’s a very brief survey. We’ll announce it in a couple of ways. We’ll announce it on our Twitter feed. That’s @threewill. We’ll also announce it on our Facebook page, Then just as a follow-up, we’ll also notify the winner via email. Then pretty much like any contest, the decision of ThreeWill concerning the giveaway is final.

With that, I’m at the end of my prepared material. For some of you, you’ve got a busy day ahead of you and you’re going to launch onto other things and that’s great. For those of you that can stick around a second, I’m going to see if I’ve got any questions. You can post those questions right now if you haven’t posted them already. If I have any, I’ll read those out loud.

I didn’t get a question so much as a comment. A couple of you thought that the Office 2013 app integration was pretty cool. I completely agree. I think that there are some enormous opportunities there for you to integrate data and make it very easy for your users to consume and let them be largely naïve about where that data is coming from. They just get what they need and they don’t have to do a lot of clicks.

Again, I’m going to stay on the line here for a couple of more minutes. If you’ve got your questions, please add those to the question manager and I’ll respond to those out loud. Otherwise, I really appreciate your time. I know that any time you invest an hour of your work life, you’re hoping to get something out of it. I hope that you’ve seen some things here that would apply to your organization. If you’ve got any questions or any interest whatsoever in letting us help you out with that, feel free to drop me an email. Again, [email protected] and I’ll be sure and connect you to the right place. Thanks again for attending.

read more
John UnderwoodSharePoint Document Generation Webinar

Enterprise SharePoint Search Webinar

John Underwood is a Technical Evangelist at ThreeWill. He has nearly thirty years of software development experience. He is an experienced technical instructor with superior presentation skills and is proficient in delivering standard curriculum as well as developing and delivering custom curriculum.

Hello folks, John Underwood here welcoming you to another of the ThreeWill Webinar series. Today, we’re going to talk about Enterprise SharePoint Search. I hope you’re having a good day wherever you’re at. I’m coming to you from just North of Atlanta, Georgia in Alpharetta and we’ve actually had a rare sunny day around here. It’s been a pretty good day. A little bit about me. For those of you who’ve not done events with me before, I’m lucky enough to be an employee of ThreeWill. My job title here is technical evangelist. That means I get to do events like this. I also do training and I even get to work on the odd project every now and then. A couple of ways to contact me obviously, you can use my email address on the screen. If you are someone that enjoys being a part of Twitter, there’s my Twitter handle. You’ll get a nice mix of hobbies and information, but mostly, you’ll get some good SharePoint information. I hope you’ll take advantage of that and follow me if you’re inclined to do so.

I know we want to get right into the material, but a couple of quick housekeeping items before we start. First of all, today’s session is being recorded. If you find that you have to drop off in the middle or if you’ve got some co-workers that were unable to attend but maybe want to see the information later, it will be available online for a certain amount of time. Essentially what you do is go back to the same URL that you used to originally visit and register for the event. You can take that invitation that you received and you can forward that to any of your friends or associates and they’ll be able to use that in order to play the video. Again, it’s interesting, even if they didn’t attend, they’ll still be able to register and view the video.

Then, one other thing. Although we hope we won’t have any technical interruptions, it’s not impossible that a phone would get disconnected or the power might go out temporarily or something like that. If any of those kinds of things happen during the event, you can notify me that you’re no longer seeing me. If you just send an email to the address you see there on the screen, that comes directly to my phone and at least, I’ll know something’s going on and we can see if we can’t get service restored and resumed. Again, that’s rare that that happens.

All right, with introductions and housekeeping items out of the way, let’s talk about what I’m going to cover with you today. First of all, I’m going to talk a little bit about search-driven applications. I think you’ll see that if you don’t already know about these, they’re going to represent what is for a lot of companies a missed opportunity. The ability to really exploit the SharePoint search engine in a way that’s going to be beneficial to you. Then, after that, we’re going to talk a a little bit about extending and customizing search.

A few of the things we’ll see here is something where things that maybe a power user could do but a lot of these are going to be more developer or even administrator oriented. Really, what I hope to do with this part of the presentation is twofold. Number one, just letting you know what’s possible and number two, take an opportunity to show you some of the things that ThreeWill has done as far as extending and customizing search on behalf of our customers.

Then, finally, I’m going to wrap up with just a quick look at some of the new things in SharePoint 2013 as relates to search. I know inevitably when there’s a new release, there are a few companies that jump on it immediately, but there are others that take their time. Since I’ve been working in the SharePoint universe, I know that’s particularly true of SharePoint moving up to a new version is not trivial.

Really, more than anything, I just want you to know what’s coming and really plan for some of those opportunities at the risk of sounding like a Microsoft product personnel, will tell you that some of the new search things in SharePoint 2013, I think are really compelling. They alone may not be a reason enough for you to upgrade, but taking in a whole lot, I think it is a worthwhile upgrade that you’ll want to undertake as soon as possible.

Then, time-permitting, we’ll do a little Q & A at the end. If time doesn’t permit or if you don’t think of any questions at that point, of course, I always welcome questions in the future at my email address, so please take advantage of that.

All right, so let’s spend the first part of our time together now talking just a little bit about search-driven applications. The hope here is that if you haven’t already thought about the search engine in this way before, after we’re done here, you’re going to see it as more than just a place where you type in words. Hopefully, you’re going to see it as a repository of some really valuable information for your company.

Let’s start with a common scenario. An application that we might need within our company. Suppose that we work at a company. We use SharePoint pretty heavily. We’ve got a corporate home page that’s hosted on SharePoint and then we have lots of site collections and sites and sub-sites and teen sites and all these different things. We have a legitimate need to present a list of sites that a particular user can see. Again, it could very well be that a user has access to a useful site and they don’t even know it. They don’t even know what’s out there.

One of the ways we might solve this problem is by writing a Web Part. Again, I’m a developer and what do developers do? They develop. They write code. Almost inevitably, if someone brings a business problem to us and says, “How are you going to solve this?” The first little thought balloon that pops up over my head is, “I’m going to write some code.” In this case, we’ve got a developer that creates a Web Part and that Web Part iterates through sites and makes a list of available sites based on the current user.

Now, we all know and love this phrase, “It passes the developers works on my machine test.” What we see here is that, a user or excuse me, a developer that’s written this may be testing it in an environment where they have a dozen sites. Then from there, we deploy it to our company’s home page where we have thousands of sites. All of a sudden, somebody goes to the home page and it’s taking forever to load because this little application is now going to try to iterate through those thousands of sites for every single page visit to the home page.

There’s really a couple of things wrong with this. Thing number one, obviously, is that, this is going to be a huge performance hit and people are going to hate SharePoint when it’s really not even SharePoint’s fault. The other thing here that’s a little more subtle but perhaps even more important and that is, there was a huge missed opportunity here. Because of the preconceived notion of how this developer viewed this problem, they didn’t see that there’s actually an efficient, simple and easy way to solve it. Easy enough in fact that in some cases, we could even rely upon a power user to do that.

What I’m talking about there is really a search-driven application. What is a search-driven application? Well, in simple terms, instead of us trying to use the lists and libraries in our SharePoint world as a source of data, we’re actually using the search engine or more correctly, the search engine’s index as our source of data. Because after all, that index contains a lot of really useful things.

Now, when we talk about a search-driven application, we’re talking about a scenario where we’re not going to go ask the users to type in some data and search for things. Rather, we’re going to search on their behalf. Really, they may not even know that it’s coming from search, and they may not even see the query. We may do it in a very subtle way so that that information is hidden from them. All they know is that they are able to undertake some actions such as clicking a link or visiting and web page and they get whatever it is that they needed.

Another nice thing about search-driven applications is that, the results are security-trimmed, by the very nature when you go into SharePoint and you search for things, you only get results for the things that you are authorized to see. In the case of as an example trying to show a user all of the sites that they have access to, by definition the search engine’s not going to let us serve any site references up to users unless they’re authorized to visit that. That just means there’s a whole level of coding that we often have to undertake that we’re just going to be able to ignore here because the search engine and its index already knows the answer to that question.

Then, another thing about a search-driven application, and while this isn’t a requirement, I think it makes it more compelling and it’s really nice. That is, results are presented to the user in a way that are ready to work with. To say that a little bit differently, certainly we could take users to a search page and we could teach them how to look for the thing that they want to look for. With the little effort on our part, we can customize the output from that process in such a way that they don’t get a lot of the other decoration that we would normally see on a search results page. Instead, they’re just going to get the data in question and the importance of that data to them.

When I do some demonstrations on this in a few minutes, I’ll reinforce what I’m talking about when I say presenting the results in a way that are ready to work with. If you’re a developer and you’re on this webinar today, here is my encouragement to you. When you start thinking about the sources of data that are available to you when you are customizing SharePoint, don’t forget the search index. Even furthermore, almost any time you’re writing some kind of customization that’s asking a question, the odds are pretty good that the search engine would know the answers to those questions.

Let’s dig a little deeper then. If it’s all about asking the right question, then how can we ask the right question? At least traditionally, when we think of a search engine, we think about a keyword-driven search. As an example, excuse me, a user might type in the word invoice hoping to find things that are related to invoices or resumes, or a series of what we would call keywords. However, the thing that a lot of people don’t know and that is really useful for the search-driven application, and that is SharePoint has a capability that allows us to specify the kind of thing that we’re looking for. Not only might we say, “I want invoice,” but we might say, “I want invoice and I only want that word where it appears in a list or where it appears in a library.” We can refine the kind of thing that we want to get back from the search engine.

If you think a moment ago about my example of trying to give a list of sites that a user has access to, we could specifically say to the SharePoint search engine, “Hey, I only want you to send me back things that represent sites. Anything else, just leave out of the results.” Then, another capability that we have in the search engine that not everyone knows about, we can also qualify details about the item that we were looking for. Not only could I say, “I only want items that appear in a task list.” I could go further and say, “I only want the ones that are authored by Kim or the ones that are assigned to Kim.” This notion that the entities that we’re looking for out there, each have some properties associated with them, and the query language actually gives us some mechanism for specifying this property.

Then, let’s be a little more specific. How is it that we’re going to qualify the type of results? Well, one very easy one and again I’ll be demonstrating this in just a moment, is the use of IsDocument. That’s pretty straightforward. It just lets me say, “I either want to explicitly return results that are documents, or are not documents.” By doing that, I may be able to go from a search that gave me a whole lot of results, to a search that gave me a set of results that are a little closer to what I’m looking for.

Now, the next one, and in my opinion, simultaneously both the most important one and the one that’s probably the most underused is the concept of a content class. You may not be aware of the fact that when you issue a keyword search in SharePoint, you’re actually able to specify the content class that you wish to receive back. Even if you’re not a real SharePoint expert, you can figure it out based on what you see here. As an example, I could say, “I want to look for the word invoice but I only want to see back a list item.” Or, “I only want to see back something from a document library.” Then, we can even go a step further. Not only could we say, “I only want to see items from a list,” but we can even provide a type that says, “I only want to see items from a list that’s a task list or a contacts list, or even a calendar.”

When you begin to think about this ability to not only ask for what you want but to specify the kinds of things that you want, that’s really a cornerstone of building these kinds of applications. Then, I handed that just a moment ago, this is just a more formal look at that notion of a property restriction. In effect, we’re saying, once we found the things we think you’re looking for, we can even be more specific about, it should have a certain file extension. It should have a certain file name. It should have a certain author and so on and so on. Taking what we already know about keywords and laying on these layers of qualification, that’s how we’re going to be able to not only write better queries to start with, but then take these queries and package them up in a way where users can conveniently consume them.

Let me take a moment now and do a quick demonstration for you. I’ll show you a little bit about some of these restrictions. Now, just a little FYI here. I’m going to be working in a virtual machine environment and I’ll just confess to you right upfront. It’s running on my local machine and it’s not got the most horsepower in the world behind it. Don’t try to draw too many conclusions here about the performance of this application. Really, let’s just focus more on what it does. Now, those of you that have worked a lot with SharePoint, probably recognize the fact that I’ve got a SharePoint 2010 Team Site that I’m working with here. You can see I’ve got a top-level site named Acme Corporate. I’ve got a couple of sub-sites underneath. Then, I’ve got a search center setup as well.

Let me see if I can demonstrate now some of the techniques that I’ve talked about in terms of refining your search and then from there, we’ll put that all together into a simple little application. Let’s suppose that I’m looking for a resume and I know that it was authored by Kim. I can start with a real simple search just by putting in the word resume and then searching for that. Then, we can see that, I get back results here that are all over the board. I’ve got different things that are actually resumes. Not only that, I’ve even gotten pages that just happened to have the words resume on them.

Now, if I go a step further, and I can say, “Well, let’s refine it so that it looks for both of those words.” Certainly, that gets us a little closer but you can see that I’m also winding up with a little more debris down here that’s not really relevant to what I’m looking for. Again, in this somewhat contrived example, I’ve gotten it down to three items. I could probably live with that, but those of you that work in very large corporate environments, the little refinement we just did there maybe took you from 10 thousand results to three thousand results. That’s a huge improvement, but you’ve still got a big hole to dig out of.

Let’s refine it just a little bit more. We want to look for the keyword resume, but we only want to get back items where the author of that document is Kim. You can see now that we’ve got a much better result at this point. We’ve got the actual resume and we can see clearly in our results here that it was authored by Kim. Just in taking that much of an approach on looking for things, you can see that we’ve really been able to take a machete and cut away some of that underbrush that we frequently get when we’re looking for search results.

Let’s take this a step further now and look at how else we might use this. Let’s say that I’m in search of something related to tasks. Now, if I just type in the word tasks, I’m going to get a lot of different things. Depending on what I was looking for, it could just be that I’m looking for something related to documents that have the word tasks in it. Or, it could be a classic problem that we have in SharePoint where I’ve got 15 different sites and each one of them has a task list and I’m just trying to figure out which lists have an assignment for me. Let me see if I can tackle both of those for you.

For the first one, let’s suppose that I truly do want to look for documents that contain the word tasks. Notice what I can do. I can put in this document and in this case, equals one means that I want it to be a document, equals zero means I want it to be a non-document. Just by adding the equals one, notice now I went from dozens of results, several pages worth down to just three. One little qualification on this one to be aware of as well and that is, notice the definition of a document. Not only do I get a Word document which is what a lot of us think of when we think of documents, we get library pages here. In the SharePoint sense of the word, they are in a library called site pages and they are in the documents.

Then, let’s take it one step further. Let’s say that we are truly interested just in Word documents. We could just go ahead and qualify the file extension on this just a little bit, and that would help us reign this in and see just the things we’re looking for. Right here I’ll just say, file type: docx. Then I’ll go ahead and search again. You can see now I’ve been able to refine it down to just the exact thing that I’m looking for.

Let me go one more step. Now I promised you that we would not only be able to find documents that have the word tasks in it, but that we’d also be able to find our task. In fact, I would tell you that of all the demos I think I’m going to do for you today, this is the one that is the most relevant and the most useful. That is, being able to find all of the tasks that are assigned to you across numerous sites. Let me start my query again. What I’m going to do now is, I’m going to change this one completely. I’m going to say content type:, excuse me, content class, and then STS_listitem_tasks.

Now, even if you are decidedly non-technical, this is a simple enough syntax that you can master it. Any time we see content class presented this way and then followed by some kind of qualifier, really what we’re saying is, “SharePoint, I only want you to bring back things of this kind.” In this case, we’re saying, “We only want you to bring list items that are actually of a task type.” Having typed that in, I’ll go ahead and search for that now. Notice, all I get are things from task lists. It’s pretty obvious that that’s the case.

Now, we could go a step further here and say, “I really want to find the ones that are assigned to me.” There’s sort of a good news, bad news going on here. The good news is, we can qualify who it’s assigned to. The bad news is that, there’s not an automatic way for us to say, “Only show me the ones that are assigned to the current user.” Those of you who’ve built these before and you know how you can filter a view where it says, “Just show items for me.” Unfortunately in the query language, we don’t have anything exactly like that. You are going to have to prompt your users for some kind of information. Or, it is possible with just a little bit of code, we could make this a fully automated process.

For this example, let’s just pretend that we are in the role of Kim. We’ll just come out here and say, “Assigned to” and “Kim.” What I’ve done now is make it possible for Kim to go to one place and see all of the tasks that are assigned to her. Again, she’s obviously not going to see any places that she’s not authorized to see. The thing here is, she’s now not going to have to go visit all these different sites to figure out if she’s got an assignment or not. She’ll just simply execute this query and she’s off and running. Now, how can we package this in such a way that it’s a bit more convenient to use? Well, let’s talk a little bit about that and then we’ll go a bit deeper and see how to get things done with that.

I’m going to switch back to Power Point here. The next part of this is called Accessing the Application. That is to say, once we figured out a way to ask for the results that we want, then we need a way to present it up to users so that they can conveniently consume it. Now, the easiest way to solve that problem is to recognize the fact that these search results can be expressed as a hyperlink. You might have noticed this when I was … demo, or you may have missed it, but if you take a look here, you can see. When I typed in all of this stuff in the search box, all of it’s represented up here on this URL. It is quite literally as simple as taking that URL and then making it available to a user.

How am I to do that? Well, I can put it on the Quick Launch menu. I could create a page and author a hyperlink there. Or, there’s even a hyperlinks list that’s available to me in SharePoint. I could put the information there. Pretty much anywhere we can store a clickable link, we’re now giving our users the ability to take advantage of this. Now, something that is a bit more sophisticated, but also brings us a bit more power and that is the Core Search Results Web Part. In particular, as developers or really high-end power users, we have the ability to customize the XSLT that is associated with that.

Now, if you have no idea what XSLT is, let me just take a quick aside here. Extensible Stylesheet Language Transformation is what it stands for. In very short terms, it’s just a template that describes how the results are going to look. As an end-user, you probably aren’t ready to engage trying to change that, but for a technical resource, it’s a pretty simple thing to do. Now, one other word about customizing the Search Results Web Part. The good news is, you can do it. You can absolutely do it. The challenge here is that, it can be difficult and if you’re doing this in a search center, you have to be aware of the fact that if someone else tries to modify that same set of XSLT, their changes might collide with your changes.

I want to drop that in your mind as a little bookmark of a problem, because when we start SharePoint 2013 conversation, we’re going to see that that problem gets solved. What I think is a better approach on this and it’s the one I’m going to demonstrate for you in just a moment is, when I’m building a search-driven app, instead of modifying the Web Part that’s in my search center on the results page, what I probably do and just did is just go to my site pages library. Create a dedicated page and put a Core Search Results Web Part on that. From there, I can configure the part so that it issues whatever query it needs to and it has whatever customization it needs. I am going to demonstrate that for you and let you see that in action. Let’s do that right now.

I’ll go back over and just continue working in the same demo environment that I’m in. Let’s do something a little more generic here. Excuse me. Let’s see if we can do a query that will show us all of the sites that someone has access to. Once again, I’m going to use the content class and I’m going to look for two content classes here. The first one I’m going to get is STS WEB and then I’m going to do an OR, and the OR does have to be capitalized. Then I’m going to do STS SITE.

Now, for those that are less technical in nature here, let me just give you a simple answer. If you want to get all the URLs, all of the sites, this is the way that you have to do it. For those of you that are a little more technically savvy when it comes to SharePoint, remember, if I did STS WEB, that’s only going to give me the sites. That’s not going to give me the top-level site for the site collection. For that, I have to use STS SITE. If I do those two together, you can see now I actually have a list of the sites that are available to me.

Now, in terms of making this a “application”, watch what I’m going to do. I’m going to copy this URL and then I’m going to navigate back over to the home page as my Acme Corporate Team Site. Then, once I’ve done that, I’m going to go in and customize this page. Once I’ve opened up page editor, I’m going to go in and insert a Web Part. What I’m going to insert is the links Web Part. I’ll put that right down here. Then, I’ll go ahead and save my work. Then once I’ve done that, I can go over to all site content and you can see that I have a links list here. I’ve got this one left over from a previous demo. I’ll just delete it out real quickly and then, add a new link here. I’ll put in the URL of the query that I just wrote and then I’ll give a description of all the sites you can access and click save.

Then once I’ve done that, now I can go back to the home page of this site and at any point, when a user comes to the home page of this site, they’d be able to click on this link and go directly to a search results page and see just the sites that they can access. To really drive home that this indeed doing what I’m claiming it’s doing, notice that I could take a second here and sign in as a different user. In this case, a user named Ed. Notice that if Ed clicks on this link, he’s only going to get two results whereas before I was logged in as an administrator and I got four results.

No worries here about writing any logic whatsoever, and we’ve solved that problem that we started talking about at the very beginning of the webinar. Now, to go a step further here, let’s see if I can log back in as administrator now and show you a little bit about how to make this a bit more beautiful. Okay, so I’ve logged back in as administrator. Now, let’s remember a point that I brought up early on in the process. I said that when we do these search-driven results, or these search-driven applications, it’s really nice to put the results in a format where they’re ready to use.

Saying that another way, if all I’m interested in are sites, then I don’t need all of this garbage. I just need a title of the site and something I can click on, and I don’t need the refiners and I don’t need all these other stuff. What I can do here is very simply create a page that contains a Core Search Results Part, and then configure that Part to issue this query. I’ve actually already pre-baked an example of that. Let me take you over and show you what that looks like.

In my site pages library, I actually have a page here called Available Sites. You can see that if I navigate to that page, it has a very simple list of the sites that are available to me. Then of course, I could click on any one of those and navigate directly to that site. Now, how is it that I’ve done this? Let me show you real quickly. I’m going to go ahead and put this page in edit mode. It’s being a little bit stubborn there. It doesn’t want to go in edit mode. Sometimes I have to switch back and forth between Internet Explorer and Chrome to get what I’m looking for. That’s fine. There we go. That’s behaving a little bit better.

Now, you’ll notice, the very first thing that I’ve got here is a Search Core Results Web Part. If I go in and actually edit this Web Part, I can show you the two things that I changed in order to make this thing work the way that I wanted it to work. Let’s drill down through some of these and have a look see here. Now, you can see there’s a section that describes my display properties. Then, you can see there’s a section where I have something to say about the results query. Let’s start with that one first.

Now, I know this text box is not big enough for you to see all of the contents, but you ought to be able to figure out pretty quickly what I’ve done there. In fact, if I select all of that and copy it, and then put it into Notepad so you can see it, that’s the exact same query that I issued before. Really, what I’m telling this Part is, the user is not going to type in a query. Instead, we’re just going to give them a fixed query, and then we’ll allow security trimming to make sure that everybody sees what they’re supposed to see.

Now, if I did only that, I would actually wind up with the same appearance and search results that I had before. Now, I want to go one step further and make it look nicer. I did that by adding my own custom XSL. Now, the point of this is not to teach you how to write this XSL. Some of you under the sound of my voice, probably would never do that, but the point here is just to let you know that it’s this simple. You can see in this care, really all I’ve done is just say, “Take all of the results. Turn each one of them into a hyperlink. Substitute the URL and then put a title on it.” That’s how I wind up with these very simple search results. Then, if I stop editing and go back into view mode, you can see that ultimately the page is just going to give me a very simple appearance and let me navigate to any one of those sites.

Then, you might have noticed when I was in the site page library, I also did a similar page here with tasks. Again, the only challenge on this one is, I’m going to have to do some kind of exercise to make sure that it works properly for different users. That’s probably going to involve a little bit of JavaScript. If you’re any kind of programmer at all, it’s not that challenging and it lets you do some very clever things. If you’re not already familiar with building those kinds of simple applications and letting the search engine do a lot of work for you, I hope that will inspire you to go forth and embrace some of these search-driven applications.

Now, as clever and cool as some of that stuff is, there comes a point where you say, “We want to do something with search that’s just a little more involved than a little bit of a power user trick.” That’s what this next part of the webinar is about. Now, one of the big ways that we can go a little above and beyond with search is integrating it with other sources. We have a couple of ways of doing that. One way to do that is to actually configure SharePoint so that it crawls external content. A classic example of this, maybe we have a line of business application where we keep data in a SQL server database. Among the things we keep in there, we keep our Part catalog information in there.

It’s actually possible to set-up SharePoint to use business connectivity services to connect into that database, to crawl the database and then to index a series of results that we can then search against. There’s a lot of good things that happen with that approach. We’ll talk about specific pros and cons on that in just a moment, but the idea here is that SharePoint is the thing that indexes and crawls the content and then any integration happens at that moment. At query time, in terms of getting the results back, it’s no different than any other SharePoint search.

Now, an alternative to that is something called Federation or federating the external content. In this example, the SharePoint search engine is not the one that’s doing the crawling and the indexing. That’s actually maintained by some external piece of software. Probably native to whatever repository we’re using. Then, instead of the integration happening at crawl time, it actually happens at query time. That is to say, when we do this kind of integration, even though they don’t realize it, the user is actually issuing two queries. They’re issuing a query in the SharePoint search engine and then on their behalf, it is issuing a query to this external engine.

What are the pros and cons of each approach? Well, first of all, the first one crawling external content. There’s several compelling reasons to do this. One, well, because it’s pre-indexed in the same way that other SharePoint data is, we just get better ranking of the content. In effect, it’s going to be subject to the same rules and the same ranking calculations that any other SharePoint data would be subject to. Another nice thing here, we have control on the frequency of crawl. As an example, let’s say that we are crawling some external data and we only need that data to be up to date within the 24-hour window. Well, we can save a lot of CPU cycles by crawling that information once a day as opposed to having some other mechanism crawl it on a more frequent basis.

This third one is particularly compelling. You’ll remember a moment ago, when I was able to do property specifiers in order to refine my search results, well if we’re crawling external data and we’re allowing SharePoint to do the crawling, we can actually specify those mappings. We can take bits of metadata from this remote source of data and bubble those up as properties that actually can be filtered in our queries. Then, in terms of maintaining the index, it’s just a single index for all of our data, and so our backup and restore operation on that is fairly simple.

The downside of crawling external content from SharePoint, well probably the biggest one would be the fact that we’ve got additional capacity requirements. Now SharePoint is going to have to have enough disk space to not only index its own content, but this external content as well. Not only is that a disk space equation, but also a CPU cycle equation. Beyond that, there is also this challenge that in some cases, we may be talking about external data that for security reasons, simply can’t be crawled. They may have blocked any external access in terms of being able to crawl through and read the data. That would drive us toward an approach where we would do federation instead.

The pros and cons here are mirror images of each other. In the case of federation, if we’re talking about a repository that already has a search engine, that search engine is probably already optimized for the kind of content that we’re searching. Rather than try to re-index it, we’ll just make a call out to the search engine at the query time and say, “Hey, tell us what you know about this.” This is particularly useful in cases where we simply cannot index the information externally because of security.

Then again, the cons here, well one of the biggest cons is that, we’re not going to be able to necessarily interleave the results on a single results page the way we would if we had SharePoint indexed it. More likely, you’re going to have a section of your search page that’ll show SharePoint results and then a section of your search page that will show external results. Not only that, when it comes to this kind of things, security trimming can be really tricky because at this point, you’re making that determination at run time.

Now, the good news, it can be done and in fact I can tell you that here at ThreeWill, we have written integrations on behalf of vendors where they took their products, integrated them on SharePoint and then when we did this run time federation or query time federation, we were actually able to make sure that the end-user still only saw the results that they were looking for. That is something that’s kind of a dark art. It’s not all that well-documented, but we’ve got a ton of experience with that and we’ve been able to help our customers get what they want there.

Now, just as before when I talked about building search-driven applications, not only can we make some trivial customizations to the Core Results Web Part, such as you saw a moment ago, but we can also do some pretty drastic customizations. This is another thing that we’ve actually done on behalf of one of our customers. I’m going to show you a pretty compelling demo of that in just a moment. Beyond that, if you’re familiar with search centers, we also have the ability to customize search center tabs that’ll let us do things like look for results in a narrowed results set or people searches or things such as that.

Then ultimately, if we just can’t get what we need out of the search infrastructure as it is, there’s always the option of writing either server code or JavaScript that makes a direct call into the search API, brings back the results we’re looking for and then we can specifically show that to our users. I can tell you again that, a particular ThreeWill use of this, we’ve actually written some application code for a customer where we did a typeahead search. Where as the user was typing in the characters of their search terms, we would go out and do a prefetch of matching items. That kind of thing is virtually difficult to do, or virtually impossible to do I should say in a search center, but can be done with custom pages and code.

Another thing that we’ve done on behalf of our customers, creating custom refiners. You’re probably already familiar with the concept that the SharePoint search engine gives you the ability to refine on different characteristics such as the date that the document was created or the format of the document. For certain custom data types, we can actually go in and create refiners from scratch.

Let me just show a little bit about how one might use this. What I’m showing you here is an example of an integration that we actually wrote on behalf of Polycom. You may or may not be aware of the fact that Polycom has a fairly sophisticated portal for keeping up with multimedia content. It’s called the Polycom Real Presence Media Manager. Particularly companies that do a lot of training videos or educational videos, video content of various types. This is a very popular piece of software that companies can use to manage that information.

Now, the product has a portal of its own for accessing that content and it’s really useful and it’s really nice. What Polycom has discovered is that, for a lot of companies, the destination of choice is going to be SharePoint. If we lock up all of our video content over in this vault where people have to visit another site, they’re not going to see it quite as often. Now, you can see on the screen here that we’ve actually created a Web Part that will let us go in and discover that content. I can look at things like recently viewed and so on.

Obviously, the demo that we’re talking about here and we’re more interested in here is this notion of search. Not only have we built a facility for browsing that content, we’ve actually integrated it in search. In this example, what we’ve done is, we’ve changed the XSLT associated with the Core Search Results Web Part so that we can not only interleave the data that’s coming back from Polycom, but also show it in an enhanced way. Let me show you what I mean by that. I happen to know as an example, that there are several videos and even some other information about Stroke. Again, obviously, the sample app here, being a healthcare app.

Now, notice you see some of the kinds of things you would expect. You see a title and you see some kind of icon, but you’ll notice that the information down here looks a little bit atypical. Not only do we have some author information and a date, we even have the duration of the clip. Then, this little link of Video Program Details. This is all custom code that we wrote on behalf of the customer. You can see the compelling story here is that, with one click, we’re able to preview that video in line and then if we so choose, we can even begin viewing it right there on the spot.

I think this is a pretty high-end example of how you can take the out-of-the-box SharePoint search bits, and then do some pretty significant customizations along with that. You’ll also note that not only did we do customizations related to the viewing of the content, but also refiners. Even though this is video content that’s held on another site and there’s really nothing SharePoint-y about it, we were able to supplement this information in such a way that we can refine it based on where it came from, who the author was and other such things. Again, the goal here isn’t so much to teach you how to do that. Obviously, that’s a pretty deep process, but rather just to alert you to the idea that it’s possible and it’s something that we have the experience and the expertise to do.

All right, we’re getting pretty late into our hour here. I’ve got one more section of content that I want to talk you through here, and that is just giving you a preview of coming attractions when it comes to Search in SharePoint 2013. I know that’s probably an over-the-horizon thing for some of you, but it is nice to know what’s coming and what’s to be expected.

Probably the biggest difference when it comes to the architecture of Search in SharePoint 2013, and that is this notion that we now have a single search engine. For those of you with a lot of deep knowledge on this, you know in the past, you’ve got SharePoint Search and FAST, and different versions of those. Now it’s just a single unified product. Really, I think that’s a good thing for the most part in that, it reduces some confusion. When you’re authoring queries now, you don’t necessarily have to be cognizant of, “Well, gee, which version of the search engine am I on?”, in order to be able to do this thing that I want to do.

Now, in terms of the languages that you use or the query languages that you use, keyword query language, which is what I’ve been demonstration for you today is obviously still supported. In addition, in prior versions, FAST had its own unique query language. Even though FAST doesn’t exist as a separate engine anymore, the FAST query language is still supported. The one that isn’t and it isn’t still supported is the SQL variant of being able to do search queries with a SQL LIKE Syntax. I think at this point, that’s one that didn’t get used a whole lot and that’s why Microsoft made the choice to drop that. If you’re under the sound of my voice and you’ve got some things that you’re doing in SharePoint that are dependent upon that dialect, then just be aware of that. That’s something that’s going to go away when you get to SharePoint 2013.

Some other things and some of these are technically meaty. Others of them are more marketing points, but they’re worth making. There’s some improvements to the results ranking model. Another one that I think is really cool, and that is the results are further refined based on click-throughs when the results came up before. Those results that have a higher rate of click-through, will show up as being more important on subsequent queries.

Another thing that’s new, a moment ago, I’ve talked to you a little bit about federating outside sources of data. That’s gotten a new name now. It’s called Result Sources. It’s really a way of just describing external data. Probably the biggest thing here that would be of use to developers is the fact that the way that we do security trimming here is a little more elegant and a little more sophisticated. In terms of being able to write customizations, that’s a good thing.

Now some other things, some refinements and changes to query rules. We have this thing called Promoted Results. This was formerly known as Best Bets. We’ve got a little more flexibility on how that we use that. We have this other thing called Result Blocks, and I’m actually going to talk about this one a little bit more on a later slide. The idea with result blocks is that you can use it to group related results. As an example, let’s suppose that somebody searched for a particular kind of content and maybe we discovered that based on certain words. Probably they’re looking to highlight some things related to Power Point presentations. Well, we could actually set up a separate result block that says, in addition to all the results we’d show them, we could actually use a result block to call out just the Power Point presentations that are in those search results.

Again, if you think about where you visited commercial search engines like Bing or Google, and they often have a way of saying, “Well, gee, you searched for this thing, but there’s a whole bunch of images that match. Would you like to see those?” That’s exactly what result blocks are all about. Another thing that’s kind of cool is that, you can actually boost rank results based on certain terms. As an example, suppose that someone goes in and types in a term to download a utility, we can actually configure a query rule that says download is recognized as an action term and based on that, if we have a particular site in the company where people are supposed to be getting their downloads from, we can actually boost that result and that would result in a likelihood that it would be higher in the result stack.

Another nice innovation here, Continuous Crawl. Again, those of you that have worked with SharePoint a long time, you know that there’s always this challenge of, you want to set your crawls frequently enough to be up to date, but not so frequently that they killed the server. Now we have this concept that, for particular sources of data where it absolutely has to be up to date, continuous crawl really is just tapping into the transaction engine from the underlying database and saying, “Anytime we have new content, we’re just going to go ahead and update the sear- index immediately.”

Now, let’s talk for a moment about a hybrid environment. This is something that I think a lot of you, particularly if you work for big companies, you’re going to encounter over the next year or two. It’s interesting not only to identify it, but to talk about some of the search implications of that. When I say hybrid environment, what do I mean? Well, we all know that Microsoft is making a big push to move customers to the cloud, to begin using SharePoint Online as a part of the Office 365 Suite.

Now, for most companies that have a significant amount of content, it’s probably not going to be possible to move exclusively to the cloud in a short-term. There’s really two big reasons for that. Number one, they’ve probably got customizations that wouldn’t run in the cloud particularly if they’re farm solutions. Number two, it’s probably not practical to take the gigabytes of data that they already have in place and simply move those up to the cloud in order to sit there. It would be very expensive to move them up there and it would be an expense to have it up there in an ongoing basis.

In order to combat those barriers, a lot of customers are using what’s called a hybrid environment. In simple terms, they’re going to take everything that’s on-premise and for the most part leave it there. Then, new content is going to be put into the cloud and used there. Now, the presumption is, if we are going to have such an environment, we really need to be able to integrate the search results from those two environments. To say that a little bit differently, we don’t want users to have to constantly say, “Well now, wait a minute. Is this thing in the cloud or is this thing on-premise?”

The idea here is that, we can use the result sources feature that I mentioned just a moment ago in order to integrate those results. How are we going to use that? Well, suppose that we’ve got an on-premise search page. What we can do is create a result sources feature and then a result block that highlights the matching cloud results. They may not be interleaved one for one, but what we’ll see is a section of highlighted results. Then, if someone observes, “Well, gee, the results that I think are relevant to me, there’s more of them on the cloud than there are on my on-premise.” They can actually click on one or more items inside of that result block and it’ll navigate them to the cloud. You can also do the same thing in reverse. You could set up a cloud search page so that it not only shows cloud results, but also has a results block that highlights on-premise.

I can tell you that we are actually doing a proof of concept of this as I speak for a company here in Atlanta. Fairly large company that you would recognize and we’re getting some really good results with that. Really kind of two good news fits there. Number one, it’s doable and it works. Number two, when you find yourself at a point where your company needs to do that, we’ve got the expertise to help you get that done.

Some other things you’ll want to watch for in SharePoint 2013. Some of these are just features that we’re going to like and take advantage of. Some of them are things where we actually have opportunities to customize. We now have this concept of a hover panel for particular content types that show up in search. You can actually hover over those and preview the results on a mouse over. We also have the ability to create our own hover panels. If I were to go back to what I showed you a moment ago on the Polycom integration, once that products gets updated to SharePoint 2013, it would be a doable thing for us to go in and write a hover panel where instead of having to click to view the video, one could simply float the mouse over and see a preview of the video playing. I think that’s a pretty cool one.

Another nice thing of SharePoint 2013 Search, if you look at the search pages I’ve been showing you up to this point today, the icons and the graphics associated with the results are pretty bland. They don’t really communicate a lot of information because they’re tiny and somewhat nondescript. Microsoft has tried really hard to put some good visual cues on your search results so that it’s real obvious, “Oh, that one’s a Word doc, and that one’s a diagram and that one’s from Visio and another one is an image.” I think that is a much needed improvement.

Some other things. I know that not all of you take advantage of the term store and the managed metadata capabilities of SharePoint 2010, but if you do, there are some greatly improved capabilities there in terms of being able to navigate and find items. Another improvement that we have, this concept of display templates. Now, you’ll remember that I’ve talked twice now about changing the XSLT in the Core Results Search Web Part either on a particular page or in a search center. Given the difficulties associated with that, it’s nice now that we have an alternative to that.

Really, what are we saying when we say display templates? Now we’re able to build what are in effect, customizable little nuggets of HTML. We’re going to be able to associate these in such a way that when a particular result set comes up, we can actually modify how it’s going to look on the results page. The nice thing about this is that, because it is associated with particular types of content, we can do it in a search center and do it in such a way that we won’t collide with other customizations. It’s more flexible, it’s more powerful, it’s less error-prone, it’s easier to do. That’s nothing but a win when it comes to doing this.

Now, the various parts that make up this, there’s really two things. First of all, you have a controlled template that’s going to describe the header and the footer and the layout of the results list. Then, an item template that we’ll use to describe the presentation of the items in that list. Those of you that do a little bit better with pictures instead of words here, this gives you an architectural representation of what we see. Then, this is a nice screenshot that shows you how you could do a search results page with an image slider. You’re able to just simply scroll back and forth through the images used and a little bit of jQuery in addition to the HTML template to show this.

If you are in a place where you do customization of search, your world just got a lot bigger. Conversely, if you’re a consumer and you need these kind of capabilities, it’s nice to know that it’s finally available in the product, and that you’ve got some folks in ThreeWill that know how to make that happen for you. Let’s talk about some takeaways real quickly, we’re getting close to the top of the hour here.

First and foremost, if you’re a developer and you’re under the sound of my voice, you need to start thinking about search-driven applications. If somebody describes a business problem to you and the answering of that business problem can be done from the search engine, it’s faster, it’s more efficient, it’s easy to write, there’s a lot of wins there. Obviously, if you’re doing a traditional data-entry app where you need to add items to a list, that’s not necessarily a search-driven application. Anything that involves asking questions. This is a good candidate.

Another thing to think about, beyond just the building of these simple search applications, if your company is in a place to make a higher technical commitment, there are a lot of other things that we can do. Customizing the search results page, writing refiners, integrating external data. Another thing to look forward to, SharePoint 2013 really offers us some much better opportunities to integrate at a deeper level. Better security trimming, better models for extending the UI, and doing those things in such a way where it’s not going to be brittle or prone to breakage if somebody else tries to do a customization as well.

Then, obviously, what I want you to take away from that is, this is what we do. It’s not that we’re going to do these things. We have already done these things on behalf of our customers, and we’ve had some really good results with that, and we want to help you get those same results as well.

read more
John UnderwoodEnterprise SharePoint Search Webinar

August 22nd Free Webinar – Enterprise SharePoint Search

John Underwood is a Technical Evangelist at ThreeWill. He has nearly thirty years of software development experience. He is an experienced technical instructor with superior presentation skills and is proficient in delivering standard curriculum as well as developing and delivering custom curriculum.


SharePoint at it’s best can be a useful repository of company information, but in some cases it can be difficult to find the right data at the right time. This presentation looks and SharePoint Search capabilities for SharePoint 2010 and SharePoint 2013, and provides insights that users and technical resources alike can employ to make search a valuable tool.


SharePoint Users/Power Users, IT Personnel


  1. Asking the question correctly
    • Basic search terms
    • Using refiners
    • Qualifying search queries
  2. Search-driven applications
    • What is a search-driven application?
    • Search-driven apps for users
    • Taking it to the next level
  3. Customizing the results UI
    • Search results web parts
    • Using Display Templates in SharePoint 2013
  4. Extending and enhancing search
    • Integrating with external data sources
    • Custom security trimming
Register Now
read more
John UnderwoodAugust 22nd Free Webinar – Enterprise SharePoint Search

Kirk’s Takeaways from #spc12

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

So much SharePoint, so little time…

I didn’t get to see all of the sessions at the SharePoint 2012 conference (spc12) that I was interested in.  I was quadruple booked for every one, so there are plenty more (recorded) sessions that I want to watch, not to mention digging in and getting dirty.

Regardless, here are my initial impressions.


I absolutely love some new usability features that I have seen in SP2013:

  • Minimal Download Strategy – by default when you navigate from page to page within a SharePoint site, very small changes are downloaded instead of the entire page.  For example, the quick nav and global nav don’t change and are not part of the HTTP response.  The result is a much faster and smoother experience for the user.  I think this is really going to help people “like” (dare I say “love”) navigating around in SharePoint.
  • Focus on Content – a simple icon is available which removes the navigation elements (quick nav and global nav) and just shows the body of the page.  This should be very handy for when you are viewing lists with a lot of columns and rows.
  • Drag and Drop – yes, you can drag and drop documents into a document library.


Yes, they are for real.  It’s not clear to me how quickly and how much they will take off, but I think you should consider creating an app anytime you are doing SharePoint development. The good news is that these are not taking away any options for developers.  We can still do Farm Solutions.  Sandbox solutions are even an option, but not recommended (oopsies).  Some great aspects of apps are:

  • Apps provide a common and familiar development model for both SharePoint apps and Office apps (javascript).  Think of it…  Several of us have done VSTO applications before, but not regularly.  Now we can take what we know about JavaScript and apply it to Office.  I think this really puts Office applications back in the mainstream at ThreeWill – and for a much larger user base than just SharePoint.
  • There is a public store hosted by Microsoft but you can also have an on-prem catalog for apps specific to a company.
  • Apps use underlying API and infrastructure to allow for server-to-server communication within apps as well as client side cross-site messaging (e.g., resize the IFrame).


OK, this doesn’t excite users, but it should excite developers; esp. developers that have connected SharePoint with other systems.  ThreeWill does a lot of this and this should make our life easier if the other side handles OAuth as well.


Now SharePoint workflows are built on top of Windows Workflow Foundation 4 and are hosted outside of SharePoint.  I think the biggest deal here, though, is that SharePoint Designer workflows have a lot more power.  They can have much more sophisticated capabilities with the ability to loop and jump to different areas of the workflow.  I see this being very useful at ThreeWill.


I hadn’t heard any buzz on search going in, but there were around 26 sessions on search and there are plenty of things to be interested in.  The big news is that FAST is now part of core search.  This will have some pain because certain things are deprecated, but I think the positives will well outweigh the negatives.  A few things to note include:

  • Query Rules – these give you a lot of power on how search results are presented.  I think this will open up the possibilities for search-based applications.
  • Display Templates – this should make it a lot easier and less intrusive to provide a different display for custom content returned from search results.  These are very powerful for displaying list content outside of search as well.
  • Security Trimming (something near and dear to my heart) – you can now have a pre-trimmer in addition to a post-trimmer.  The post-trimmer is similar to ISecurityTrimmer2 in SharePoint 2010.  The pre-trimmer will let you provide claims information and map it to claims that can now come in from the BCS crawl.  A much improved and faster approach than the post-trimmer (if feasible for your situation).

Other Notes

  • Access – its baaaack!  … or maybe it never left.  But this appears to be more useful for real applications.  I can’t believe I just wrote that, but the fact that is uses actual SQL Server tables looks promising.
  • Remote debugging for SharePoint!
  • Napa Office 365 Developer Tools – looks pretty interesting; it’s a browser based development environment for O365.
  • If I see one more Bing Maps demo I think I will throw up.

What did you takeaway?

Feel free to share some of your takeaways from the conference in the comments section – I’d love to hear them.

read more
Kirk LiemohnKirk’s Takeaways from #spc12

ContentType SQL Query Tip

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

Whenever I used ContentType in the WHERE clause of a FullTextSQLQuery the query would return no results. However, I found a fix:

  1. Go to Central Admin > Search Service Application > Metadata Properties.
  2. Edit ContentType and checked “Allow this property to be used in scopes”.
  3. Do a full crawl.

In some tests I reset the index, but that does not appear to be required (I did one test without resetting the index).

Note that this is not required for many of the other managed properties if you want to use them in the WHERE clause.

FWIW, I looked at the content type properties in PowerShell. Checking the checkbox changed the two properties in red below.

PS C:\Users\Administrator> $ctype = Get-SPEnterpriseSearchMetadataManagedProperty -SearchApplication “Search Service Application” -Identity ContentType

PS C:\Users\Administrator> $ctype | fl

ID : ContentType

PID : 60

Name : ContentType

SplitStringCharacters :

SplitStringOnSpace : False

Description :

ManagedType : Text

FullTextQueriable : True

Retrievable : False

UseAAMMapping : False

MappingDisallowed : True

DeleteDisallowed : True

QueryIndependentRankCustomizationDisallowed : True

EnabledForScoping : True

NameNormalized : False

RespectPriority : False

RemoveDuplicates : True

HasMultipleValues : False

OverrideValueOfHasMultipleValues : True

IsInDocProps : True

IncludeInMd5 : False

NoWordBreaker : False

UserFlags : 0

Weight : 0

LengthNormalization : 0

EnabledForQueryIndependentRank : False

DefaultForQueryIndependentRank : 0

IsInFixedColumnOptimizedResults : False

RetrievableForResultsOnly : False

PutInPropertyBlob : False

QueryPropertyBlob : False

UsePronunciationString : False

MaxCharactersInPropertyStoreIndex : 64

MaxCharactersInPropertyStoreNonIndex : 4000

EqualityMatchOnly : False

MaxCharactersInPropertyStoreForRetrieval : 65536

DecimalPlaces : 3

IsBacked : True

The blog that led me to the solution is here: My issue occurred with only a couple of conditions and not 10+ conditions as mentioned in this blog. In addition, I am using SharePoint 2010 and not SharePoint 2007. It was the same fix for both scenarios nevertheless.

While researching this I found several posts about ContentType where people have a hard time selecting the ContentType in the full text SQL query. Using ContentType in scopes does not help with that problem. The solution others have found is to create your own metadata property and have it map to the same crawled properties that ContentType maps to. Here are a couple of posts that mention this:

read more
Kirk LiemohnContentType SQL Query Tip

Security Trimming Secrets

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

Introduction – What is Security Trimming?

Security trimming is simply the act of filtering out content that should not be accessible (typically read-able) for a given user. It is a core concept within SharePoint that affects what navigation elements you see, what sites you have access to, what lists and libraries you have access to, and what list items you see. It is also a core concept within SharePoint search. Security trimming in SharePoint search comes in two flavors: indexed security trimming and query-time security trimming.

Indexed Security Trimming

Security trimming within the index is possible when the crawler can obtain Access Control Lists (ACLs) for each item and store them in the index. It is the preferred approach because it is faster. There is little that is “dirty” about this approach. This is used for SharePoint content as well as file shares and other content sources. Outside of some minor storage and crawl processing costs, the only real downside is that permission changes are not reflected until the next incremental crawl.

Query-Time Security Trimming

Sometimes you do not have ACLs available at index time and you must resort to query-time security trimmers. This is the case when crawling web sites since there is no way to ask a website “who has access to this page?” This may also be the case with Business Connectivity Services (BCS) when you are crawling content from a database or web service. BCS can use indexed security trimming, but only if you can make ACLs available through your external system.

The rest of this post focuses on query-time security trimming and how this has changed with SharePoint 2010. Most of it is straightforward, but there are a couple of little “dirty secrets” you’ll need to be aware of. First I’ll give an implementation overview, then I’ll show what I have observed, and finally I’ll focus on two gotchas that are in SharePoint 2010.

Implementing a Custom Security Trimmer

Query-time security trimming in SharePoint 2010 is very similar to SharePoint 2007. They both require the server edition of the product (no WSS 3.0 or SharePoint Foundation). In both cases you write a class that implements an interface and then you register that security trimmer with a crawl rule. For SharePoint 2007 you use the ISecurityTrimmer interface, whereas in SharePoint 2010 you use the ISecurityTrimmer2 interface. A good reference for writing security trimmers can be found in Writing a Custom SecurityTrimmer for SharePoint Server Search.

This interface is very simple. It has an Initialize() method that runs once and can provide your class information specified when registering the security trimmer. This method is identical for SharePoint 2007 and SharePoint 2010.

The only other method is a CheckAccess method. This is the more interesting of the two. This is where the query is actually happening. This method returns a BitArray describing whether the current user has access to each URL provided. It has some differences between SharePoint 2007 and SharePoint 2010; most notably that in SharePoint 2010 you are provided an IIdentity. In SharePoint 2007 you had to use WindowsIdentity.GetCurrent() or HttpContext.Current.User depending on the authentication method.

CheckAccess Details

The CheckAccess method can run multiple times for a single query. What the query engine is doing is providing your security trimmer a batch of URLs expecting to find out which ones can be provided to the current user for search results. So, your security trimming code is running during the query – potentially multiple times. If the security trimmer does not give the query engine enough hits, it will provide the next batch of URLs to your security trimmer – and will continue to do so until it has what it considers enough URLs or until it has exhausted all hits for the search request.

Since the security trimmer is running during the query, it needs to be quick and efficient. However, while your security trimmer code is running you are likely doing some processing to find out if the current user has access to a URL. This may involve a web service call to another system. This may be fine, but I recommend that you test the performance thoroughly just to be sure. Unfortunately, since SharePoint 2010 only provides an IIdentity, you are fairly limited since this does not provide you as much information as what you have access to in SharePoint 2007. The context the security trimmer runs in is not the same context you are used to for your typical server-side SharePoint code, so you may have to do additional calls to lookup user information or other context-specific data. If so, you’ll want to factor that into your performance costs.

In SharePoint 2007, the security trimmer was typically run enough so that the query engine could provide a page-worth of URLs to the user running the query. Since the default query page size was 10, it may have stopped after having 20 or so URLs. In SharePoint 2010, however, it not only wants enough URLs to show a page-worth of results to the user, but it also wants enough to show appropriate refiners. By default, I believe that this means that the SharePoint 2010 query engine wants 50 results (after trimming).

CheckAccess Observations

Depending on how many search results there are before trimming and how many are trimmed out, your security trimmer may be called a large number of times. Using the default settings with a large result set and with the majority of search results trimmed out, my observations have shown me that the first time the security trimmer is run it is provided 50 URLs. If you trim some out, the security trimmer is immediately called again with another 75 URLs – and this is done before the user sees any search results. That continues with batches of 75 URLs until enough results are satisfied or until the total result set is exhausted. Using trace statements and DebugView I can watch what is going on. The chart below shows how many URL are requested for a single query. In this case the security trimmer is called 6 times and given a total of 425 URLs.

In the output below, the first line is when Initialize is called. Each subsequent line indicates an individual call to CheckAccess.

00000000 0.00000000 [9292] [CustomSecurityTrimmer] Initializing
00000001 0.00353188 [9292] [CustomSecurityTrimmer] Count = 50 - Total = 50.
00000002 0.05990715 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 125.
00000003 0.10741841 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 200.
00000004 0.16418955 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 275.
00000005 0.21421386 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 350.
00000006 0.26570737 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 425.

The algorithm changes once the user clicks to view the second page of results. The initial count of URLs provided to the security trimmer for the second page is 60. It is 70 for the third, and then seems to max out at 76. The pattern is a little more complex, so it is probably best just to show you some data. Notice how the total resets on subsequent page requests. That is because for each page we are starting a new set of processing as far as the security trimmer is concerned.

00000000 0.00000000 [11200] [CustomSecurityTrimmer] Initializing
00000001 0.00489843 [11200] [CustomSecurityTrimmer] Count = 50 - Total = 50.
00000002 0.06208209 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 125.
00000003 30.23514748 [11200] [CustomSecurityTrimmer] Count = 60 - Total = 60.  (Page 2)
00000004 30.27280998 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 135.
00000005 45.76335526 [11200] [CustomSecurityTrimmer] Count = 70 - Total = 70.  (Page 3)
00000006 45.79971313 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 145.
00000007 59.88526917 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 4)
00000008 59.88538742 [11200] [CustomSecurityTrimmer] Count = 4 - Total = 80.
00000009 59.92379761 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 155.
00000010 59.97734451 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 230.
00000011 82.77108765 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 5)
00000012 82.77122498 [11200] [CustomSecurityTrimmer] Count = 14 - Total = 90.
00000013 82.84067535 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 165.
00000014 82.93080902 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 240.
00000015 99.88347626 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 6)
00000016 99.88369751 [11200] [CustomSecurityTrimmer] Count = 24 - Total = 100.
00000017 99.94698334 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 175.
00000018 100.04191589 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 250.
00000019 106.22407532 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 7)
00000020 106.22446442 [11200] [CustomSecurityTrimmer] Count = 34 - Total = 110.
00000021 106.30342865 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 185.
00000022 106.39843750 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 260.
00000023 111.46296692 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 8 )
00000024 111.46327972 [11200] [CustomSecurityTrimmer] Count = 44 - Total = 120.
00000025 111.52843475 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 195.
00000026 111.62356567 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 270.

If we are processing too much within a page (a single request), then then you can throw a PluggableAccessCheckException. This is a way for the security trimmer to throw up its hands and basically give up. A good overview of how to use this is found on Walkthrough: Using a Custom Security Trimmer for SharePoint Server Search Results, but read further below as there is a problem with this in SharePoint 2010. With this exception you can provide a message to show the end user; it basically tells them to refine their search. This is an important part of a security trimmer because you don’t want to leave the user hanging too long. If the query lasts too long, the thread will be aborted. By default this is 90 seconds.

OK, now it’s time to dish some dirt…

Dirty Secret #1 – PluggableAccessCheckException

As mentioned above, the purpose of the PluggableAccessCheckException is to allow the security trimmer to tell the query engine to stop processing so a response can be given to the user performing the query without taking too much time. Unfortunately, with SharePoint 2010 the opposite occurs. If you throw a PluggableAccessCheckException what happens is that your security trimmer will be called with all URLs in the result set (typically in batches of 75) unless the thread is aborted before you reach the end of the result set. This is even the case if the security trimmer does not trim out any results after throwing the exception. In the output below I have set a low limit of 150 URLs after which I throw the exception.

00000000 0.00000000 [4500] [CustomSecurityTrimmer] Initializing
00000001 0.00339907 [4500] [CustomSecurityTrimmer] Count = 50 - Total = 50.
00000002 0.05012119 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 125.
00000003 20.21066093 [4500] [CustomSecurityTrimmer] Count = 60 - Total = 60.  (Page 2)
00000004 20.25085640 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 135.
00000005 40.40690613 [4500] [CustomSecurityTrimmer] Count = 70 - Total = 70.  (Page 3)
00000006 40.45227051 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 145.
00000007 57.72388458 [4500] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 4)
00000008 57.72399139 [4500] [CustomSecurityTrimmer] Count = 4 - Total = 80.
00000009 57.76459122 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 155.
00000010 57.76465225 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 155.  Throwing PluggableAccessCheckException
00000011 57.93753815 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 230.
00000012 57.93761063 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 230.  Throwing PluggableAccessCheckException
                ... (skipping 8 sets of trace statements) ...
00000029 58.56541061 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 905.
00000030 58.56551361 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 905.  Throwing PluggableAccessCheckException
00000031 58.65005875 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 980.
00000032 58.65012360 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 980.  Throwing PluggableAccessCheckException
00000033 58.73397064 [4500] [CustomSecurityTrimmer] Count = 47 - Total = 1027.
00000034 58.73407364 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 47 - Total = 1027.  Throwing PluggableAccessCheckException

Last I heard, the product group confirmed the behavior and said that since multiple pluggable trimmers can be registered, search cannot be stopped on throwing the exception. I assume that they are still trying to prevent continuously calling the security trimmer that threw the exception and hope that they will consider all results trimmed that are associated to a crawl rule in which a security trimmer threw the exception.

Bottom line: You may want to consider the risks of your custom security trimmer being called too many times and running too long. A workaround may be to simply return False for every URL that is provided to your security trimmer once you determine that you have been running too long. This could be done without doing any expensive processing.

Dirty Secret #2 – FAST Search for SharePoint 2010

Unfortunately, query time security trimmers are not supported with FAST for SharePoint (FS4SP). The FAST pipeline is different and although FAST is a flexible platform, I am told that FAST Search for SharePoint 2010 does not currently have a feasible way to make this happen.

I believe this is because FS4SP has deep refinements (among other features). Without FAST, SharePoint provides refiner information based on the first 50 results, but with FAST the exact count can be provided for each refiner. This exact count cannot be provided unless all of the results are processed, which just isn’t feasible with a query-time security trimmer (which can conceivably have tens of thousands of hits or more for a single query).

Last I heard this was not currently supported, but that R&D is actively working on a design change to allow for this functionality in a future release.


Security trimming is a great thing and a powerful feature used liberally throughout SharePoint. Security trimming with SharePoint search is crucial, especially when search results show hit highlighting of content. If possible, use security trimming at index time. If you must use query-time security trimming, go in with your eyes wide open.

read more
Kirk LiemohnSecurity Trimming Secrets

Breaking MOSS Search

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

If you have registered a search security trimmer with a crawl rule and then prevented SharePoint from accessing the security trimmer (such as removing it from the GAC), you will get a nasty error when trying to execute a search that has results that meet your crawl rule.

To aid those searching for this error I’ll restate it:

Your search cannot be completed because of a service error. Try your search again or
 contact your administrator for more information.

To fix this simply make sure the security trimmer assembly is available (put it back in the GAC). You’ll likely have to do an IISRESET for things to work again. If you don’t have access to the security trimmer you may need to reset your index (hopefully there is another option, but I suspect not).

I had to dig through the SharePoint ULS logs to figure this out:

09/18/2008 11:15:42.01 w3wp.exe (0x0EB0) 0x0EF4 Search Server Common MS Search Query Processor 97zu
Unexpected Error occurred when executing pluggable security trimmer with Id 33402:

09/18/2008 11:15:42.14 w3wp.exe (0x0EB0) 0x0EF4 Search Server Common MS Search Query Processor 97zv
Exception System.IO.FileNotFoundException: Could not load file or assembly
'Atlassian.Confluence.SharePoint.Search, Version=, Culture=neutral,
PublicKeyToken=96f82bd35ec392e7' or one of its dependencies. The system cannot
find the file specified. File name: 'Atlassian.Confluence.SharePoint.Search,
Version=, Culture=neutral, PublicKeyToken=96f82bd35ec392e7' at
System.RuntimeTypeHandle._GetTypeByName(String name, Boolean throwOnError, Boolean
ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, Boolean loadTypeFromPartialName)
at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase,
Boolean reflectionOnly, StackCrawlMark& stackMark) at System.RuntimeType.PrivateGetType
(String typeName, Boolean throwOnError, Boolean ignoreCas...

09/18/2008 11:15:42.14* w3wp.exe (0x0EB0) 0x0EF4 Search Server Common MS Search Query Processor 97zv
Exception ...e, Boolean reflectionOnly, StackCrawlMark& stackMark) at System.Type.GetType
(String typeName, Boolean throwOnError) at Microsoft.Office.Server.Search.Administration.

09/18/2008 11:15:42.18 w3wp.exe (0x0EB0) 0x0EF4 Windows SharePoint Services Web Parts 89a1
Monitorable Error while executing web part: Microsoft.Office.Server.Search.WebControls.
SrhdcGenericException: Your search cannot be completed because of a service error.
Try your search again or contact your administrator for more information. --->
System.IO.FileNotFoundException: Could not load file or assembly 'Atlassian.Confluence.
SharePoint.Search, Version=, Culture=neutral, PublicKeyToken=96f82bd35ec392e7'
or one of its dependencies. The system cannot find the file specified. File name:
'Atlassian.Confluence.SharePoint.Search, Version=, Culture=neutral,
PublicKeyToken=96f82bd35ec392e7' at System.RuntimeTypeHandle._GetTypeByName(String name,
Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark,
Boolean loadTypeFromPartialName) at System...

read more
Kirk LiemohnBreaking MOSS Search

Viewing Scopes

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

I learned something yesterday while trying to understand a client issue with MOSS search.

With the work we did with the SharePoint Connector for Confluence, we created functionality to let you search Confluence from SharePoint. A lot of this work used out of the box features with MOSS 2007 Enterprise Search. However, we did have to create custom configuration screens to allow the user to create a crawl rule that used forms based authentication (FBA). In addition, we needed to create a custom security trimmer because crawling web sites does not allow for the indexing of ACLs.

What I learned yesterday was something interesting with the security trimmer. I knew that custom security trimmers are executed when an end user performs a query as opposed to when the search engine crawls and indexes the content. What I didn’t know is that simply viewing scopes within the search administration interfaces will also execute the security trimmer.

Within your shared service provider (SSP), you can view the scopes as shown below:

In my test environment, I had only crawled the “TW Confluence” content source which had an associated “TW Confluence” scope as shown above. My total index had 104 items, but no items were showing up in the scopes. I was expecting to see counts for both “All Sites” and “TW Confluence”.

What was happening was that simply viewing this page invokes the security trimmer assigned to my crawl rule for all items in the index that map to the crawl rule. In the case of the SharePoint Connector for Confluence security trimmer, it needs to ask Confluence if the current user has access to each URL. Unfortunately, if the security trimmer is invoked from the shared service provider (as it is done in this case) it does not know how to connect to Confluence because that configuration is available within a typical site collection, not within a SSP.

Interestingly, there is also a way to view scopes from within a typical site collection:

As you can see here, we do have counts for our “All Sites” and “TW Confluence” scopes. Once again, our security trimmer is executed when viewing this page, but this time it is able to find configuration data on how to connect to Confluence. The count of 9 is much less than what is in the index because the current user does not have access to all of the URLs; access to them was denied by the custom security trimmer.

Note that the only way I knew for sure that the security trimmer runs in these cases is through some tracing capability we have had in the product for quite some time.

If you have more interest in learning more about the SharePoint Connector for Confluence, check out the links above or visit If you are in the Atlanta area and want to learn more about MOSS 2007 Enterprise Search, keep an eye on our Event Calendar for upcoming presentations.

read more
Kirk LiemohnViewing Scopes

SharePoint Wildcard Searching

Tim is a Senior Consultant at ThreeWill. He has 15 years of consulting experience designing and developing browser-based solutions using Microsoft technologies. Experience over the last 8 years has focused on the design and implementation of SharePoint Intranets, Extranets and Public Sites.

The Ontolica download and install is a straight-forward process. It can be downloaded for free from the Ontolica site. Once installed, the Ontolica Wildcard search will automatically take the place of the search defined in the default master page of most of the SharePoint Site templates such as the Team Site. However, some of the Publishing master pages such as BlueBand.master will need to be modified for the Ontolica Wildcard search to work.

The reason the Wildcard search will “override” the default search in some sites and not others is due to the fact that the master page defined for Team Sites uses a delegate control and the masterpage for some of the publishing sites use a WebControl. Delegate controls are designed with the intent that they can be easily overridden. Delegate Controls are overridden in SharePoint by installing a control with the same id and a lower sequence number. The Ontolica Wildcard feature installs the Ontolica Wildcard search with a sequence number of 1 which overrides the default control that has a sequence number of 100.

Here’s a copy of the xml:

To get this to work in master pages such as the BlueBand.master, you will need to replace the Web Control reference in the PlaceHolderSearchArea content place holder with the Delegate control as demonstrated below:

Once this change is saved in BlueBand.master, the Ontolica Search will be “substituted” for the default “SmallSearchInputBox” that comes out of the box with MOSS 2007.
Another issue I ran into when using the Ontolica Wildcard Search was getting the licensing properly configured so that Users didn’t receive an error message on the search results page. As part of the Ontolica Wildcard Search install, an evaluation copy of some of the richer features of Ontolica Search are included. To get rid of the annoying message, you must “toggle” the “current license” from the Free Ontolica Evaluation License to the Free Ontolica Wilcard License. This action is performed in the Central Admin site under the Operations tab. You will find an Ontolica for SharePoint section in the bottom right corner of the screen. Select the Manage Ontolica Licenses option to see the screen below:

By default, the Free Ontolica Evaluation License will be selected. You will likely want to “toggle” this to the Free Ontolica Wildcard License unless you choose to purchase the richer-featured Ontolica Search capabilities. To toggle the licensing to the Free Ontolica Wildcard License, select the Free Ontolica Wildcard License drop-down and choose the “set as active license” option as demonstrated below

The resulting License Manager screen should have a check-mark beside the Free Ontolica Wildcard License and no check-mark beside the Free Ontolica Evaluation License as demonstrated below:

read more
Tim CoalsonSharePoint Wildcard Searching

Enable Advanced Search in SharePoint 2007

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

One of the problems that we ran into having created the default publishing site, is that you do not get the option for advanced search. For example, we were unable to search for just people. The one for the publishing portal does not have the advanced search option.

Here are the steps we took to correct this.

  1. Login to the main web site and login as the site collection administrator. (SPAdmin)
  2. Click Site Actions -> Site Settings -> Modify All Site Settings.
  3. Under Site Collection Administration Select Site Collection Features
  4. Active all Features that are not active. Of course you don’t have to do all but dang gum it’s just easier.
  5. When you create a publishing portal by default you can only create publishing sites. You have to enable the other site templates.
  6. From root site , click Site Actions -> Site Settings -> Modify All Site Settings
  7. Under Look and Feel select Page layouts and site templates
  8. Select “Subsites can use any site template” and “Pages in this site can use any layout”
  9. You of course can change this if you need to; the only template you will need is the search template.
  10. Return to Root site
  11. Click on Site Actions -> Create site
  12. For Title and description enter Search Center, for url enter SearchCenter as one word
  13. For Template Selections click the enterprise tab and select Search Center with Tabs
  14. Leave the rest to the default and click Create
  15. Click Site Actions -> Site Settings -> Modify All Site Settings
  16. Under Site Collection Administration click Search Settings
  17. Put the radio button in “Use custom scopes. Display richer results using the following Search Center:”. In the space below enter “/SearchCenter/Pages” without the quotes
  18. Click OK
  19. On the Site Settings page Select Master Page
  20. Change the Master Page to default.master, Put a check mark in Reset all subsites
  21. Leave the rest default and click OK
read more
Kirk LiemohnEnable Advanced Search in SharePoint 2007

MOSS Enterprise Search Presentation

Kirk Liemohn is a Principal Software Engineer at ThreeWill. He has over 20 years of software development experience with most of that time spent in software consulting.

The slide deck can be found here.

The presentation gives an overview of MOSS Search and discusses the search architecture as well. The first demo shows many of the ways search can be configured through the Shared Service Provider search settings as well as through the Site Collection search settings. The second demo shows a custom security trimmer in action against a web site content source.

During the presentation I mention how you can crawl against a web site that uses forms-based authentication. This requires a hotfix.

For more information see:
To download the hotfix see:
Much of these slides were taken from an earlier presentation I gave along with others at the Next Level event we held at Microsoft in early November.

SharePoint is a web application platform in the Microsoft Office server suite. Launched in 2001, SharePoint combines various functions which are traditionally separate applications: intranet, extranet, content management, document management, personal cloud, enterprise social networking, enterprise search, business intelligence, workflow management, web content management, and an enterprise application store. SharePoint servers have traditionally been deployed for internal use in mid-size businesses and large departments alongside Microsoft Exchange, Skype for Business, and Office Web Apps; but Microsoft’s ‘Office 365’ software as a service offering (which includes a version of SharePoint) has led to increased usage of SharePoint in smaller organizations.

While Office 365 provides SharePoint as a service, installing SharePoint on premises typically requires multiple virtual machines, at least two separate physical servers, and is a somewhat significant installation and configuration effort. The software is based on an n-tier service oriented architecture. Enterprise application software (for example, email servers, ERP, BI and CRM products) often either requires or integrates with elements of SharePoint. As an application platform, SharePoint provides central management, governance, and security controls. The SharePoint platform manages Internet Information Services (IIS) via form-based management tooling.

Since the release of SharePoint 2013, Microsoft’s primary channel for distribution of SharePoint has been Office 365, where the product is continuously being upgraded. New versions are released every few years, and represent a supported snapshot of the cloud software. Microsoft currently has three tiers of pricing for SharePoint 2013, including a free version (whose future is currently uncertain). SharePoint 2013 is also resold through a cloud model by many third-party vendors. The next on-premises release is SharePoint 2016, expected to have increased hybrid cloud integration.

Office 365 is the brand name used by Microsoft for a group of software plus services subscriptions that provides productivity software and related services to its subscribers. For consumers, the service allows the use of Microsoft Office apps on Windows and OS X, provides storage space on Microsoft’s cloud storage service OneDrive, and grants 60 Skype minutes per month. For business and enterprise users, Office 365 offers plans including e-mail and social networking services through hosted versions of Exchange Server, Skype for Business Server, SharePoint and Office Online, integration with Yammer, as well as access to the Office software.

After a beta test that began in October 2010, Office 365 was launched on June 28, 2011, as a successor to Microsoft Business Productivity Online Suite (MSBPOS), originally aimed at corporate users. With the release of Microsoft Office 2013, Office 365 was expanded to include new plans aimed at different types of businesses, along with new plans aimed at general consumers wanting to use the Office desktop software on a subscription basis—with an emphasis on the rolling release model.

read more
Kirk LiemohnMOSS Enterprise Search Presentation