John Underwood is a Technical Evangelist at ThreeWill. He has nearly thirty years of software development experience. He is an experienced technical instructor with superior presentation skills and is proficient in delivering standard curriculum as well as developing and delivering custom curriculum.
Hello folks, John Underwood here welcoming you to another of the ThreeWill Webinar series. Today, we’re going to talk about Enterprise SharePoint Search. I hope you’re having a good day wherever you’re at. I’m coming to you from just North of Atlanta, Georgia in Alpharetta and we’ve actually had a rare sunny day around here. It’s been a pretty good day. A little bit about me. For those of you who’ve not done events with me before, I’m lucky enough to be an employee of ThreeWill. My job title here is technical evangelist. That means I get to do events like this. I also do training and I even get to work on the odd project every now and then. A couple of ways to contact me obviously, you can use my email address on the screen. If you are someone that enjoys being a part of Twitter, there’s my Twitter handle. You’ll get a nice mix of hobbies and information, but mostly, you’ll get some good SharePoint information. I hope you’ll take advantage of that and follow me if you’re inclined to do so.
I know we want to get right into the material, but a couple of quick housekeeping items before we start. First of all, today’s session is being recorded. If you find that you have to drop off in the middle or if you’ve got some co-workers that were unable to attend but maybe want to see the information later, it will be available online for a certain amount of time. Essentially what you do is go back to the same URL that you used to originally visit and register for the event. You can take that invitation that you received and you can forward that to any of your friends or associates and they’ll be able to use that in order to play the video. Again, it’s interesting, even if they didn’t attend, they’ll still be able to register and view the video.
Then, one other thing. Although we hope we won’t have any technical interruptions, it’s not impossible that a phone would get disconnected or the power might go out temporarily or something like that. If any of those kinds of things happen during the event, you can notify me that you’re no longer seeing me. If you just send an email to the address you see there on the screen, that comes directly to my phone and at least, I’ll know something’s going on and we can see if we can’t get service restored and resumed. Again, that’s rare that that happens.
All right, with introductions and housekeeping items out of the way, let’s talk about what I’m going to cover with you today. First of all, I’m going to talk a little bit about search-driven applications. I think you’ll see that if you don’t already know about these, they’re going to represent what is for a lot of companies a missed opportunity. The ability to really exploit the SharePoint search engine in a way that’s going to be beneficial to you. Then, after that, we’re going to talk a a little bit about extending and customizing search.
A few of the things we’ll see here is something where things that maybe a power user could do but a lot of these are going to be more developer or even administrator oriented. Really, what I hope to do with this part of the presentation is twofold. Number one, just letting you know what’s possible and number two, take an opportunity to show you some of the things that ThreeWill has done as far as extending and customizing search on behalf of our customers.
Then, finally, I’m going to wrap up with just a quick look at some of the new things in SharePoint 2013 as relates to search. I know inevitably when there’s a new release, there are a few companies that jump on it immediately, but there are others that take their time. Since I’ve been working in the SharePoint universe, I know that’s particularly true of SharePoint moving up to a new version is not trivial.
Really, more than anything, I just want you to know what’s coming and really plan for some of those opportunities at the risk of sounding like a Microsoft product personnel, will tell you that some of the new search things in SharePoint 2013, I think are really compelling. They alone may not be a reason enough for you to upgrade, but taking in a whole lot, I think it is a worthwhile upgrade that you’ll want to undertake as soon as possible.
Then, time-permitting, we’ll do a little Q & A at the end. If time doesn’t permit or if you don’t think of any questions at that point, of course, I always welcome questions in the future at my email address, so please take advantage of that.
All right, so let’s spend the first part of our time together now talking just a little bit about search-driven applications. The hope here is that if you haven’t already thought about the search engine in this way before, after we’re done here, you’re going to see it as more than just a place where you type in words. Hopefully, you’re going to see it as a repository of some really valuable information for your company.
Let’s start with a common scenario. An application that we might need within our company. Suppose that we work at a company. We use SharePoint pretty heavily. We’ve got a corporate home page that’s hosted on SharePoint and then we have lots of site collections and sites and sub-sites and teen sites and all these different things. We have a legitimate need to present a list of sites that a particular user can see. Again, it could very well be that a user has access to a useful site and they don’t even know it. They don’t even know what’s out there.
One of the ways we might solve this problem is by writing a Web Part. Again, I’m a developer and what do developers do? They develop. They write code. Almost inevitably, if someone brings a business problem to us and says, “How are you going to solve this?” The first little thought balloon that pops up over my head is, “I’m going to write some code.” In this case, we’ve got a developer that creates a Web Part and that Web Part iterates through sites and makes a list of available sites based on the current user.
Now, we all know and love this phrase, “It passes the developers works on my machine test.” What we see here is that, a user or excuse me, a developer that’s written this may be testing it in an environment where they have a dozen sites. Then from there, we deploy it to our company’s home page where we have thousands of sites. All of a sudden, somebody goes to the home page and it’s taking forever to load because this little application is now going to try to iterate through those thousands of sites for every single page visit to the home page.
There’s really a couple of things wrong with this. Thing number one, obviously, is that, this is going to be a huge performance hit and people are going to hate SharePoint when it’s really not even SharePoint’s fault. The other thing here that’s a little more subtle but perhaps even more important and that is, there was a huge missed opportunity here. Because of the preconceived notion of how this developer viewed this problem, they didn’t see that there’s actually an efficient, simple and easy way to solve it. Easy enough in fact that in some cases, we could even rely upon a power user to do that.
What I’m talking about there is really a search-driven application. What is a search-driven application? Well, in simple terms, instead of us trying to use the lists and libraries in our SharePoint world as a source of data, we’re actually using the search engine or more correctly, the search engine’s index as our source of data. Because after all, that index contains a lot of really useful things.
Now, when we talk about a search-driven application, we’re talking about a scenario where we’re not going to go ask the users to type in some data and search for things. Rather, we’re going to search on their behalf. Really, they may not even know that it’s coming from search, and they may not even see the query. We may do it in a very subtle way so that that information is hidden from them. All they know is that they are able to undertake some actions such as clicking a link or visiting and web page and they get whatever it is that they needed.
Another nice thing about search-driven applications is that, the results are security-trimmed, by the very nature when you go into SharePoint and you search for things, you only get results for the things that you are authorized to see. In the case of as an example trying to show a user all of the sites that they have access to, by definition the search engine’s not going to let us serve any site references up to users unless they’re authorized to visit that. That just means there’s a whole level of coding that we often have to undertake that we’re just going to be able to ignore here because the search engine and its index already knows the answer to that question.
Then, another thing about a search-driven application, and while this isn’t a requirement, I think it makes it more compelling and it’s really nice. That is, results are presented to the user in a way that are ready to work with. To say that a little bit differently, certainly we could take users to a search page and we could teach them how to look for the thing that they want to look for. With the little effort on our part, we can customize the output from that process in such a way that they don’t get a lot of the other decoration that we would normally see on a search results page. Instead, they’re just going to get the data in question and the importance of that data to them.
When I do some demonstrations on this in a few minutes, I’ll reinforce what I’m talking about when I say presenting the results in a way that are ready to work with. If you’re a developer and you’re on this webinar today, here is my encouragement to you. When you start thinking about the sources of data that are available to you when you are customizing SharePoint, don’t forget the search index. Even furthermore, almost any time you’re writing some kind of customization that’s asking a question, the odds are pretty good that the search engine would know the answers to those questions.
Let’s dig a little deeper then. If it’s all about asking the right question, then how can we ask the right question? At least traditionally, when we think of a search engine, we think about a keyword-driven search. As an example, excuse me, a user might type in the word invoice hoping to find things that are related to invoices or resumes, or a series of what we would call keywords. However, the thing that a lot of people don’t know and that is really useful for the search-driven application, and that is SharePoint has a capability that allows us to specify the kind of thing that we’re looking for. Not only might we say, “I want invoice,” but we might say, “I want invoice and I only want that word where it appears in a list or where it appears in a library.” We can refine the kind of thing that we want to get back from the search engine.
If you think a moment ago about my example of trying to give a list of sites that a user has access to, we could specifically say to the SharePoint search engine, “Hey, I only want you to send me back things that represent sites. Anything else, just leave out of the results.” Then, another capability that we have in the search engine that not everyone knows about, we can also qualify details about the item that we were looking for. Not only could I say, “I only want items that appear in a task list.” I could go further and say, “I only want the ones that are authored by Kim or the ones that are assigned to Kim.” This notion that the entities that we’re looking for out there, each have some properties associated with them, and the query language actually gives us some mechanism for specifying this property.
Then, let’s be a little more specific. How is it that we’re going to qualify the type of results? Well, one very easy one and again I’ll be demonstrating this in just a moment, is the use of IsDocument. That’s pretty straightforward. It just lets me say, “I either want to explicitly return results that are documents, or are not documents.” By doing that, I may be able to go from a search that gave me a whole lot of results, to a search that gave me a set of results that are a little closer to what I’m looking for.
Now, the next one, and in my opinion, simultaneously both the most important one and the one that’s probably the most underused is the concept of a content class. You may not be aware of the fact that when you issue a keyword search in SharePoint, you’re actually able to specify the content class that you wish to receive back. Even if you’re not a real SharePoint expert, you can figure it out based on what you see here. As an example, I could say, “I want to look for the word invoice but I only want to see back a list item.” Or, “I only want to see back something from a document library.” Then, we can even go a step further. Not only could we say, “I only want to see items from a list,” but we can even provide a type that says, “I only want to see items from a list that’s a task list or a contacts list, or even a calendar.”
When you begin to think about this ability to not only ask for what you want but to specify the kinds of things that you want, that’s really a cornerstone of building these kinds of applications. Then, I handed that just a moment ago, this is just a more formal look at that notion of a property restriction. In effect, we’re saying, once we found the things we think you’re looking for, we can even be more specific about, it should have a certain file extension. It should have a certain file name. It should have a certain author and so on and so on. Taking what we already know about keywords and laying on these layers of qualification, that’s how we’re going to be able to not only write better queries to start with, but then take these queries and package them up in a way where users can conveniently consume them.
Let me take a moment now and do a quick demonstration for you. I’ll show you a little bit about some of these restrictions. Now, just a little FYI here. I’m going to be working in a virtual machine environment and I’ll just confess to you right upfront. It’s running on my local machine and it’s not got the most horsepower in the world behind it. Don’t try to draw too many conclusions here about the performance of this application. Really, let’s just focus more on what it does. Now, those of you that have worked a lot with SharePoint, probably recognize the fact that I’ve got a SharePoint 2010 Team Site that I’m working with here. You can see I’ve got a top-level site named Acme Corporate. I’ve got a couple of sub-sites underneath. Then, I’ve got a search center setup as well.
Let me see if I can demonstrate now some of the techniques that I’ve talked about in terms of refining your search and then from there, we’ll put that all together into a simple little application. Let’s suppose that I’m looking for a resume and I know that it was authored by Kim. I can start with a real simple search just by putting in the word resume and then searching for that. Then, we can see that, I get back results here that are all over the board. I’ve got different things that are actually resumes. Not only that, I’ve even gotten pages that just happened to have the words resume on them.
Now, if I go a step further, and I can say, “Well, let’s refine it so that it looks for both of those words.” Certainly, that gets us a little closer but you can see that I’m also winding up with a little more debris down here that’s not really relevant to what I’m looking for. Again, in this somewhat contrived example, I’ve gotten it down to three items. I could probably live with that, but those of you that work in very large corporate environments, the little refinement we just did there maybe took you from 10 thousand results to three thousand results. That’s a huge improvement, but you’ve still got a big hole to dig out of.
Let’s refine it just a little bit more. We want to look for the keyword resume, but we only want to get back items where the author of that document is Kim. You can see now that we’ve got a much better result at this point. We’ve got the actual resume and we can see clearly in our results here that it was authored by Kim. Just in taking that much of an approach on looking for things, you can see that we’ve really been able to take a machete and cut away some of that underbrush that we frequently get when we’re looking for search results.
Let’s take this a step further now and look at how else we might use this. Let’s say that I’m in search of something related to tasks. Now, if I just type in the word tasks, I’m going to get a lot of different things. Depending on what I was looking for, it could just be that I’m looking for something related to documents that have the word tasks in it. Or, it could be a classic problem that we have in SharePoint where I’ve got 15 different sites and each one of them has a task list and I’m just trying to figure out which lists have an assignment for me. Let me see if I can tackle both of those for you.
For the first one, let’s suppose that I truly do want to look for documents that contain the word tasks. Notice what I can do. I can put in this document and in this case, equals one means that I want it to be a document, equals zero means I want it to be a non-document. Just by adding the equals one, notice now I went from dozens of results, several pages worth down to just three. One little qualification on this one to be aware of as well and that is, notice the definition of a document. Not only do I get a Word document which is what a lot of us think of when we think of documents, we get library pages here. In the SharePoint sense of the word, they are in a library called site pages and they are in the documents.
Then, let’s take it one step further. Let’s say that we are truly interested just in Word documents. We could just go ahead and qualify the file extension on this just a little bit, and that would help us reign this in and see just the things we’re looking for. Right here I’ll just say, file type: docx. Then I’ll go ahead and search again. You can see now I’ve been able to refine it down to just the exact thing that I’m looking for.
Let me go one more step. Now I promised you that we would not only be able to find documents that have the word tasks in it, but that we’d also be able to find our task. In fact, I would tell you that of all the demos I think I’m going to do for you today, this is the one that is the most relevant and the most useful. That is, being able to find all of the tasks that are assigned to you across numerous sites. Let me start my query again. What I’m going to do now is, I’m going to change this one completely. I’m going to say content type:, excuse me, content class, and then STS_listitem_tasks.
Now, even if you are decidedly non-technical, this is a simple enough syntax that you can master it. Any time we see content class presented this way and then followed by some kind of qualifier, really what we’re saying is, “SharePoint, I only want you to bring back things of this kind.” In this case, we’re saying, “We only want you to bring list items that are actually of a task type.” Having typed that in, I’ll go ahead and search for that now. Notice, all I get are things from task lists. It’s pretty obvious that that’s the case.
Now, we could go a step further here and say, “I really want to find the ones that are assigned to me.” There’s sort of a good news, bad news going on here. The good news is, we can qualify who it’s assigned to. The bad news is that, there’s not an automatic way for us to say, “Only show me the ones that are assigned to the current user.” Those of you who’ve built these before and you know how you can filter a view where it says, “Just show items for me.” Unfortunately in the query language, we don’t have anything exactly like that. You are going to have to prompt your users for some kind of information. Or, it is possible with just a little bit of code, we could make this a fully automated process.
For this example, let’s just pretend that we are in the role of Kim. We’ll just come out here and say, “Assigned to” and “Kim.” What I’ve done now is make it possible for Kim to go to one place and see all of the tasks that are assigned to her. Again, she’s obviously not going to see any places that she’s not authorized to see. The thing here is, she’s now not going to have to go visit all these different sites to figure out if she’s got an assignment or not. She’ll just simply execute this query and she’s off and running. Now, how can we package this in such a way that it’s a bit more convenient to use? Well, let’s talk a little bit about that and then we’ll go a bit deeper and see how to get things done with that.
I’m going to switch back to Power Point here. The next part of this is called Accessing the Application. That is to say, once we figured out a way to ask for the results that we want, then we need a way to present it up to users so that they can conveniently consume it. Now, the easiest way to solve that problem is to recognize the fact that these search results can be expressed as a hyperlink. You might have noticed this when I was … demo, or you may have missed it, but if you take a look here, you can see. When I typed in all of this stuff in the search box, all of it’s represented up here on this URL. It is quite literally as simple as taking that URL and then making it available to a user.
How am I to do that? Well, I can put it on the Quick Launch menu. I could create a page and author a hyperlink there. Or, there’s even a hyperlinks list that’s available to me in SharePoint. I could put the information there. Pretty much anywhere we can store a clickable link, we’re now giving our users the ability to take advantage of this. Now, something that is a bit more sophisticated, but also brings us a bit more power and that is the Core Search Results Web Part. In particular, as developers or really high-end power users, we have the ability to customize the XSLT that is associated with that.
Now, if you have no idea what XSLT is, let me just take a quick aside here. Extensible Stylesheet Language Transformation is what it stands for. In very short terms, it’s just a template that describes how the results are going to look. As an end-user, you probably aren’t ready to engage trying to change that, but for a technical resource, it’s a pretty simple thing to do. Now, one other word about customizing the Search Results Web Part. The good news is, you can do it. You can absolutely do it. The challenge here is that, it can be difficult and if you’re doing this in a search center, you have to be aware of the fact that if someone else tries to modify that same set of XSLT, their changes might collide with your changes.
I want to drop that in your mind as a little bookmark of a problem, because when we start SharePoint 2013 conversation, we’re going to see that that problem gets solved. What I think is a better approach on this and it’s the one I’m going to demonstrate for you in just a moment is, when I’m building a search-driven app, instead of modifying the Web Part that’s in my search center on the results page, what I probably do and just did is just go to my site pages library. Create a dedicated page and put a Core Search Results Web Part on that. From there, I can configure the part so that it issues whatever query it needs to and it has whatever customization it needs. I am going to demonstrate that for you and let you see that in action. Let’s do that right now.
I’ll go back over and just continue working in the same demo environment that I’m in. Let’s do something a little more generic here. Excuse me. Let’s see if we can do a query that will show us all of the sites that someone has access to. Once again, I’m going to use the content class and I’m going to look for two content classes here. The first one I’m going to get is STS WEB and then I’m going to do an OR, and the OR does have to be capitalized. Then I’m going to do STS SITE.
Now, for those that are less technical in nature here, let me just give you a simple answer. If you want to get all the URLs, all of the sites, this is the way that you have to do it. For those of you that are a little more technically savvy when it comes to SharePoint, remember, if I did STS WEB, that’s only going to give me the sites. That’s not going to give me the top-level site for the site collection. For that, I have to use STS SITE. If I do those two together, you can see now I actually have a list of the sites that are available to me.
Now, in terms of making this a “application”, watch what I’m going to do. I’m going to copy this URL and then I’m going to navigate back over to the home page as my Acme Corporate Team Site. Then, once I’ve done that, I’m going to go in and customize this page. Once I’ve opened up page editor, I’m going to go in and insert a Web Part. What I’m going to insert is the links Web Part. I’ll put that right down here. Then, I’ll go ahead and save my work. Then once I’ve done that, I can go over to all site content and you can see that I have a links list here. I’ve got this one left over from a previous demo. I’ll just delete it out real quickly and then, add a new link here. I’ll put in the URL of the query that I just wrote and then I’ll give a description of all the sites you can access and click save.
Then once I’ve done that, now I can go back to the home page of this site and at any point, when a user comes to the home page of this site, they’d be able to click on this link and go directly to a search results page and see just the sites that they can access. To really drive home that this indeed doing what I’m claiming it’s doing, notice that I could take a second here and sign in as a different user. In this case, a user named Ed. Notice that if Ed clicks on this link, he’s only going to get two results whereas before I was logged in as an administrator and I got four results.
No worries here about writing any logic whatsoever, and we’ve solved that problem that we started talking about at the very beginning of the webinar. Now, to go a step further here, let’s see if I can log back in as administrator now and show you a little bit about how to make this a bit more beautiful. Okay, so I’ve logged back in as administrator. Now, let’s remember a point that I brought up early on in the process. I said that when we do these search-driven results, or these search-driven applications, it’s really nice to put the results in a format where they’re ready to use.
Saying that another way, if all I’m interested in are sites, then I don’t need all of this garbage. I just need a title of the site and something I can click on, and I don’t need the refiners and I don’t need all these other stuff. What I can do here is very simply create a page that contains a Core Search Results Part, and then configure that Part to issue this query. I’ve actually already pre-baked an example of that. Let me take you over and show you what that looks like.
In my site pages library, I actually have a page here called Available Sites. You can see that if I navigate to that page, it has a very simple list of the sites that are available to me. Then of course, I could click on any one of those and navigate directly to that site. Now, how is it that I’ve done this? Let me show you real quickly. I’m going to go ahead and put this page in edit mode. It’s being a little bit stubborn there. It doesn’t want to go in edit mode. Sometimes I have to switch back and forth between Internet Explorer and Chrome to get what I’m looking for. That’s fine. There we go. That’s behaving a little bit better.
Now, you’ll notice, the very first thing that I’ve got here is a Search Core Results Web Part. If I go in and actually edit this Web Part, I can show you the two things that I changed in order to make this thing work the way that I wanted it to work. Let’s drill down through some of these and have a look see here. Now, you can see there’s a section that describes my display properties. Then, you can see there’s a section where I have something to say about the results query. Let’s start with that one first.
Now, I know this text box is not big enough for you to see all of the contents, but you ought to be able to figure out pretty quickly what I’ve done there. In fact, if I select all of that and copy it, and then put it into Notepad so you can see it, that’s the exact same query that I issued before. Really, what I’m telling this Part is, the user is not going to type in a query. Instead, we’re just going to give them a fixed query, and then we’ll allow security trimming to make sure that everybody sees what they’re supposed to see.
Now, if I did only that, I would actually wind up with the same appearance and search results that I had before. Now, I want to go one step further and make it look nicer. I did that by adding my own custom XSL. Now, the point of this is not to teach you how to write this XSL. Some of you under the sound of my voice, probably would never do that, but the point here is just to let you know that it’s this simple. You can see in this care, really all I’ve done is just say, “Take all of the results. Turn each one of them into a hyperlink. Substitute the URL and then put a title on it.” That’s how I wind up with these very simple search results. Then, if I stop editing and go back into view mode, you can see that ultimately the page is just going to give me a very simple appearance and let me navigate to any one of those sites.
Now, as clever and cool as some of that stuff is, there comes a point where you say, “We want to do something with search that’s just a little more involved than a little bit of a power user trick.” That’s what this next part of the webinar is about. Now, one of the big ways that we can go a little above and beyond with search is integrating it with other sources. We have a couple of ways of doing that. One way to do that is to actually configure SharePoint so that it crawls external content. A classic example of this, maybe we have a line of business application where we keep data in a SQL server database. Among the things we keep in there, we keep our Part catalog information in there.
It’s actually possible to set-up SharePoint to use business connectivity services to connect into that database, to crawl the database and then to index a series of results that we can then search against. There’s a lot of good things that happen with that approach. We’ll talk about specific pros and cons on that in just a moment, but the idea here is that SharePoint is the thing that indexes and crawls the content and then any integration happens at that moment. At query time, in terms of getting the results back, it’s no different than any other SharePoint search.
Now, an alternative to that is something called Federation or federating the external content. In this example, the SharePoint search engine is not the one that’s doing the crawling and the indexing. That’s actually maintained by some external piece of software. Probably native to whatever repository we’re using. Then, instead of the integration happening at crawl time, it actually happens at query time. That is to say, when we do this kind of integration, even though they don’t realize it, the user is actually issuing two queries. They’re issuing a query in the SharePoint search engine and then on their behalf, it is issuing a query to this external engine.
What are the pros and cons of each approach? Well, first of all, the first one crawling external content. There’s several compelling reasons to do this. One, well, because it’s pre-indexed in the same way that other SharePoint data is, we just get better ranking of the content. In effect, it’s going to be subject to the same rules and the same ranking calculations that any other SharePoint data would be subject to. Another nice thing here, we have control on the frequency of crawl. As an example, let’s say that we are crawling some external data and we only need that data to be up to date within the 24-hour window. Well, we can save a lot of CPU cycles by crawling that information once a day as opposed to having some other mechanism crawl it on a more frequent basis.
This third one is particularly compelling. You’ll remember a moment ago, when I was able to do property specifiers in order to refine my search results, well if we’re crawling external data and we’re allowing SharePoint to do the crawling, we can actually specify those mappings. We can take bits of metadata from this remote source of data and bubble those up as properties that actually can be filtered in our queries. Then, in terms of maintaining the index, it’s just a single index for all of our data, and so our backup and restore operation on that is fairly simple.
The downside of crawling external content from SharePoint, well probably the biggest one would be the fact that we’ve got additional capacity requirements. Now SharePoint is going to have to have enough disk space to not only index its own content, but this external content as well. Not only is that a disk space equation, but also a CPU cycle equation. Beyond that, there is also this challenge that in some cases, we may be talking about external data that for security reasons, simply can’t be crawled. They may have blocked any external access in terms of being able to crawl through and read the data. That would drive us toward an approach where we would do federation instead.
The pros and cons here are mirror images of each other. In the case of federation, if we’re talking about a repository that already has a search engine, that search engine is probably already optimized for the kind of content that we’re searching. Rather than try to re-index it, we’ll just make a call out to the search engine at the query time and say, “Hey, tell us what you know about this.” This is particularly useful in cases where we simply cannot index the information externally because of security.
Then again, the cons here, well one of the biggest cons is that, we’re not going to be able to necessarily interleave the results on a single results page the way we would if we had SharePoint indexed it. More likely, you’re going to have a section of your search page that’ll show SharePoint results and then a section of your search page that will show external results. Not only that, when it comes to this kind of things, security trimming can be really tricky because at this point, you’re making that determination at run time.
Now, the good news, it can be done and in fact I can tell you that here at ThreeWill, we have written integrations on behalf of vendors where they took their products, integrated them on SharePoint and then when we did this run time federation or query time federation, we were actually able to make sure that the end-user still only saw the results that they were looking for. That is something that’s kind of a dark art. It’s not all that well-documented, but we’ve got a ton of experience with that and we’ve been able to help our customers get what they want there.
Now, just as before when I talked about building search-driven applications, not only can we make some trivial customizations to the Core Results Web Part, such as you saw a moment ago, but we can also do some pretty drastic customizations. This is another thing that we’ve actually done on behalf of one of our customers. I’m going to show you a pretty compelling demo of that in just a moment. Beyond that, if you’re familiar with search centers, we also have the ability to customize search center tabs that’ll let us do things like look for results in a narrowed results set or people searches or things such as that.
Another thing that we’ve done on behalf of our customers, creating custom refiners. You’re probably already familiar with the concept that the SharePoint search engine gives you the ability to refine on different characteristics such as the date that the document was created or the format of the document. For certain custom data types, we can actually go in and create refiners from scratch.
Let me just show a little bit about how one might use this. What I’m showing you here is an example of an integration that we actually wrote on behalf of Polycom. You may or may not be aware of the fact that Polycom has a fairly sophisticated portal for keeping up with multimedia content. It’s called the Polycom Real Presence Media Manager. Particularly companies that do a lot of training videos or educational videos, video content of various types. This is a very popular piece of software that companies can use to manage that information.
Now, the product has a portal of its own for accessing that content and it’s really useful and it’s really nice. What Polycom has discovered is that, for a lot of companies, the destination of choice is going to be SharePoint. If we lock up all of our video content over in this vault where people have to visit another site, they’re not going to see it quite as often. Now, you can see on the screen here that we’ve actually created a Web Part that will let us go in and discover that content. I can look at things like recently viewed and so on.
Obviously, the demo that we’re talking about here and we’re more interested in here is this notion of search. Not only have we built a facility for browsing that content, we’ve actually integrated it in search. In this example, what we’ve done is, we’ve changed the XSLT associated with the Core Search Results Web Part so that we can not only interleave the data that’s coming back from Polycom, but also show it in an enhanced way. Let me show you what I mean by that. I happen to know as an example, that there are several videos and even some other information about Stroke. Again, obviously, the sample app here, being a healthcare app.
Now, notice you see some of the kinds of things you would expect. You see a title and you see some kind of icon, but you’ll notice that the information down here looks a little bit atypical. Not only do we have some author information and a date, we even have the duration of the clip. Then, this little link of Video Program Details. This is all custom code that we wrote on behalf of the customer. You can see the compelling story here is that, with one click, we’re able to preview that video in line and then if we so choose, we can even begin viewing it right there on the spot.
I think this is a pretty high-end example of how you can take the out-of-the-box SharePoint search bits, and then do some pretty significant customizations along with that. You’ll also note that not only did we do customizations related to the viewing of the content, but also refiners. Even though this is video content that’s held on another site and there’s really nothing SharePoint-y about it, we were able to supplement this information in such a way that we can refine it based on where it came from, who the author was and other such things. Again, the goal here isn’t so much to teach you how to do that. Obviously, that’s a pretty deep process, but rather just to alert you to the idea that it’s possible and it’s something that we have the experience and the expertise to do.
All right, we’re getting pretty late into our hour here. I’ve got one more section of content that I want to talk you through here, and that is just giving you a preview of coming attractions when it comes to Search in SharePoint 2013. I know that’s probably an over-the-horizon thing for some of you, but it is nice to know what’s coming and what’s to be expected.
Probably the biggest difference when it comes to the architecture of Search in SharePoint 2013, and that is this notion that we now have a single search engine. For those of you with a lot of deep knowledge on this, you know in the past, you’ve got SharePoint Search and FAST, and different versions of those. Now it’s just a single unified product. Really, I think that’s a good thing for the most part in that, it reduces some confusion. When you’re authoring queries now, you don’t necessarily have to be cognizant of, “Well, gee, which version of the search engine am I on?”, in order to be able to do this thing that I want to do.
Now, in terms of the languages that you use or the query languages that you use, keyword query language, which is what I’ve been demonstration for you today is obviously still supported. In addition, in prior versions, FAST had its own unique query language. Even though FAST doesn’t exist as a separate engine anymore, the FAST query language is still supported. The one that isn’t and it isn’t still supported is the SQL variant of being able to do search queries with a SQL LIKE Syntax. I think at this point, that’s one that didn’t get used a whole lot and that’s why Microsoft made the choice to drop that. If you’re under the sound of my voice and you’ve got some things that you’re doing in SharePoint that are dependent upon that dialect, then just be aware of that. That’s something that’s going to go away when you get to SharePoint 2013.
Some other things and some of these are technically meaty. Others of them are more marketing points, but they’re worth making. There’s some improvements to the results ranking model. Another one that I think is really cool, and that is the results are further refined based on click-throughs when the results came up before. Those results that have a higher rate of click-through, will show up as being more important on subsequent queries.
Another thing that’s new, a moment ago, I’ve talked to you a little bit about federating outside sources of data. That’s gotten a new name now. It’s called Result Sources. It’s really a way of just describing external data. Probably the biggest thing here that would be of use to developers is the fact that the way that we do security trimming here is a little more elegant and a little more sophisticated. In terms of being able to write customizations, that’s a good thing.
Now some other things, some refinements and changes to query rules. We have this thing called Promoted Results. This was formerly known as Best Bets. We’ve got a little more flexibility on how that we use that. We have this other thing called Result Blocks, and I’m actually going to talk about this one a little bit more on a later slide. The idea with result blocks is that you can use it to group related results. As an example, let’s suppose that somebody searched for a particular kind of content and maybe we discovered that based on certain words. Probably they’re looking to highlight some things related to Power Point presentations. Well, we could actually set up a separate result block that says, in addition to all the results we’d show them, we could actually use a result block to call out just the Power Point presentations that are in those search results.
Again, if you think about where you visited commercial search engines like Bing or Google, and they often have a way of saying, “Well, gee, you searched for this thing, but there’s a whole bunch of images that match. Would you like to see those?” That’s exactly what result blocks are all about. Another thing that’s kind of cool is that, you can actually boost rank results based on certain terms. As an example, suppose that someone goes in and types in a term to download a utility, we can actually configure a query rule that says download is recognized as an action term and based on that, if we have a particular site in the company where people are supposed to be getting their downloads from, we can actually boost that result and that would result in a likelihood that it would be higher in the result stack.
Another nice innovation here, Continuous Crawl. Again, those of you that have worked with SharePoint a long time, you know that there’s always this challenge of, you want to set your crawls frequently enough to be up to date, but not so frequently that they killed the server. Now we have this concept that, for particular sources of data where it absolutely has to be up to date, continuous crawl really is just tapping into the transaction engine from the underlying database and saying, “Anytime we have new content, we’re just going to go ahead and update the sear- index immediately.”
Now, let’s talk for a moment about a hybrid environment. This is something that I think a lot of you, particularly if you work for big companies, you’re going to encounter over the next year or two. It’s interesting not only to identify it, but to talk about some of the search implications of that. When I say hybrid environment, what do I mean? Well, we all know that Microsoft is making a big push to move customers to the cloud, to begin using SharePoint Online as a part of the Microsoft 365 Suite.
Now, for most companies that have a significant amount of content, it’s probably not going to be possible to move exclusively to the cloud in a short-term. There’s really two big reasons for that. Number one, they’ve probably got customizations that wouldn’t run in the cloud particularly if they’re farm solutions. Number two, it’s probably not practical to take the gigabytes of data that they already have in place and simply move those up to the cloud in order to sit there. It would be very expensive to move them up there and it would be an expense to have it up there in an ongoing basis.
In order to combat those barriers, a lot of customers are using what’s called a hybrid environment. In simple terms, they’re going to take everything that’s on-premise and for the most part leave it there. Then, new content is going to be put into the cloud and used there. Now, the presumption is, if we are going to have such an environment, we really need to be able to integrate the search results from those two environments. To say that a little bit differently, we don’t want users to have to constantly say, “Well now, wait a minute. Is this thing in the cloud or is this thing on-premise?”
The idea here is that, we can use the result sources feature that I mentioned just a moment ago in order to integrate those results. How are we going to use that? Well, suppose that we’ve got an on-premise search page. What we can do is create a result sources feature and then a result block that highlights the matching cloud results. They may not be interleaved one for one, but what we’ll see is a section of highlighted results. Then, if someone observes, “Well, gee, the results that I think are relevant to me, there’s more of them on the cloud than there are on my on-premise.” They can actually click on one or more items inside of that result block and it’ll navigate them to the cloud. You can also do the same thing in reverse. You could set up a cloud search page so that it not only shows cloud results, but also has a results block that highlights on-premise.
I can tell you that we are actually doing a proof of concept of this as I speak for a company here in Atlanta. Fairly large company that you would recognize and we’re getting some really good results with that. Really kind of two good news fits there. Number one, it’s doable and it works. Number two, when you find yourself at a point where your company needs to do that, we’ve got the expertise to help you get that done.
Some other things you’ll want to watch for in SharePoint 2013. Some of these are just features that we’re going to like and take advantage of. Some of them are things where we actually have opportunities to customize. We now have this concept of a hover panel for particular content types that show up in search. You can actually hover over those and preview the results on a mouse over. We also have the ability to create our own hover panels. If I were to go back to what I showed you a moment ago on the Polycom integration, once that products gets updated to SharePoint 2013, it would be a doable thing for us to go in and write a hover panel where instead of having to click to view the video, one could simply float the mouse over and see a preview of the video playing. I think that’s a pretty cool one.
Another nice thing of SharePoint 2013 Search, if you look at the search pages I’ve been showing you up to this point today, the icons and the graphics associated with the results are pretty bland. They don’t really communicate a lot of information because they’re tiny and somewhat nondescript. Microsoft has tried really hard to put some good visual cues on your search results so that it’s real obvious, “Oh, that one’s a Word doc, and that one’s a diagram and that one’s from Visio and another one is an image.” I think that is a much needed improvement.
Some other things. I know that not all of you take advantage of the term store and the managed metadata capabilities of SharePoint 2010, but if you do, there are some greatly improved capabilities there in terms of being able to navigate and find items. Another improvement that we have, this concept of display templates. Now, you’ll remember that I’ve talked twice now about changing the XSLT in the Core Results Search Web Part either on a particular page or in a search center. Given the difficulties associated with that, it’s nice now that we have an alternative to that.
Really, what are we saying when we say display templates? Now we’re able to build what are in effect, customizable little nuggets of HTML. We’re going to be able to associate these in such a way that when a particular result set comes up, we can actually modify how it’s going to look on the results page. The nice thing about this is that, because it is associated with particular types of content, we can do it in a search center and do it in such a way that we won’t collide with other customizations. It’s more flexible, it’s more powerful, it’s less error-prone, it’s easier to do. That’s nothing but a win when it comes to doing this.
Now, the various parts that make up this, there’s really two things. First of all, you have a controlled template that’s going to describe the header and the footer and the layout of the results list. Then, an item template that we’ll use to describe the presentation of the items in that list. Those of you that do a little bit better with pictures instead of words here, this gives you an architectural representation of what we see. Then, this is a nice screenshot that shows you how you could do a search results page with an image slider. You’re able to just simply scroll back and forth through the images used and a little bit of jQuery in addition to the HTML template to show this.
If you are in a place where you do customization of search, your world just got a lot bigger. Conversely, if you’re a consumer and you need these kind of capabilities, it’s nice to know that it’s finally available in the product, and that you’ve got some folks in ThreeWill that know how to make that happen for you. Let’s talk about some takeaways real quickly, we’re getting close to the top of the hour here.
First and foremost, if you’re a developer and you’re under the sound of my voice, you need to start thinking about search-driven applications. If somebody describes a business problem to you and the answering of that business problem can be done from the search engine, it’s faster, it’s more efficient, it’s easy to write, there’s a lot of wins there. Obviously, if you’re doing a traditional data-entry app where you need to add items to a list, that’s not necessarily a search-driven application. Anything that involves asking questions. This is a good candidate.
Another thing to think about, beyond just the building of these simple search applications, if your company is in a place to make a higher technical commitment, there are a lot of other things that we can do. Customizing the search results page, writing refiners, integrating external data. Another thing to look forward to, SharePoint 2013 really offers us some much better opportunities to integrate at a deeper level. Better security trimming, better models for extending the UI, and doing those things in such a way where it’s not going to be brittle or prone to breakage if somebody else tries to do a customization as well.
Then, obviously, what I want you to take away from that is, this is what we do. It’s not that we’re going to do these things. We have already done these things on behalf of our customers, and we’ve had some really good results with that, and we want to help you get those same results as well.