azure-search-primer.jpg

Azure Search Primer with Eric Bowden

Danny Ryan

Host – Danny Ryan

Bio – LinkedIn – Twitter

Sam Marshall

Guest – Eric Bowden

Bio – LinkedIn

Key Points

Four Pillars of Azure Search:

  1. Create a Search Service
  2. Create indexes and fields
  3. Push content into the index via the API
  4. Search the index via the API

Conversation Highlights

  • Adding Search Ability  – 2:36
  • Security and Security Trimming  – 8:59
  • How to start with Azure Search – 13:22

Danny :Hello and welcome to The Two Bald Brothers and a Microphone podcast. This is one of the bald brothers, Danny. And I’m here with someone from ThreeWill. I’m here with Eric Bowden. How are you doing, Eric?

 

Eric:Oh, I’m doing fantastic, Danny. How are you today?

 

Danny :I’m doing great. It’s a Friday afternoon. All is well. I can’t complain. It was a great week, and so just looking forward to the weekend. I’ve got a date tomorrow night. Amy and I are gonna go back down to Buckhead and go where we used to live, and go wander around. Go do a little bit of shopping. I’m looking forward to that. How about you?

 

Eric:All right. Time in the old neighborhood.

 

Danny :Yes.

 

Eric:Oh, it’s gonna be a weekend of rest and relaxation.

 

Danny :I love it. I love it, I love it, I love it. What I wanted to talk to you about is Azure Search. Now, we’ve done plenty of content on the website about SharePoint Search. I’d like to understand more about what you’re doing. Why did you start wanting to learn about this? Let’s get started with just sort of what prompted going and learning more about Azure Search.

 

Eric:Sure, sure. Yeah. You know, of course, Search is a great feature of SharePoint. It’s a strong feature. The Fast search engine became part of SharePoint in 2010. You know, there’s a lot that can be built on Search and SharePoint. We’ve had projects which are really search-focused projects in SharePoint. That is the only exposure that I really have to the fast search engine. I understand that it is a technology that precedes its usage in SharePoint, but it’s a well-respected feature in SharePoint. Then I’m hearing, okay there is the fast search engine. There is Azure Search, which is a lot like the search that you’re accustomed to seeing and working with in SharePoint. And that really peaked my interest. I’m saying, “Well what” … SharePoint Search already has a great reputation with me. It’s a highly used feature, but what is this Azure Search? I’m hearing it’s a lot like SharePoint and I want to find out more about it.

 

Danny :Awesome. Awesome. How did you get started with learning more about it?

 

Eric:With my favorite go-to learning source, which is Pluralsight.

 

Danny :Awesome.

 

Eric:I head right into Pluralsight and found a great course called Adding Search abilities to Your Apps with Azure Search by Chad Campbell. This was a great course. Provided a really good high-level and a deep understanding of how I can get started with Search in Azure.

 

Danny :Cool. Let’s dive into it. Let’s maybe start at the 25,000 foot level. We’ll call it that. What were some of the things to get started with Azure Search? What were some of the things that you picked up?

 

Eric:Sure. Yeah. The good news is that, for one thing, if you are familiar with SharePoint Search, Azure Search will be equally familiar. There’s some translation of terms. Some things, you know, seem to have been re-named, for example in Azure, they use the word score, whereas I’m more used to hearing the word ranking in SharePoint. They use facets in Azure, where they use ranking. I mean, excuse me, where they use refiners in SharePoint, so there’s some translation to occur. But in large part, it’s very similar to SharePoint.

 

Now, I start off by sort of building, you know there are really four pillars of knowledge with Azure Search. The first is there is, you’re gonna create a search service in Azure. You go in, create a search service. It’s actually a fairly minimal set of inputs to create a search service. You’re gonna give it a name. You’re gonna give it a resource group and a choose the pricing, the sizing of your Azure Search. A few other fields, but it’s fairly minimal. There’s your search service, pillar one.

 

Pillar two is you’re gonna create indexes, which in SharePoint we’re not used to creating indexes. In SharePoint, you have one index, but in Azure Search you could create more than one. Then from there, you’re creating fields for that index. This is equivalent, if you kind of translate over into the SharePoint world, this is the equivalent of configuring managed properties, or crawled properties and managed properties. It’s really both of those you’re doing in Azure Search. You’re configuring your fields. At that point, you have a search service and you have an index.

 

Then the third step is you’re gonna need to get content into your index. That’s accomplished with the API. There’s an API call. You’re gonna push content up into Azure. There is also, there are indexers. There’s indexers capability, which on the SharePoint world you would know that as a crawler. Then finally, that leads me to the fourth pillar, which is searching the index. You’re gonna execute API queries against the Azure Search service to find your results. And of course, all four of those topics are very rich. They’re, certainly indexes and searching are very, very rich and you can go very deep in what Azure provides for you. I think that a really good key takeaway for somebody who is familiar with SharePoint Search and are thinking, you know, “What is this Azure Search thing and what are the complexities and really how can I get into it?” Just keep in mind those four sort of pillars of knowledge there.

 

Danny :Awesome. Are you indexing SharePoint? Is it just a website that you’ve created or an app? What’s some of the typical things for you to index?

 

Eric:It can pretty much be anything. I am at the beginner, the intro level of knowledge myself with this, but you know, the way that I see Azure Search being used versus SharePoint Search is Azure Search is I think gonna be appropriate for these sort of isolated solutions. It’s a solution that has an isolated set of data. It’s not, you know, part of your internet. It’s not part of an aggregate. You know, your enterprise fortress of content. You can push any record, so it can be documents or it can be records with fields. Then also, of course, you can configure the indexers that I talked about, so you can crawl content sources. It can crawl sequel databases, cosmos database, blog storage, a couple of other content sources. It’s a variety of content that you can push into Azure, into the index.

 

Danny :Then, I imagine you have some way of scheduling when the crawls occur and whether they’re incremental or full crawls, and those types of things?

 

Eric:Yes, exactly. Very similar process, as you’re familiar with in SharePoint. You can schedule the indexers, you know when those are gonna crawl over your content sources.

 

Danny :I wonder, how does this fit into … I guess if somebody is using this Azure service and they’re indexing content, and I guess for customers who might be, let’s take SharePoint on prem, and they’re on prem for a certain reason, where they want to keep that data within their environments. This wouldn’t be a solution they could go with because it’s you’re actually pulling that content up into Azure. Am I reading that right?

 

Eric:Yep, you’re right. Yes. It’s a Cloud service and you are pulling. What you’re pulling into the Cloud are those elements that you want to search. You know, which most likely, if you have concerns about data and you’re keeping that data on prem, most likely the aspects of it that you want to search are also sensitive. But it is, yeah it’s a Cloud hosted solution in Azure.

 

Danny :How does security and security trimming fit into this?

 

Eric:That is a good question and something that I haven’t fully explored yet. I was actually discussing that with the team yesterday. You know, there are two aspects of security to keep in mind with Azure Search. The first is your access to the services. Access to the service itself is covered using access keys. A number of services, Azure functions is one that I’m familiar with in Azure, that support the idea of access keys. Essentially it’s a course access mechanism, where you can create these access keys, which simply give you access, you know, to call those services. But then the next step is gonna be saying, “Well, what about the specific items in the index? How are the specific items security trimmed?” Folks that are accustomed to SharePoint, we’re accustomed to, you know the SharePoint Search index being security trimmed. You execute a search in SharePoint, you don’t see results back for content that you don’t have access to. I don’t know what that story is yet in Azure. You know, I would imagine that, I think you’re on your own, but I don’t know that for sure.

 

Danny :Okay.

 

Eric:You know, where you’re gonna need to be coding in either a mechanism to store the access control list that’s part of the search index, so that’s sort of a front-loading method. Or the other side is, you know, when you retrieve those contents, you’ll have to have a mechanism to decipher if the users have access to it.

 

Danny :Any projects that’s from your past, now that you’re starting to learn about this, where you see where you might have used this on the project? Or maybe taken a closer look at this?

 

Eric:Well, sure. I mean, a recent project was a knowledge-based solution built on SharePoint. It was essentially a search-focused project. A lot of filtering, refiners. A lot of logic. We were doing promoting of results. Increasing the ranking of specific results and doing some tuning, and so forth. This potentially could have been a good candidate for that, and there are a couple of reasons for that. One reason is that the solution didn’t have deep security requirements, security trimming requirements.

 

Danny :Gotcha.

 

Eric:You know, we needed to secure content in buckets. For this project, they were by client. It wasn’t down to the individual level. You know, the security trimming aspects wouldn’t have been an issue. But then also, you know, this solution, the knowledge-based solution, it was a very specific built application for this knowledge-based purpose, for a mission critical purpose at this company for our client. You know, looking at building it on SharePoint, SharePoint is more of an enterprise-wide internet solution. This might have been a better fit in Azure because it kind of takes what is this purpose built mission critical application and kind of puts it into its own isolated environment, where you can really tune things and you can make configuration changes that only affect this application. It has no possibility of affecting the rest of your SharePoint environment.

 

Danny :Gotcha. Anything else as far as getting started with Azure Search? I mean, you mentioned Pluralsight as being a great resource. Anything else as you’ve started to take a look at this?

 

Eric:You know, I just recommend just dive in. When you get there, there is a, of the pricing tiers, there’s a free pricing tier for developers. As you’ll see from the Pluralsight course, and there are other walk-throughs that you can find. Postman is a great resource for executing those search queries.

 

Danny :Gotcha. Wouldn’t the search … What are you writing search queries in? I mean, is it a certain … What is that?

 

Eric:It’s actually, it’s called the Lucene search syntax. Well let me take a step back. There are actually two.

 

Danny :Okay.

 

Eric:There’s what they call, there’s a simple query language and there’s a full text, a full query language. Which to kind of translate those onto the SharePoint side, you know you might be familiar with Keyword Query syntax or Keyword Queries. And then there’s Full-Text Query. That’s what we had in SharePoint. Then, you know, in Azure we have what they call Simple and they call Full. That Full syntax is using what they call the Lucene Search Syntax, which Lucene is a search engine that goes back a number of years that has eventually been ported into, I guess into dot net. Then we’re able to access it through Azure.

 

Danny :Lucene, was that like an open … I thought I’ve heard of that in the past, like an open source search engine. I could be wrong.

 

Eric:Right, right. Yeah. It was actually, it started, I think in the late ’90’s, ’99 or so. I think was originally a Java-based project, and then it was ultimately open sourced and ported to a number of, probably half a dozen or so, platforms.

 

Danny :I thought it was originally a show – like Lucene and Shirley, right?

 

Eric:Exactly. Well, it’s pretty funny. I was even taking another direction. I was looking up … I was trying to understand the history of it myself, and I misspelled it. And there is apparently a protein called lucien, if you spell it a little bit differently. And believe it or not, there is a search engine out there for the protein called Lucien. I don’t know if they use that.

 

Danny :Nice, nice, nice. Well I think this was great. This is great for getting us started off. I appreciate you taking the time to do this, sir.

 

Eric:Sure.

 

Danny :Absolutely. Thanks, everybody, for listening. Have a wonderful day. Hopefully good luck with learning more about Azure Search. And have a wonderful day. Thank you so much. Bye bye.

 

Additional Credits

Podcast Producer – Oliver Penegar
Intro/Outro Music – Daniel Bassett

read more
empty.authorAzure Search Primer with Eric Bowden
Search.jpg

Find Anything in SharePoint with Amazon-Like Faceted Search Webinar


 

Danny:

 

Hello everyone, and welcome to this webinar on finding anything in SharePoint with Amazon-Like faceted search, wow I got that out.

 

This is Danny Ryan, I’ll be your host for today, and I am the co-founder and VP of Business Development for ThreeWill. I have here with me a partner, Ross Leher, he’s the CEO and chairman of WAND INC. I’ve also included in on this conversation as well Bo George. Bo is a principal consultant for ThreeWill. He leads up our portals practice lead. I wanted him in on the conversation as well.

 

The way this is set up today is, it’s set up as a conversation. I wanted to have Bo just be able to chime in, maybe on some recent project experience, or some things that we’ve seen, and have him chime in that way, and look forward to showing this to you guys today.

 

Thank you everybody for joining. A couple of logistical things, before we jump into the content … we will be recording the webinar, so you can share it with colleagues. We’ll send a link out to it, so look for a followup next week, with the link. I think we might also be putting this on the podcast as well.

 

The deck for the webinar is available, it’s in the downloads panel … let’s make sure we’re all set there. You should be able to see … actually, it’s in the handouts. You can see in the handouts panel, you can download it from there, a PDF of the presentation.

 

If you’ve got any questions at all, I’ve got our producer here, Oliver is behind me. Today’s Oliver’s birthday, it’s his 21st birthday, happy birthday Oliver. He’ll be handling your questions, or if you have anything you’d like to chat about as well, he’ll keep an eye on that.

 

You’re not drinking yet, are you? You’re 21 years old … not til tonight. Okay, he’s going to wait til tonight.

 

So if you’ve got anything there, just let us know, and we’ll respond to the questions towards the end of the webinar. So that gets us kicked off with a little bit of logistics.

 

The agenda for today … what we wanted to start off with is our perspective and the backstory on this, which is why we wanted to share more information about what Ross and his company does, and why we think this is so important to our customers and to people in general who are using SharePoint and things. So I wanted to give a little bit of a backstory to get us started out with the webinar today. The next part, I’ll let Ross take over, and he’ll help define for us what the enterprise need is, around findability. I also want him to give a little bit of a background on sort of how WAND fits into the picture, what they do. And then, for some people this might be a new topic for them, as far as what taxonomy is, so I just want to give sort of an overview of what that is, and really just be able to make sure that we’re all using the same vocabulary when we’re talking. And then, sort of looking at what this overall webinar’s about, we’re looking at trying to create something, like Amazon-like faceted search.

 

We all know, the net time you go look for a new pair of shoes within Amazon, it’s amazing how quickly you can narrow it down to the correct ones. So we wanted to say, “How can we do something similar with the content that we have on our intranets, inside of SharePoint?” And then he’s going to go through and just share a little bit about the business functions in industries, the different taxonomies for those, and then I’ll ask him to do a demonstration of their taxonomy portal.

 

And then from there, sort of like, “How do you get started with all this stuff?” There’s a two-to-three week quick-start project, which we’ll talk a little bit about, and what does that look like, and how do you get this whole thing started out in the right way?

 

And then we’ve got a case study that we’d like to review about Goodwill industries, and review what some of the things were that we found out from that case study.

 

Then we’ll wrap it up with, I’ve just got some conclusions, just some things, what do I want everybody to walk away with from this webinar, and just some of the conclusions. At that point in time, we’ll go through and see if there’s any questions that folks have, and we’ll answer the questions at that time.

 

I’ve got it open right now. I like sort of conversational. I’ll hand it over, after I’m done with the backstory, over to Ross. But Ross or Bo, feel free to jump in, if there’s anything that you’d like to add to the conversation.

 

Hey Bo, how’s it going?

 

Bo:Good, good. How are you, Danny?

 

Danny:Good, good, good. And Ross, you’re there?

 

Ross:We are here in the mile-high city of Denver, Colorado. It’s a beautiful day.

 

Danny:It is? Okay, awesome. Great to have you guys here.

 

Let’s get started with the backstory. For folks who are on the line, I ended up inviting a lot of people to this who are current client, or folks that we have interacted with through the years. Most people know we’ve been working with clients on building intranets and extranets on SharePoint for a while now. It’s been a while.

 

One of the most common complaints that we hear from people is, “I can’t find anything in SharePoint.” Bo, you hear that quite a bit?

 

Bo:Yeah, for sure. I was thinking, as soon as you have two people working on a single document, you’re going to have the, “I can’t find anything in SharePoint.” That’s how simple it is, is two people working on something in SharePoint.

 

Danny:Never mind an organization with thousands or hundreds of thousands of people.

 

Bo:Right.

 

Danny:So the taxonomy is going to enable ways for us to present and find content. Bo, I know you wanted to specifically call out these things: search, navigation, automatic tagging, and document routing.

 

Bo:Yeah. When people talk about, “findability,” immediately you gravitate towards, “search,” and that’s kind of the most obvious thing … conversational search, your Amazon scenario, is clicking things and refining your results. So that one’s a huge one with taxonomy. The other thing that I think taxonomy enables in SharePoint that people don’t think about as much is navigation, which could be search, or it could be other things. And then automatic tagging, so where I put a document, it could be automatically tagged with terms and taxonomy stuff, which reduces the need for people to fill out metadata, because it’s done for them, auto-magically.

 

And even document routing features in SharePoint, which I think are awesome, like … I upload it, I tag it with something, and it puts it in specific places, so that it’s organized based on how I tag it.

 

So it’s not just search, but that’s a huge part, but it also enables all kinds of other features in SharePoint.

 

Danny:And all this is going to be missing if you don’t have a common taxonomy that’s been defined.

 

Bo:Exactly, yep.

 

Danny:You seem as though this is a daunting task, ’cause this is not typically something that we learned in school, or people taking this on. I just wanted to mention, I know a recent example where you were working with a client and asking for, “Give us your top 20 document types,” and even that was a difficult thing for them to go through.

 

Bo:Yeah, and it’s been my experience, not many of us have graduate school with a degree in Information Sciences, that talk about classification systems and things like that, that’s just not our day job. So when you say, “Give me your top 10 document types,” on the surface it sounds like an easy question, but it may not be.

 

Some customers will just punt on it and not do anything, which is maybe the worst thing to do. The other flip side is, I’ve worked with customers that, they’ve taken on that daunting task, and it’s taken them years, potentially, to define a taxonomy, because they started with a blank slate and di all that work.

 

So it can be daunting to the point where you just don’t do anything, or it just takes forever. I’ve seen both sides for sure.

 

Danny:And Ross, not everyone’s like you, where they eat taxonomy for breakfast.

 

Ross:Well, it’s not that difficult, but we’ll have some fun today. Look forward to it.

 

Danny:Awesome, appreciate it.

 

So starting with what we found, and this is a ley thing from meeting up with Ross, is really understanding … if we can start with a pre-defined taxonomy, a lot of … you just do this in general, consulting-wise, instead of starting with a blank slate, having something that is in initial skeleton for you to work with, or a straw-man that’s out there, it’s really a way to make this a reality.

 

This is why we wanted to … we didn’t have this webinar initially scheduled for this year, but as soon as Bo and I met up with Ross a couple times, I was like, “We’ve really got to share this with folks in the ThreeWill community.” That sort of brings us to where we are today, and why we wanted to share what Ross has for you.

 

Ross, I’m going to go ahead and switch over and make you the presenter. I’ll give you control, and if you want to turn on your webcam, that’s fine as well. It’s up to you.

 

Ross:Sounds good, I hope I’m not dominating the screen too much.

 

Danny:No, you look great.

 

Ross:Well, thank you.

 

Danny:You don’t look that great. Wait a minute.

 

Ross:Listen, first things first. I want to thank Danny and his team for organizing everything today. It’s terrific. Hopefully we’ll have a little bit of fun today.

 

The Enterprise Need – The Pain Point. What the heck are taxonomies, and what does it mean to all of us? The problem, I think as Danny just talked about, is search. We can’t find stuff. If we think about the physical assets of an enterprise, we have inventory control systems for all of our physical assets, we know where every component is, if we’ve got a manufacturing line … all of the physical assets, they’re on our ballot sheet, real estate, equipment, so on and so forth.

 

All the physical assets of an enterprise have been tagged and classified. Why? So we can find them. The macro-problem, 75% … and this is probably understated, 75% of information workers say that finding the right information, not the physical assets, but the intellectual, the information assets, is critical to the organization’s success. We would suggest taxonomies as one of the key success factors for effective search and findability.

 

What are the obstacles to finding our information, our intellectual assets well, everyone knows poor search functionality, inconsistency in how we tag content. Lack of adequate tags. We waste a lot of time when we can’t find the documents we’re looking for, and we get very, very frustrated. When we can’t find the documents that we’re looking for, we either do without them, or we try and recreate them. But in every case, it’s extraordinarily frustrating. So how do we solve this problem?

 

Little bit of background about our company first. WAND has been building taxonomies since 1983. We began to build taxonomies a long time ago, because we had large insurance companies here in the United States that we provided software programs to, that required taxonomies. Taxonomy libraries that we’re going to look at today, there’s 128,000 curated terms, address virtually every industry vertical and all of the operational areas of an organization. For some of our clients, we’re necessary. We provide taxonomy professional services. Our client base all over the world, taxonomies can be used in SharePoint, but they can be used in big data applications, eCommerce cataloging … search, as we’ve discuss, BI analytics, the list goes on and on and on.

 

Out of the box, the taxonomy term store in SharePoint is empty. There’s nothing in it. WAND provides taxonomies that are formatted for direct input into that empty term store. But the enterprise may have many, many applications into which taxonomies can be imported, BI analytics, big data, the list goes on and on and on. We format the taxonomies for direct import into those various applications.

 

Taxonomy is a mystery for most people. I think most of the folks on this call know what they are. I hope I do, but we all face the burden of explaining to people in our companies what the heck that word, “taxonomies,” means, and what difference it makes to what we’re doing.

 

Say, for the sake of argument, we’ve got a company that’s got 2,000 employees. I’m going to put those 2,000 employees into a room, and I’m going to ask them the question, “Everybody that knows what a taxonomy is, raise your hand.” Not too many hands go up.

 

Then I’m going to say to those folks, “Everybody that’s used Amazon.com, raise your hand.” Now everybody raises their hand.

 

I’m going to say to the people, “When you do a search for the word, ‘shoes,’ on Amazon, you’re going to get 550,000 results. On the left-hand side of the screen, are taxonomies. A taxonomy of size, I want a size 11. Taxonomy of colors, I choose black. Type of shoe, I select men’s dress shoe. Taxonomy of brands, I choose Allen Edmonds. 550,000 results, narrowed down to the five that are relevant to the user.”

 

I say to those 2,000 people, “As long as you’ve been using Amazon, you’ve been using taxonomy. You’ve just never put that word to it before. It didn’t matter whether you were searching for a pair of shoes, a computer, a television … the taxonomy’s always on the left-hand side of the screen, to enable you to find the results that are relevant for you.”

 

I’d say to the folks, “Think about the feeling of control we have over our search when we use Amazon. We know we’re going to find what we’re looking for, because those taxonomies are there to help us navigate the result sets. Imagine having the same feeling of control over the search for our enterprise documents.” This is what the WAND taxonomies, in conjunction with SharePoint, enable.

 

The analogy is a very good one, and if you think about the problem that Amazon had when they started, they had stacks of shoes, stacks of computers, libraries of televisions, so on and so forth, and they needed taxonomies to be able to tag those various libraries of merchandise to enable that taxonomy-driven search.

 

The enterprise has the same problem. Instead of libraries of shoes, computers, televisions, the enterprise has libraries of HR documents, accounting documents, IT documents. If you’re in the building and construction business, building and construction documents, so on and so forth. And WAND provides taxonomies to tag those various libraries of documents.

 

The analogy is another good one if we think about a content type. We might think that, “shoes,” is a content type. If we have a library of HR documents, let’s say they’re compensation documents. We may say, “HR compensation documents.” That’s a content type.

 

So, remember that analogy. It explains everything.

 

One slide we like to show folks, you can show this to the business side, the technical side, and people immediately get it. Everybody knows what the Amazon experience looks like. Do a search, you’ve got refiners … let’s see what this actually looks like.

 

In this case, Amazon decide they’re going to have a library of shoes. What we decided is, we’re going to have a library of HR documents. A library of documents, nothing confidential, just documents, insurance plans, company handbooks, time off forms, so on and so forth. And when we check documents into that library, we tag them with the appropriate taxonomy terms. The same terms that we use to tag the documents, navigation trees, in the search process.

 

When we do a search for the term, “HR,” in that library, you can imagine we’re going to get everything on the left-hand side of the screen, just like Amazon. There’s the refiners … and I need an expense form. Instead of calling my manager, instead of calling HR, I can now find all of the documents that are relevant, open-source documents for employees.

 

The reason we always like to start with a library like this is because it’s a success story, and it’s relevant to every individual in your company. We like to break taxonomies down even further to their absolute basic levels.

 

We would suggest taxonomies are the data model for unstructured text. Now I’m going to say to everyone, there’s no such thing as unstructured text. If I put a ThreeWill brochure on the table, if I put a Wall Street Journal article on the desk, everybody’s saying to me, “Ross, two perfect examples of unstructured text, what ar you talking about?”

 

And I’d say to everybody on the call, “Every one of us can read the ThreeWill brochure, we understand it perfectly. We can all read the Wall Street Journal article. It’s not unstructured text for the human brain.”

 

So make a small modification to this first statement. Taxonomies are the data model for unstructured text that we can use in computer programs like SharePoint, to tag documents in a structure that replicates how the human brain organizes the domain of information.

 

We would suggest implicit taxonomies are how the human brain organizes information. These implicit taxonomies have been developed ever since the day we were born.

 

I have a five year old granddaughter. The first terms she knew were the terms, “wet,” and “dry.” The terms, “hungry,” me stomach is full. She didn’t put words to those concepts, but from the day she was born, she started organizing her knowledge base.

 

She’s five years old. She knows there’s a top-level term of, “food,” a narrower term, “fruit … apples, oranges, bananas … ” Narrower term to food, “Baked goods, cookies, brownies.” She’s organized that domain of information, “food,” into logical, broader, narrower, narrower, narrower terms.

 

We do the same thing with geography. Continents, countries, states, cities, districts. And we do the same thing with all the enterprise domains of information. The explicit taxonomies that WAND has developed are simply representations of how the human brain and enterprises organize information. These explicit taxonomies are used with enterprise applications like SharePoint for tagging documents, for taxonomy-driven search and libraries, and of many, many other applications for text-to-knowledge purposes.

 

We want those 2,000 people to say to themselves, “Boy, four or five minutes ago, I had no idea what the heck a taxonomy was. But I’ve been using the all my life. Been using them when I use Amazon. Heck, I’m going to the grocery store after work tonight, there’s taxonomies everywhere … meat department, dairy department, milk, cheese … different types of cheese. There’s no mystery about that word, ‘taxonomies,’ except one … why have I never used it before, to describe how information’s organized?”

 

The reason I spend time on this, four or five minutes, is because we find, within our clients, when everybody understands, it eliminated the mystery of that word, they see, “Boy, we’re going to a great search, it’s going to be just like Amazon.” It creates a tremendous amount of momentum, and it makes your jobs a lot easier to implement a great search, and to implement these types of processes within SharePoint.

 

We take every document, we’ll say [inaudible 00:28:09] your telecommunications company. If we take every document in your telecommunications company, we say, “We’re going to put al those documents, we’re going to put them into one room, just like Amazon did. They decided, ‘We’re going to put those into stacks, we’re going to put them into libraries.’ We’re going to have libraries of accounting, environmental, HR, IT, libraries of documents, every organization has, WAND provides taxonomies that address all those libraries of documents.

 

Also going to have to have libraries of documents that address the business activities of your organization. Telecommunication documents, a lot of telecommunications companies have retail store, sell retail stores. Going to have real estate documents, going to have building and construction documents, every company build and constructs assembly lines, remodels, whatever it might be.

 

In a nutshell, WAND provides taxonomies that address all of the domains of knowledge for an organization.

 

With that, we’re going to go to a demonstration of our WAND Taxonomy Library Portal. What this provides is online access to all of our WAND taxonomies, our client can browse, download the taxonomies in formats that can be directly imported into SharePoint. We provide regular updates, release new taxonomies on an ongoing basis, and the library can be licensed for a single application, or it can be licensed for any, many applications that the enterprise might have.

 

I’m assuming that everyone on this call has an understanding of where taxonomies reside within SharePoint, and that’s the taxonomy term store. We’ll go to the taxonomy library. Each one of our clients gets a username and a passcode. This particular user, Ross Leher, is licensed to download taxonomies that can be imported into these various SharePoint applications. Just for context, we format the taxonomies for about 100 different enterprise applications, BI analytics, big data, the list goes on and on and on and on.

 

We partner with many of these application providers, probably our highest-profile partner, WAND provides taxonomies that can be imported into the [inaudible 00:30:34] engine applications.

 

Taxonomies improve the performance of many, many enterprise applications, and they’re a very, very necessary ingredient. What our users are able to do is, go to the taxonomy library and browse the various taxonomy titles.

 

Because there’s so many of them, we’ve grouped them into industry verticals. We’ll take financial banking organization, investment bank organization … we’ll say to them, “This is a list of all of the domains of knowledge within your organization. You’re going to have real estate documents, libraries of procurement documents, libraries of finance and investment banking documents, customer service … every organization provides health insurance, life insurance have to have casualty insurance, accounting documents, libraries or facilities management.”

 

These represent all of the libraries, all the domains of knowledge. And always start with IT for the demonstration. I’ll explain to you why in a moment.

 

User is able to click into any one of the taxonomy titles and explore the various taxonomy trees. Numbers of terms in a taxonomy hierarchy on the right hand side, in parentheses.

 

We’ll drill down on IT administration. Many of our clients are using the second and the third-level terms as a basis for setting up libraries. I want to set up a library for all of my IT policies, make sure we’ve got them all. When we set that library up, these are the terms that we’ll be able to tag documents with, when they’re checked in. And just like that HR search we were looking at, these are the terms that will be used as the refiners on the search process in SharePoint.

 

Some of our clients will come back and say, “Ross, we didn’t have a smartphone policy, we didn’t have some of the access control policies,” soon and so forth. We began to checklist, because in a lot of cases, they had a lot of these policies, but they were in this file share, they were over here, they were someplace else, or they hadn’t formalized them, or they hadn’t updated them. So a great byproduct, it became a wonderful checklist to make sure that they did have everything.

 

The reason I always start with IT, I make the argument, and every day when you read the newspapers, IT security, most important taxonomy for any organization. We don’t have documents, we don’t have workflows … if we don’t have expertise within the organization that we can tag with these terms, we may want to take a second look at our IT security policies.

 

What the user is able to do is explore the various taxonomy titles. We can then highlight at any level, and whatever we highlight, we can click on Download. Or, if I want to take all of IT administration at one time, I highlight that, click on Download … select the application into which it’s going to be imported. I confirm it.

 

Starting to add multiple languages select our language. Click on Continue To Download, and what’s taking place now, IT administration is being formatted for SharePoint. It takes three or four minutes for the formatting to take place. The user can continue browsing, or we can go to our delivery page.

 

We formatted this just yesterday, IT administration formatted for SharePoint. We can download the file directly to our desktop, and then proceed to import that term set directly into the taxonomy term store in SharePoint, and proceed to use it in our SharePoint implementation.

 

You kind of want to think about, the taxonomy portal represents like a grocery store. Everything’s there, you can pick and choose what you need, and if you want to think about … go back to that Amazon analogy, each one of these represents … shoes, computers and this and that, all the different domains of knowledge, and I may want to have a library for … just like we look at, employee benefits, and the library for company policies.

 

This is kind of our menu. What libraries do we want to set up? I want to set up an IT policy library, I want to set up a library for all of our IT infrastructure, all of the different components within that infrastructure, the documents, so on and so forth.

 

Danny:Ross, quick question as we’re looking at this. From your experience, is there a specific executive inside of larger companies that owns the overall taxonomy of the company? I know there’s certain departments, like an HR department where they can sort of define their own. Who’s the person responsible, overall in these large organizations, who’s responsible for defining the taxonomy? Is there typically an executive?

 

Ross:It’s not a matter of defining the taxonomy per se, but it’s a matter of realizing, recognizing, we have intellectual assets. We have knowledge within the organization. I don’t think it’s the IT department. What we’re seeing more and more, there’s an increasing … frankly, increasing sales, because there’s a recognition of, Chief Information Officer, we’re seeing new officers, Chief Knowledge Officers, Chief Innovation Officers … and they look at the requirement and say, “We’ve got all this information, and no one can find it.” It’s crazy.

 

Danny:And then they would work with the different departments, or they would work with their industry specialist to go out and define these taxonomies.

 

Ross:That’s it. What we’re seeing is … and we call these, “foundation taxonomies,” our clients tell us, it gets the 85%, 90% of the way there. Our client are telling us, it literally saves them months and month and months and months. They’re able to customize the HR taxonomy list in a day. Our clients tell us, “It gets us 85%, 90% of the way there.” Some of our clients tell us, “It’s got us 140% of the way there,” because there’s terms and phrases they would have never thought about, had they built it from scratch.

 

This is the key point: people are expert at what they do, not experts in taxonomy building, editing … the pre-built taxonomies provides an easy, contextual reference.

 

Danny:It’s almost like when you hear, “I know it when I’ll see it.” Like you have to see it first. And as I mentioned earlier, it’s commonly. From a consulting standpoint, “Never show up with a blank slate.” Show up with something that you can at last work with, and take apart and build up. This provides that to you.

 

Ross:I’m going to do, cause we’re in the finance area, “I need a taxonomy for risk.” Huh. Risk. Risk can have many, many meanings. The search result on the left-hand side of the screen, the terms and phrases. On the right-hand side of the screen, the taxonomy they’re located in, from a contextual point of view.

 

I don’t care about insurance, I can are about credit risk in context of banking. I click on that term, and it takes me directly to that term, “credit risk,” in the banking taxonomy.

 

Some probably need to have a library for risk management processes. And a library for internal process, and so on and so forth. A library for the various bank regulations, so we don’t have to go searching for them, so on and so forth. A library for the different bank products.

 

Remember, we can tag a document with more than one taxonomy term. We can have a mortgage product, and a brochure about that mortgage product.

 

I’m going to show you one other search. This is an example or a multi-faceted search. What we’ve one here is, we’ve indexed a number of SharePoint conferences. When we were looking at the HR library, we just had one refiner. Now, when we tagged the documents that went into this library, we tagged them with the type of document, the conference locations, and the event.

 

And now I want to find document, conferences in Chicago. I don’t care about SharePoint Fest, I care about Microsoft Ignite. And I don’t care about sponsor schedules, I need to get my registration form.

 

So if we think about this we have a mortgage product … and we have different types of mortgage products, credit cards, so on and so forth. We may want to tag the various products that we have, and because we’ve got a great sales and marketing team, but they can’t find stuff … when we check that document in, it’s about a mortgage, and the document could be a case study, it could be a data sheet, it could be a sales brochure, it could be a testimonial.

 

When a salesperson is looking, “I’m trying to sell this product to my client … ” Well, here’s the product, here’s all the documents for the product, and here’s the lead-behinds, and here’s the product data sheets, and so on and so forth. It could be the price list, the price list exist in here, so on and so forth.

 

And the marketing people … I’m focused on marketing mortgages in the Georgia area. I probably want to be able to find my research, demand forecasting. Or maybe I’ve got the international, I want to see demand forecasting for the UK marketing, so on and so forth.

 

You want to think about, “How do I organize information?” At the grocery store, how do they organize stuff? They’ve got meat in one area …

 

Think about your house. You’ve got a kitchen. We’ve got different things in the kitchen. We’ve got a place where we put our spices. We’ve got a place in SharePoint where we put our sales collateral documents. It’s the same kind of a concept.

 

It’s not difficult because we do it every day. We just have not thought about it, in these types of a context.

 

Danny:What if your house is not that organized? You see how you could benefit from it.

 

Ross:This is where you have to start. You’ve got to recognize … and I say this to people, if you’ve been living in your house for 20 years … and this is a great time for a migration strategy, and you’re moving. When you move from House A to House B, you make an evaluation on House A, “What don’t I want to move? What have I not used?” And then, “Where do I want to put it in House B?”

 

I want to be very straightforward. WAND is a company, we provide great taxonomies. But experts like ThreeWill, they provide the How. Our clients, they want to have great search? Why? “Because we’re wasting a lot of time, a lot of frustration.” But every environment is different, every organization is different.

 

I don’t know how many engagements, hundreds if not thousands of engagements ThreeWill has been involved in, they’ve done it a bunch of times. Hundreds of times, so this is the expertise, and this is how we integrate with folks like Danny and his team.

 

Danny:A couple of other things Ross. You just mentioned migrations, and that’s probably a good time … when I think about organizing. We’ve been doing the latest, everybody’s moving typically to SharePoint Online, so I think that’s a great time for you to … a lot of customers want to do cleanup at that time, probably reorganizing, and it’s probably really a good time for you to introduce that taxonomy to the company.

 

Ross:Absolutely. At the end of the day, we want to be able to find stuff.

 

Danny:The other thing is, I noticed … if you go back to that search page that you were on, is that Microsoft is starting to, based upon things like the document type and the author and created date, they’re trying to implicitly start to create some facets for you. If you look at the bottom of it, it’s almost like there’s certain things that Microsoft can do like the author and modified date. I think this is something relatively new to SharePoint Online.

 

There’s been some things that have been around for a while, but there are some new stuff. I find this incredibly helpful, to have that modified date, ’cause I’m looking for a document I know I worked on last week, and it’s not even fining things that other people have created, it’s my own stuff. It’s a huge help.

 

Ross:I think what you want to think about, when we think about metadata or taxonomy is, we could call this, “administrative metadata.” It tells us that it is an Excel document, it tells us who the author is, and it tells us when it was modified. But it doesn’t tell us what the document is about. This is what we would call, “descriptive metadata.”

 

When we go back to our slide on the shoes, this is the descriptive metadata. It tells us the size, the color. It tells us what’s in that box of shoes. This is telling us, what’s in that Excel document that Danny authored on June 26th. This describes what it is. Or if it’s a specification sheet on XYZ computer that I’m a salesman for. So this is the descriptive metadata versus the so-called administrative metadata.

 

Danny:And that administrative data, although helpful, it’s definitely helpful in that, along with sort of a free-text search is helpful … it just doesn’t get people to where they really need to be. It’s like one of those things, if you don’t take that last step, the experience is just … commonly what we’re running into, which is, there’s and time spent towards this, and they’re trying to use that administrative metadata, and they’re trying to use what’s inside the content of the document, but it just doesn’t get them to a place where they can easily find things.

 

Ross:Another aspect of search … and the guys at ThreeWill, they know I’m not a SharePoint specialist. That’s the first thing I tell people. But in the term store, we call these foundation taxonomies, and in your company, you may have a special location, Georgia, rule and regulations. So you can add terms, you can delete terms, so on and so forth.

 

Another important point is … we may think of the concept, “leave and time off.” Or, HR. I call it HR, Danny calls is Human Resources, Bo calls is Personnel. So, within SharePoint, we can add synonyms. Retirement plans, whatever the synonyms …

 

Danny:A real word, for reference, is we would call it, “People.” We sort of call our HR, “People.”

 

Ross:There you go. But this is where the customizations, the adding of additional terms, adding vocabulary that’s used within the organization. Call it Information Technology, call it IT, so on and so forth. You simply add synonyms.

 

Bo:Ross and Danny, I wanted to jump in on, when you guys were talking about the administrative taxonomy versus the descriptive taxonomy. Another thing that it made me think, where we help a lot is, taking WAND’s taxonomy and, like Ross said, you apply it to a company, and every company is different in terms of what they call stuff. But they’re also different in terms of their people and their people’s willingness to tag things with the descriptive metadata part.

 

Administrative metadata, I also, internally, call it the, “free metadata,” the stuff that it doesn’t require a used to anything other than upload it. One of those areas when I was talking earlier is, simply by virtue of where you put something, that could give you the ability for more free metadata, i.e., if you uploaded a document to an HR document library, we might automatically tag it with some level of that HR taxonomy, versus if you had uploaded it to an IT one. So your SharePoint site-structure, document libraries and so on can enable getting some free descriptive metadata, I think is what that kind of boils down to.

 

Ross:I think that’s a great point, Bo. If we think about our houses … again, I use this metaphor, the house. I use this analogy, here’s the spice rack, and in the garage we’ve got our tools. We’ve got wrenches, screwdrivers, so on. It’s a pain to tag things, it’s a real pain when I do some work in the basement on the water heater, to get all the tools back and put them back where they belong. It’s not a lot of fun, but at the end of the day, if I don’t put those tools where they belong, if my wife doesn’t put the spices where … we can’t find them.

 

When people realize, “This isn’t that difficult, and I’m going to be able to find my stuff, I’m going to be able to find document that Bo checked in, that Danny checked in, we’re all going to be able to find … we’re going to start to eliminate this frustration.” Then, there’s a lot momentum, because it dues alleviate a lot of frustration, and it enables us to do our work a lot better.

 

Overview For a Successful Managed Metadata Project. We always tell folks, “Get a quick win.” And what we want to be able to say to our organizations, “We’re not going to have deliverable results in two to three years, not two to three months … two to three weeks is what we’re looking for.”

 

Danny:Now you’re talking.

 

Ross:And our goal is findability. We want to tag our documents. Why? So we can quickly find them in search. How do we get that started?

 

And if we go back to this financial organization, the folks do a library on banking, the banking folks are going to be interested in that. The procurement people ae going to be interested in the procurement libraries, the accounting people the accounting libraries. So there’s going to be a limited group of people interested in these various libraries … with that one exception. Every employee at the bank, from the CEO to the latest new hire, wants to quickly be able to find their policy documents, benefit documents … again, nothing confidential.

 

So, that’s the group we start with. We download the HR taxonomy, get somebody from Human Resources to help customize it, generally that takes three to four hours. You’ve got a limited number of terms there. Takes 30 minutes to configure the SharePoint taxonomy columns and libraries, search refiners. Have that same person from HR assist in the tagging of the documents. Generally, there’s 250, 300 documents of that character, three or four hours to do that, maybe a little longer.

 

Then we announce to every employee in the company, “Library’s open.” And they see that search works, and they look at this and they say, “My goodness. If search can work for this library, why can’t they do that for my sales documents?” And the IT people the IT documents, and everybody starts to project what this is going to look like in their day-to-day world, within the organization.

 

It’s not an IT project anymore, it’s not a SharePoint project. It’s a project that everybody sees, “What’s in it for me?” It generates a huge amount of enterprise momentum … and then your problem is going to be, everybody’s going to be tugging on your shirt, prioritizing additional groups for implementation. Deliverable results, two to three weeks, and now we want to replicate it enterprise-wide.

 

Nice case study, and there’s a link to this … I’ll see if I can find it here. Nielsen Norman selected the 10 best intranets every year, and Goodwill Industries International was selected as one of those. In the real world, we all know this, we want to be able to find our stuff, we want to be able to find our documents, we want to be able to find the things we have in our houses. Search, findability, findability, search … it’s part of what living is all about, and SharePoint’s a great environment for that. We think we’ve got wonderful taxonomies. The folks that organized this, ThreeWill, Danny and his team, they’ve got the expertise to show you how to get it done.

 

Danny, I’m going to turn it back to you. I think I’ve talked enough.

 

Danny:That was great Ross, thank you so much. I’m sure everybody got … it’s such a great presentation that you do, and I really wanted to share that with everyone else.

 

Just to get us wrapped up here, we’ve got about 10 minutes left before the start of the hour. I wanted to just go through some of what I sort of took away as the conclusions, when listening to Ross and to Bo, and talking with Bo about this.

 

The key component to addressing a findability or search problem is to define a common taxonomy for organizing information inside your company. We have to do that. Taxonomy enables … and I pulled this from Bo, Bo ha pointed this put before the webinar, which was, it provides powerful ways to present and find content, including not just search, it’s also navigation, it’s also automatic tagging and the document routing.

 

Coming back to the overall webinar and what we were trying to present in this webinar, which was, how do you turn SharePoint and how this Amazon-like faceted search available to you? It’s there so that your users can find a document the same way they find their next pair of new shoes.

 

As part of that defining that custom taxonomy from scratch, we’ve just found, is a really daunting task for most companies. The companies who do go after it, as Bo mentioned earlier, they might spend a couple of years doing that. So it’s really a daunting task, and from a lot of what we’ve seen, it’s just commonly not done, and all they’re using is the administrative taxonomy.

 

Starting with a pre-built taxonomy, it’s an effective way to jumpstart this process, and to get to the point where, if you’ve got 75%, 80% of the way there and you’re going there, and you’re culling and you’re adding, and you’re just making it your own, is really the route that we would recommend doing.

 

Also, one of the things that’s sort of the last point here is, taxonomies do change over time, and one of the neat things that I saw about Ross’s product is, it will also, over time, as industries update and business changes come, you can keep those taxonomies up-to-date, which is really important, because the business changes.

 

Anything else to add to that, Ross or Bo?

 

Ross:I think the updates, we have six masters of library information science taxonomists that review each of the taxonomies on a quarterly or semi-annual basis, deepening on the volatility. If we just think about, 20 years ago, the vocabulary that we were using. We weren’t talking about, “online,” we weren’t talking about, “internet.” Al the vocabulary that didn’t exist. Today, machine learning, artificial intelligence, internet of things, so on and so forth, it’s just all new technologies that we keep an eye on, and we add them to our vocabulary sets.

 

As you can tell, I’m an older guy, and you can think about … when I was going to high school, they had computer card. You think about your kids, the technology you grew up with, and then think about the technology your kids are growing up with. It’s really remarkable, the changes that take pace. Sorry for pontificating. Back to you, Danny.

 

Danny:That’s great anything else that you’d add, Bo?

 

Bo:No, the only thing I was thinking is, for those who are on the fence about pursuing a taxonomy, I would say go for it. I think the worst thing you can do is be complacent and do nothing, and somewhere down the road have hundreds or tens of thousands of documents that aren’t tagged, and then you want to do the taxonomy, it’s probably harder. The sooner you start with something, the sooner you’ll get benefits from it.

 

Danny:One of the great points … in the chat window, I shared one of the neat things that Ross and his company does is, they do make available to everyone a general business taxonomy. It’s sort of a starting place that has general business terms, and a taxonomy for that. So if you wanted to download that there, as an example, I think that’s a great place for people to start as well.

 

Ross:If I can just expand on that a bit … Microsoft announced SharePoint 2010 in 2010. We developed the genesis of this general business taxonomy, it’s also available on three blog postings on the Microsoft site. We developed it in conjunction with Microsoft, as a partner. What they wanted to do is, they wanted to provide their client base something for their clients to download to put in that empty term tore for this new feature that they just developed. We’ve actually had about 8,500 downloads over the last five, six years. Danny, that’s a great idea. That gives you a start, give you a flavor of what you can do.

 

Danny:Awesome. Any questions that folks have? And if you’ve been waiting until now, feel free to ask them now. I don’t think … what do you have, Oliver?

 

Oliver:Can it help fine-tune an existing taxonomy?

 

Danny:Ross?

 

Bo:I was going to jump in and answer. I saw the question in the window, too. I don’t know if it would necessarily help you fine-tune an existing taxonomy. I think it might show you a general taxonomy for your particular needs and user input, but I think the tuning of your taxonomy is probably going to be more closely-aligned with your specific company and needs. But I don’t know what your thoughts are, Ross.

 

Ross:What I would say is, companies that have developed taxonomies, there’s not very many of them. They generally develop them based upon file plans, the terms that they use for file plans. If we think about a file plan, a file plan is an analog way or organizing information. So, while those exiting taxonomies can be interesting, the formal curated taxonomy that’s designed for digital tagging of documents is much more effective, but you can fold the terms in as synonyms, and that can be helpful in the transition.

 

Danny:Awesome. Anything else? I think that’s it. I appreciate the question, that was great. It looks like we are getting to the end of the hour here, so again, we’ll send this deck out next week. If you want more information, feel free to follow up with myself or Ross. Here’s our websites.

 

Ross, thank you so much. It was really informative. I really appreciate you taking the time to do this.

 

Ross:It’s a pleasure. Thank you, and thanks to all the people that have attended, and to Bo as well.

 

Danny:Absolutely. Thanks, Bo. Thank you everybody for attending, look for an email next week, and have a wonderful weekend, everyone. Take care.

 

Ross:Have a great Fourth of July. Bye bye now.

 

Danny:Absolutely. Bye bye.

 

read more
Bo GeorgeFind Anything in SharePoint with Amazon-Like Faceted Search Webinar
sorry-heart.jpg

SharePoint Search Error – “Sorry, something went wrong”

There are times when SharePoint Search can get out of whack and you need to go in and get it cleaned up and running again. This manifested itself at one our customers recently and I thought I’d document the steps I needed to take so that I would remember them for the next time and maybe help someone else as well.

The error being returned in the UI was the usual ubiquitous message that simply said ‘Sorry, something went wrong.’ and ‘Search has encountered a problem …’ – very, very helpful messages.

The error being thrown in Event Viewer and in the ULS logs was as follows:

Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance (09fe67b2-d3d8-4d45-a87a-bda7d45a9ad0).
 

Reason: An update conflict has occurred, and you must re-try this action. The object SearchDataAccessServiceInstance was updated by XXX\xxxxx, in the OWSTIMER (17324) process, on machine XX-XXXX. View the tracing log for more information about the conflict.
 

Technical Support Details:
 

Microsoft.SharePoint.Administration.SPUpdatedConcurrencyException: An update conflict has occurred, and you must re-try this action. The object SearchDataAccessServiceInstance was updated by XXX\xxxxx, in the OWSTIMER (17324) process, on machine XX-XXXX. View the tracing log for more information about the conflict.
 

 at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
 

 at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(Boolean isAdministrationServiceJob)

In simplistic terms, this is basically saying that the SharePoint search index and the local configuration cache on the server are out of sync.

I then went into the Search Service Application within Central Administration and I saw that there was a warning sign under Index Partition.

So there was definitely something wrong with the search index. I found that this can typically be corrected by doing two things – clearing the cache and resetting the search index. Here are the steps necessary to complete both of these.

Clearing the Configuration Cache

The first step is to stop the timer service. This is most easily done via command prompt (run as administrator) and running this command:

net stop SPTimerV4

Once done, open Windows Explorer and navigate to the following directory:

%SystemDrive%\ProgramData\Microsoft\SharePoint\Config

In this folder, you’ll see several GUID folders. Open the one that has the most recent Date Modified value.

Select all the files in this folder with the exception of the cache.ini file and delete them. If this makes you uncomfortable, backup the entire directory to a temp folder before doing this.

Edit the cache.ini file in your favorite text editor and reset the random number found there to 1. Save and close the file.

Go back to the command prompt and run this command to restart the timer service:

net start SPTimerV4

Soon you should begin to see .xml files begin to repopulate in this directory. The cache is now refreshed so that just leaves the search index to reset.

Resetting the Search Index

To reset the search index, go into Central Administration / Application Management / Manage service applications.

Scroll down the list until you find Search Service Application and click on it to open.

On the left hand side, click the Content Sources link. You should see a screen like the one below. If any of the sources are set for Continuous Crawl, turn this off by right clicking on that content source and selecting ‘Disable Continuous Crawl‘. After doing this, it will immediately kick off an incremental crawl which you will also need to stop. Right click again and select ‘Stop Crawl‘ to cancel it.

Do this for each of the content sources until they all show as idle.

Once all the crawls are stopped, click on the Index Reset link found on the left, a few entries below the Content Sources link.

In the dialog, make sure the Deactivate search alerts during reset checkbox is checked and then click Reset Now.

Click Ok to confirm and start the reset. This can take a few minutes to complete so be patient.

Once complete, a full crawl will need to be started on all the content sources. Right-click each source and select Start Full Crawl. To verify the crawl started successfully, peruse the crawl logs for successful items.

Once the full crawls have completed you should set any content sources that were initially set to continuously crawl back to this.

To do this click on the Content Sources link again then click on the name of the Content Source (e.g. ‘Local SharePoint sites’) to open the Edit Content Source screen. Scroll down to the Crawl Schedules section and select Enable Continuous Crawls, then click Ok.

Your search should now be all set until the next blip knocks it out again (you know it will!).

Here are some references to a few sites that helped me get this all sorted out.

  1. http://sharepointviews.com/clear-configuration-cache-in-sharepoint-farm-2010-2013/
  2. https://vigneshsharepointthoughts.com/2016/12/15/resetting-search-index-in-sharepoint-2013/
  3. https://technet.microsoft.com/en-us/library/jj219652.aspx

Let me know in the comments if this helped you out (or if you have anything to add).

read more
Caroline SosebeeSharePoint Search Error – “Sorry, something went wrong”
ba-insight.jpg

Interview with Sean Coleman from BA Insight (Enterprise Search)

 

Danny:                  Hello, and welcome to the AppExchange podcast, this is your host Danny Ryan. Today we have Sean Coleman from BA Insight here with us. Thanks for joining us Sean.

Sean:                     No, thank you Danny.

Danny:                  You betcha. Today, we’re focusing in on, is it search for Salesforce? That’s the particular app that we’re going to show off today.

Sean:                     Yes, that’s right, our search for Salesforce App certainly.

Danny:                  Awesome. This will be very interesting to me, and for some of the listeners who know our backgrounds around Microsoft technologies, and learning more about the integration between Salesforce and Microsoft. It’s always an interesting topic to go into. I know a lot of customers are very interested in this topic. Hit it at a 10,000-foot level. Just give us an overview of what the app does for customers.

Sean:                     Yeah, absolutely. I think where the idea for the app came was really with our usage, right? We are a Salesforce user. We use it for our customer support. We build products for SharePoint, so really for SharePoint search and portals and things like that. What we realized was all of the trouble that our customers was having was described to us in Salesforce. All of the solutions to the trouble they were having were stored in our SharePoint [crosstalk 00:01:22] our documents. That was where we use TSS for our bug tracking. We’re a software company.

We realized all of the content that we needed to solve the problems, our documentation and everything, just wasn’t in Salesforce. I don’t think that’s uncommon, I think that people run into that everywhere, document management systems and things like that. We said, you know you could either try to jam all that stuff in Salesforce, but you run into the problem with a lot of organizations, not everyone in the organization has access to Salesforce.

Danny:                  You’re absolutely correct.

Sean:                     They are contributing in other ways. While SharePoint’s a great portal, SharePoint has a great search engine. What if we would integrate that search engine inside of Salesforce, so the Salesforce users don’t have to leave to be able to access content that’s stored in SharePoint? Then through the B-inside technology that we already have, we can extend SharePoint to go to your document management system. Box.com, we have got like 50 odd connectors, we can go to a bunch of backend systems, document them, and back to Salesforce to integrate it and provide this single view of all your organizational data available to either your support reps or your sales reps to go out and solve whatever problem they’re having. Are they putting together a proposal, and they need some data to pull together? Or they are solving a customers problem or case and they need to send them a document, or they want to point them somewhere. Give them access to that information in line contextually with what they’re doing. That’s really where it came from.

Danny:                  That’s great, so it sounds like initially the concept of putting the app together was your just trying to scratch your own back, since you guys are running into this issue internally and figured customers would be looking for something similar right?

Sean:                     Yeah absolutely. We see Salesforce, we use it extensively, it’s got a lot of penetration in the cerium market, obviously Microsoft and Salesforce are starting to work together, and we realize that there is a good symbiotic relationship there. You don’t typically run into people that say, we don’t have SharePoint, we have Salesforce. It’s not that, they have them both, there’s likely SharePoint has the larger content footprint, right? Because everyone in the organization has access to SharePoint , and not everyone has access to Salesforce. Our philosophy is go to where the content is and bring the content to the people. That’s what we were trying to do.

Danny:                  Very nice. I’m looking here on the APP exchange on your listing. So this is, so you can initiate the search from Salesforce? Or what’s typically the way folks interact with the APP?

Sean:                     Yeah, so it’s actually one better than that. What you’re looking at right there is our main configuration screen.

Danny:                  Okay.

Sean:                     What that does is that it ties together … You kind of say  “Here’s my SharePoint portal address”. The neat part about that is that the next thing they do is a field mapping, because it’s not just “can you search the stuff” it’s actually a little bit of a predictive search. Meaning what you can do is you can embed it on a case object. Embed it on the case object page, when someone launches that case, what it’s going to do is it’s going to pull data off of the case itself and feed it to the SharePoint search and return a result.

Danny:                  Oh, nice.

Sean:                     You could say, send across the product names, send across the product version, send across the description and title that they’re looking for, and on the SharePoint side, it’s going to feed all that stuff in and do a search index. They’re going to start their case already with results based on what Salesforce knows about the case that was filed to them, instantaneously. They load the case up, right there is likely information that’s contextual for what they’re looking at.

Danny:                  I could see you using this for accounts as well, where you’re searching for the account name, for all the documents that are associated to that account … Definitely could see things like that, with the opportunities … There’s a lot of different Salesforce  objects where you could tie in. It’s almost like we’re … We’ve seen folks build search-based applications, its almost like it feels like an APP that you’re using. You’re just tying search in there to build out that app. Very cool stuff. Very nice.

Sean:                     We see a lot of search-based applications and things like that, having the access to the data. No one really … Whenever you run a search, it’s because you have problem to solve, right? So you have to go do a search, or you have an issue and you go to Google and you try and solve it. When you’re looking at a case, you’re trying to solve that. So bring the data that search gives you … Almost make the fact that you’re doing a search secondary to the data. That’s what you see in this screen shot right here. Here’s a test case where someone’s like, “I’ve got an install problem,” right? This is the reps view, and what he sees already is, “here’s some PDFs that are install documents.” With refiners and things to drill down, and you can configure, that this is highly configurable. So it can configure to whatever properties someone uses in their Sales Force to whatever their backend data is, we can do that mapping, and really they can find the information they’re looking for right there in line.

Danny:                  Very nice. So you can go over here, select a certain version and that would show up over here and refine it down to what in particular you’re looking for.

Sean:                     Yeah, absolutely.

Danny:                  That’s great, what a neat idea. I agree with you 100%, there’s so many organizations who … I think I’ve yet to run into a large organization that at least doesn’t have SharePoint somewhere.

Sean:                     Right.

Danny:                  Whether it being used in only particular departments but SharePoint’s all over the place. With Salesforce it’s starting off in the sales organization and maybe making its way over to marketing. They need to be able to get to the appropriate asset at the right time, so this would really help them do that.

Sean:                     Yeah, absolutely. Another thing we see is … One of the problems that we saw and we leveraged in the Salesforce … The way Salesforce search is, right so if you go and do a search in Salesforce, well here’s contact, and here’s accounts, here’s opportunity, right here’s cases that are against what you asked for. So if you know that object you are looking for, you’re in pretty good shape. If you don’t, if you’re like, “I just want to learn about this account” or “I just want to go learn about this case we just had” or something like that, it’s not what we call combined and relevancy ranked which is big from the search side.

Give me what I am interested in, but give it to me in the proper order that the system should give it to me in, rather than giving me a chunk of this, and a chunk of that. We actually have ability to take … We’re obviously exposing the SharePoint search index here and delivering it back, but inside of that search index can include all of Salesforce data too, so you can inline see documents from SharePoint, account records, contact records, case records, documents from your documents management system, documentsinbox.com, documents in other CRM’s … all inline relevancy ranked. So you actually get the full “enterprise picture” of what you’re looking for rather than the [inaudible 00:08:50] system picture.

Danny:                  Nice.

Sean:                     So Salesforce has a bit of a picture, SharePoint has a bit of a picture, Box has a bit of a picture. So we say, no, you don’t want to have to go to all of these three systems and assemble your puzzle. Put it all in one, and get the full cooperate enterprise picture of what’s going on in that one result. You’re going to get data faster, you are going to be able to make decisions faster, you’re going to make your customers happier, hopefully you’re going to close more deals. That’s what we’re after.

Danny:                  And its not even a federated search where you have to jump to different screens to go see the different results at, it’s all in one result set.

Sean:                     Exactly. All inline, relevancy rank, one result set.

Danny:                  That’s nice. That’s sweet! Very cool. With seeing how your customers are using this, any surprising usages that you’ve seen through the years?

Sean:                     I mean we’ve talking to a number of customers about the way they use it. I think the more surprising things you start to learn are that people use Salesforce in a lot of different ways. They’ve done custom objects, they’ve done things like that, and they’re always delighted to find out that data works just as fine in here as it does anything else.

I can embed this on my custom object page just like I can on the case page, and leverage searches that are relative to that. They always say what we find is …  A lot of apps we find, they work great with the core Salesforce objects. What we do, this custom thing over here, and the way that we built this is that you can embed our visual force pages and you can do that integration, and you can scope your search. You can say, in this use case, I only really want to find this particular bits of data. It’s very refined, they want to go for that. We can configure it so that it only looks at certain data sources when it does that search. A financial application, someone who’s running their leases, or anything through Salesforce.

We do all of our invoicing through sale source, great. I can give you a invoice specific history search that just pulls invoices, gives you documents previews of them, so that they can do that faster. I always say this to financing, whenever you’re doing one invoice, how often do you go to look at the last one you did? Or do a [inaudible 00:10:55] you did, so you can go look at the last one you did to compare it. Well I can give you that capability inline. Right there, so that it’s already found them based on you you’re looking at and based on their passed history was. Then you can mouse over them and get more data on it. They were excited to find that kind of stuff out.

Danny:                  80% of the sales people out there will … Their next contract they write will look somewhat like their last contract or their last statement they wrote. For some reason, I wonder why.

You are absolutely correct. This is great stuff! I’m going to go ahead and switch over to you. I’d love to see a demo from you if that’s okay.

Sean:                     Yeah, that sounds great.

Danny:                  All right, I’ll give you control here. You have the conch.

Sean:                     Okay, let me know when that shows up.

Danny:                  Absolutely. So while we’re switching over here, are you out of Boston? Or where are you based out of?

Sean:                     I’m out of Raleigh, North Carolina.

Danny:                  Oh, nice.

Sean:                     But I work kind of remotely down there. But our corporate office is in Boston, so we have a nice office up there. Right near Boston Commons.

Danny:                  That’s great.

Sean:                     It’s gorgeous this time of year.

Danny:                  I can see your screen.

Sean:                     Okay, excellent. So what we’re looking at here is a story of a case worker. Case worker that’s gotten a case in someone that’s looking for help in their network configuration. This is just an example of … I load this right up, and inline of what I here are the search results already. So this is very similar to the screen shot we looked at earlier. But here are some of the things, here’s the connector framer work release guide, connector configuration. If its certain documents types that I was looking for, I could say, you know what I DO think release notes is what I’m after, so let me filter that down. I really want things that are in this specific version, so 432, I know the customer has that. And it gets down to what I’m looking for. Now some of the cool things that we didn’t show previously is that we have the ability to now do an action with it. So I found it, this is exactly what the customer needs.

Now what I have is one example of an integration, so what I do now is I click the sales force button. It’s going to tell me, “hey, we created a solution for you.” Okay, excellent. What does that mean? That means that we took that information and we actually authored if he happened to be using the solution capability inside the sales force. We offer this solution as a draft, right here and attach it to the case. If you have workflow set up, that would email the customer about that, it would send them the release notes. You can actually go in there and edit it, make it … Maybe it’s not for this specific customer, maybe it’s for other customers, but it’s taken that data you had and put it into solution. I could just as easily configure this to say send an email, or attach this document. Or I could click an email that would take this link and attach it, put it in the email, and I could send it. OR I could attach it. Through our configuration we are actually able to extend that, and do any number of actions that someone would want.

Now that I’ve found the data, what do I do with it? If that makes sense.

So this example is one type of integration. We’ve integrated it also in the service cloud console. Just wanted to show that, in case your sale reps are console users. That have seen this nice interface here. Instead of having it inline on the left, you can have it open on the right. Gives you a little more screen real estate in here. Lets do some other stuff. If you’re someone very much into screen real estate, you can split that up and put more data over here. Same things apply, you can configure what data will feed that connector.

And I wanted to show that. So if I go back to my other app, and go to my configurations, this is where we can figure what URLs we bring in, and what objects we show and bring over. So what we’ve done here in [inaudible 00:14:54] for the case object, if someone has chosen what web browser they want, we’re going to map that over to a search and put client browser. That’s a filter that I have in my share point search, so we’re going to pass that in. If it’s got a … That’s who created it. Any number of these fields we have full access to them. So you count objects, here’s all the different fields, and you can do that mapping. What that does is it makes the initial search that much more intelligent. So it pulls the data off the of case, or off of the account and feeds into the search, so you’re going to get that much more relevant data right to the user when they get there.

Another cool thing to say is that maybe that compact search is great, and you say, “hey Sean, you’ve told me all about all the ways that I can do all my general searching in here. Do I need a larger interface for that?” And you certainly do, so what we’ve done is a larger embedding where we can actually embed … This is our full clamp portal but we can go right to our search center. If we did that same documentation, the results I am able to get here.

Danny:                  You’re probably going to show this, but does this mean I can search Sales Source from SharePoint? Does that make sense? I guess is that what you’re showing?

Sean:                     Yeah that’s right. So basically anything that’s in here … Right now we’re searching documentation. We’ve got some of our [inaudible 00:16:27] with bigger screen real estate we can do things like our document preview, inline preview, we don’t have to do it … Hit highlighting, search highlighting in here. If I click on this … My search key word, I was looking for documentation, but I’m interested in federator, it’ll show me were that’s at. Oh, it’s on page nine. So take me to page nine directly. We’ve got some more of these enhanced things that we can do in there. But also, you can query … you can index Salesforce.

So, just as easy as this document, this could be an account record, this could be an email object, it could be an attachment. Anything in there, a case itself, all could be integrated in and through filtering and things. You can come in and decide I’m only looking for sales [inaudible 00:17:18] objects or things like that.

So full enterprise search capability integrated. Obviously this is our styling, but you can style this and make it look however you want. We actually have a new lightening based interface that’s coming which we’re going to roll out soon, which will take this search center and kind of make it look … And [inaudible 00:17:39] new lightening interface, people are checking that out.

Danny:                  Very cool.

And is this … We’ve been talking in general about SharePoint 2013. Is this work with Office 365 yet?

Sean:                     It does. If you have a share point online search center, you can absolute integrate it, much the same way to do share point 2013. The single sign-on stuff exists there, so yeah! SharePoint online or SharePoint 2013.

Danny:                  Well this was a great [inaudible 00:18:08] Sean, thank you for taking the time to do that.

Sean:                     No problem at all, happy to do it.

Danny:                  So for folks who are really looking to get more out of … Who are using SharePoint for managing a lot of their documents, which is most organizations … People who are using sales force really want to get more out of backend documentation, all that content that you might be storing on SharePoint. This just looks like a great way for you to make up … Hold the relevant content up to the right places inside a sales source. It just looks like a great tool for those folks.

Sean:                     We hope so! That’s what we built it for.

Danny:                  That’s great! Well, thank you for doing this, I am going to … Let’s close this out here. For folks that want to see the transcript for this, and for other episodes, you can go to www.appexchangepodcast.com. Thanks Sean again for taking the time to do this, and the great job the BAN insight has done with this product. Congratulations.

Sean:                     Awesome, thanks so much Danny.

Danny:                  You betcha. Every body have a great day, and thank you so much for listening. Bye bye.

 

read more
Danny RyanInterview with Sean Coleman from BA Insight (Enterprise Search)
customer.jpg

Top 5 Reasons to Manage Customer Experience in SharePoint

Customer Experience in SharePoint

Customers interact with a business in numerous ways, whether browsing through the store or online catalog, asking questions in person or through email, purchasing product at point-of-sale or online and returning or exchanging merchandise after the sale. At each of these touch points is an opportunity to capture information about the experience your customer has with the process or people. Whether the customer experience is entered directly by the customer or via an astute employee, here are the top 5 reasons Microsoft SharePoint can assist with capturing this information:

  1. Lists provide a versatile repository for structured data

    SharePoint lists are the place to record the “who, what, when and where” of the customer interaction. As needs change, the list data can be enhanced, in place, to capture additional information about the customer experience without the need for data conversion or migration. Files attached to the list can help document the experience, whether they are photographs, email correspondence, or other content. SharePoint lists contain the “database” of customer information.

  2. Search for similar customer experiences

    Different customers often have similar experiences. SharePoint full-text search can be used to identify other related interactions to consolidate or remove duplicate information. Search can be executed interactively or programmatically depending on how it best fits the processing workflow. Results ranking can be used to allow the most pertinent results appear at the top of the results list. User experience can be greatly enhanced by integrating search results directly into the data entry screens, enabling search-as-you-type or search for duplicates at the initial point of data entry.

  3. Workflows allow for custom routing, review and approval

    SharePoint workflows are a method to automate, augment and enhance business processes. These workflows can send email notifications of newly added items, trigger approval steps as list items are reviewed and data is changed, and perform a myriad of other tasks, many without the need for custom programming, to support the business processes.

  4. Retention policies can trigger reminder emails or even archive older content

    SharePoint retention policies are triggers on list data or documents that occur when an item is deemed to have “expired.” Typical uses for retention policies include moving content to an archive repository when the content reaches a certain age and to send email reminders based on a number of days since the last interaction or the number of days an item has remained in a particular state or queue. Retention policies run once a week by default, but can easily be configured to run once a day.

  5. Integration with Microsoft Office

    A standard feature of SharePoint lists is the ability to export of data into Microsoft Excel. Additionally, Microsoft Access can read, query, process and report on SharePoint data, in real time, without the need to export and process the data locally. Microsoft Word can process Excel data, allowing for robust mail merge capability. Via the OpenXML SDK for Microsoft Office (software development kit) programmatic creation of Excel, Word and PowerPoint slides provide unlimited possibilities for presentation of customer experience data.

Summary

Microsoft SharePoint provides a rich out-of-the-box infrastructure for data related to customer interactions and experiences (as well as for many any other data-related needs.) The combination of list data, full-text search, workflows, retention policies and integration with Microsoft Office can cover a wide range of customer experience use cases, empowering your organization to streamline your business data collection and processing.

ThreeWill has the ability and experience to transform standard SharePoint lists and forms into enterprise level applications. Contact us to let us know how we can help you.

read more
Matthew ChestnutTop 5 Reasons to Manage Customer Experience in SharePoint
hybrid-car.jpg

SharePoint 2013 Hybrid Environments and Search

Looking for a New Hybrid to Drive?

Hybrid automobiles are all the rage these days. By combining a familiar technology with newer technologies, we see a new kind of vehicle that provides better mileage and less exhaust emissions.

Office 365 provides us with a similar opportunity – to bring together the familiarity of on-premises servers such as Exchange, Lync, and SharePoint with their cloud-based counterparts in a “hybrid” environment that draws on the best of both worlds.

This article focuses on a hybrid environment with regards to SharePoint and in particular as it relates to the search experience for users. We’ll look at some opportunities as well as limitations, and we’ll discuss approaches on how to configure an environment to improve the search experience.

Taking a Look Under the Hood

Hybrid O365 environments come in 3 flavors called “topologies”:

  • One-way outbound: On-premises SharePoint 2013 farm(s) can reach SharePoint Online, but 
SharePoint Online cannot reach on-premises SP 2013 farm(s).
  • One-way inbound:  SharePoint Online can reach on-premises SharePoint 2013 farm(s), but 
on-premises SP 2013 farm(s) cannot reach SharePoint Online.
  • Two-way: On-premises SharePoint 2013 farm(s) can reach SharePoint Online and visa-versa.

For many enterprises, the only option is one-way outbound because it is not feasible or desirable to open up the firewall and allow Office 365 to communicate directly to the on-premises SharePoint servers. Because we believe many enterprises will have the same restriction and because our experience is around this topology, the rest of this blog will assume a one-way outbound technology. To learn more about the various topologies, check out this guide.

Kicking the Tires

There are currently 3 integration points with SharePoint Online:

  • Search: allows search results from SharePoint Online to show up in SharePoint on-premises search results (and visa-versa) depending on the topology. This can work in any of the topologies listed above.
  • Business Connectivity Services (BCS): allows on-premises data sources to be accessible to SharePoint Online via BCS. This only works with inbound and two-way topologies.
  • Duet Enterprise Online: allows on-premises SAP content to be available to SharePoint Online. As with BCS, this only works with inbound and two-way topologies.

While BCS and Duet are interesting and useful in their own right, our focus for now is on Search.

SharePoint 2013 Hybrid Environments – Checking Over the List of Available Options

Given our stated focus on a one-way outbound topology, we know the following with regards to search:

  • Our on-premises search center(s) can show results from on-premises SharePoint as well as SharePoint Online.
  • Our SharePoint Online search center(s) can only show results from SharePoint Online.

We’ll be using out-of-the-box SharePoint Federated Search for getting results from both of our SharePoint environments and showing them in our on-premises search center. It is possible to use third-party solutions to provide different alternatives on indexing and federating, but that topic is beyond the scope of our discussion.

Here are the prerequisites for configuration:

  • A SharePoint Online tenant
  • A SharePoint 2013 on-premises farm
  • Proper configuration of server-to-server authentication

Point number three is seriously important: If server-to-server authentication isn’t setup correctly then we won’t get any results, even if everything else is setup properly. While attempting to set this up is beyond the scope of this article, this guide and this article will give us what we need to know.

Once we’ve met the prerequisites we’ll configure our SharePoint 2013 on-premises server to crawl any content we wish to show in the search results. While this is typically the local farm, it could also include other SharePoint 2013 farms, SharePoint 2010 farms, and local BCS data sources (such as line-of-business applications). Result Sources for each of these sources of data are typically created in the Search Service Application (the steps for doing this are well-known and well-documented, so they are omitted here for brevity).

After we’ve configured all of the desired local content sources, we’ve got one more left: the one for our SharePoint Online data. The configuration details are specified below and shown in the accompanying screen shot:

  • Result Source Name: SharePoint Online Results (or whatever you like)
  • Protocol: Remote SharePoint
  • Remote Service URL: <your SharePoint Online URL> (such as https://mycompany.sharepoint.com)
  • Type: SharePoint Search Results
  • Query Transform: {searchTerms}
  • Credentials Information: Default Authentication

Next, we must answer the following question: which data is more important in terms of presentation – the on-premises data or that from SharePoint Online? This option is pretty significant because it will drive how search results are presented to users. While the answer to this question will vary by circumstances, the key here is to specify the “default” data source in order to indicate which one will take the preeminent place on the results page (as we shall see, this also affects refiners and other capabilities). The screen shot below illustrates how to set the SharePoint Online Results as the default, but for the remainder of this blog we will assume that the Local SharePoint Results are the default.

The way we’ll identify our secondary source is by creating a Query Rule in the Search Service Application (for this exercise, we’re assuming the secondary source is SharePoint Online):

  • Name: “All SharePoint Online Results”
  • Context: All sources, categories, and segments (we are staying that we want the query rule to always run)
  • Query Conditions: none (remove the default condition provided)
  • Publishing: Accept default value
  • Actions: add a Result Block
    • Block Title: SharePoint Online results
for “{subjectTerms}”
    • Configure Query: {subjectTerms}
    • Search this Source: SharePoint Online Results (Service)
    • More Link (substitute the link for your SharePoint Online instance):
      https://mycompany.sharepoint.com/
      search/Pages/results.aspx?k={subjectTerms}
    • Block Location: This block is always shown above core results
    • Routing: leave blank

Note that with SharePoint 2013, much of this configuration can be done within a site collection and doesn’t have to be done at the farm or tenant level.  We’re just configuring at the farm level to make it the default for all site collections.  These configuration changes can be overridden at a site collection as we’ll see in a later blog post.

Let’s Take This Cream-puff for a Spin

Issuing a query in the search center should show results as follows (assuming we’ve entered search terms that get matches in both environments):

Search Results

As we exercise the results, it’s worth noting that the default Result Source sets the agenda for refiners; they are based on the default result source, not secondary. However, if we click one of the refiners, both the default and secondary sources refresh based on the refiner selected.

In addition, even though we can page through results from the default result source, we only see the block of results from the secondary result source on the top of the first page of results.  Incidentally, the Result Block  can be configured to show “within core results”.  In this case the secondary results will still be in one block, but may not be at the top of the page or even on the first page based on relevance.

What if You Run in the Ditch?

Here are some things to check if the results don’t match expectations:

  • Misconfiguration of Query Rule or Result Source can cause missing or incorrect results
  • The hybrid server-to-server has to be configured properly – some good troubleshoot tips are documented here.

More Options to Close the Deal

What about People Search? Videos? Conversations? How do we get search results initiated from SharePoint Online to show us integrated results containing both on-premises and online content? We’ll consider these in future installments. For now, get out there and drive the wheels off that new “hybrid”.

read more
Kirk LiemohnSharePoint 2013 Hybrid Environments and Search
sharepoint-document-generation.jpg

SharePoint Document Generation Webinar


Well, good afternoon or morning or evening to all of you, depending on where you’re at. I’m John Underwood and I’m going to be your tour guide today as we go through the SharePoint Document Generation webinar. I’ve had the good fortune of having some of you attend my webinars and classes in the past and if you’ve come back, I thank you very much.

For those of you that are new, just a little bit about me. I’m lucky enough to be working at ThreeWill. My job there is Technical Evangelist and, really, that involves two things. I get to do some engineering chores where I write code and then also get the privilege of doing a lot of presentations, be they webinars or videos or even face-to-face training classes. That’s something that I really enjoy doing.

If you need to contact me for any reason, you see a couple of touch points there. My email, [email protected] Also, if you like, you can follow me on Twitter @jbutweets. You may want to scribble that email address down really for two reasons. Number one, if we were to have any technical problems during the presentation, you could use that as an alternate way to reach me in a moment of desperation. Beyond that, you may have some questions, some follow-up, et cetera, after the webinar. Always welcome your questions. Always have to give the disclaimer that I can’t act as your off-site help desk, but generally if a customer follows up with a specific question about a topic I’ve taught, I always wind up learning something myself. If you have some questions on today’s material, I would welcome those emails in the future.

Now, some of you may have heard that we were giving away an Xbox today. My little bruised ego, I’m not going to think that you showed up just for that, but on the other hand, if it did motivate you, that’s a good thing. Just letting you know that we’ll give you the details of the giveaway and how that’s all going to happen at the end of the presentation. Obviously, we’d like for you to stay engaged until the very end so that we can share some useful information to you.

Who am I going to be speaking to today? First of all, I’m expecting that some of you are in the role of developer and you may be a developer that’s experienced with SharePoint or maybe you’re kind of new to it. Regardless, I’m going to show you some code today. We’re going to do some things in Visual Studio, but even more than that, I’m going to show you some concepts that will let you go back and help your SharePoint users get the most out of SharePoint.

The second audience that I hope to address today are those of you that just showed up and you’re just SharePoint users and you want to learn some cool new stuff. I think what you’re going to see today is really two things. Number one, some tips and techniques that you can actually go use this afternoon after the webinar is over, and then beyond that, just kind of a knowledge of what’s possible. Sometimes that’s really helpful even if you’re not the person that’s going to write this little application that does this wonderful thing. Just knowing what it possible will then allow you to go back and have a conversation with your developers about what you want to accomplish.

Then, finally, some of you are just kind of more in a leadership or a business decision-maker role and more than anything, through the code and the concepts and the tips and the techniques and the knowing what’s possible, my goal is to help you get the most out of your investment in SharePoint. I think SharePoint has some really compelling capabilities, but I think it’s also kind of a shame that a lot of enterprises under-utilize that and that’s part of what we’re hoping to do in this webinar series is not only raise some awareness about ThreeWill, but also some awareness about some of the cool things that you can do with SharePoint inside your company.

With that, what are we going to talk about today? Well, SharePoint is all about collaboration. That’s one of the principle reasons that it exists. We’re going to talk about SharePoint in light of another tool that we use for collaboration and that is email. Whether it’s right or wrong and whatever the reasons for it, I would make the argument that this is the way that most of us collaborate in business. We put an attachment on an email and we ship it off and the process begins. What I’m going to try to do today is convince you that that’s not the right answer in all circumstances and that if you have access to SharePoint, you have a much better mechanism at your disposal for doing that kind of collaboration.

Then from there, we’re going to build on that and talk about what we call around here “a single source of truth”. If we’re going to collaborate on something, then we all kind of have to agree upon what the truth is, what’s the actual data, and what state that data is in. If you’re not already aware of it, I think you’re going to see that SharePoint gives us a really, really easy way at arriving at that. Easy enough, in fact, that we don’t even necessarily need to involve a developer for this part. If we can get our SharePoint users to the point where they understand and can implement this part of the story, then the next part gets pretty interesting and that is the producing of documents.

What I’m going to do is talk you through a couple of application scenarios where we can take data that’s been stored in SharePoint and then use that to produce PDFs or spreadsheets or whatever it is we’re trying to create, and in doing so, automate a repeatable business process. That’s kind of a cornerstone of what I’m talking to you about today. These are not things that are kind of one-offs, but they’re things that have to happen over and over and over. If we have to do a report on our quarterly sales numbers, we have to do that every quarter. If we can make that process simple and easy to consume, that’s just going to make life better for both those that produce the data and those that consume the data.

That’s really the over-arching story of this entire webinar. Even though we may talk about technical details or end-user details. Ultimately, it is about a repeatable process where we can automate the generation of documents that we use in the running of our businesses.

Then, with that, let’s talk a little bit about the collaborating via email model. I’m not trying to hide my disdain for it here, at least in certain circumstances. The reason for that, if we go back through the history of email, originally it was a text medium. It was just a way to send a message to someone and get a response. Really, I can remember in my career I first started using email, the thing that I thought was so awesome about it was that it was in asynchronous form of communication where I could send off a message and when this person got time, they could respond. I didn’t have to interrupt them by calling them on the phone, nor did I have to sit and wait on the phone while they tried to formulate an answer. In some respect, it’s a very, very efficient way for human beings to communicate with each other.

As email evolved, it got some extensions. Most of you are familiar with this acronym of multipurpose internet mail extensions or MIME. Now, several really good things came out of that. The ability to have different character sets. The ability to have some more sophisticated capabilities in mail. Depending on its intended use, one possible unfortunate side effect is the advent of attachments.

I know you guys love attachments. You use them a lot and I use them a lot, too, but I say “unfortunate” in a couple of respects. Really the problem with attachments is that they don’t require a lot of forethought. They don’t require us to put anything in place in order to carry out the process. We can just at the drop of a hat say, “Oh, you need that document? Boom, I’m going to send you an attachment.”

In a general sense, there are three, maybe more, kind of issues with this that I want to talk about. I’m going to focus on the first one. The versioning or the accuracy of the data. That is really hard to do when you’re working with attachments. Give Microsoft credit. Over the years they’ve tried to put some capabilities into their office products where they can track changes and be aware of the fact that it’s being circulated by email, but in the end, the problem here is just this wasn’t meant to be a collaborative authoring tool.

Even if you can get past that, another problem with attachments, and I have to confess I do this all the time, I know a coworker sent me something and I know it’s attached to an email, but gosh, which folder did I put it in or did I accidentally delete it? This whole accountability of being able to locate a piece of important data quickly. If I don’t religiously get those things out of the email and into some well-known place, it’s very, very easy for me to lose track of them. At that point, I’m either at the mercy of the organizational system in my inbox or I’m at the mercy of the search facility in my inbox in order to locate something that’s really important.

Another one that’s a challenge with email attachments, and I’ve experienced this firsthand, you’ve got someone that’s kind of responsible for some important data, they have it in their inbox in a series of attachments, and they depart the company. Before we really get a chance to contemplate what it might mean, their email gets decommissioned. Their inbox gets blown away. This data that might really be important to us is kind of lost there.

My encouragement to you in a very general sense is before you start attaching business documents and shipping them off, think a little bit. Think about maybe there’s a better mechanism that we could use to accomplish that kind of collaboration. Now, the flip side of that, if we’re not really going to be changing the data, if I’m just sending a map of directions to my friends so they can come to my party, attachments are awesome for that. I think it’s just another case of let’s make sure we’re using the right tool for the right job.

Then let’s take a typical scenario and I can say typical not only because of I’ve experienced this, but because literally we’ve written applications for several of our customers that follow this exact pattern. We start out with a series of sales managers. We’ll say that we have a manager of each one of our regions and each one of these managers has got to contribute to a spreadsheet of quarterly numbers. Ultimately, the consumer of this spreadsheet will be people that are high in the company, executives. It’s really important that the information be correct and up-to-date.

How we usually start this, well, one of the managers will step up and say, “Okay, I’ll start the spreadsheet.” He or she goes in and puts in their numbers and then they do an email and they send it to all of the other users at once. Then they all take a solemn oath that they’re going to make sure that they keep track of who’s making what changes so that we can keep everything up-to-date. The reality is that rarely happens. The original author forgot they made a change and they went back in and did it and now two other people don’t know about it and so in a more specific sense, we have multiple copies, multiple changes, but no single, authoritative copy.

It’s funny. I’ve seen people go to just great lengths to make this work. They’ll start attaching version numbers to the end of the documents or they’ll start attaching their initials and a version number. Again, that’s just all treating the symptom instead of treating the actual problem here. Then put yourself in the role of the VP of Sales. This person has now got to try to figure out which one of those spreadsheets has the most correct data and then they’ve got to use that and forward it to the executive team. If we’re willing to do a small amount of work up front, what we can do is make it so that this information can be more reliably entered, more reliably modified, and then more reliably reported to those that are interested in it.

Let’s look at the problem a little bit differently. We said that the goal here was to produce a spreadsheet that had quarterly sales numbers that were accurate and usable, but part of the problem is that we started with the end. Let’s don’t start with the spreadsheet. Instead, let’s start at a higher level. We have a scenario where we’re trying to author or create some content and we have many contributors to that. Whenever you think about problems in business and how you’re going to solve them with technology, that’s something you always want to consider. Who are my authors and how many of them am I going to have? In this case, if we have regions, we’re going to have five authors. If we have thirty regions, we’re going to have thirty authors. We need a mechanism that works as well for two as it does for twenty-two. Something that will scale in the human sense.

Another way of looking at this problem is not only do we have many data contributors, but we have a relatively-small number of reviewers or approvers. In fact, in this case we have exactly one. The Vice President of Sales. Not only do we need to have a mechanism that will allow many people to contribute, we also have a mechanism that will allow those who are responsible for doing so to be able to review, approve, and reject content without a lot of botheration.

Another thing that’s really important here, if we’re going to make this work, we’ve got to have one location for the data and there has to be a mechanism for managing the changes in that data. This is also a really important one. We want to do this in such a way that those who would consume the data can’t see it until it is approved. That way we don’t get illegitimate numbers out in the wild and then we have to play this game of, “Oh my goodness, who all has this and how do I chase them down and tell them how to fix the numbers?”

Only then, once we’ve gone through the process of authoring, approving, and making the data fit to be used, do we worry about the producing of the documents. Then what you’re going to see in the little demonstrations that I’m going to do for you is that in a very basic sense, we can use SharePoint itself to produce a document or we can write some applications that would give us a great deal of power and flexibility on how we produce those documents.

Then let’s contemplate for a moment how we’re going to use SharePoint to drive us toward this single source of truth so that we can all agree upon the data that we’re acting on. Well, in my scenario what we’re going to do is, instead of entering those numbers directly into a spreadsheet, instead we’re going to enter them into a SharePoint. As you’re going to see in the demonstration in just a moment, this is a really, really simple SharePoint list that any user can go in and create on their own. I think that’s one of the most cool things about SharePoint and it’s something that you really need to be tapped in if you’re an end-user and that is you can get this process rolling into a usable state without having to rely upon a developer. Then once you do get it to a usable state, we can bring the developers in and even automate it further.

The user’s going to create the list and then once the list is created, now we have an easy path for all of those that are submitting the data. Quite literally, we can circulate a URL and say, “From now on, when you need to submit quarterly numbers, this is the place you go.” Not only could we give them the URL of the SharePoint site, we can even give them the URL to the specific list. It really doesn’t get much easier than that in terms of them knowing where to go in order to submit the data.

As you’re going to see in the demonstration, SharePoint has some built-in approval capabilities that you may or may not be aware of. They really take two levels. At a very, very industrial-strength level we have this notion of publishing and publishing sites, but at the list and library level, we also have this concept of approval. It’s really a breeze to set up. It just says, “I’m going to make it so that list items are only visible to the public at large or to the consuming public once we’ve approved them.” By default, when the data is submitted, it’s in a state where it’s not going to be seen by anyone other than the contributor and those that are authorized to approve it.

Let me go in and just show you a little bit about how this is going to work. Now, a couple of disclaimers as I go through my demonstrations here. Disclaimer number one is that a lot of what I’m going to be showing you today, I’m actually running on a local VM on my system. Because I’m running it on a laptop, sometimes it might not be the fastest. I wouldn’t draw any negative conclusions about SharePoint on that. It’s just the limitations of running in a VM.

Another thing that I want to point out is as I go through my various demos, I’m actually running some of them against my local SharePoint instance and some of them against Office 365. I think the important point there is just whatever you’re going to learn today, it would actually apply equally to both of those environments. Whether you’re cloud-based or not, it’s not really going to matter.

Maybe a third quick point to make about my demos. These are all very-much focused on SharePoint 2013, however, many of the concepts and much of the code that I’m going to demonstrate could also be applied to a 2010 environment. Some of the application bits that I’m going to do would not be quite as cool or quite as integrated in a SharePoint 2010 environment, but the basic concepts would still exist.

I’m going to start with a very simple demo where I go into the contents for my site and I add a custom list. I’ll start the process by doing Add an App and then choose Custom List as my template. Then, for those of you who haven’t done this before, really all I have to do is give it a name and start customizing it. I’ll just call this, “Sales Numbers List,” and create it. Once I’ve created that list, I can navigate to it. I can click on the List tab and then go to List Settings. Now, obviously the presumption here is that as an end-user, you’re authorized to do this. If you’re not, then that’s something that you wouldn’t want to talk to your administrator about. You don’t have to be a site owner or an administrator to be able to change the capabilities of a list.

There’s two things that I would potentially do here in order to implement this. The first thing I would do is go down and create some columns. Obviously, I would want to go in, as example, and create a column for my gross sales with currency and my net sales and the year and the quarter and all those other things that I want to be able to track. Then once I’ve created the necessary columns, the other thing that I’ve got to do is set up my approval mechanism. Again, this just couldn’t be easier because literally I’m going to go into the Versioning Settings and right here, I’m going to turn on Require Content Approval for Submitted Items. That’s literally all that’s involved in getting this thing up and going.

Now, kind of like they do on the cooking shows here, I’ve shown you the basics of how I would do this. Let me know go over to a site where I’ve actually got it set up and I can show you this list in action. This particular one is named, “Quarterly Numbers,” and you can see that all of my sales managers have done a really, really good job of getting in their numbers for the third quarter. Now we find ourselves approaching the end of the fourth quarter and I’m going to put myself in the role of a sales manager and I’m going to go in and add some data. I’ll just simply go up and add a new list item and then notice the warning that the user gets here. It basically says, “This is a list that’s using the approval mechanism, so when you submit your data, it’s only going to be seen by you and those that are eligible to do approvals.” I’ll talk about that a little more in just a moment, those that are eligible to do approvals.

I’ll go ahead and put in my fourth quarter numbers here and we’ll choose a particular number value for each one of these. Again, I don’t pretend these are accurate numbers. I’m just trying to put in something for demonstration purposes. Now, notice if you will, when I save this, it immediately goes into a pending state. I can see it as an author and then anyone else that has approval permissions would be able to see it.

Now, if I switch roles real quickly, now I’m in the VP of Sales role and I’ve got to go in and make a decision about whether this thing is legit or not. I’ve done a little research and what I’ve discovered is that this number is actually incorrect based on some other things that I’ve seen. I’ll select the item, I’ll go to the Items tab on the ribbon, and then I’ll choose “Approve and Reject”. What I can do is set it to a rejected state and then put in some details about what’s wrong with this and what needs to be fixed. It’s a very simple process. Those that are contributors can easily see whether their data has been approved or not. In fact, they can even set up an alert on this list to let them know when data has changed that they care about. Later on, if they do realize their mistake and they come in and fix it, it’s a pretty simple matter. All they’ve got to do is go in and edit the item and recognize that they put one too many zeros on that and save it. Now it’s gone back to a pending state and someone could go in and approve it.

Those of you out there that are more on the end-user scale of things, that’s a ton of functionality and, again, it’s maybe ten minutes’ worth of work to set it up. If you’re not already familiar with those capabilities of using a list that way, I want to strongly encourage you to dig a little deeper on that and give it a try. I think what you’ll see is that you can go a long way toward accomplishing what it is that you’re trying to accomplish.

Now, the last thing I wanted to show you is how we get permissions for these. I know that some of you would be the ones that would set up permissions. Others of you, you’re not going to be doing this, but you just need to know a little about it so that you can ask the right questions. If I go to my Site Settings page and then I select Site Permissions, what I want to show you is just real simply how you might set this up. What I can do is I can set up a group for people that should be allowed to approve the content. I can name that whatever I wanted to name and I could put in whatever individuals I wanted to put in.

The key part, though, is right down here. Notice the various permission levels. Obviously, we don’t want to give them full control, but in order for someone to be approver, they have to have at least design-level permissions. Obviously, my VP of Sales, he or she is going to be in the role of approving this content. They would need to have this effective permission in order to do that. Most likely, those that are contributing data are just going to have contribute capabilities. Those that might be responsible for configuring lists would need edit capabilities. All of it’s there and it’s out of the box, you just got to know how to stack things up correctly in order to take advantage of it.

Now that we’ve reached a point where we can pretty easily capture that data and feel pretty confident that it’s going to be accurately disseminated, let’s now talk a little bit about how we can get at the data. If you’re lucky enough to have a developer at your disposal, we’re going to talk in a few minutes about some of the amazing things that you could do with that. Even if you don’t have a developer at hand, once you get the list looking the way that you want it to look, it’s a pretty simple thing to get it into Excel. Literally, I’m going to go over into Quarterly Numbers and I’m going to go to the List tab, and then I’m going to choose Export to Excel.

The cool thing about this approach is that it’s not actually downloading the data, rather it’s downloading a query that pulls that data from SharePoint. The idea here is I can now slice and dice this data and accomplish whatever I want. What I would tell you is that in the real world, this is an approach that’s really cool and probably okay for someone that’s a bit of a SharePoint jockey, but for someone that’s more maybe in an executive role and they just want to push a button and have the stuff at their disposal, this probably draws them a little farther into the SharePoint experience than they want to go. For that audience, that’s the case at which we’d probably need to think about writing some applications. Let’s go have a look at that.

How is it that we’re going to produce our documents? Well, in a very general sense, what we’re going to do is write an application that uses SharePoint as its source of data. No doubt many of you have either written or worked on applications where someone wrote a “front end” that knows how to read out a SQL server or some line of business application. We’re essentially doing the same thing. In this case, instead of reading the data and putting it on screen in a data entry form, these are apps that are going to read the SharePoint data and produce some kind of document from it.

I’m going to show you two approaches that you can use to accomplish this in SharePoint 2013. The first of those is a SharePoint 2013 app. The advantage to this one is really kind of multi-fold. Number one, it integrates directly with the SharePoint site. The same destination that someone might go to to enter data, they can go to that same destination and with a couple of mouse clicks extract the data that they need.

Another thing that’s nice about the SharePoint 2013 app approach is that because it has some knowledge of SharePoint, not only do we have the ability to programmatically read data out of the SharePoint list to produce the document, but then we can turn around and take the produced document and store it in the document library. The person responsible can generate that document, but then any other party that’s interested in it can know that if they go to that library, they’re going to see an approved and released version of that document.

The other approach, and this one is completely new as well, something called an Office 2013 app. I have to confess, even though these have been out for a little while, it’s only now that I’ve gotten an opportunity to start working with these. I have to admit I’m actually pretty excited about some of the things that you can do here. The idea with this is instead of having some kind of app that integrates with the SharePoint site directly, instead I’m going to write an app that actually integrates with the desktop version of Excel or PowerPoint or whatever app I’m writing it for. The really cool part of this, as you’ll see in my demonstration, is that the consumer doesn’t have to visit SharePoint. They don’t have any concept that SharePoint is even coming to the party. All they know is that they can launch their Office app and make a few mouse clicks and wind up with the data that they want. I think you’re going to like what we can do with those.

I know some of you have been to prior webinars or maybe you just read up on your own enough that you already know about this, but on the chance that a few of you kind of don’t know the SharePoint 2013 app story, let me just take one slide here and talk about that briefly. First of all, when we say “app,” we kind of joke around the office of App with a capital “A” versus app with a lower case “a”. This is an app with a capital “A”. It’s a particular kind of programming model that’s new for SharePoint 2013. Apps are created by developers. This is not a power user thing, but rather a developer thing that is created on behalf of users. When we write SharePoint 2013 apps, we’re writing code that runs out-of-process or, perhaps betters said, runs outside of SharePoint. There’s a lot of reasons why that’s a really good thing, not the least of which is it kind of insulates our SharePoint server from being slowed down or brought to its knees by bad code.

From a developer perspective, when I say “out-of-process,” what does that mean? Well, it really means your code is going to be running in one of a couple of places. It’s either going to be JavaScript that’s running in the browser or it may be code that’s running on another server. Depending on the kind of server it is, it might likely be .NET codes such as C#. It’s also possible that you could have a non-Microsoft server that’s integrated into this process. Either way, you could just think about this model as being, “I’m going to write my code and then that code is going to remotely call into SharePoint and get whatever data it needs.”

How do you users get these apps? There’s really two models. First of all, you would have something within your enterprise called the App Catalog. Essentially, the App Catalog is like a giant library where all authorized users can find apps and install them to their sites. There’s also a global SharePoint Store. Most companies probably wouldn’t open up to their users for fear of what they might download, but the cool thing about the store is that as a user, I can actually visit there, find an app that I like, and then make a request for it in my company’s app catalog. At that point, an administrator would see that request, see that I want that app, they could go review it, and if it makes sense, they could actually purchase it and put it in the app catalog.

There’s a lot of exciting things going on there. For users, your developers are going to be able to build you some really cool things that are very easy for you to consume. For developers, what you just got is a much better mechanism on rolling out and updating your apps and having a central place where people can actually discover those apps.

For the specifics of the app that I’ve written for you that I’m going to demonstrate here in just a moment, we’ll start with the list that I showed you just a moment ago. I’ve got a SharePoint list and it contains sales numbers for a variety of quarters and years. Then from there, I’m going to write what we would call a cloud-hosted app. That is to say, I’m going to have some server code that runs in another server. It’s going to call in the SharePoint and then it’s going to read the sales numbers from that list for a particular year and for a particular quarter. Then those of you that are developers, what I’m using here is the C# client-side object model. I’m just making a remote call into SharePoint.

Now once I’ve pulled down the data that I need from SharePoint, the next step is to produce the PDF in this case. I’m going to do that via a third-party PDF library. Once I’ve actually produced that document, I’m going to turn right around and have my application write it to a document library and in that same SharePoint site. Then from there, the users will have two approaches to getting at that data. For the user that runs my application, they’ll actually be presented with a hyperlink that will easily let them read that document. In addition to that, anybody on the team that has access to that particular document library can navigate directly to it and see all of the contents within.

All right. Let me take a moment now and show you how this works. Now, those of you that are developers, you’re probably very excited to see that I’m in Visual Studio now. Those of you who aren’t developers, don’t let your eyes glaze over. I’m not going to stay in here very long. I’m just using this as a point to kind of help our developer friends understand what I’m doing here.

For the sake of our demonstration, I want to just focus on this page right here, default.aspx. This is going to run in my remote web and it has a very simple UI that’s just going to allow the user to pick a year, pick a quarter, and then generate the information that they’re interested in. Now, if we go and look at the code behind that is associated with this, even if you haven’t done any SharePoint 2013 programming just yet, if you’ve done some client-side object model in SharePoint 2010, it’s essentially the same here. The ultimate goal is for me to create a client context object. This is what will allow me to call into SharePoint and request the data that I’m interested in.

In this case, the data that I’m interested in, I’m going to extract that via a CAML query. Even if you have no idea what a CAML query is, basically we’re saying, “I want a particular year and I want a particular quarter.” Those are going to come from whatever the user happened to choose on the screen. Once I’ve asked for my data and received it back, then I’m just going to write a simple loop and in that loop I’m going to take that data and render it into a PDF. I’m going to take that PDF and write it to a memory stream. Then finally, I’m going to upload that file. Literally, I’m going to use the client-side object model to take a document that’s buffered in memory and upload it.

There should be two reactions to that. If you’re a developer, that’s pretty cool. I like the fact that I can do that. Then if you’re an end-user and you’re a little bewildered, let me kind of set your mind at ease. The reason you care about this as an end-user is that you didn’t have to think about where you stored the app. You didn’t have to worry about if you stored it in the right place or the right location on your hard drive. The app was responsible for putting it in the correct place and your job was just to ask for what you want and then get it. It doesn’t get much better than that.

All right, with that, let me go ahead and run this. Again, obviously those of you in the user role, you would not be running it from Visual Studio. This app would already be installed on your SharePoint site and you’d be able to navigate to it. Now, if I attempt to generate data for which I don’t really have any data, it’s just going to give me a little message saying that we don’t have any numbers for that particular time unit. Instead, let’s go get the third quarter of 2013 and I’ll click on “Generate”. Notice our users get two things here. They get a little indication that the PDF has been generated and then they also have this click-to-open link. As I click on that, you can see that I literally generated a PDF that just gave them a formatted list of the available numbers.

Really important here. Don’t get hung up on the format of the document. I wasn’t trying to make an artistic statement here so much as I was just trying to show you that you can programmatically create these things and upload them to SharePoint fairly easily. Obviously, we have a particular look and feel that we wanted to achieve and we can certainly do that on behalf of our users.

Now the other thing to think about is if I returned to the site on which this app is running, at any future point, if I want to see this document, I can literally go into the document’s library and there’s the PDF that I just generated. Couple of things that I think are kind of cool about this, not only did I generate the document and it knows that it was me that generated it, there’s another little cool part here and that is, notice that last column, “App Modified By”. Those of you that are developers, you probably want to make a mental or written note of this. When you start writing apps, SharePoint not only has the concept of the user’s identity, but also the app’s identity. There are some really, really clever things that you can do with that. That’s a capability that I think is pretty exciting and holds out some interesting ways to do things on behalf of the user.

All right, so there’s the process. Going forward, how would our users consume this? They’d consume it in one of two ways. They would either go to their SharePoint site, go to the Site Contents area, and click on the app in question, or we could even simply provide them with a link. They use the link and it takes them directly to the app.

I’m going to keep plugging ahead here. Just an FYI, I noticed maybe one or two of you have posted some questions out there. That’s awesome. Please continue to do that. I’m going to save those until the very end just so that I make sure that I get through all of my material. Please don’t think I’m ignoring you. I’m just holding those until the end.

All right, so the SharePoint app is cool enough, but let’s face it. There are some people out there, right or wrong, that are just a little biased against SharePoint. Maybe they had a bad experience. Maybe they just don’t like the way it looks. Doesn’t matter. The point is we can implement this same model that we just saw, a very convenient way for a consumer to get at the data that they want without them ever having to visit SharePoint.

That’s really what the Office 2013 apps model is going to be about here. Conceptually, it’s similar to SharePoint. You’re writing an app. It’s going to enhance this product or customize this product that Microsoft is providing to you. The difference here is instead of integrating through the web browser onto a SharePoint site, it’s actually going to integrate directly with Word or Excel or PowerPoint or whatever you target this to.

When you write these apps, what you’re really doing is you’re writing a web page. Developers, everything you already know about server code and HTML and JavaScript, those would all still apply. The difference here is that the web page that you’re authoring is actually going to be visible or hosted inside of Excel or inside of Word. As you’ll see in the code that I’m going to demonstrate for you in just a moment, you’ll actually have a JavaScript library that will allow your page to interact with the hosting application. I can put together logic that would allow me to inject data into a spreadsheet. I could put together logic that would let me read data out of a Word document and then do some kind of action based on that.

What is the integration point here? Well, because we’re using standard-based web tools here, we can take these pages that we’re going to add to Word or Excel and we can integrate them with a variety of back-end data sources, most notably for this webinar, SharePoint. The idea here is that the user’s going to be completely sheltered from SharePoint. They’re not even going to know it’s a part of the party, but we’ll be able to go out to SharePoint on their behalf, pull down whatever data they need, and present it in an easy-to-use format.

Now there are essentially three different kinds of applications that you can write for Office 2013. The first of those is called a task pane app. That’s the kind of app that I’m going to be building for you. As you can see, the app renders in a Window to the right of your main document and then, as I mentioned a moment ago, this app is going to have both the capabilities of putting things into the document as well as reading things out of the document and taking some kind of answer on them. Then this back-end data source, you can see in this little simulated example, someone’s actually written a task pane app that will take text, call Bing, and translate it. Pretty much any back-end service that’s a breachable service by a REST call or a web service call, we’re going to be able to do those kinds of integrations from within Office.

A second approach is one where we right what’s called a content app. In this case, instead of being in a separate pane, this is actually an application that embeds directly in the document itself. In this example, we’ve actually embedded this app that someone has authored that will read data about states and sales and then actually give us a graphical map that shows where those sales are taking place. Maybe another way for you to think about these if you’re a developer, anytime you’ve got a user that’s trying to write a formula that’s really, really complicated in Excel or they’re trying to create a chart that has some highly-customized capabilities, it could be that you could author a content app on their behalf that would do all of those things and for them, it’s literally a couple of clicks to put it in place and use it.

Then, ironically enough, I kind of ragged on mail and attachments earlier in the presentation, but there are some cases where we’d want to be able to do some clever things with email integration. That’s what a mail app is about. A classic example here. As a user, I could highlight addresses and then I could have a mail-out that would actually go out and do a Bing look-up and show the user the map for that. Anything that you’re doing that is legitimately built around the email process, we can do some very clever things with the mail app.

Once again, for those of you that do better with pictures as opposed to words, we’re starting in the same place here. We’re starting with a SharePoint list that contains the sales numbers that you saw at the beginning of my presentation. Now in this case, instead of the user navigating to a particular place, they’re going to launch Excel. Then their Excel instance will be configured in such a way that they actually have access to this task pane app that I’ve written on their behalf. For them, it’s as simple as picking the right options, clicking a button, the task pane app then will go out and read the SharePoint data on their behalf, pull it out, and then populate it into a spreadsheet or whatever kind of office document that we’d like to put it in.

Let me give you a couple of demonstrations here. I’m going to show you how you can use this process to generate an Excel spreadsheet. I’m also going to show you how you can generate PowerPoint data. I can tell you again, literally, we have written apps on behalf of customers that do both of these things. The story here is just now that we’ve got these Office 2013 apps, the process gets really, really simple and really, really easy.

All right, so I’m going to go back in now and I’m going to open up the first of my two demo apps. Then, for the people in the room that are more on the developer track, you’ll notice that when you create an office app, structurally it looks very similar to a cloud-hosted SharePoint app. That’s a pretty decent analogy. To put that in another way, most of the code that you’re going to be writing is going to be down in this app web and then really the top part is just declaratively describing some information about your app. Among the things that it would tell you, what products can it work with. The fact that this says “workbook” here, that tells me that this is for Excel and that it’s not for PowerPoint. They don’t have to be mutually exclusive. You could actually write an app that would integrate with Word and Excel and PowerPoint all at the same time.

Then the other part of this, the actual logic here for creating the data and pushing it into the document. Again, I don’t want to belabor this too much because I know I have a mixed audience here. Those of you that looked at the code earlier, here’s what you need to take away. I’m using the exact same client-side object model code to fetch the data. Same query and everything. The only difference here is that once I’ve got the data, I’m going to format it in adjacent array format and then send that back as a string. Then I’ll actually have some JavaScript code that can take that, decompose it into an array, and then insert it into the document.

How is it that things get into the document? For those of you that are going to be writing this kind of code, this is probably the most important sentence of all in this demo and that is everything that allows my app to interact with Office is encapsulated in this office.js script. Literally it’s providing me with an object model that will allow me to read and write data from JavaScript down into the document or vice versa.

In this example, I’ve set my data back in a container named “Payload”. I’m going to take that, parse it into an array, and then notice what I’m doing in this example. Because it’s Excel, Excel has this concept of a matrix. To put that in simple terms, you can create a two-dimensional array, put data in it, hand that over to Excel, and Excel knows automatically how to format that data correctly. The method that I’m using here is called “Set Selected Data ASYNC”. Those of you that write a lot of code, you know that when you see that word “ASYNC,” you can optionally provide a callback that would let you do a success or a failure message. For this example, I’m leaving off the callback and I’m just putting the data in directly.

All right. Let’s give this a try here. I’m going to run it now. Again, those of you that are in the user role, you’re not going to run this from Visual Studio. You would just have Excel already configured to have availability to this app. Notice what I get here. When I run it, I’m running Excel. Then the markup you saw just a moment ago has now actually rendered in this window. If I try to get data for a year-end quarter that doesn’t have any associated data, I get the same message that I got before. On the other hand, if I go in and pick some valid settings here, notice that it retrieves the data and immediately puts it into the spreadsheet.

Now if I’m in the role here of the Sales Vice President, watch what I’m going to do. I’m going to go right here and select some data and I’m going to insert a recommended chart and maybe change a title or two and I’m done. Save my work and publish this information to whoever needs to get it. Then when I’ve got to do this again next quarter, I fire up Excel, I create a new spreadsheet, I click the mouse the same three times like I did before, and I’m done. To me, the exciting part of this is just that we’re getting the collaborative capabilities of SharePoint to manage the data, but for those who want to consume it, we consume it very close to the source. They don’t have to know that much about where it came from and how it got constructed.

The model for how we would do this on PowerPoint is almost identical. The one little bit of code that’s going to be different here is that PowerPoint really doesn’t understand the concept of anything other than text data. You’ll remember a moment ago when I wrote my data, I had a coercion type of matrix. In this case, I’m just writing string data. Literally what I’m doing here is just writing a loop that will construct the data and then I’m going to inject that into a PowerPoint document.

Let’s suppose that you’re the VP of Sales and you have to produce a PowerPoint every quarter where you report on how your divisions are doing and then one of the slides has to have your numbers. Well, you’re in PowerPoint. You’re authoring the different parts of the PowerPoint and then once you get to the point in time where you want to add that particular slide, you just simply come over here and click “New Slide,” and then you click in the body of the slide. You choose the year that you’re interested in and the quarter and you click “Get Data”. Notice what it did. It actually read that data from SharePoint, formatted it in a way that is appropriate for my PowerPoint, and so now all I have to do is put on a title and then continue the process. Again, I think that is a really compelling story on how you can generate these things very easily.

All right, let’s hit a few wrap-up points here. We’re getting near the top of the hour. Kind of recapping what we talked about. First of all, if you know you’re going to be collaborating on something where there’s going to be more than one author, don’t use email. Spin the ten or fifteen minutes that it would take to set up a SharePoint list or a library and keep the data there. You’re just going to be way ahead of the curve even if you don’t have the luxury of writing applications. That single source of truth. Really in this one, I focused on lists because the data was more appropriate for a list. If you truly have to author a spreadsheet and that’s the only way you know to do it is just to author the spreadsheet, you still ought to be keeping it in a library as well. Then you can use the check-in and check-out mechanism just to make sure that you’re not trampling each other’s changes. These are going to be much better alternatives than trying to track it by email.

Another thing, take advantage of that SharePoint approval mechanism. Literally one click and you’ve got yourself in a circumstance where people can’t see the data until it’s been legitimized by some reviewing party. Then from there, for those of you that are developers, I hope I kind of whetted your appetite a little bit there in terms of writing SharePoint apps and Office apps as a way to generate these documents. Then, if you’re more on the user side of the house, while you’re not going to be authoring these applications, you now kind of know what’s possible and this is something that you can take back to your developers and kind of blaze a trail on getting these things done. Obviously, part of the point of these webinars is just the fact that ThreeWill does this. We’ve done this for several our customers and we’ve got a really good track record in making it so that people can get these documents produced repeatably and reliably without having to exert a lot of effort.

I hope enough of you stayed awake to this part of the presentation to kind of hear a little bit about our giveaway. Again, in order for you to be eligible for the giveaway, not only did you have to attend the webinar, but you’ve got to properly complete the end-of-webinar survey. You’ll get prompted for that when you exit the webinar. If for some reason you miss that or you lose that link, you can email me and I can send you an alternate link. I encourage you to go ahead and do it. Notice it says, “properly complete.” There is actually a question in there where you have to say, yes, you’d like to be registered for the drawing. The reason that we’re doing that is that we occasionally have customers that work for companies where they’re not allowed to receive gifts over a certain amount. If you’re at a company where you’re not allowed to do that, you may want to forego the contest, but for the rest of you, that’s how you’re going to opt-in.

A couple of details. We’ll perform the drawing tomorrow, so that will give you the rest of today and the evening to get the survey filled out. It shouldn’t take long at all. It’s a very brief survey. We’ll announce it in a couple of ways. We’ll announce it on our Twitter feed. That’s @threewill. We’ll also announce it on our Facebook page, facebook.com/threewill. Then just as a follow-up, we’ll also notify the winner via email. Then pretty much like any contest, the decision of ThreeWill concerning the giveaway is final.

With that, I’m at the end of my prepared material. For some of you, you’ve got a busy day ahead of you and you’re going to launch onto other things and that’s great. For those of you that can stick around a second, I’m going to see if I’ve got any questions. You can post those questions right now if you haven’t posted them already. If I have any, I’ll read those out loud.

I didn’t get a question so much as a comment. A couple of you thought that the Office 2013 app integration was pretty cool. I completely agree. I think that there are some enormous opportunities there for you to integrate data and make it very easy for your users to consume and let them be largely naïve about where that data is coming from. They just get what they need and they don’t have to do a lot of clicks.

Again, I’m going to stay on the line here for a couple of more minutes. If you’ve got your questions, please add those to the question manager and I’ll respond to those out loud. Otherwise, I really appreciate your time. I know that any time you invest an hour of your work life, you’re hoping to get something out of it. I hope that you’ve seen some things here that would apply to your organization. If you’ve got any questions or any interest whatsoever in letting us help you out with that, feel free to drop me an email. Again, junder[email protected] and I’ll be sure and connect you to the right place. Thanks again for attending.

read more
John UnderwoodSharePoint Document Generation Webinar
enterprise-sharepoint-search.jpg

Enterprise SharePoint Search Webinar


Hello folks, John Underwood here welcoming you to another of the ThreeWill Webinar series. Today, we’re going to talk about Enterprise SharePoint Search. I hope you’re having a good day wherever you’re at. I’m coming to you from just North of Atlanta, Georgia in Alpharetta and we’ve actually had a rare sunny day around here. It’s been a pretty good day. A little bit about me. For those of you who’ve not done events with me before, I’m lucky enough to be an employee of ThreeWill. My job title here is technical evangelist. That means I get to do events like this. I also do training and I even get to work on the odd project every now and then. A couple of ways to contact me obviously, you can use my email address on the screen. If you are someone that enjoys being a part of Twitter, there’s my Twitter handle. You’ll get a nice mix of hobbies and information, but mostly, you’ll get some good SharePoint information. I hope you’ll take advantage of that and follow me if you’re inclined to do so.

I know we want to get right into the material, but a couple of quick housekeeping items before we start. First of all, today’s session is being recorded. If you find that you have to drop off in the middle or if you’ve got some co-workers that were unable to attend but maybe want to see the information later, it will be available online for a certain amount of time. Essentially what you do is go back to the same URL that you used to originally visit and register for the event. You can take that invitation that you received and you can forward that to any of your friends or associates and they’ll be able to use that in order to play the video. Again, it’s interesting, even if they didn’t attend, they’ll still be able to register and view the video.

Then, one other thing. Although we hope we won’t have any technical interruptions, it’s not impossible that a phone would get disconnected or the power might go out temporarily or something like that. If any of those kinds of things happen during the event, you can notify me that you’re no longer seeing me. If you just send an email to the address you see there on the screen, that comes directly to my phone and at least, I’ll know something’s going on and we can see if we can’t get service restored and resumed. Again, that’s rare that that happens.

All right, with introductions and housekeeping items out of the way, let’s talk about what I’m going to cover with you today. First of all, I’m going to talk a little bit about search-driven applications. I think you’ll see that if you don’t already know about these, they’re going to represent what is for a lot of companies a missed opportunity. The ability to really exploit the SharePoint search engine in a way that’s going to be beneficial to you. Then, after that, we’re going to talk a a little bit about extending and customizing search.

A few of the things we’ll see here is something where things that maybe a power user could do but a lot of these are going to be more developer or even administrator oriented. Really, what I hope to do with this part of the presentation is twofold. Number one, just letting you know what’s possible and number two, take an opportunity to show you some of the things that ThreeWill has done as far as extending and customizing search on behalf of our customers.

Then, finally, I’m going to wrap up with just a quick look at some of the new things in SharePoint 2013 as relates to search. I know inevitably when there’s a new release, there are a few companies that jump on it immediately, but there are others that take their time. Since I’ve been working in the SharePoint universe, I know that’s particularly true of SharePoint moving up to a new version is not trivial.

Really, more than anything, I just want you to know what’s coming and really plan for some of those opportunities at the risk of sounding like a Microsoft product personnel, will tell you that some of the new search things in SharePoint 2013, I think are really compelling. They alone may not be a reason enough for you to upgrade, but taking in a whole lot, I think it is a worthwhile upgrade that you’ll want to undertake as soon as possible.

Then, time-permitting, we’ll do a little Q & A at the end. If time doesn’t permit or if you don’t think of any questions at that point, of course, I always welcome questions in the future at my email address, so please take advantage of that.

All right, so let’s spend the first part of our time together now talking just a little bit about search-driven applications. The hope here is that if you haven’t already thought about the search engine in this way before, after we’re done here, you’re going to see it as more than just a place where you type in words. Hopefully, you’re going to see it as a repository of some really valuable information for your company.

Let’s start with a common scenario. An application that we might need within our company. Suppose that we work at a company. We use SharePoint pretty heavily. We’ve got a corporate home page that’s hosted on SharePoint and then we have lots of site collections and sites and sub-sites and teen sites and all these different things. We have a legitimate need to present a list of sites that a particular user can see. Again, it could very well be that a user has access to a useful site and they don’t even know it. They don’t even know what’s out there.

One of the ways we might solve this problem is by writing a Web Part. Again, I’m a developer and what do developers do? They develop. They write code. Almost inevitably, if someone brings a business problem to us and says, “How are you going to solve this?” The first little thought balloon that pops up over my head is, “I’m going to write some code.” In this case, we’ve got a developer that creates a Web Part and that Web Part iterates through sites and makes a list of available sites based on the current user.

Now, we all know and love this phrase, “It passes the developers works on my machine test.” What we see here is that, a user or excuse me, a developer that’s written this may be testing it in an environment where they have a dozen sites. Then from there, we deploy it to our company’s home page where we have thousands of sites. All of a sudden, somebody goes to the home page and it’s taking forever to load because this little application is now going to try to iterate through those thousands of sites for every single page visit to the home page.

There’s really a couple of things wrong with this. Thing number one, obviously, is that, this is going to be a huge performance hit and people are going to hate SharePoint when it’s really not even SharePoint’s fault. The other thing here that’s a little more subtle but perhaps even more important and that is, there was a huge missed opportunity here. Because of the preconceived notion of how this developer viewed this problem, they didn’t see that there’s actually an efficient, simple and easy way to solve it. Easy enough in fact that in some cases, we could even rely upon a power user to do that.

What I’m talking about there is really a search-driven application. What is a search-driven application? Well, in simple terms, instead of us trying to use the lists and libraries in our SharePoint world as a source of data, we’re actually using the search engine or more correctly, the search engine’s index as our source of data. Because after all, that index contains a lot of really useful things.

Now, when we talk about a search-driven application, we’re talking about a scenario where we’re not going to go ask the users to type in some data and search for things. Rather, we’re going to search on their behalf. Really, they may not even know that it’s coming from search, and they may not even see the query. We may do it in a very subtle way so that that information is hidden from them. All they know is that they are able to undertake some actions such as clicking a link or visiting and web page and they get whatever it is that they needed.

Another nice thing about search-driven applications is that, the results are security-trimmed, by the very nature when you go into SharePoint and you search for things, you only get results for the things that you are authorized to see. In the case of as an example trying to show a user all of the sites that they have access to, by definition the search engine’s not going to let us serve any site references up to users unless they’re authorized to visit that. That just means there’s a whole level of coding that we often have to undertake that we’re just going to be able to ignore here because the search engine and its index already knows the answer to that question.

Then, another thing about a search-driven application, and while this isn’t a requirement, I think it makes it more compelling and it’s really nice. That is, results are presented to the user in a way that are ready to work with. To say that a little bit differently, certainly we could take users to a search page and we could teach them how to look for the thing that they want to look for. With the little effort on our part, we can customize the output from that process in such a way that they don’t get a lot of the other decoration that we would normally see on a search results page. Instead, they’re just going to get the data in question and the importance of that data to them.

When I do some demonstrations on this in a few minutes, I’ll reinforce what I’m talking about when I say presenting the results in a way that are ready to work with. If you’re a developer and you’re on this webinar today, here is my encouragement to you. When you start thinking about the sources of data that are available to you when you are customizing SharePoint, don’t forget the search index. Even furthermore, almost any time you’re writing some kind of customization that’s asking a question, the odds are pretty good that the search engine would know the answers to those questions.

Let’s dig a little deeper then. If it’s all about asking the right question, then how can we ask the right question? At least traditionally, when we think of a search engine, we think about a keyword-driven search. As an example, excuse me, a user might type in the word invoice hoping to find things that are related to invoices or resumes, or a series of what we would call keywords. However, the thing that a lot of people don’t know and that is really useful for the search-driven application, and that is SharePoint has a capability that allows us to specify the kind of thing that we’re looking for. Not only might we say, “I want invoice,” but we might say, “I want invoice and I only want that word where it appears in a list or where it appears in a library.” We can refine the kind of thing that we want to get back from the search engine.

If you think a moment ago about my example of trying to give a list of sites that a user has access to, we could specifically say to the SharePoint search engine, “Hey, I only want you to send me back things that represent sites. Anything else, just leave out of the results.” Then, another capability that we have in the search engine that not everyone knows about, we can also qualify details about the item that we were looking for. Not only could I say, “I only want items that appear in a task list.” I could go further and say, “I only want the ones that are authored by Kim or the ones that are assigned to Kim.” This notion that the entities that we’re looking for out there, each have some properties associated with them, and the query language actually gives us some mechanism for specifying this property.

Then, let’s be a little more specific. How is it that we’re going to qualify the type of results? Well, one very easy one and again I’ll be demonstrating this in just a moment, is the use of IsDocument. That’s pretty straightforward. It just lets me say, “I either want to explicitly return results that are documents, or are not documents.” By doing that, I may be able to go from a search that gave me a whole lot of results, to a search that gave me a set of results that are a little closer to what I’m looking for.

Now, the next one, and in my opinion, simultaneously both the most important one and the one that’s probably the most underused is the concept of a content class. You may not be aware of the fact that when you issue a keyword search in SharePoint, you’re actually able to specify the content class that you wish to receive back. Even if you’re not a real SharePoint expert, you can figure it out based on what you see here. As an example, I could say, “I want to look for the word invoice but I only want to see back a list item.” Or, “I only want to see back something from a document library.” Then, we can even go a step further. Not only could we say, “I only want to see items from a list,” but we can even provide a type that says, “I only want to see items from a list that’s a task list or a contacts list, or even a calendar.”

When you begin to think about this ability to not only ask for what you want but to specify the kinds of things that you want, that’s really a cornerstone of building these kinds of applications. Then, I handed that just a moment ago, this is just a more formal look at that notion of a property restriction. In effect, we’re saying, once we found the things we think you’re looking for, we can even be more specific about, it should have a certain file extension. It should have a certain file name. It should have a certain author and so on and so on. Taking what we already know about keywords and laying on these layers of qualification, that’s how we’re going to be able to not only write better queries to start with, but then take these queries and package them up in a way where users can conveniently consume them.

Let me take a moment now and do a quick demonstration for you. I’ll show you a little bit about some of these restrictions. Now, just a little FYI here. I’m going to be working in a virtual machine environment and I’ll just confess to you right upfront. It’s running on my local machine and it’s not got the most horsepower in the world behind it. Don’t try to draw too many conclusions here about the performance of this application. Really, let’s just focus more on what it does. Now, those of you that have worked a lot with SharePoint, probably recognize the fact that I’ve got a SharePoint 2010 Team Site that I’m working with here. You can see I’ve got a top-level site named Acme Corporate. I’ve got a couple of sub-sites underneath. Then, I’ve got a search center setup as well.

Let me see if I can demonstrate now some of the techniques that I’ve talked about in terms of refining your search and then from there, we’ll put that all together into a simple little application. Let’s suppose that I’m looking for a resume and I know that it was authored by Kim. I can start with a real simple search just by putting in the word resume and then searching for that. Then, we can see that, I get back results here that are all over the board. I’ve got different things that are actually resumes. Not only that, I’ve even gotten pages that just happened to have the words resume on them.

Now, if I go a step further, and I can say, “Well, let’s refine it so that it looks for both of those words.” Certainly, that gets us a little closer but you can see that I’m also winding up with a little more debris down here that’s not really relevant to what I’m looking for. Again, in this somewhat contrived example, I’ve gotten it down to three items. I could probably live with that, but those of you that work in very large corporate environments, the little refinement we just did there maybe took you from 10 thousand results to three thousand results. That’s a huge improvement, but you’ve still got a big hole to dig out of.

Let’s refine it just a little bit more. We want to look for the keyword resume, but we only want to get back items where the author of that document is Kim. You can see now that we’ve got a much better result at this point. We’ve got the actual resume and we can see clearly in our results here that it was authored by Kim. Just in taking that much of an approach on looking for things, you can see that we’ve really been able to take a machete and cut away some of that underbrush that we frequently get when we’re looking for search results.

Let’s take this a step further now and look at how else we might use this. Let’s say that I’m in search of something related to tasks. Now, if I just type in the word tasks, I’m going to get a lot of different things. Depending on what I was looking for, it could just be that I’m looking for something related to documents that have the word tasks in it. Or, it could be a classic problem that we have in SharePoint where I’ve got 15 different sites and each one of them has a task list and I’m just trying to figure out which lists have an assignment for me. Let me see if I can tackle both of those for you.

For the first one, let’s suppose that I truly do want to look for documents that contain the word tasks. Notice what I can do. I can put in this document and in this case, equals one means that I want it to be a document, equals zero means I want it to be a non-document. Just by adding the equals one, notice now I went from dozens of results, several pages worth down to just three. One little qualification on this one to be aware of as well and that is, notice the definition of a document. Not only do I get a Word document which is what a lot of us think of when we think of documents, we get library pages here. In the SharePoint sense of the word, they are in a library called site pages and they are in the documents.

Then, let’s take it one step further. Let’s say that we are truly interested just in Word documents. We could just go ahead and qualify the file extension on this just a little bit, and that would help us reign this in and see just the things we’re looking for. Right here I’ll just say, file type: docx. Then I’ll go ahead and search again. You can see now I’ve been able to refine it down to just the exact thing that I’m looking for.

Let me go one more step. Now I promised you that we would not only be able to find documents that have the word tasks in it, but that we’d also be able to find our task. In fact, I would tell you that of all the demos I think I’m going to do for you today, this is the one that is the most relevant and the most useful. That is, being able to find all of the tasks that are assigned to you across numerous sites. Let me start my query again. What I’m going to do now is, I’m going to change this one completely. I’m going to say content type:, excuse me, content class, and then STS_listitem_tasks.

Now, even if you are decidedly non-technical, this is a simple enough syntax that you can master it. Any time we see content class presented this way and then followed by some kind of qualifier, really what we’re saying is, “SharePoint, I only want you to bring back things of this kind.” In this case, we’re saying, “We only want you to bring list items that are actually of a task type.” Having typed that in, I’ll go ahead and search for that now. Notice, all I get are things from task lists. It’s pretty obvious that that’s the case.

Now, we could go a step further here and say, “I really want to find the ones that are assigned to me.” There’s sort of a good news, bad news going on here. The good news is, we can qualify who it’s assigned to. The bad news is that, there’s not an automatic way for us to say, “Only show me the ones that are assigned to the current user.” Those of you who’ve built these before and you know how you can filter a view where it says, “Just show items for me.” Unfortunately in the query language, we don’t have anything exactly like that. You are going to have to prompt your users for some kind of information. Or, it is possible with just a little bit of code, we could make this a fully automated process.

For this example, let’s just pretend that we are in the role of Kim. We’ll just come out here and say, “Assigned to” and “Kim.” What I’ve done now is make it possible for Kim to go to one place and see all of the tasks that are assigned to her. Again, she’s obviously not going to see any places that she’s not authorized to see. The thing here is, she’s now not going to have to go visit all these different sites to figure out if she’s got an assignment or not. She’ll just simply execute this query and she’s off and running. Now, how can we package this in such a way that it’s a bit more convenient to use? Well, let’s talk a little bit about that and then we’ll go a bit deeper and see how to get things done with that.

I’m going to switch back to Power Point here. The next part of this is called Accessing the Application. That is to say, once we figured out a way to ask for the results that we want, then we need a way to present it up to users so that they can conveniently consume it. Now, the easiest way to solve that problem is to recognize the fact that these search results can be expressed as a hyperlink. You might have noticed this when I was … demo, or you may have missed it, but if you take a look here, you can see. When I typed in all of this stuff in the search box, all of it’s represented up here on this URL. It is quite literally as simple as taking that URL and then making it available to a user.

How am I to do that? Well, I can put it on the Quick Launch menu. I could create a page and author a hyperlink there. Or, there’s even a hyperlinks list that’s available to me in SharePoint. I could put the information there. Pretty much anywhere we can store a clickable link, we’re now giving our users the ability to take advantage of this. Now, something that is a bit more sophisticated, but also brings us a bit more power and that is the Core Search Results Web Part. In particular, as developers or really high-end power users, we have the ability to customize the XSLT that is associated with that.

Now, if you have no idea what XSLT is, let me just take a quick aside here. Extensible Stylesheet Language Transformation is what it stands for. In very short terms, it’s just a template that describes how the results are going to look. As an end-user, you probably aren’t ready to engage trying to change that, but for a technical resource, it’s a pretty simple thing to do. Now, one other word about customizing the Search Results Web Part. The good news is, you can do it. You can absolutely do it. The challenge here is that, it can be difficult and if you’re doing this in a search center, you have to be aware of the fact that if someone else tries to modify that same set of XSLT, their changes might collide with your changes.

I want to drop that in your mind as a little bookmark of a problem, because when we start SharePoint 2013 conversation, we’re going to see that that problem gets solved. What I think is a better approach on this and it’s the one I’m going to demonstrate for you in just a moment is, when I’m building a search-driven app, instead of modifying the Web Part that’s in my search center on the results page, what I probably do and just did is just go to my site pages library. Create a dedicated page and put a Core Search Results Web Part on that. From there, I can configure the part so that it issues whatever query it needs to and it has whatever customization it needs. I am going to demonstrate that for you and let you see that in action. Let’s do that right now.

I’ll go back over and just continue working in the same demo environment that I’m in. Let’s do something a little more generic here. Excuse me. Let’s see if we can do a query that will show us all of the sites that someone has access to. Once again, I’m going to use the content class and I’m going to look for two content classes here. The first one I’m going to get is STS WEB and then I’m going to do an OR, and the OR does have to be capitalized. Then I’m going to do STS SITE.

Now, for those that are less technical in nature here, let me just give you a simple answer. If you want to get all the URLs, all of the sites, this is the way that you have to do it. For those of you that are a little more technically savvy when it comes to SharePoint, remember, if I did STS WEB, that’s only going to give me the sites. That’s not going to give me the top-level site for the site collection. For that, I have to use STS SITE. If I do those two together, you can see now I actually have a list of the sites that are available to me.

Now, in terms of making this a “application”, watch what I’m going to do. I’m going to copy this URL and then I’m going to navigate back over to the home page as my Acme Corporate Team Site. Then, once I’ve done that, I’m going to go in and customize this page. Once I’ve opened up page editor, I’m going to go in and insert a Web Part. What I’m going to insert is the links Web Part. I’ll put that right down here. Then, I’ll go ahead and save my work. Then once I’ve done that, I can go over to all site content and you can see that I have a links list here. I’ve got this one left over from a previous demo. I’ll just delete it out real quickly and then, add a new link here. I’ll put in the URL of the query that I just wrote and then I’ll give a description of all the sites you can access and click save.

Then once I’ve done that, now I can go back to the home page of this site and at any point, when a user comes to the home page of this site, they’d be able to click on this link and go directly to a search results page and see just the sites that they can access. To really drive home that this indeed doing what I’m claiming it’s doing, notice that I could take a second here and sign in as a different user. In this case, a user named Ed. Notice that if Ed clicks on this link, he’s only going to get two results whereas before I was logged in as an administrator and I got four results.

No worries here about writing any logic whatsoever, and we’ve solved that problem that we started talking about at the very beginning of the webinar. Now, to go a step further here, let’s see if I can log back in as administrator now and show you a little bit about how to make this a bit more beautiful. Okay, so I’ve logged back in as administrator. Now, let’s remember a point that I brought up early on in the process. I said that when we do these search-driven results, or these search-driven applications, it’s really nice to put the results in a format where they’re ready to use.

Saying that another way, if all I’m interested in are sites, then I don’t need all of this garbage. I just need a title of the site and something I can click on, and I don’t need the refiners and I don’t need all these other stuff. What I can do here is very simply create a page that contains a Core Search Results Part, and then configure that Part to issue this query. I’ve actually already pre-baked an example of that. Let me take you over and show you what that looks like.

In my site pages library, I actually have a page here called Available Sites. You can see that if I navigate to that page, it has a very simple list of the sites that are available to me. Then of course, I could click on any one of those and navigate directly to that site. Now, how is it that I’ve done this? Let me show you real quickly. I’m going to go ahead and put this page in edit mode. It’s being a little bit stubborn there. It doesn’t want to go in edit mode. Sometimes I have to switch back and forth between Internet Explorer and Chrome to get what I’m looking for. That’s fine. There we go. That’s behaving a little bit better.

Now, you’ll notice, the very first thing that I’ve got here is a Search Core Results Web Part. If I go in and actually edit this Web Part, I can show you the two things that I changed in order to make this thing work the way that I wanted it to work. Let’s drill down through some of these and have a look see here. Now, you can see there’s a section that describes my display properties. Then, you can see there’s a section where I have something to say about the results query. Let’s start with that one first.

Now, I know this text box is not big enough for you to see all of the contents, but you ought to be able to figure out pretty quickly what I’ve done there. In fact, if I select all of that and copy it, and then put it into Notepad so you can see it, that’s the exact same query that I issued before. Really, what I’m telling this Part is, the user is not going to type in a query. Instead, we’re just going to give them a fixed query, and then we’ll allow security trimming to make sure that everybody sees what they’re supposed to see.

Now, if I did only that, I would actually wind up with the same appearance and search results that I had before. Now, I want to go one step further and make it look nicer. I did that by adding my own custom XSL. Now, the point of this is not to teach you how to write this XSL. Some of you under the sound of my voice, probably would never do that, but the point here is just to let you know that it’s this simple. You can see in this care, really all I’ve done is just say, “Take all of the results. Turn each one of them into a hyperlink. Substitute the URL and then put a title on it.” That’s how I wind up with these very simple search results. Then, if I stop editing and go back into view mode, you can see that ultimately the page is just going to give me a very simple appearance and let me navigate to any one of those sites.

Then, you might have noticed when I was in the site page library, I also did a similar page here with tasks. Again, the only challenge on this one is, I’m going to have to do some kind of exercise to make sure that it works properly for different users. That’s probably going to involve a little bit of JavaScript. If you’re any kind of programmer at all, it’s not that challenging and it lets you do some very clever things. If you’re not already familiar with building those kinds of simple applications and letting the search engine do a lot of work for you, I hope that will inspire you to go forth and embrace some of these search-driven applications.

Now, as clever and cool as some of that stuff is, there comes a point where you say, “We want to do something with search that’s just a little more involved than a little bit of a power user trick.” That’s what this next part of the webinar is about. Now, one of the big ways that we can go a little above and beyond with search is integrating it with other sources. We have a couple of ways of doing that. One way to do that is to actually configure SharePoint so that it crawls external content. A classic example of this, maybe we have a line of business application where we keep data in a SQL server database. Among the things we keep in there, we keep our Part catalog information in there.

It’s actually possible to set-up SharePoint to use business connectivity services to connect into that database, to crawl the database and then to index a series of results that we can then search against. There’s a lot of good things that happen with that approach. We’ll talk about specific pros and cons on that in just a moment, but the idea here is that SharePoint is the thing that indexes and crawls the content and then any integration happens at that moment. At query time, in terms of getting the results back, it’s no different than any other SharePoint search.

Now, an alternative to that is something called Federation or federating the external content. In this example, the SharePoint search engine is not the one that’s doing the crawling and the indexing. That’s actually maintained by some external piece of software. Probably native to whatever repository we’re using. Then, instead of the integration happening at crawl time, it actually happens at query time. That is to say, when we do this kind of integration, even though they don’t realize it, the user is actually issuing two queries. They’re issuing a query in the SharePoint search engine and then on their behalf, it is issuing a query to this external engine.

What are the pros and cons of each approach? Well, first of all, the first one crawling external content. There’s several compelling reasons to do this. One, well, because it’s pre-indexed in the same way that other SharePoint data is, we just get better ranking of the content. In effect, it’s going to be subject to the same rules and the same ranking calculations that any other SharePoint data would be subject to. Another nice thing here, we have control on the frequency of crawl. As an example, let’s say that we are crawling some external data and we only need that data to be up to date within the 24-hour window. Well, we can save a lot of CPU cycles by crawling that information once a day as opposed to having some other mechanism crawl it on a more frequent basis.

This third one is particularly compelling. You’ll remember a moment ago, when I was able to do property specifiers in order to refine my search results, well if we’re crawling external data and we’re allowing SharePoint to do the crawling, we can actually specify those mappings. We can take bits of metadata from this remote source of data and bubble those up as properties that actually can be filtered in our queries. Then, in terms of maintaining the index, it’s just a single index for all of our data, and so our backup and restore operation on that is fairly simple.

The downside of crawling external content from SharePoint, well probably the biggest one would be the fact that we’ve got additional capacity requirements. Now SharePoint is going to have to have enough disk space to not only index its own content, but this external content as well. Not only is that a disk space equation, but also a CPU cycle equation. Beyond that, there is also this challenge that in some cases, we may be talking about external data that for security reasons, simply can’t be crawled. They may have blocked any external access in terms of being able to crawl through and read the data. That would drive us toward an approach where we would do federation instead.

The pros and cons here are mirror images of each other. In the case of federation, if we’re talking about a repository that already has a search engine, that search engine is probably already optimized for the kind of content that we’re searching. Rather than try to re-index it, we’ll just make a call out to the search engine at the query time and say, “Hey, tell us what you know about this.” This is particularly useful in cases where we simply cannot index the information externally because of security.

Then again, the cons here, well one of the biggest cons is that, we’re not going to be able to necessarily interleave the results on a single results page the way we would if we had SharePoint indexed it. More likely, you’re going to have a section of your search page that’ll show SharePoint results and then a section of your search page that will show external results. Not only that, when it comes to this kind of things, security trimming can be really tricky because at this point, you’re making that determination at run time.

Now, the good news, it can be done and in fact I can tell you that here at ThreeWill, we have written integrations on behalf of vendors where they took their products, integrated them on SharePoint and then when we did this run time federation or query time federation, we were actually able to make sure that the end-user still only saw the results that they were looking for. That is something that’s kind of a dark art. It’s not all that well-documented, but we’ve got a ton of experience with that and we’ve been able to help our customers get what they want there.

Now, just as before when I talked about building search-driven applications, not only can we make some trivial customizations to the Core Results Web Part, such as you saw a moment ago, but we can also do some pretty drastic customizations. This is another thing that we’ve actually done on behalf of one of our customers. I’m going to show you a pretty compelling demo of that in just a moment. Beyond that, if you’re familiar with search centers, we also have the ability to customize search center tabs that’ll let us do things like look for results in a narrowed results set or people searches or things such as that.

Then ultimately, if we just can’t get what we need out of the search infrastructure as it is, there’s always the option of writing either server code or JavaScript that makes a direct call into the search API, brings back the results we’re looking for and then we can specifically show that to our users. I can tell you again that, a particular ThreeWill use of this, we’ve actually written some application code for a customer where we did a typeahead search. Where as the user was typing in the characters of their search terms, we would go out and do a prefetch of matching items. That kind of thing is virtually difficult to do, or virtually impossible to do I should say in a search center, but can be done with custom pages and code.

Another thing that we’ve done on behalf of our customers, creating custom refiners. You’re probably already familiar with the concept that the SharePoint search engine gives you the ability to refine on different characteristics such as the date that the document was created or the format of the document. For certain custom data types, we can actually go in and create refiners from scratch.

Let me just show a little bit about how one might use this. What I’m showing you here is an example of an integration that we actually wrote on behalf of Polycom. You may or may not be aware of the fact that Polycom has a fairly sophisticated portal for keeping up with multimedia content. It’s called the Polycom Real Presence Media Manager. Particularly companies that do a lot of training videos or educational videos, video content of various types. This is a very popular piece of software that companies can use to manage that information.

Now, the product has a portal of its own for accessing that content and it’s really useful and it’s really nice. What Polycom has discovered is that, for a lot of companies, the destination of choice is going to be SharePoint. If we lock up all of our video content over in this vault where people have to visit another site, they’re not going to see it quite as often. Now, you can see on the screen here that we’ve actually created a Web Part that will let us go in and discover that content. I can look at things like recently viewed and so on.

Obviously, the demo that we’re talking about here and we’re more interested in here is this notion of search. Not only have we built a facility for browsing that content, we’ve actually integrated it in search. In this example, what we’ve done is, we’ve changed the XSLT associated with the Core Search Results Web Part so that we can not only interleave the data that’s coming back from Polycom, but also show it in an enhanced way. Let me show you what I mean by that. I happen to know as an example, that there are several videos and even some other information about Stroke. Again, obviously, the sample app here, being a healthcare app.

Now, notice you see some of the kinds of things you would expect. You see a title and you see some kind of icon, but you’ll notice that the information down here looks a little bit atypical. Not only do we have some author information and a date, we even have the duration of the clip. Then, this little link of Video Program Details. This is all custom code that we wrote on behalf of the customer. You can see the compelling story here is that, with one click, we’re able to preview that video in line and then if we so choose, we can even begin viewing it right there on the spot.

I think this is a pretty high-end example of how you can take the out-of-the-box SharePoint search bits, and then do some pretty significant customizations along with that. You’ll also note that not only did we do customizations related to the viewing of the content, but also refiners. Even though this is video content that’s held on another site and there’s really nothing SharePoint-y about it, we were able to supplement this information in such a way that we can refine it based on where it came from, who the author was and other such things. Again, the goal here isn’t so much to teach you how to do that. Obviously, that’s a pretty deep process, but rather just to alert you to the idea that it’s possible and it’s something that we have the experience and the expertise to do.

All right, we’re getting pretty late into our hour here. I’ve got one more section of content that I want to talk you through here, and that is just giving you a preview of coming attractions when it comes to Search in SharePoint 2013. I know that’s probably an over-the-horizon thing for some of you, but it is nice to know what’s coming and what’s to be expected.

Probably the biggest difference when it comes to the architecture of Search in SharePoint 2013, and that is this notion that we now have a single search engine. For those of you with a lot of deep knowledge on this, you know in the past, you’ve got SharePoint Search and FAST, and different versions of those. Now it’s just a single unified product. Really, I think that’s a good thing for the most part in that, it reduces some confusion. When you’re authoring queries now, you don’t necessarily have to be cognizant of, “Well, gee, which version of the search engine am I on?”, in order to be able to do this thing that I want to do.

Now, in terms of the languages that you use or the query languages that you use, keyword query language, which is what I’ve been demonstration for you today is obviously still supported. In addition, in prior versions, FAST had its own unique query language. Even though FAST doesn’t exist as a separate engine anymore, the FAST query language is still supported. The one that isn’t and it isn’t still supported is the SQL variant of being able to do search queries with a SQL LIKE Syntax. I think at this point, that’s one that didn’t get used a whole lot and that’s why Microsoft made the choice to drop that. If you’re under the sound of my voice and you’ve got some things that you’re doing in SharePoint that are dependent upon that dialect, then just be aware of that. That’s something that’s going to go away when you get to SharePoint 2013.

Some other things and some of these are technically meaty. Others of them are more marketing points, but they’re worth making. There’s some improvements to the results ranking model. Another one that I think is really cool, and that is the results are further refined based on click-throughs when the results came up before. Those results that have a higher rate of click-through, will show up as being more important on subsequent queries.

Another thing that’s new, a moment ago, I’ve talked to you a little bit about federating outside sources of data. That’s gotten a new name now. It’s called Result Sources. It’s really a way of just describing external data. Probably the biggest thing here that would be of use to developers is the fact that the way that we do security trimming here is a little more elegant and a little more sophisticated. In terms of being able to write customizations, that’s a good thing.

Now some other things, some refinements and changes to query rules. We have this thing called Promoted Results. This was formerly known as Best Bets. We’ve got a little more flexibility on how that we use that. We have this other thing called Result Blocks, and I’m actually going to talk about this one a little bit more on a later slide. The idea with result blocks is that you can use it to group related results. As an example, let’s suppose that somebody searched for a particular kind of content and maybe we discovered that based on certain words. Probably they’re looking to highlight some things related to Power Point presentations. Well, we could actually set up a separate result block that says, in addition to all the results we’d show them, we could actually use a result block to call out just the Power Point presentations that are in those search results.

Again, if you think about where you visited commercial search engines like Bing or Google, and they often have a way of saying, “Well, gee, you searched for this thing, but there’s a whole bunch of images that match. Would you like to see those?” That’s exactly what result blocks are all about. Another thing that’s kind of cool is that, you can actually boost rank results based on certain terms. As an example, suppose that someone goes in and types in a term to download a utility, we can actually configure a query rule that says download is recognized as an action term and based on that, if we have a particular site in the company where people are supposed to be getting their downloads from, we can actually boost that result and that would result in a likelihood that it would be higher in the result stack.

Another nice innovation here, Continuous Crawl. Again, those of you that have worked with SharePoint a long time, you know that there’s always this challenge of, you want to set your crawls frequently enough to be up to date, but not so frequently that they killed the server. Now we have this concept that, for particular sources of data where it absolutely has to be up to date, continuous crawl really is just tapping into the transaction engine from the underlying database and saying, “Anytime we have new content, we’re just going to go ahead and update the sear- index immediately.”

Now, let’s talk for a moment about a hybrid environment. This is something that I think a lot of you, particularly if you work for big companies, you’re going to encounter over the next year or two. It’s interesting not only to identify it, but to talk about some of the search implications of that. When I say hybrid environment, what do I mean? Well, we all know that Microsoft is making a big push to move customers to the cloud, to begin using SharePoint Online as a part of the Office 365 Suite.

Now, for most companies that have a significant amount of content, it’s probably not going to be possible to move exclusively to the cloud in a short-term. There’s really two big reasons for that. Number one, they’ve probably got customizations that wouldn’t run in the cloud particularly if they’re farm solutions. Number two, it’s probably not practical to take the gigabytes of data that they already have in place and simply move those up to the cloud in order to sit there. It would be very expensive to move them up there and it would be an expense to have it up there in an ongoing basis.

In order to combat those barriers, a lot of customers are using what’s called a hybrid environment. In simple terms, they’re going to take everything that’s on-premise and for the most part leave it there. Then, new content is going to be put into the cloud and used there. Now, the presumption is, if we are going to have such an environment, we really need to be able to integrate the search results from those two environments. To say that a little bit differently, we don’t want users to have to constantly say, “Well now, wait a minute. Is this thing in the cloud or is this thing on-premise?”

The idea here is that, we can use the result sources feature that I mentioned just a moment ago in order to integrate those results. How are we going to use that? Well, suppose that we’ve got an on-premise search page. What we can do is create a result sources feature and then a result block that highlights the matching cloud results. They may not be interleaved one for one, but what we’ll see is a section of highlighted results. Then, if someone observes, “Well, gee, the results that I think are relevant to me, there’s more of them on the cloud than there are on my on-premise.” They can actually click on one or more items inside of that result block and it’ll navigate them to the cloud. You can also do the same thing in reverse. You could set up a cloud search page so that it not only shows cloud results, but also has a results block that highlights on-premise.

I can tell you that we are actually doing a proof of concept of this as I speak for a company here in Atlanta. Fairly large company that you would recognize and we’re getting some really good results with that. Really kind of two good news fits there. Number one, it’s doable and it works. Number two, when you find yourself at a point where your company needs to do that, we’ve got the expertise to help you get that done.

Some other things you’ll want to watch for in SharePoint 2013. Some of these are just features that we’re going to like and take advantage of. Some of them are things where we actually have opportunities to customize. We now have this concept of a hover panel for particular content types that show up in search. You can actually hover over those and preview the results on a mouse over. We also have the ability to create our own hover panels. If I were to go back to what I showed you a moment ago on the Polycom integration, once that products gets updated to SharePoint 2013, it would be a doable thing for us to go in and write a hover panel where instead of having to click to view the video, one could simply float the mouse over and see a preview of the video playing. I think that’s a pretty cool one.

Another nice thing of SharePoint 2013 Search, if you look at the search pages I’ve been showing you up to this point today, the icons and the graphics associated with the results are pretty bland. They don’t really communicate a lot of information because they’re tiny and somewhat nondescript. Microsoft has tried really hard to put some good visual cues on your search results so that it’s real obvious, “Oh, that one’s a Word doc, and that one’s a diagram and that one’s from Visio and another one is an image.” I think that is a much needed improvement.

Some other things. I know that not all of you take advantage of the term store and the managed metadata capabilities of SharePoint 2010, but if you do, there are some greatly improved capabilities there in terms of being able to navigate and find items. Another improvement that we have, this concept of display templates. Now, you’ll remember that I’ve talked twice now about changing the XSLT in the Core Results Search Web Part either on a particular page or in a search center. Given the difficulties associated with that, it’s nice now that we have an alternative to that.

Really, what are we saying when we say display templates? Now we’re able to build what are in effect, customizable little nuggets of HTML. We’re going to be able to associate these in such a way that when a particular result set comes up, we can actually modify how it’s going to look on the results page. The nice thing about this is that, because it is associated with particular types of content, we can do it in a search center and do it in such a way that we won’t collide with other customizations. It’s more flexible, it’s more powerful, it’s less error-prone, it’s easier to do. That’s nothing but a win when it comes to doing this.

Now, the various parts that make up this, there’s really two things. First of all, you have a controlled template that’s going to describe the header and the footer and the layout of the results list. Then, an item template that we’ll use to describe the presentation of the items in that list. Those of you that do a little bit better with pictures instead of words here, this gives you an architectural representation of what we see. Then, this is a nice screenshot that shows you how you could do a search results page with an image slider. You’re able to just simply scroll back and forth through the images used and a little bit of jQuery in addition to the HTML template to show this.

If you are in a place where you do customization of search, your world just got a lot bigger. Conversely, if you’re a consumer and you need these kind of capabilities, it’s nice to know that it’s finally available in the product, and that you’ve got some folks in ThreeWill that know how to make that happen for you. Let’s talk about some takeaways real quickly, we’re getting close to the top of the hour here.

First and foremost, if you’re a developer and you’re under the sound of my voice, you need to start thinking about search-driven applications. If somebody describes a business problem to you and the answering of that business problem can be done from the search engine, it’s faster, it’s more efficient, it’s easy to write, there’s a lot of wins there. Obviously, if you’re doing a traditional data-entry app where you need to add items to a list, that’s not necessarily a search-driven application. Anything that involves asking questions. This is a good candidate.

Another thing to think about, beyond just the building of these simple search applications, if your company is in a place to make a higher technical commitment, there are a lot of other things that we can do. Customizing the search results page, writing refiners, integrating external data. Another thing to look forward to, SharePoint 2013 really offers us some much better opportunities to integrate at a deeper level. Better security trimming, better models for extending the UI, and doing those things in such a way where it’s not going to be brittle or prone to breakage if somebody else tries to do a customization as well.

Then, obviously, what I want you to take away from that is, this is what we do. It’s not that we’re going to do these things. We have already done these things on behalf of our customers, and we’ve had some really good results with that, and we want to help you get those same results as well.

read more
John UnderwoodEnterprise SharePoint Search Webinar
search1.jpg

August 22nd Free Webinar – Enterprise SharePoint Search

Overview

SharePoint at it’s best can be a useful repository of company information, but in some cases it can be difficult to find the right data at the right time. This presentation looks and SharePoint Search capabilities for SharePoint 2010 and SharePoint 2013, and provides insights that users and technical resources alike can employ to make search a valuable tool.

Audience

SharePoint Users/Power Users, IT Personnel

Outline

  1. Asking the question correctly
    • Basic search terms
    • Using refiners
    • Qualifying search queries
  2. Search-driven applications
    • What is a search-driven application?
    • Search-driven apps for users
    • Taking it to the next level
  3. Customizing the results UI
    • Search results web parts
    • Using Display Templates in SharePoint 2013
  4. Extending and enhancing search
    • Integrating with external data sources
    • Custom security trimming
Register Now
read more
John UnderwoodAugust 22nd Free Webinar – Enterprise SharePoint Search
box-takeaway-e1425508188683.jpg

Kirk’s Takeaways from #spc12

So much SharePoint, so little time…

I didn’t get to see all of the sessions at the SharePoint 2012 conference (spc12) that I was interested in.  I was quadruple booked for every one, so there are plenty more (recorded) sessions that I want to watch, not to mention digging in and getting dirty.

Regardless, here are my initial impressions.

Usability

I absolutely love some new usability features that I have seen in SP2013:

  • Minimal Download Strategy – by default when you navigate from page to page within a SharePoint site, very small changes are downloaded instead of the entire page.  For example, the quick nav and global nav don’t change and are not part of the HTTP response.  The result is a much faster and smoother experience for the user.  I think this is really going to help people “like” (dare I say “love”) navigating around in SharePoint.
  • Focus on Content – a simple icon is available which removes the navigation elements (quick nav and global nav) and just shows the body of the page.  This should be very handy for when you are viewing lists with a lot of columns and rows.
  • Drag and Drop – yes, you can drag and drop documents into a document library.

Apps

Yes, they are for real.  It’s not clear to me how quickly and how much they will take off, but I think you should consider creating an app anytime you are doing SharePoint development. The good news is that these are not taking away any options for developers.  We can still do Farm Solutions.  Sandbox solutions are even an option, but not recommended (oopsies).  Some great aspects of apps are:

  • Apps provide a common and familiar development model for both SharePoint apps and Office apps (javascript).  Think of it…  Several of us have done VSTO applications before, but not regularly.  Now we can take what we know about JavaScript and apply it to Office.  I think this really puts Office applications back in the mainstream at ThreeWill – and for a much larger user base than just SharePoint.
  • There is a public store hosted by Microsoft but you can also have an on-prem catalog for apps specific to a company.
  • Apps use underlying API and infrastructure to allow for server-to-server communication within apps as well as client side cross-site messaging (e.g., resize the IFrame).

OAuth

OK, this doesn’t excite users, but it should excite developers; esp. developers that have connected SharePoint with other systems.  ThreeWill does a lot of this and this should make our life easier if the other side handles OAuth as well.

Workflow

Now SharePoint workflows are built on top of Windows Workflow Foundation 4 and are hosted outside of SharePoint.  I think the biggest deal here, though, is that SharePoint Designer workflows have a lot more power.  They can have much more sophisticated capabilities with the ability to loop and jump to different areas of the workflow.  I see this being very useful at ThreeWill.

Search

I hadn’t heard any buzz on search going in, but there were around 26 sessions on search and there are plenty of things to be interested in.  The big news is that FAST is now part of core search.  This will have some pain because certain things are deprecated, but I think the positives will well outweigh the negatives.  A few things to note include:

  • Query Rules – these give you a lot of power on how search results are presented.  I think this will open up the possibilities for search-based applications.
  • Display Templates – this should make it a lot easier and less intrusive to provide a different display for custom content returned from search results.  These are very powerful for displaying list content outside of search as well.
  • Security Trimming (something near and dear to my heart) – you can now have a pre-trimmer in addition to a post-trimmer.  The post-trimmer is similar to ISecurityTrimmer2 in SharePoint 2010.  The pre-trimmer will let you provide claims information and map it to claims that can now come in from the BCS crawl.  A much improved and faster approach than the post-trimmer (if feasible for your situation).

Other Notes

  • Access – its baaaack!  … or maybe it never left.  But this appears to be more useful for real applications.  I can’t believe I just wrote that, but the fact that is uses actual SQL Server tables looks promising.
  • Remote debugging for SharePoint!
  • Napa Office 365 Developer Tools – looks pretty interesting; it’s a browser based development environment for O365.
  • If I see one more Bing Maps demo I think I will throw up.

What did you takeaway?

Feel free to share some of your takeaways from the conference in the comments section – I’d love to hear them.

read more
Kirk LiemohnKirk’s Takeaways from #spc12
database.jpg

ContentType SQL Query Tip

Whenever I used ContentType in the WHERE clause of a FullTextSQLQuery the query would return no results. However, I found a fix:

  1. Go to Central Admin > Search Service Application > Metadata Properties.
  2. Edit ContentType and checked “Allow this property to be used in scopes”.
  3. Do a full crawl.

In some tests I reset the index, but that does not appear to be required (I did one test without resetting the index).

Note that this is not required for many of the other managed properties if you want to use them in the WHERE clause.

FWIW, I looked at the content type properties in PowerShell. Checking the checkbox changed the two properties in red below.

PS C:\Users\Administrator> $ctype = Get-SPEnterpriseSearchMetadataManagedProperty -SearchApplication “Search Service Application” -Identity ContentType

PS C:\Users\Administrator> $ctype | fl

ID : ContentType

PID : 60

Name : ContentType

SplitStringCharacters :

SplitStringOnSpace : False

Description :

ManagedType : Text

FullTextQueriable : True

Retrievable : False

UseAAMMapping : False

MappingDisallowed : True

DeleteDisallowed : True

QueryIndependentRankCustomizationDisallowed : True

EnabledForScoping : True

NameNormalized : False

RespectPriority : False

RemoveDuplicates : True

HasMultipleValues : False

OverrideValueOfHasMultipleValues : True

IsInDocProps : True

IncludeInMd5 : False

NoWordBreaker : False

UserFlags : 0

Weight : 0

LengthNormalization : 0

EnabledForQueryIndependentRank : False

DefaultForQueryIndependentRank : 0

IsInFixedColumnOptimizedResults : False

RetrievableForResultsOnly : False

PutInPropertyBlob : False

QueryPropertyBlob : False

UsePronunciationString : False

MaxCharactersInPropertyStoreIndex : 64

MaxCharactersInPropertyStoreNonIndex : 4000

EqualityMatchOnly : False

MaxCharactersInPropertyStoreForRetrieval : 65536

DecimalPlaces : 3

IsBacked : True

The blog that led me to the solution is here: http://laneyvb.blogspot.com/2008/11/sharepoint-fulltextsqlquery-malformed.html. My issue occurred with only a couple of conditions and not 10+ conditions as mentioned in this blog. In addition, I am using SharePoint 2010 and not SharePoint 2007. It was the same fix for both scenarios nevertheless.

While researching this I found several posts about ContentType where people have a hard time selecting the ContentType in the full text SQL query. Using ContentType in scopes does not help with that problem. The solution others have found is to create your own metadata property and have it map to the same crawled properties that ContentType maps to. Here are a couple of posts that mention this:

read more
Kirk LiemohnContentType SQL Query Tip
secrets1.jpg

Security Trimming Secrets

Introduction – What is Security Trimming?

Security trimming is simply the act of filtering out content that should not be accessible (typically read-able) for a given user. It is a core concept within SharePoint that affects what navigation elements you see, what sites you have access to, what lists and libraries you have access to, and what list items you see. It is also a core concept within SharePoint search. Security trimming in SharePoint search comes in two flavors: indexed security trimming and query-time security trimming.

Indexed Security Trimming

Security trimming within the index is possible when the crawler can obtain Access Control Lists (ACLs) for each item and store them in the index. It is the preferred approach because it is faster. There is little that is “dirty” about this approach. This is used for SharePoint content as well as file shares and other content sources. Outside of some minor storage and crawl processing costs, the only real downside is that permission changes are not reflected until the next incremental crawl.

Query-Time Security Trimming

Sometimes you do not have ACLs available at index time and you must resort to query-time security trimmers. This is the case when crawling web sites since there is no way to ask a website “who has access to this page?” This may also be the case with Business Connectivity Services (BCS) when you are crawling content from a database or web service. BCS can use indexed security trimming, but only if you can make ACLs available through your external system.

The rest of this post focuses on query-time security trimming and how this has changed with SharePoint 2010. Most of it is straightforward, but there are a couple of little “dirty secrets” you’ll need to be aware of. First I’ll give an implementation overview, then I’ll show what I have observed, and finally I’ll focus on two gotchas that are in SharePoint 2010.

Implementing a Custom Security Trimmer

Query-time security trimming in SharePoint 2010 is very similar to SharePoint 2007. They both require the server edition of the product (no WSS 3.0 or SharePoint Foundation). In both cases you write a class that implements an interface and then you register that security trimmer with a crawl rule. For SharePoint 2007 you use the ISecurityTrimmer interface, whereas in SharePoint 2010 you use the ISecurityTrimmer2 interface. A good reference for writing security trimmers can be found in Writing a Custom SecurityTrimmer for SharePoint Server Search.

This interface is very simple. It has an Initialize() method that runs once and can provide your class information specified when registering the security trimmer. This method is identical for SharePoint 2007 and SharePoint 2010.

The only other method is a CheckAccess method. This is the more interesting of the two. This is where the query is actually happening. This method returns a BitArray describing whether the current user has access to each URL provided. It has some differences between SharePoint 2007 and SharePoint 2010; most notably that in SharePoint 2010 you are provided an IIdentity. In SharePoint 2007 you had to use WindowsIdentity.GetCurrent() or HttpContext.Current.User depending on the authentication method.

CheckAccess Details

The CheckAccess method can run multiple times for a single query. What the query engine is doing is providing your security trimmer a batch of URLs expecting to find out which ones can be provided to the current user for search results. So, your security trimming code is running during the query – potentially multiple times. If the security trimmer does not give the query engine enough hits, it will provide the next batch of URLs to your security trimmer – and will continue to do so until it has what it considers enough URLs or until it has exhausted all hits for the search request.

Since the security trimmer is running during the query, it needs to be quick and efficient. However, while your security trimmer code is running you are likely doing some processing to find out if the current user has access to a URL. This may involve a web service call to another system. This may be fine, but I recommend that you test the performance thoroughly just to be sure. Unfortunately, since SharePoint 2010 only provides an IIdentity, you are fairly limited since this does not provide you as much information as what you have access to in SharePoint 2007. The context the security trimmer runs in is not the same context you are used to for your typical server-side SharePoint code, so you may have to do additional calls to lookup user information or other context-specific data. If so, you’ll want to factor that into your performance costs.

In SharePoint 2007, the security trimmer was typically run enough so that the query engine could provide a page-worth of URLs to the user running the query. Since the default query page size was 10, it may have stopped after having 20 or so URLs. In SharePoint 2010, however, it not only wants enough URLs to show a page-worth of results to the user, but it also wants enough to show appropriate refiners. By default, I believe that this means that the SharePoint 2010 query engine wants 50 results (after trimming).

CheckAccess Observations

Depending on how many search results there are before trimming and how many are trimmed out, your security trimmer may be called a large number of times. Using the default settings with a large result set and with the majority of search results trimmed out, my observations have shown me that the first time the security trimmer is run it is provided 50 URLs. If you trim some out, the security trimmer is immediately called again with another 75 URLs – and this is done before the user sees any search results. That continues with batches of 75 URLs until enough results are satisfied or until the total result set is exhausted. Using trace statements and DebugView I can watch what is going on. The chart below shows how many URL are requested for a single query. In this case the security trimmer is called 6 times and given a total of 425 URLs.

In the output below, the first line is when Initialize is called. Each subsequent line indicates an individual call to CheckAccess.

00000000 0.00000000 [9292] [CustomSecurityTrimmer] Initializing
00000001 0.00353188 [9292] [CustomSecurityTrimmer] Count = 50 - Total = 50.
00000002 0.05990715 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 125.
00000003 0.10741841 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 200.
00000004 0.16418955 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 275.
00000005 0.21421386 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 350.
00000006 0.26570737 [9292] [CustomSecurityTrimmer] Count = 75 - Total = 425.

The algorithm changes once the user clicks to view the second page of results. The initial count of URLs provided to the security trimmer for the second page is 60. It is 70 for the third, and then seems to max out at 76. The pattern is a little more complex, so it is probably best just to show you some data. Notice how the total resets on subsequent page requests. That is because for each page we are starting a new set of processing as far as the security trimmer is concerned.

00000000 0.00000000 [11200] [CustomSecurityTrimmer] Initializing
00000001 0.00489843 [11200] [CustomSecurityTrimmer] Count = 50 - Total = 50.
00000002 0.06208209 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 125.
00000003 30.23514748 [11200] [CustomSecurityTrimmer] Count = 60 - Total = 60.  (Page 2)
00000004 30.27280998 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 135.
00000005 45.76335526 [11200] [CustomSecurityTrimmer] Count = 70 - Total = 70.  (Page 3)
00000006 45.79971313 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 145.
00000007 59.88526917 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 4)
00000008 59.88538742 [11200] [CustomSecurityTrimmer] Count = 4 - Total = 80.
00000009 59.92379761 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 155.
00000010 59.97734451 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 230.
00000011 82.77108765 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 5)
00000012 82.77122498 [11200] [CustomSecurityTrimmer] Count = 14 - Total = 90.
00000013 82.84067535 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 165.
00000014 82.93080902 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 240.
00000015 99.88347626 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 6)
00000016 99.88369751 [11200] [CustomSecurityTrimmer] Count = 24 - Total = 100.
00000017 99.94698334 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 175.
00000018 100.04191589 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 250.
00000019 106.22407532 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 7)
00000020 106.22446442 [11200] [CustomSecurityTrimmer] Count = 34 - Total = 110.
00000021 106.30342865 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 185.
00000022 106.39843750 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 260.
00000023 111.46296692 [11200] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 8 )
00000024 111.46327972 [11200] [CustomSecurityTrimmer] Count = 44 - Total = 120.
00000025 111.52843475 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 195.
00000026 111.62356567 [11200] [CustomSecurityTrimmer] Count = 75 - Total = 270.

If we are processing too much within a page (a single request), then then you can throw a PluggableAccessCheckException. This is a way for the security trimmer to throw up its hands and basically give up. A good overview of how to use this is found on Walkthrough: Using a Custom Security Trimmer for SharePoint Server Search Results, but read further below as there is a problem with this in SharePoint 2010. With this exception you can provide a message to show the end user; it basically tells them to refine their search. This is an important part of a security trimmer because you don’t want to leave the user hanging too long. If the query lasts too long, the thread will be aborted. By default this is 90 seconds.

OK, now it’s time to dish some dirt…

Dirty Secret #1 – PluggableAccessCheckException

As mentioned above, the purpose of the PluggableAccessCheckException is to allow the security trimmer to tell the query engine to stop processing so a response can be given to the user performing the query without taking too much time. Unfortunately, with SharePoint 2010 the opposite occurs. If you throw a PluggableAccessCheckException what happens is that your security trimmer will be called with all URLs in the result set (typically in batches of 75) unless the thread is aborted before you reach the end of the result set. This is even the case if the security trimmer does not trim out any results after throwing the exception. In the output below I have set a low limit of 150 URLs after which I throw the exception.

00000000 0.00000000 [4500] [CustomSecurityTrimmer] Initializing
00000001 0.00339907 [4500] [CustomSecurityTrimmer] Count = 50 - Total = 50.
00000002 0.05012119 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 125.
00000003 20.21066093 [4500] [CustomSecurityTrimmer] Count = 60 - Total = 60.  (Page 2)
00000004 20.25085640 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 135.
00000005 40.40690613 [4500] [CustomSecurityTrimmer] Count = 70 - Total = 70.  (Page 3)
00000006 40.45227051 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 145.
00000007 57.72388458 [4500] [CustomSecurityTrimmer] Count = 76 - Total = 76.  (Page 4)
00000008 57.72399139 [4500] [CustomSecurityTrimmer] Count = 4 - Total = 80.
00000009 57.76459122 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 155.
00000010 57.76465225 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 155.  Throwing PluggableAccessCheckException
00000011 57.93753815 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 230.
00000012 57.93761063 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 230.  Throwing PluggableAccessCheckException
                ... (skipping 8 sets of trace statements) ...
00000029 58.56541061 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 905.
00000030 58.56551361 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 905.  Throwing PluggableAccessCheckException
00000031 58.65005875 [4500] [CustomSecurityTrimmer] Count = 75 - Total = 980.
00000032 58.65012360 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 75 - Total = 980.  Throwing PluggableAccessCheckException
00000033 58.73397064 [4500] [CustomSecurityTrimmer] Count = 47 - Total = 1027.
00000034 58.73407364 [4500] [CustomSecurityTrimmer] Exceeded Limit of 150.
     Count = 47 - Total = 1027.  Throwing PluggableAccessCheckException

Last I heard, the product group confirmed the behavior and said that since multiple pluggable trimmers can be registered, search cannot be stopped on throwing the exception. I assume that they are still trying to prevent continuously calling the security trimmer that threw the exception and hope that they will consider all results trimmed that are associated to a crawl rule in which a security trimmer threw the exception.

Bottom line: You may want to consider the risks of your custom security trimmer being called too many times and running too long. A workaround may be to simply return False for every URL that is provided to your security trimmer once you determine that you have been running too long. This could be done without doing any expensive processing.

Dirty Secret #2 – FAST Search for SharePoint 2010

Unfortunately, query time security trimmers are not supported with FAST for SharePoint (FS4SP). The FAST pipeline is different and although FAST is a flexible platform, I am told that FAST Search for SharePoint 2010 does not currently have a feasible way to make this happen.

I believe this is because FS4SP has deep refinements (among other features). Without FAST, SharePoint provides refiner information based on the first 50 results, but with FAST the exact count can be provided for each refiner. This exact count cannot be provided unless all of the results are processed, which just isn’t feasible with a query-time security trimmer (which can conceivably have tens of thousands of hits or more for a single query).

Last I heard this was not currently supported, but that R&D is actively working on a design change to allow for this functionality in a future release.

Conclusion

Security trimming is a great thing and a powerful feature used liberally throughout SharePoint. Security trimming with SharePoint search is crucial, especially when search results show hit highlighting of content. If possible, use security trimming at index time. If you must use query-time security trimming, go in with your eyes wide open.

read more
Kirk LiemohnSecurity Trimming Secrets
broken.jpg

Breaking MOSS Search

If you have registered a search security trimmer with a crawl rule and then prevented SharePoint from accessing the security trimmer (such as removing it from the GAC), you will get a nasty error when trying to execute a search that has results that meet your crawl rule.

To aid those searching for this error I’ll restate it:

Your search cannot be completed because of a service error. Try your search again or
 contact your administrator for more information.

To fix this simply make sure the security trimmer assembly is available (put it back in the GAC). You’ll likely have to do an IISRESET for things to work again. If you don’t have access to the security trimmer you may need to reset your index (hopefully there is another option, but I suspect not).

I had to dig through the SharePoint ULS logs to figure this out:

09/18/2008 11:15:42.01 w3wp.exe (0x0EB0) 0x0EF4 Search Server Common MS Search Query Processor 97zu
Unexpected Error occurred when executing pluggable security trimmer with Id 33402:

09/18/2008 11:15:42.14 w3wp.exe (0x0EB0) 0x0EF4 Search Server Common MS Search Query Processor 97zv
Exception System.IO.FileNotFoundException: Could not load file or assembly
'Atlassian.Confluence.SharePoint.Search, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=96f82bd35ec392e7' or one of its dependencies. The system cannot
find the file specified. File name: 'Atlassian.Confluence.SharePoint.Search,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=96f82bd35ec392e7' at
System.RuntimeTypeHandle._GetTypeByName(String name, Boolean throwOnError, Boolean
ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, Boolean loadTypeFromPartialName)
at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase,
Boolean reflectionOnly, StackCrawlMark& stackMark) at System.RuntimeType.PrivateGetType
(String typeName, Boolean throwOnError, Boolean ignoreCas...

09/18/2008 11:15:42.14* w3wp.exe (0x0EB0) 0x0EF4 Search Server Common MS Search Query Processor 97zv
Exception ...e, Boolean reflectionOnly, StackCrawlMark& stackMark) at System.Type.GetType
(String typeName, Boolean throwOnError) at Microsoft.Office.Server.Search.Administration.
Security.PluggableSecurityTrimmerManager.Instantia

09/18/2008 11:15:42.18 w3wp.exe (0x0EB0) 0x0EF4 Windows SharePoint Services Web Parts 89a1
Monitorable Error while executing web part: Microsoft.Office.Server.Search.WebControls.
SrhdcGenericException: Your search cannot be completed because of a service error.
Try your search again or contact your administrator for more information. --->
System.IO.FileNotFoundException: Could not load file or assembly 'Atlassian.Confluence.
SharePoint.Search, Version=1.0.0.0, Culture=neutral, PublicKeyToken=96f82bd35ec392e7'
or one of its dependencies. The system cannot find the file specified. File name:
'Atlassian.Confluence.SharePoint.Search, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=96f82bd35ec392e7' at System.RuntimeTypeHandle._GetTypeByName(String name,
Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark,
Boolean loadTypeFromPartialName) at System...


read more
Kirk LiemohnBreaking MOSS Search