John Underwood is a Technical Evangelist at ThreeWill. He has nearly thirty years of software development experience. He is an experienced technical instructor with superior presentation skills and is proficient in delivering standard curriculum as well as developing and delivering custom curriculum.
Hello and welcome to today’s webinar, my name is John Underwood and today’s topic is Hybrid Environments in SharePoint 2013. I appreciate each of you taking the time out of your Friday afternoon or it’s Friday midday to join me. I see from the attendee list some of you are familiar faces that have attended our events before. If you’re in that category thank you so much for doing that. On the other hand if you’re new just a little bit about me. I work at ThreeWill, my title there is technical evangelist. In simple terms that means I get to do a fun combination of coding technical work and also presentations, video production, speaking at groups and so on.
ThreeWill’s a great company and I feel really fortunate to be an employee there. Contact information if you need to reach me during or after the event you can use my email address. I don’t anticipate any outages today but sometimes things happen. If we were to have an interruption and you couldn’t hear me or you couldn’t see what was going on in the screen in the worst case you could shoot me an email and it would show up on my. You can also use that after the event if you have some follow up questions or questions about other events that we might be holding in the future. Then always happy to pick up Twitter followers so you can see my Twitter handle there if you’d like to follow that.
All right. Usually when I build a presentation I make some assumptions about the audience and I like to share those. It gives you some expectation of what I’m trying to address and do and how I’m trying to reach you. The first members of audience today to consider are those that might be in a management position or a business owner or even a user. For you really the question is what on earth is a hybrid environment? Why would it be used and why would I even consider having one in my enterprise? We’re going to talk about that. If you’re a SharePoint administrator you’re likely going to think anytime we talk about doing something in SharePoint what’s it going to mean to me? What’s it going to mean to our network? What’s it going to mean to our servers, to our security profile and so on.
We’ll talk a bit about that. Then the roles that I find myself in often, that of a developer, what can I do to help? How can I take these basic [inaudible 00:02:22] of a hybrid environment and customize it to make that as useful and seamless as possible for my customers and my users? Like a lot of things that are one size fits all there maybe some parts that are perfectly relevant to you and others that may be less so. I hope at the end of our roughly hour together that you’ll be able to say, “Okay, in this role I now understand a little bit better about what a hybrid environment is and how we might use it within our company.” Given that what are going to talk about today?
We’ll, first we’re going to talk in a somewhat general sense about moving things to the cloud. Really this will be a review for some of you and an introduction for some of you on why on earth we’d even think about doing this and what are some of the decision points? What I find as I deal with our customers is that this is perhaps the most important part and it’s just trying to decide whether you should be going to the cloud or not. Some things sit very well there and maybe other things not quite as much. Then once we’ve talked about having things in the cloud in a general sense then we’re going to talk specifically about SharePoint 2013 Hybrid Environment and what that’s going to mean to you as an owner or user of SharePoint environment.
Then from there we’ll talk about some of the practical challenges. We’re going to see that the hybrid environment topologies or architectures that Microsoft has put together for us provide a lot of functionality. There are some places at the edge where we may have to make some customizations or maybe just have our eyes open as to what the capabilities are and are not. Then ultimately these practical challenges are going to set up case study. We’ve actually had the good fortune of implementing a hybrid environment for a company here in Atlanta. I’m going to talk a little bit in some general terms about what we did for them and the architecture and how we were able to smooth some of those rough edges and provide them with a really useful setup.
Let’s talk first in a general sense about going to the cloud with any of our enterprise systems. There are a lot of positive things about going to the cloud in theory. The first of those is just getting out of the data center business. I can tell you that I work at a fairly small company and this was a constant concern to us of, “Okay, we’ve got servers on site but we don’t really have the resources to mange and protect them the way that we might like to.” In some cases be it an economic decision or a safety decision it just makes sense to get out of the datacenter business and let that be someone else’s problem. An analogy I can think of if you were to go back to some point in our history a lot of big manufacturing companies would actually have a power plant there on the site.
In effect they would make their own electricity but somewhere along the way it just got easier to make that someone else’s problem. That might be a motivation for you using or moving some of your systems to the cloud. Another one and I think this one is somewhat of a … well, it’s big one. The burden of managing and patching software. Moving to the cloud gives us an opportunity to reduce that because in effect we’re turning our software over to a party who is solely responsible for doing that kind of thing. When we think about patching software we’re certainly thinking about maybe getting the latest new whiz-bang teacher. Even more than that is this notion of risk reduction on the un-patched software that we have.
I’m sure Microsoft and a lot of other vendors could tell you stories about the percentage of their customers that don’t apply patches on a regular basis and what kind of security risk that represents. By moving to the cloud we’re really engaging in a practice that will let us lower that risk. The other thing, I know this is going to be a cornerstone of SharePoint 2013 going forward is this notion of continuous innovation. Instead of having to wait two or three years for the next big bang release in order to get a new useful feature those features will come more regularly. Because we’re in cloud environment that is somewhat controlled by the vendor or the provider of that software they’re able to roll out those innovations more quickly.
For us we’re able to take those on more quickly without this giant conversion effort to move up to that. Another benefit of the cloud and this one is a big one is that you pay only for what you use. Pardon me. I’ve seen various studies that talk about the number of computers that we have in an enterprise and what percentage of their CPU capability is actually utilized on an ongoing basis. In some cases it’s a tiny fraction. We’re paying to power and cool and run these devices but we’re really not using them to their optimum. Likewise we have this notion that we have documents sitting around, we’re paying for the entire disk even though we may be using only some portion of that disk.
Bottom line here is that when it’s applied correctly we have a lot of opportunity for good things when it comes to moving to the cloud. Now, I’ve been in the industry long enough, since literally the early to mid 80s that I’m skeptical of any new panacea. I guess maybe a better way to say that is with every major advance we make there are some major positives that come out of that. We also want to have our eyes wide open to the risks or the downside. I want to just let you think about this a little bit. The theory is that everything is going to be golden but the reality is that there are some hard questions that we should probably ask.
The first of those is if we get out of the datacenter business do we engage a vendor that can provide the needed service levels and can they do it better than us? On the one hand you might say, “Well, gee they have a big staff and the expertise and they can do a better job than us.” On the other hand it is a one size fits all environment so knowing the vendor that you’re engaging is a pretty big deal. Another thing that not only maybe has some security around it as far as practical matters but maybe even just feeling in control versus the loss of control and that is my company’s important data is on a physical location now that I don’t control.
The reality is we may actually be turning it over to an environment that is physically hardened and is actually safer than where we have it now but because we can’t walk down the hallway and touch it there’s a little bit of anxiety maybe that comes on our part. Some other things to consider, yeah, the software gets patched but if we’re talking about software were we’ve done a lot of heavy customization there’s always the risk that those patches or those updates would break our customization. What that means is that if we’re just using the software as is they’re probably in great shape. If we’re customizing it we have to keep our finger on the pulse of when patches are rolling we have to be sensitive and attentive to the fact that those changes are coming.
Continuous innovation is a good thing but it probably means there’s some expectations sitting on the part of our users. A user goes on to a page and all of a sudden there’s a new feature there and they don’t know about it and they may be taken by surprise or maybe a little disoriented by that. That probably is a burden that we have to carry within the company that we’re just making sure that we’re keeping our users informed on what is coming. Now, for the pay only for what you use I think there’s two sides to this. There’s mostly a positive side where the user is paying only for what you use. On the other hand if you’ve got a big server that’s got lots and lots of documents on it the last thing you want to do is just take a bulldozer and shift those up to the cloud.
Because the reality is a lot of those documents for all practical purpose are never going to get used again. They contain obsolete data and the cost of keeping them in the cloud even though you’re paying only for what you’re using is probably going to be higher than keeping them in some kind of on-premise stores mechanism. Another thing to think about even if you legitimately need to move all your data to the cloud depending on where and how you’re talking about moving it you may not have the opportunity to ship some kind of device and have it transferred locally at a datacenter. You might actually be looking at having to upload that and depending on the pricing model of your environment you’re going to be paying a heavy price to move that much data through the pipe.
Then finally both the curse and the blessing of an online or a cloud environment is that it’s accessible from anywhere. What that means is that perhaps that’s a delicious inviting text surface to a would be hacker particularly depending on the kind of business that you may be in. Now, don’t get me wrong at all these questions the truth is I’m pretty big on what the cloud has to offer [inaudible 00:11:28] and particularly in some business settings it makes total sense together. ThreeWill as a company is actually betting on that but always we want to just go into things with our eyes wide open and we want to make sure we’re asking the right questions. Now, here’s what I think for a lot of companies.
For a lot of companies if we honestly investigate and answer these questions what we’re going to find is that the answer is some of the stuff should go in the cloud for a variety of reasons and some of the stuff ought to stay on site. In a general sense when we talk about a hybrid approach or a hybrid environment we’re talking about that very thing. Some set of data or some set of apps remaining on-premise and others being moved to a cloud environment. Now, there might be a couple of reasons that you would want to do that. Maybe it’s just a better return on your investment for your dollars. The pay as you go model suits your world better.
Maybe we have international users that are in small clusters and we just can’t maintain datacenters close to them. Maybe we’ve got mobile users and a cloud approach would give us a better way of meeting their needs. Another thing that would be a hallmark of such a hybrid approach is for the things that don’t really need to go to the cloud we’ll keep them on-premise. Maybe that’s data of a sensitive nature, maybe that’s a data that has to remain within a certain country’s borders. You may or may not be aware of the fact that there are certain kinds of data that are not supposed to be transported across boundaries between countries. That will drive us toward an on-premise.
Then if we’re going to do some kind of hybrid approach the biggest thing we want to do is make is seamless. We don’t want our users having to wring their hands about what goes where. Either that means some education and some guideline or wherever possible maybe even some software assistance that says, “You put your stuff here and we’re going to make sure that it gets to the right place.” Those of you who are familiar with the concept of a document center and a record center and a notion of a drop off over inside of SharePoint that might be a model that works great here. Where you always have your users take their documents to a particular destination.
Then we have a set of automated rules set up to decide whether those end up on-premise or in the cloud. Then probably the last one is the biggest and most important and in fact something we’re going to focus on in today’s talk and that is making the data findable. Making it easy to find and making it easy to find in such a way that a user doesn’t have to get a lot of conscious thought to where is that data? They just know the criteria of what they want and then we ought to be able to serve that up to them. Then given that kind of overview about online environments or cloud-based environment in general and some of the things that we might consider when we do a hybrid model what we’re going to talk about next is the specifics on how Microsoft has provided that capability in SharePoint 2013.
We refer to that as a hybrid environment. In simple terms it is a combination of SharePoint Online or if you prefer Microsoft 365 and SharePoint on-premise. The notion there is that we’re going to let some things stay on site and some things move to the cloud. Now, a little more detail here on why you might not move certain things to the cloud. The first sub bullet that you see there custom farm solutions, that is a big one. We’ve done a lot of SharePoint customization over the nearly 12 years that we’ve been in business and a lot of those were written as farm solutions. Those weren’t great in an on-premise setting but they’re never going to be supported in the cloud.
If you have existing SharePoint applications that are built around farm solutions those are going to have to stay behind until you reach a point where it makes economic sense to rewrite or re-engineer those for cloud. Another thing is when we’ve written SharePoint customizations or maybe even just used BCS to integrate with internal applications. Now, you will see that we do have some BCS capabilities in the cloud and I’ll talk about those. There may be cases where the data that we’re integrating with is buried so far within your networks, so far behind a firewall that it just doesn’t make practical sense to try to integrate that from the cloud.
Another reason that we might remain on-premise and I spoke about this a little bit a moment ago and that is migration cost. In particular those of you that have done a lot of SharePoint work and you’re thinking to yourself, “Well, we’ll just detach the content database if they’ll reattach it to another syncing server and boom we’ve migrated.” That’s not really going to happen in Microsoft 365. You’re not only thinking about the storage cost of that data over the long term but you’re also thinking about the bandwidth cost to move things up. I think that kind of [inaudible 00:16:33] forward or drives forward a model where a lot of companies are going to treat their cloud instance of SharePoint or Microsoft 365 as kind of a green field where they’re starting clean.
That may give us some nice opportunities to be a little more organized, to maybe try to avoid some SharePoint sprawl that we’ve had in the past and then we let the stuff on site continue to stay there and live out its useful life. Then the last thing to think about when it comes to on site versus in the cloud, I kind of hinted at this a moment ago. Beyond just the notion of whether it is more secure in my building or in Microsoft’s building there is this notion that there are some legal parameters about where you store certain kinds of data. There’s the Safe Harbor agreement that has something to say about how data crosses international borders. In a more general sense that’s known as data sovereignty.
Bottom line is that you may not even know that this is a risk because you never really thought about putting your data in cloud before. Microsoft has the capability to make sure that you’re able to use the cloud environment and still remain in legal compliance here. Bottom line is you’ve got options and a hybrid environment is just going to let you make a good choice about what you move up and what you keep down. Now, if we’re going to go with this hybrid environment why or maybe how do we connect them? What do we really mean here when we say a hybrid environment? Well, one of the ways that we can integrate is by making use of our own premise active directory with SharePoint Online.
Now, there’s a couple of ways that this integration can go but the bottom line on this is that what we’re trying to do is make it so that we only have to maintain information once. I think what you’ll see is that as we run across the gambit here some of this is going to be automated, some of this is going to be a manual process and some of this may result in us having to write some custom code. Then beyond that really the thing that your users are going to see the integration points. Now, I would tell you that for most of us based on what I’ve experienced with a hybrid environment the biggest integration point is going to be search.
We’ve got some population of data that’s in the cloud for whatever reason, we’ve got some population data that’s on-premise and we want to make the searching of that data as seamless as we can possibly be. When we get into our case study in a few minutes you’re going to see that we exerted a great deal of effort there and I think that we were also able to really give our users a seamless experience. On that there’s also integration questions of BCS. Then for those of you that use SAP there’s a product called Duet that allows you to do some integration between SharePoint and SAP. A part of this hybrid environment just has to do with whether or not Duet will work with a certain topology.
I’m not going to focus on that one a lot because we were not providing that capability for our customers but for those of you that might do that you’ll see it mentioned on a few sites and you can just makes note as to whether it’s going to work in the topology that you choose. Then what do I mean by topologies? Well, you might prefer the word architecture. Essentially Microsoft has come up with some recommend practices for connecting an on-premise and Microsoft 365. Then what we’re going to see is that which one you choose is going to be driven in large part by what functionality you’re willing or needing to have and what kind of security trade-offs your willing to make in order to achieve that.
The three options that we’re going to see the first is called a One- Way Outbound. The second is called a One-Way Inbound and then finally the Two-Way or bidirectional. Now, one of the challenges with labeling things with Outbound and Inbound is that you can get a little confused about the perspective. This may be a note worth making throughout this whole discussion of topologies when we say Outbound and Inbound it’s always from the perspective of your on-premise server. That’s your ‘home’ and then we’ll be going Outbound or Inbound based on what we’re trying to accomplish. Then with that you can see a little diagram here that’s been provided by Microsoft called a One-Way Outbound topology.
In simple non-technical terms SharePoint server 2013 that’s on-premise can see Microsoft 365 and it can see your SharePoint Online data in side of that. Then it’s eligible to provide some kind of integration based on that. Now, the very simple version of that is that you’re going to get search but you’re only going to get integrated search in one place. Notice because we don’t have any connectivity from the cloud to the on-premise we can’t show on-premise results in Microsoft 365. However, the opposite is true when we’re serving up on-premise search result pages they can not only show search results from the local farm or farms but also from SharePoint Online.
You can imagine this being a model where you would always want to drive your users toward a local search page that’s in the on-premise farm and then whatever it is that we’re searching for we’ll be able to find it both locally and in Microsoft 365. I’ll just go ahead and spoil the surprise for you and tell you that for the customer that we worked with this is the approach we chose and we’ll talk a little bit about that later. What that means is that as we’re dealing with their users we’re really trying in a lot of cases to redirect their users into using the on-premise search pages so that we can present both sets of results to them. Now, the downside here on this topology BCS connectivity from the cloud to the local instance is not going to happen [inaudible 00:22:36] integration.
Now, I would say based on our experience with this customer and the discussion that we’ve had with other customers this is probably going to be the most common way of doing this integration and the reason for that is really simple. Most people no matter how well it’s thought out, how well it’s presented and how well it’s implemented they’re going to have a problem with the cloud environment being able to connect in through their local environment. On the other hand if a company has the desire, the need and the wherewithal to do that that’s really what the other two options are about, some kind of inbound connection. Then the second one is a One-Way Inbound.
You can see clearly from the diagram here that the differences that Microsoft 365 is going to have to be able to go through your environment and be able to connect to your on-premise environment. The capabilities here are just the opposite. Now we can indeed show both SharePoint Online results and on-premise result on a SharePoint Online search result page. We would not be able to do the opposite of that locally. If you think about the two that I’ve just shown you here probably the biggest takeaway was this, if you’re going to be in a world where SharePoint Online is really the place to be and then, “Oh yeah we have some stuff locally that we’d like to be able to serve up” then this is the model that you want to take.
On the other hand if having an inbound connection is just a show stopper or the on-premise is still your primary place and Microsoft 365 is more of a secondary place then you would use the One-Way Outbound topology that we saw earlier. Then finally obviously we get the most functionality by having a bidirectional kind of topology. In this case both environments are capable of showing search results from their own environment and from the other environment. We have all of our BCS connectivity options open, we also have our options open when it comes to doing connectivity using Duet. Really the bottom line here is just what am I willing to take in terms of open or not opening my firewall? Then based on that I’m going to have certain levels of functionality.
Luckily for us the customer found they were really adamant about not opening their firewall. The truth was we were able to give them what they wanted just by using the first of three topologies. All right. Before I go into the case study and talk some of the details of that now I want to just talk about some practical challenges. Ultimately this is going to be a set up for how we solved things. I hope you’ll get two things out of this, number one you’ll see how we were able to go in and solve some of these problems. Then beyond that you’ll also think about in your own world, “Okay, if we do things this was what’s it going to mean to us?” The first of those has to do with user profiles.
Right now when you talk about a hybrid environment you’re talking about two separate sets of user profiles, two completely unconnected [inaudible 00:25:47]. Now, Microsoft has talked about it in some of the technical briefings that over the horizon they see a need for solving this problem and that there’s going to be some future integration opportunities. For today those don’t exist out of the box so at the very least if you’re going to have user profiles in two places and you’re going to have users that are going to be in both places you’re going to have to do something about keeping that straight. Maybe that’s just a set of procedures and policies that have to be followed to the letter or maybe that’s some kind of code or integration or customization that helps them do the right thing.
You’re going to see in what we did for our particular customer that this was a big part of what we did for them. Trying to make it so that the user profile is only updated in one place and then we properly propagate those changes both to the cloud and to the local on-premise environment. Another thing that we think about here both in user profiles and data is this notion of the single source of truth. This speaks to the fact that we’ve either got to have some automation or some darn good policies to make sure that people put data in the right place. Perfect example is we might have a proposal we’ve done, person one stores it on-premise, person two expects to find it in the cloud.
They don’t see it in the cloud so they create it from scratch and now we’ve got two copies of the truth and which one is the correct one. Again, some combination of automation and polices help us manage this. Then when it comes to user profiles if we have the policies in place and we tell our users to follow them we may be with a really attentive, really educated user community we could get that done. The truth is even attentive, well meaning users are probably going to reach this point where they go to change something and they don’t even realize they’re changing it in the wrong place. They just know they’re on a profile page and they’re going to update their expertise so that people will know about it.
At some level you probably reach a point where if you’re truly dedicated to keeping these user profiles in sync you’re going to have to have some sort of mechanism to force users to make the change in only one place. What you’re going to see is that that’s going to be some combination of out of the box capabilities along with some custom code to make that work. Then the last thing making sure that people search works correctly you may consider showing up people results on our search results pages. We want to be careful that we’re doing that from the so called single source of truth. Whether it’s going to be the cloud profile or the on-premise profile that rules as the master, we want to make sure that we’re featuring those properly in our people search.
Now, one other kind of challenge that we need to think about when it comes to this sort of thing is SharePoint 2010. If you looked really carefully at the diagrams that I provided for you just a moment ago they all exclusively said SharePoint 2013. The hybrid topologies are built around that both locally and in cloud. Then what are those companies that are slow to upgrade? [inaudible 00:29:04] if you’ve worked in the SharePoint community you know that there’s a lot of drag when it comes to going from one version to another. That’s not necessarily saying anything negative about the software or the customer, it’s just a big piece of software and it’s a non trivial task to upgrade it.
In that case we’re going to have to think about how to integrate those two. What you’re going to see in the solution that we actually proposed and are implementing for our customers in that at some level it’s going to be important. It’s going to be a requirement that we have SharePoint 2013 on site. Then if the customer is not in a place where they’re ready to move their entire on-premise to SharePoint 2013 what we will have to have is at least one server there that can act as an integration point. I’ll talk a little bit more about that as we get into the case study and even back that up with a few diagrams.
Now, when it comes to showing search results we’ve got to think about how we’re going to integrate the local results and the cloud results. We’ve already seen from our previous slides that it’s going to depend on the topology we’ve chosen as to which of these are even possible. Beyond that we’re going to see that the concept of a refiner is going to be a little bit different in this world. As much as we would like to be able to do refiner simultaneously on cloud and local data the truth is there are some limitations there. Yes they work and yes they are going to refine the results but we’ll see that we have to think about what are the implications of using or depending on refiners in a hybrid environment.
Another thing we have to be aware of is security trimming. Now, the good news is we do have security trimming and our experience and our proof of concepts leading up to this project with our customer definitely proved that out. The one that you have to be careful about is not so much that somebody is going to see something that they’re not supposed to see but the opposite problem. That we may in some cases have some over-trimming depending on how and when a person has authenticated. There’s always this notion that you’ve got to be aware of them authenticating both against the cloud environment and the on-premise environment. That’s going to influence us properly trimming their search results.
With that let’s take a moment and talk through our case study and we’ll get into some specifics on how we were able to implement this system for our customer. As I said Fortune 500 company they have a big on-premise investment in SharePoint 2010 and then for a variety of reasons they’ve decided that Microsoft 365 and SharePoint Online is a product that makes good sense for them. They’re in the process of beginning to move some of their data into that environment. As far as their local on-premise servers they’re not in a position where those can be upgraded to SharePoint 2013 immediately. As I talked about before we’re going to have to use SharePoint 2013 on-premise as a shim to integrate with those 2010 servers until some future point when they’re able to upgrade.
Then of all the things that we provide in this solution the two biggest are integrating search results and dealing with user profiles. The things that I mentioned earlier when we were talking about some of the details. As much as possible we want to be able to show integrated search results between cloud and on-premise data and we want to have a single user profile identity. We want to make it so that it is impossible for somebody to do the wrong thing with that profile. Then I’m going to give you the words first and then I’ll give you the pictures because people do better with one or the other. If you remember back to our three topologies earlier we chose the One-Way Outbound topology.
The decision on that was very simple. It gave us enough functionality and at the same time they were simply unwilling to consider an option that meant opening their firewall. When it came to dealing with user profiles the SharePoint Online user profile was dubbed to be the profile of record. That was the source of truth. One of the reasons they chose to go with that approach is that there will be some users in this community that are cloud [inaudible 00:33:34] users so that made that decision simple. Then around that we were able to provide some additional customizations. Some of that out of the box and some of that custom code that would enforce that so that when a user went to look at a profile or went to change a profile they would always be seeing the correct data and they would always be changing the data in the correct place.
I’m going to say that again because that’s really important. That’s not to say that someone would never see data that came from a local profile because in some cases that’s going to happen. When someone is making a change we always had to make sure that they were making that change in the cloud and then at some point those cloud changes would propagate to the on-premise environment. Then as I hinted at a couple of slides ago we did have to [inaudible 00:34:24] a SharePoint server 2013 instance inside of their company to act as a search integration point. Think of it this way it says though we’ve got a SharePoint server that has a search center and nothing else it’s not going to be serving up any non-search sites, it’s not there for teen sites.
It’s just a way for us to get our hybrid topology set up. Then obviously that SharePoint 2013 server is going to be responsible for crawling the 2010 content for indexing it and then presenting that alongside whatever other data we might want to show. Again, those of you that do better with pictures as opposed to words that’s the 30,000 foot view of what we did. We’ve got Microsoft 365, we’ve got SharePoint 2013 on-premise just for the purpose of search integration. Those are put together with a One-Way Outbound topology. Then some things that I’ve not yet mentioned but I’m going to talk in great detail about in just a moment and that is in the lower right corner of this diagram.
Most of you may be aware of this but for those who are not when you go to modify your profile as a user what you’re really doing is going to some place inside of the my site host where all of your my sites are hosted. Essentially what we did here in order to ensure that they’re always modifying their profile in the cloud as opposed to locally is we actually put some custom redirection in here. In effect when a user clicks on something that will take them to that place we’re going to catch that and send them to the proper place. There’s really not a practical way for a user to arrive at the local profile, every time they’re going to get shipped off to the right place.
We’ll talk a little bit about how we did that and how that happens. All right. We’ve already said that we decided for practical reasons to make the cloud profile the one that was the profile of record. However, we’re talking about a new environment where we’ve got a lot of existing user profiles on-premise. As the cloud is going to be the place of record then we’re going to have to get those into the cloud. This was one of the cases where we had the rights and custom code on behalf of our customer. Essentially we wrote a program that does a one-time read time version. It’s going to go in, it’s going to read all of that local profile information and then it’s going to create the profile in the cloud and spill in all of the details.
At some moment in time in the process of throwing the switch and opening the new environment we would run the spin one last time. Everything would be in sync and then immediately we would have to put in the mechanism that would keep a user from changing things locally. Now, the ongoing challenge there is that we also had to write code that’s going to synchronize changes from the cloud back to on-premise. We’ve put up the proper barriers to make sure that users are always customizing and updating their profiles online and then given some lag of time eventually our synchronization job will run. It will find all those changes in the cloud and it’ll pump those back to on-premise.
Again, it’s one of those things where when you run it you just have to decide what sort of latency you’re willing to live with. I would say in most cases when we’re talking about updating user profile data hours at most days would be suitable, certainly not weeks or months. That’s just a factor that would govern how often we would run that synchronization program. Then as I mentioned in the diagram a moment ago the on-premise profiles are practically hidden. We made it so that it’s impossible for a user to reach one of those. Again, let’s look at it in picture form. The first thing we’ve got to do is we’re going to go into on-premise and we’re going to configure the location of our My Site host.
This is something we would always do but in this case the importance here is that when you look at the URL for the My Site host what’s going to happen is when a user goes to that URL there’s not actually going to be a My Site there. There’s not actually going to be a profile page. Instead what there’s going to be is a bit of custom code. It’s essentially sitting there waiting for those requests and then anytime it sees that a user is trying to navigate to their profile information on their local My Site it’s just going to catch that and redirect it intelligently to the right place online. For the user all they know is that they clicked on something that was going to take them to their profile and they landed on a profile page.
In fact if we see the usage that’s the very experience they get. They go up and they click on their name, they choose to go to their profile or to their about me page, that’s going to take them to the local My Site host where the redirection logic is waiting. That will then forward their request to the cloud and they’re presented with a profile page. If you had a super, super observant user they might be able to notice this in one of two ways. One way they might be able to notice it is that the URL would be different. I would argue that for most users that’s not something they would pick up on. The other thing that would be a little bit different in the case of our customer is the fact that they might have started on a 2010 page and wound up on a 2013 profile page. Otherwise it’s seamless.
Again, because of the way we implemented the solution even if a clever user were to be able to deduce the actual URL of their local profile information as soon as they tried to manually type in a URL and navigate there they’d just get redirected right back to this place. It’s a pretty sound way of doing things. Then again once our test user in this case Kim has made whatever changes she needs to make from there the scheduled sync job will go in, find any changes she’s made and then replicate those or synchronize those back to the on-premise. Ultimately her data is going to be the same in all the places. Now, let’s talk a little bit more about some of the practical aspects of how we were able to integrate search on behalf of our customer.
As we’ve talked about before we installed an instance of SharePoint server 2013 on-premise, it was responsible for crawling and indexing the 2010 content that was on-premise. Then it was also there to serve up search results. That last part is really important, we’ll talk a bit about why that is so important. Bottom line here is that it’s a SharePoint server that’s really only serving one role. Now, something that would be really nice but just isn’t practical today and that is the interleaving of on-premise and cloud results. To that another way it would be awesome if you could serve that up as though it was one giant result search. There’s really not a way to do that, at least not at this time.
The better way to think about this is that as you’re integrating data each set of data that you’re going to integrate are going to be configured as result sources. You can think about a result source for cloud documents or for cloud users or for cloud social activity or any of those things. Then in effect we’re going to be able to take those result sources and integrate them on a certain page. Now, the other thing to think about, because of the topology that we’ve chosen here if we show them an on-premise results page it is able to show data both from a local result source and a cloud result source. In fact the way to think about this is the local data would be the default result source and then the cloud data would be shown as a secondary result source.
We’ll see with the screenshots here in just a moment. Really what we’re talking about with that secondary result source is just showing it in a featured box and we’re going to talk about the configurability of that. then the last option when we’ve got someone on a cloud page due to the One-Way topology that cloud search result page can only show things from the cloud. Another thing that we attempted to do on behalf of our customers here is as much as possible we try to send them on-premise to do their searching. There is actually some custom logic in the cloud search center that as when a user arrives here if they have on-premise capabilities then we’re going to redirect them to the on-premise search center to do their search.
In this case by connected or by having capabilities we’re really talking two things, do they have a VPN connection and then do they have the proper credentials. If it’s cloud only user they’re going to see cloud only result [inaudible 00:43:14]. If it’s a user where it’s appropriate even if they manually navigate to the cloud search center, in effect they’re going to wind up redirected back to on-premise so that we have the opportunity to show them the data from both places. Again, just trying to make it as easy and fool-proof as possible for our users to get what they need. Again, just a little screenshots here that show some of the configuration.
You can see that in our own premise server we have actually set up the various result sources. SharePoint local data is set up as the default and then any of these cloud result sources that we’ve configured and actually we use as secondary or featured results that we show on the page. Then from there what does a featured result look like? Well, it’s really just a block and you can see a little screenshot that’s on the screen here. In this example it’s configured where the cloud results are showing in a block on top of the page. Then from there the on-premise results are below that. Now, I’ll talk about some practical limitations of this in just a moment but for now a couple of things that you want to be aware of.
Number one, you have some configuration choices on where this block goes. The options that’s been chosen on this example is to simply place it at the top of the page. User goes to an on-premise search page, they’re always going to see cloud results in a block at the top and then that’s going to be followed by on-premise results. Another possibility here is that you can actually take this box that’s featuring the secondary result source and you could configure it to show in line based on its relevancy. Again, I have to say we’ve done a little research there and it’s not immediately obvious whether it’s the relevancy in the first item or the relevancy of all the items.
Essentially what would happen is in that case it would be shown further down the page or even on subsequent pages depending on the relevancy of the data. Again, you just have to make a choice as an organization do you want to present this say, “Hey this was always in the cloud” versus showing it in a block in line and maybe having it further down the page. Another nice thing though about the block is that regardless of where it is if we need to see more detail on that the end of the block always has a show more link. If someone said, “No, I want to go to the cloud and I want to drill in just on this” then they would have the ability to do so. From there they’d have all the full capabilities that they needed.
I would tell you that based on our experience and based on the criteria that they user set forth or that the customer set forth we did a really good job of providing them with a seamless environment. I think a lot of that just speaks to the thoroughness of the hybrid topologies that Microsoft has provided and the good out of box support that we have. Having said that there were a few loose ends just things to think about not necessarily show stoppers but some things that we’d want to be aware of. One of our biggest challenges was getting the profile picture right. Based on some research we’ve done it appears that deep in the dark halls of SharePoint that in some cases that picture is being stored in as many as four different places.
Depending on how you place it and where you put it and what happens we still may wind up with differing results on pictures. A couple of other things that can contribute to that conversation not only do we have this concept of a SharePoint having a picture but Outlook and Lync. If you’re using those in Microsoft 365 they also keep their won profile pictures. Trying to get all of that perfectly synchronized can be a challenge. Another thing that was a challenge was trying to synchronize the picture between the online profile and the local profile. One of the ways that we were able to solve that is we actually wrote a very simple little web service that would allow the local profile to use a URL that would request the picture from the host.
Instead of depending on these locally stored picture we would just dynamically [inaudible 00:47:24] from the cloud anytime you needed it. A little bit of custom code involved that are not terribly complicated and it did get us much closer to that idea of having the same picture everywhere. Really more than anything that’s probably just a user expectation of, “Hey if you go to your SharePoint profile and you change your picture that picture is going to show up in a lot of places but it may not show up every single place. Really probably the better procedure is if you want to change your picture change it in SharePoint On line, Outlook Online and Lync Online.
All of those in Microsoft 365, and changing all three of those is probably going to give one the best opportunity to have the same picture everywhere. Another thing to be aware of you’re probably used to seeing search pages where we have a few results and then we have a people section and a video section. While we still have that capability we can’t have that capability nested within our results, that box that’s we have at the top of the page. In effect you would get that for the local data or the on-premise data but you wouldn’t necessarily get it on the cloud data unless you actually clicked through and went to the show more. Once you arrive in the cloud page then you get the full fidelity of that data.
Another thing to be aware of, refiners, I hinted at this a moment ago. When you go to a page that is showing both on-premise data and cloud data you’re going to see refiners. The set of refiners that you see are going to be based only on the default result set. If there were something in our cloud data that might hint to another refiner we’re not going to see that refiner if we’re on an on-premise page. We’d actually have to go to the cloud to see that additional refiner. Now, having said that the refiners that we do see actually work on both sets of data. If we were to go over to the left and click on a refiner for word docs only when the page refreshed not only would our on-premise be subject to that refiner but the cloud data that we were showing would be subject to that as well.
I don’t think that’s a huge show stopper it just means that the number of available refiners maybe under-reported in some cases if they were being driven by data from the cloud. Then I believe I hinted at this one earlier as well. Security trimming works but depending on the authentication sequence there may be some circumstances where over-trimming would occur and someone might not see a search result. Even though technically speaking as a human being they were eligible to see that data. All right. We’re pretty close to the top of the hour here let me just do a quick recap for you.
The biggest thing as exciting as it might be to close your data center and move everything to the cloud as a business person, as a user, as an administrator you want to make sure that you know why you’re going there and that you’ve asked the right questions and you’ve got a good reason for making that move. If you decide to make that move in the context of SharePoint Microsoft has done a great job of giving us topologies and guidance on how to integrate those environments to the greatest degree possible. When it comes to getting everything seamless, getting it right with your user profiles are going to require the greatest level of customization.
Pretty much everything we did with search with the exception of maybe some redirection in this out of the box stuff that we configured. For this particular customer we did have to write some code to manage the user profiles. Then hopefully the last thing you’ll take away here is just the fact that we’ve done this. We’ve actually done it for a customer, they’re using it and we did a good job of meeting your expectations. If you’re looking some guidance on how to make this happen in your world we hope you’ll turn to ThreeWill. At this point I know that for all of you it’s a Friday afternoon and so the likelihood of you sticking around any extra time is probably pretty slim.
Having said that I’m always happy to answer some questions. It does look like there a couple of questions in the question manager so what I’ll do is open those up and read them. For those of you that are going to depart let me just thank you again for joining us. It’s always a big deal to take time out of your day and so I hope you were able to get some good information. Remember, if you have any follow up questions you can reach me [email protected] Also if you’d like to be able to share this presentation with others we’ll do some video editing of it and we’ll have it up on the Internet in the coming days.
If you have any questions about finding that you can either go to threewill.com or you can email me directly. Again, for those that have to depart thanks for your time. Then for the rest of you I’m going to open up and just have a quick look at some questions here. One of the questions was why have the redirect instead of entering the address directly in SharePoint? That’s a good point. At some level we could have just pointed the local to the cloud. I think what was really the driving force in the redirect is that there was always this fear that a user could get there some other way. As a practical example maybe a user went to their profile in the old 2010 days and they’ve bookmarked it.
When they go to that bookmark they’re going to go right back to that page as opposed to going through something to get there. By setting up a dedicated place that was a dummy My Host and then having our won logic there waiting for things to redirect it just made sure that nothing slipped through the crack. That was a really good question there, thanks for asking that. Then that looks like all of the questions that we had so I’ll hang around for a moment and see if there are any others. Again, for those of you that have got to depart thanks for your time.
For the rest of you I’ll hang around for about five minutes and just make sure I’ve answered any of your questions. Look for our next webinar we’re already getting ready to schedule that for the month of March so I hope you’ll join us and I hope all of you have a good weekend.