Moving Sandboxed Solutions to the Cloud

Danny:Hello, and welcome to ThreeWill podcast. This is your host Danny Ryan. I’m here with Matthew Chestnut. Hey Matthew, how you doing?


Matthew :Hello again Danny.


Danny:Hello again, it’s another quarter.


Matthew :It is.


Danny:Good to see you.


Matthew :Same here.


Danny:Good to catch up on things. So I would classify today’s topic maybe would be “the devils in the details” or something.


Matthew :That’s a good way to categorize it-


Danny:Along those lines, which is we’re just going to catch up on a project that you have been working on, or planning out. I would put it that way.


And it’s a project where we’re working with a larger client, and we’re looking at upgrading from SharePoint 2010 to 2013-


Matthew :Yeah.


Danny:2013, wait it’s 2017-


Matthew :Understood.


Danny:What’s up with you guys?


Matthew :You know these big enterprise companies work. They move rather slowly.


Danny:Uh huh, so we’re finally upgrading to 2013, and I just wanted to talk through … I think some of this, in reality, this is what you’re going through. A lot of people are going through when they’re looking at going to SharePoint online.


Matthew :Absolutely.


Danny:So this is a pertinent topic for a lot of people, and preparing themselves for the future.


So, give me a little bit of the backstory on this whole thing, and let’s talk this thing through.


Matthew :Well this is a project that we’ve worked on over the last five years plus. This particular application.


Danny:Mm-hmm (affirmative)


Matthew :And it’s going through another iteration, because of the enterprise desire to move to SharePoint 2013 on premise.




Matthew :And, you know back in the old days quote on quote of SharePoint 2010, there was a certain set of best practices. Things like sand boxed event receivers, or sand boxed processes.


This is where code that used to run on the farm, in SharePoint 2007 … Now the IT group had the ability to sandbox, or put a wrapper around it. Which prevented applications from running wild, and bringing down the farm.


So in the SharePoint 2010 days sand boxed processes were the way to go, and that’s how we implemented this particular solution.


And the whole idea about a sand boxed process is, you’re adding a list item-


Danny:Mm-hmm (affirmative)


Matthew :To a SharePoint list, and SharePoint lists are ubiquitous. I mean that is the thing, along with documents, list data is very important.


And the idea is, is that when this item is created on an application that’s non-trivial, you may want to do other actions-




Matthew :The actions may be create a notification record, write some data to another list, do some processing on this data to make it more enterprise ready.


And the event receivers were awesome. In the sense that, when a list item was added, updated, or in the process of being added, or updated these events were fired, and code could run.


In the olden days, back in the days of, you know when we had just, or even asp. And we depended on SQL Server-


Danny:Uh huh.


Matthew :We had triggers, store procedures that would execute when a data base was added, updated, etcetera. So this is a similar concept. So it was great. We had the ability to run code.


Well now here comes SharePoint 2013, and more importantly here comes Microsoft changing their best practices. And they’re really making these best practiced decisions based on SharePoint Online Office 365. Just like you mentioned.


They don’t want code running on their farms, in the online cloud, because that would be a maintenance nightmare. How do you segregate it? How do you keep it secure? With the variety of tenants.


So they’ve come out, and said that, “Sandbox Online, in the cloud, is not allowed at all.” They’re on premise in 2013 SharePoint, and even SharePoint 2016 is allowed. So they are still allowing this user code host process, AKA the Sandbox to be configured, and run on premise.


But this particular company, our customer decided that they wanted to deprecate it now. So in other words they don’t want to wait until the application, or the capability goes away. They said let’s get rid of it now.


Well that’s a great opportunity for us to figure out how we’re going to solve this problem now-




Matthew :We have all this code, we have all this business logic, what are we going to do with it? Oh, and by the way this particular application was written using info path. Which is yet another technology that Microsoft has deprecated. Still works, it will probably still work through, you know the year 2025. But the customer wants to get rid of that technology.


So now we start moving into some of this newer technology that you may have talked, with other Three Wheel consultants about. Things such as Angular-




Matthew :JavaScript frameworks-




Matthew :And how SharePoint really participates very well in that world, and Microsoft SharePoint has got frameworks now, API’s. The patterns and practices one, is one I’m thinking of currently, that provides a great API for giving you JavaScript access to your SharePoint data. And of course with JavaScript it’s a programming language. You can do whatever you want with it.


So here we go, so we started in evaluating our choices. And we’re trying to keep the customers’ pocketbook in mind. Certainly, in our grandiose world, we could change this thing, and make it do all kinds of stuff. But we want to keep it affordable for them.


So we looked at remote event receivers. So this is a new technology that we haven’t used in this particular situation, but the remote event receiver’s gives the IT group the ability to stand up a server-


Danny:Mm-hmm (affirmative)


Matthew :That gives us some WCFN points that we can then call from a SharePoint list item event. So instead of the event calling code that runs on the SharePoint farm itself, even if it’s sand boxed, it runs somewhere else.


And it does some of the same things that it would do if it were running on the farm. It has the same client connectivity, it could read list items, update list items, et cetera.


Here’s where that got to be a little not desirable for us-


Danny:Uh huh.


Matthew :The remote event receivers worked great, except when they don’t. And here’s how they can not work.


For whatever reason let’s say the connection between our SharePoint farm, and the remote event receiver server is disabled. It’s down, or whatever. The challenge is the call to the remote event receiver fails. But it silently fails. And that of course is the worst thing that can happen to a developer, “Why does my application data not look right? “We don’t know, because it silently fails.”


Danny:So what you’re saying is if a remote event receiver fires in a forest, and nobody hears it-


Matthew :Right, does anybody care?


Danny:Does anybody care!


Matthew :Yes, the customer cares.


Danny:Yes, the customer cares.


Matthew :Yeah, so-


Danny:Did it really happen?


Matthew :Did it really happen?


Danny:Did it really fire? If it’s a fire’s in a forest, and nobody hears it, did it really happen?


Matthew :So that’s the challenge. When data does not get updated, and it’s supposed to be updated by this thing, and this thing didn’t fire. What was the problem? So-


Danny:Uh huh.


Matthew :Now of course we could add logging on our remote event receiver. So we would know if it got called, here’s why it failed. But if it never gets called we have no control over that.




Matthew :Yes, we could look at the ULS logs, and there might be some information there, but once again this is an enterprise solution. We don’t have access to those logs. Those logs are generating tons of data, too hard to wade through.


And the other challenge with the remote event receivers is that there’s like a 30 second time out. In other words, it’ll give you 30 seconds to do your work, and you better be done. Or else, it’ll just say, “Thank you very much. We’re moving onto the next thing.”


Now that wouldn’t have been a problem. We could of spawned another task that would run asynchronously. You know in the background without any user interaction. But that turned us away from remote event receivers. Just because they’re not guaranteed, or at least the guarantee was very, very loosely coupled-




Matthew :So then we started working backwards, and said, “Okay, well what about work flows? What can work flows do for us?” In fact, we were using work flows pretty heavily with this existing solution. That the challenge was work flows work great, but the work flow language is a little bit restricted.


But in the old days of SharePoint 2010, when we could write, and we were allowed to write sand boxed code. We were writing custom actions on events that a work flow would fire, as well.


So the work flow action could simply be, “Send notification.” And the work flow action was custom code, written in the sand box-


Danny:Mm-hmm (affirmative)


Matthew :Written in C sharp, deployed on the server. That would take that one line work flow command, send notification, and it would figure out whatever it needed to do. It’d find out oh, what type of notification do you have here? Who’s it need to go to, based on all these criteria?


So once again, workflows are great, but they’re limited in the language. They’re limited in their ability. They can certainly update list item, the current list item. They can create other list items, but if you start talking about program logic it kind of falls down.




Matthew :So we moved away from that. So basically now we’re going back to, “Okay, we’ve got JavaScript, we have the ability to communicate with JavaScript. We have to change this form anyway from info path, to Angular two. Let’s just write this logic in JavaScript itself.” And that’s what we’re doing.


And that’s going to actually work out very well. The Angular development using SharePoint as the back-end database, is working just as we expect it would.


Our code really doesn’t matter where it’s writing to, although we are using the SharePoint library. That could easily be a SQL Server library. We could write the data elsewhere, we’re sticking with obviously the SharePoint solution. Because the data structures are in place, and all the supporting lists are in place, to support this main form.


And that brings me to, well some of these supporting list items, these little look-up tables-




Matthew :Also have business logic, and we’re not wanting to rewrite those. We were heavily leveraging SharePoints’ ability list items. Add, edit, delete, view-


Danny:Mm-hmm (affirmative)


Matthew :Filter, all that stuff we get right out of the box. We still want to use that. We don’t want to rewrite those. So what we’re doing with those scenarios, is we have to participate in the event model, of a standard SharePoint list form.


And a standard SharePoint list form give you the ability, with JavaScript. You inject the JavaScript into the page using a couple of different techniques, but it’s pretty standard. And I think we even have some blog posts on our, that talks about using jQuery with SharePoint forms.




Matthew :But the idea there is there’s a pre save action, and on pre save we can do certain things like populate data values on a list. Or quite frankly we can do whatever we want, but that is a pre save action.


We also have when the document form is loaded, we have that action. So if we have some fields we want to hide, or show, or repopulate, great we have all those abilities.


Here’s where it came to be a tricky scenario. Is after the item has been save, and we want to do some other actions on that item, how do we do that?


Well our traditional way has been event receivers, but we can’t do that. Our other way has been work flows, but we don’t have the ability to do them as robustly.


So what we’re doing is we’re also using the ability on a SharePoint list form with new, and add it. Where on the save, and or cancel for that matter, you can redirect to another form.


By default it’s simply going to close the existing form, and take you back to the list page where it was launched from. What we’re going to do is transfer control from the page, the new, or edit form, and go to another page, and that page is going to have the logic. That’ll do whatever it is we need to get done. Whether it’s add additional list data to other lists, or create notification records, what have you.


So in essence we’re daisy chaining the application-


Danny:Mm-hmm (affirmative)


Matthew :So once we press save here, we use a standard out of the box SharePoint technique of redirecting to another page-


Danny:Mm-hmm (affirmative)


Matthew :And that page is going to pick up the work, and do whatever remains to be done-


Danny:Now I’ve talked with other people about this, did you think ten years ago that your career, you would be writing so much JavaScript.


Matthew :Well its funny, yeah.


Danny:And Angular’s based off of JavaScript-


Matthew :Yeah in fact, you know JavaScript was kind of invented to help out the browser experience-




Matthew :Do some things tweaking the DOM, the object model so you could hide show fields in a programmatic fashion-


Danny:Uh huh.


Matthew :It’s become the language du jour, if you will.


Danny:Uh huh.


Matthew :And we’re actually using in the Angular world, we’re using the TypeScript flavor on top of it. TypeScript’s just a layer, it’s another language that is an industry standard. But it trans piles itself at compile time, or transpile time.


It converts TypeScript code into JavaScript code, but TypeScript code gives us a little bit of language niceties. Like validation of data types, and certain constructs that what end up happening is when it’s transpiled, it converts itself into JavaScript. Which runs of course on every browser. But it makes the developer life a little bit easier.


Danny:I don’t think I’ve heard of transpiled-


Matthew :Transpile, yeah that’s a-


Danny:A new one for me-


Matthew :There’s always some-




Matthew :Buzz words, right? You know you’ll probably use that quite frequently, yeah, probably incorrectly, but you’ll use it frequently.


Danny:I use it as a marketing term-


Matthew :Right.


Danny:See what that transpiles-


Matthew :Transpile your business.


Danny:Transpile your business. So probably somebody listening to this is probably thinking, what I was thinking as well, which is why aren’t you using Azure for this?


Matthew :Oh yeah, that’s a great point. Azure would be an excellent solution, and even SharePoint 2016 on premise would be a little bit better, because it’s got some extra capabilities in regards to events. Like web hooks, that are more guaranteed delivery, etcetera. Out in the cloud-




Matthew :It would be great. We could have used remote event receivers even in the cloud. We were talking about using remote event receivers hosted locally, and there’s two different ways, a low trust, and a high trust. How you can figure all this stuff. But it would’ve worked.


In this particular scenario we’re not using the cloud because the customer’s not using the cloud right now-




Matthew :They maybe moving to that in the future, but we’ve got other customers who are cloud based, and we’re doing some of the same techniques that we’re talking about.


The challenge here is we have a working application. Works great. Has been in use for five plus years, and the IT group is changing the infrastructure slightly, and they’re putting new rules in place.


And so our business customers look at us and, “Well why do we have to do this?” And we’re kind of saying, “Well-” We had meetings with all the parties involved, us, them, the business, and the IT group so they could hear it straight from the horses’ mouth.


And so they understand, it’s a business need. It’s positioning themselves for the future, it’s a little bit of pain right now-




Matthew :But we have a solution, there’s always a way to fix a problem-


Danny:I had this visual of the business coming to you, and saying, “We need to do this.” And you’re like, “Aw, that’s not a problem. It’ll take a couple days to do that.” And then it’s almost like what I always joke around that part of my job is slowing everybody down around here.


I can just almost have this visual of somebody taking your argument and saying, “Yeah, but you can’t use this, and you can’t use this.”


Matthew :You’re absolutely right. It really is a standard, and straightforward SharePoint 2010 to 2013. If we had all the same technologies-




Matthew :In place … Which are still available. Yes they might be deprecated, but they’re still supported-




Matthew :If we’d of done that, if we had that ability, we would be done. It would be a easy upgrade. But what ends up happening is the changes we’re making now, the next step is going to be so much easier-


Danny:So they’re ready, they’re getting ready for-


Matthew :More cloud ready, yeah.


Danny:So even though they’re not even going to SharePoint 2016, or SharePoint Online, the next jump that they make it’ll be-


Matthew :Yeah.


Danny:Much easier. That will take a couple days-


Matthew :Yeah, and if we were doing this again, if they were coming to us brand new-




Matthew :And said, “We want this application” we would not do it the way we’ve done it.


So we’re trying to balance the technology that they have in place, the training that they’ve already done. The way the application works, what is the least amount of effort we can do, to get this thing done in the new environment.


Danny:Just a side note, since we’re doing a lot of these … I mean basically our applications are turning into little JavaScript applications. That probably makes them even … You know where they’re hosted in SharePoint.


As people maybe look at different platforms, or moving to different platforms … I know there’s the back end is different-


Matthew :Mm-hmm (affirmative)


Danny:But it seems like even get them into a situation where … If it was all server site code, then making that move is a big deal. But if it’s all JavaScript, then it might be easier to look at different ways of hosting these applications.


Matthew :We’ve done some amazing things with SharePoint-




Matthew :You know our applications are really enterprise applications that we do, and we really leverage SharePoint to it’s full extent. We utilize its capabilities of data management, as well as search, and document management, and all the policies they have in place.


But if a customer did have SQL Server, and they wanted to use SQL Server, because they had other enterprise applications that were using that. When we rewrite this application using JavaScript, and it’s Angular based forms, then we become more platform agnostic-




Matthew :And in many ways that’s what Microsoft is trying to do with SharePoint. Is trying to make SharePoint fit better with that model. It’s nice to able to store data in SharePoint, cause you automatically get search, and you get some of this other stuff.


But quite frankly you get that with Microsoft SQL Server as well. The search language is slightly different, the configuration is slightly different. With SharePoint you get all these management tools that kind of come with it, but with these other platforms you get similar things as well.


Danny:Mm-hmm (affirmative)


Matthew :So we’ve written some very complex applications, some very feature rich applications in SharePoint. This new technology, this new model is going to future proof us. To be able to deliver applications in the future, regardless of the underlying infrastructure.


Danny:What is the … I know I’ve heard people talk about the SharePoint frame work, and that sort of thing. How does that fit into this stuff? You’re probably not utilizing it yet, because it’s not a … How does that fit into this?


Matthew :My opinion as a developer-


Danny:Uh huh.


Matthew :SharePoint, or excuse me Microsoft used to dictate to us, “Here’s how things should be done.”




Matthew :But we always questioned, “Have they ever done this thing themselves?” Because if they did they would say there’s no way this would even work.


Danny:Oh that hurts.


Matthew :Yeah.


Danny:But it works in a demo-


Matthew :Yeah right.


Danny:I can really kick ass-


Matthew :So I think what they’re realizing is, wait a minute they’re trying to eat their own dog food, if you will. They’ve tried it themselves, their consultants have come in, and they’re listening to them saying, “Wait a minute. You know, we’re saying do it this way, but have you ever tried to do it that way? It’s hard to do.”


So they’re coming up with these frame works to help with that. The patterns, and practices. The SharePoint library, all these frameworks that the goal is to make it easy to incorporate user processes, user data inside the SharePoint platform.


Danny:Mm-hmm (affirmative)


Matthew :And so all this is great news for us.




Matthew :As developers.


Danny:Anything else you wanna … This has been really interesting, it’s kind of-


Matthew :Oh this project I’ve worked on in the past-


Danny:Uh huh.


Matthew :I’m a domain expert on this particular application-


Danny:Uh huh.


Matthew :I’m applying new technology that’s new to me-


Danny:Uh huh.


Matthew :In certain areas, which is great, it’s a great learning experience. I came off a more traditional enterprise application, that was


Danny:Uh huh.


Matthew :Microsoft SQL Server, etcetera. So it’s nice getting into the modern world if you will.




Matthew :It’s nice doing this, because I think it is going to be the future.


Danny:Yeah, and this whole … Again all this stuff will apply to other applications, and moving to the cloud, and talk … I just wonder in general what is … It maybe just people will start moving applications to the cloud when they need to rewrite them, or when they sunset old applications.


But it seems to me probably a lot of people are going through this, which is you know they’ve got an older style application. They want to move it over to the cloud, and they’re getting these … Well it’s going to take us three to six months to do this, and you’re just going to get the same thing that’s on the cloud, as what you have right now. How do you justify that?


Matthew :And that is a huge challenge, yeah, because you have a working application that looks a certain way, and you have to move it somewhere else, and it’s going to be this big expense.


One thing that’s at least nice about this transition that this particular customer’s doing, and what we’re doing here as a company at Three Wheel. Is we are getting our tool belt even bigger with more tools, and solutions.


We like to come up with patterns, and practices ourselves to help deliver solutions to the customers quickly. You know we’re not a think tank, we don’t like to spend two or three years working on a solution. The customers want action now.




Matthew :And we realize that, we use agile processes in our project management, and because we know, decisions get made, changes are absolute. Changes are going to happen, and we have to be able to adapt.




Matthew :So this whole thing that we’re going through with this particular customer is kind of like you said, it’s kind of like what’s happening with the industry.


But once you get over that initial hump-




Matthew :Unless they take away JavaScript for whatever reason, then we’re really in trouble. Go back to Cobalt. Once we make this step it will be smaller steps in the future.


Danny:Yeah, interesting. This has been fascinating, thank you for sharing what’s going on, and the project hasn’t … You’re just in the planning phase right now-


Matthew :We’re in the middle of it-




Matthew :We’ve gone through a planning stage-




Matthew :And we’ve gone through the initial development-




Matthew :We’re going gang busters right now, getting things done.


Danny:Awesome, well good luck. I look forward to getting a update in a couple of months here, and thank you for taking the time to do this.


Matthew :Good talking to you Danny.


Danny:Awesome, thank you everybody for listening, and have a wonderful day. Take care, bye.


Danny RyanMoving Sandboxed Solutions to the Cloud

1 comment

Join the conversation

Join the conversation

This site uses Akismet to reduce spam. Learn how your comment data is processed.