choice.png

Jive to SharePoint Migration Tool

Danny Ryan:                       Hi, this is Danny Ryan, and welcome to the ThreeWill Podcast. Today I’ve got Chris Edwards here with me today. He’s a Senior Software Engineer. Chris, thanks for joining me.

Chris Edwards:                  Sure. Glad to be here.

Danny Ryan:                       Great. I appreciate you taking time. I know you’re very busy right now, so I appreciate you taking the time to do this.

Chris Edwards:                  It’s all good.

Danny Ryan:                       I’m just another direction that you’re getting pulled into to do. I wanted to today … For today’s conversation I wanted to focus in on the Jive to SharePoint Migration Tool.

Chris Edwards:                  Sure. Okay.

Danny Ryan:                       Okay. Just give me maybe just a description. I know we’ve had this conversation with a lot of customers, and I just wanted to capture this all together at once. Just give me maybe a description of what the tool is at a high level.

Chris Edwards:                  Okay, sure.

The tool itself is a single command line or console based tool. It was actually done … We produced it originally for our own internal use. You know? For our own internal migration of Jive specific content to our own Office 365. We had a specific need. We had to move quickly. Move quickly means, “Let’s go ahead and get something that’s quick and dirty out and working.”

A console based tool was kind of a chosen technology. Very easy to create, right? That’s very easy to customize, very easy to kind of wire up other APIs. That sort of thing.

It’s kind of nice that it’s turned into something that’s pretty decent because a lot of times we get into customer organizations, customer that want to do very specific things. The tool itself does some pretty reusable stuff, but very easy to extend, right?

Danny Ryan:                       Mm-hmm (affirmative).

Chris Edwards:                  Not really focused on the UI and all the kind of the mud that you kind of get caught in with user expectations and all that stuff. This is specifically trying to get work done, and specifically migration work done.

That’s what it does and tries to do it well.

Danny Ryan:                       Awesome. Tell me a little bit about sort of the way the tool works as far as extracting out data. I know it sounds like we’ve done some things where we gather some information about a client’s Jive environment, and then we go and later on maybe pull the actual binaries and stuff like that. Tell me a little bit more about that whole process.

Chris Edwards:                  The nice thing about the tool is it actually you can run multiple flavors of the command. The command line itself you can easily pack different parameters in. It’s very traditional, nothing new there, but my thing about it is that we can use the tool … The way we structured our process around the tool is really to go after pulling the content first. The nice thing about it as well is you can pull different levels of that content.

Let’s say you just want to get a back-up of all your content in Jive. That’s really how it’s designed to work. What we can do is we can go after and get all of the people references, so basically all the users that are in the database. We can go after and gather those. Then typically we go after all of the …. I call them places. Jive is a concept of space, product, and group, so I’ve used the generic term place to represent all three of those.

What we do is we typically get the list of all the places … That is … Something we try to do is keep people and places up front as early on the customers as possible. Then they can see that this is the volume, this is how many … This is how much stuff is out there at a high level, right? Then we use that information to map it into where it’s going to end up residing, whether it’s Office 365 or on premises SharePoint. It may be in different places. We can map those places. It’s mainly a manual exercise, right?

The tool itself is … you know, with a console based tool, we’re not trying to put everything in it. We’re weaning back on using Excel and leaving manual process and certain aspects to do the mapping in that exercise.

Danny Ryan:                       It gives you like a listing of all these different places then, and then you can map them to where they go into in SharePoint?

Chris Edwards:                  Exactly. You can choose this particular place from Jive. Might not go into SharePoint at all, maybe it just gets archived. that’s what it supports.

Typically we get people places, we use that information for analysis and working with the customer to map. Then we do another pull … I call this an inventory pull, where we pull people places again and getting the updates, any deltas that have occurred maybe since the last time you ran it. Then it actually can go through and pull all the content from those places out of Jive into SQL Server and then to a file system.

Then anything … You can actually run another even layer on top of that if you want to pull all the social aspects. If you want to pull commenting that was done on a document or a discussion, you can go after that type of detail. If people have liked things, you can go after that type of detail. It’s kind of layers of pull and inventory. How it works.

You can stop me if you want to. The beautiful thing is that we had to get everything out, we wanted to get an inventory that we could work with, right? When we originally designed the tool. Therefore, we can go back and … Let’s say we shut the drive off completely. Well, we can take that inventory that we already have and start pushing it into SharePoint.

It allows us to kind of get off of Jive, get things organized, and then when we’re ready start pushing things into the target environment, whether it’s Office 365 or SharePoint on premises or whatever. It could be anything, right?

Danny Ryan:                       That inventory is stored in a SQL Server database and the file system? Is that the two places primarily?

Chris Edwards:                  Yes. SQL Server database for all the metadata and the structuring. If you think about the concept of Jive, you get places or groups within other groups. You can do that kind of thing. It maintains the integrity and the relationships between those things. A comment may have comments on other comments, right? We have to be able to maintain the integrity of that relationship. SQL Server just makes it very easy from a relational perspective, just to store that information and query it later.

It’s really being able to get that information in something that … In this case, something that I was comfortable with. I was comfortable with SQL Server, so queries against to manipulate it. It was a logical choice to use that for this metadata.

Now, physical files like images and Word documents and things like that or physical content? That we just put in the file system, and then we structure it.

Danny Ryan:                       Okay.

Chris Edwards:                  It’s structured in a way that … A database can reference and point to that file structure, but it’s actually organized in a way that even if you looked at it on it’s own, you could still kind of figure out where  things came from.

Danny Ryan:                       What types of Jive content have we moved? Are there certain types that we haven’t moved yet, or what have we done so far?

Chris Edwards:                  Well, I mean, we’re focusing on the big stuff. You know, like Jive collaborative documents. That typically gets converted. Something that can be rendered as a HTML document. We’ve also pulled binary, like I said Word document images. Just anything that can be uploaded to Jive and commented on and socialized. Yeah, we’ve pulled that into a file system.

We’ve also done discussions, and there’s some …. Actually doing some work right now trying to get it to … A Jive discussion convert to a SharePoint discussion. Then we’ve actually converted those to Wiki as well, and that’s been pretty successful with that.

Those are the main, big things we’ve pulled. We have pulled private messages and some basic profile properties that are Jive. There’s some other things we’re actually looking at doing now, too. More of the tasks. I’ve done tasks mainly. A lot of people don’t necessarily want to retain that information from Jive in the move forward, but we’re actually looking to add that to the tool, where you can actually pull tasks information from Jive projects.

As well as video content. Video content’s a special content in Jive that we need to spend a little bit more time on. It’s actually something that we’re going to be focusing on here pretty soon, but we’re not currently moving that. Unless we can fit it as a binary file, that’s different, but actual Jive video, that’s something we’re focusing on for the next generation of this tool.

Danny Ryan:                       Great, great. Yeah. What else can you tell me about anything else that … Sounds like they’re basic components are this command line tool that we run, we’ve got some Excel sort of mapping things that are involved. It’s a SQL Server database. It’s a file system. Any other major components of what this tool is or does?

Chris Edwards:                  Well, certainly. It does … It has another additional command line options that if you wanted to do custom navigation updates in SharePoint. Once it’s done the … Actually, you know what? Let me try to take a step back.

One thing it does do is it if it tries to migrate content into Office 365 or to a SharePoint site of any form, you can actually add a flag to say, “Okay, if this particular site doesn’t exist or [inaudible 00:09:42] first-

Danny Ryan:                       Oh, that’s nice.

Chris Edwards:                  -and then layer in the content.

Danny Ryan:                       That’s nice.

Chris Edwards:                  It kind of has the ability to do that. One of the enhancements we’re looking at possibly doing is allowing you to choose the template for each one of those maps places from Jive. Maybe it’s associated to a default template on SharePoint, or maybe it’s associated to a custom template. That’s something that we’re looking at doing. It can provision basically a default template now in SharePoint, as well as it lays in all the libraries or it creates … It can create new libraries, document libraries in SharePoint or it can actually layer into the existing libraries if the SharePoint file exists.

It can also update navigation. If you wanted to set up custom navigation to these new libraries, it says so up on the quick launch. Real basic stuff there. Then … Some of the things we’re actually looking at doing, too, and I think it would be pretty cool, is something I’d love to see in the tool itself, is the ability to actually get the overview pages out of Jive, in some form or fashion in SharePoint. That’s something that I’ve heard of people ask about.

I think it’s actually an important thing. A lot of people spend time organizing and really tailoring their Jive to look a certain way. If we can somehow get that into SharePoint,  i think that’d be pretty awesome. I’ve got some ideas how to do that, but it’s in development now.

Danny Ryan:                       Awesome. Tell me some about profiles and security. How does that work into this, because I know Jive profiles are important? Then how does the security map … Do we have some way of mapping over security that’s setup in Jive?

Chris Edwards:                  One of the first things we do in Jive is we … Like I said at the very beginning, we actually go get all the people. We get a list of all the people.

Danny Ryan:                       Mm-hmm (affirmative).

Chris Edwards:                  That includes email address, anything that’s assigned to their user account up at the highest level. The basic information like username, email address, just basic … Maybe their avatar image. We go after that information. If we need to dig deeper, that’s another customization.

Danny Ryan:                       Mm-hmm (affirmative).

Chris Edwards:                  We get that basic information up at the forefront.

One of the things we do is we take that information and say, “Okay, let’s look. Let’s see if we can look up by email address or by some other attribute that we pulled from Jive, let’s see if we can look up on the corresponding SharePoint the targeting SharePoint system to make sure we can identify and map that user.” Email address might be one thing we look up by. It’s more like a AD active directory look up to find the user.

If we can find the user, then we’ve got a match and then we can basically associate any content that we’re bringing over to that perspective user.

Danny Ryan:                       Nice.

Chris Edwards:                  One of the things we do when we bring over the content is that if we do have to adjust how user references and things like that are in the Jive content … Maybe there’s a Jive collaborative document that’s got some embedded user links in it. Well, we actually convert those into the appropriate form in SharePoint. Maybe a Jive user in Jive, but then we bring it over to SharePoint, and they get the Yammer profile link. Or maybe it’s a just a standard SharePoint profile link. Or maybe it’s a direct link to their active directory profile.

We can … We have flexibility in how we convert that over. It’s all about finding by email address or finding by one specific identifier from Jive into the target system.

Danny Ryan:                       Let me see if I’ve got this correct. If you have a file like is a metadata about a file in Jive that can be moved over like the last person who modified it or person who created it, that can be mapped and moved over into SharePoint? Is that correct?

Chris Edwards:                  At a high level. You can definitely move the last modified. Created, we typically do like a separate property for that because-

Danny Ryan:                       Okay.

Chris Edwards:                  -SharePoint’s a little bit tricky in terms of setting the actual author of content. It wants the actual user to be doing that.

Danny Ryan:                       Okay.

Chris Edwards:                  You can impersonate. You can do some tricks that kind of make that work, but we do the best we can with making sure we maintain that information.

Danny Ryan:                       Would it … How is the where you talked a little bit about how we run the tool at different times … Where is … Is the tool run from the client’s side, or where is it typical … Where do people run the tool from?

Chris Edwards:                  It can pretty much … The main thing about having a console is that it can pretty much run anywhere.

Danny Ryan:                       Okay.

Chris Edwards:                  In the fact that we’re dependent on the rest base libraries … libraries for SharePoint and then the rest, the Jive rest base API, we can pretty much run it anywhere.

A lot of times it’s run on a certain machine, in a client environment, that has substantial amount of disk space. We’re running, you know, maybe pulling a ton of content over out of Jive. We want to have significant space.

Danny Ryan:                       Yeah.

Chris Edwards:                  Typically we target a VM that has a substantial amount of disk space and decent CPU memory, so they can run this process. It can run anywhere.

Danny Ryan:                       It sounds like you said it can run once and later on it can run a delta based off the changes that have been made? Is that correct? Did I get that right?

Chris Edwards:                  Yeah, it basically … It checks to see if it’s pulled content previously. If it has, then you can override, force it to be updated, or you can set points to tell it to move on. We have options for all those.

Try to be smart about, “Let’s have different flavors, different options.” So if we did want to pull just the stuff that has not been pulled before, we can do that. Or we can go ahead and say, “Give me everything again.” The flavors of the command line that you can do to be this combination.

Danny Ryan:                       Mm-hmm (affirmative).

Chris Edwards:                  The nice thing that works together, you can focus on just a specific Jive place, right? And say, here’s a series of command lines back to back that you can use to go and pull content, massage that content into some form that’s acceptable by SharePoint, and then another command that pushes it into SharePoint. You can do different flavors of those three or four different commands that you have. Then build a script to do all that.

Let’s say you want to do that for maybe a top ten list of Jive places, it could be very easy to build a script that does those four commands per place. Right? If you wanted … Say you needed to restart for some reason, you got an error, it’d be very easy to pick the specific place and to start from that point on.

Danny Ryan:                       Gotcha.

Chris Edwards:                  It’s just keeping being smart about being … That flexibility and applying that to the process.

Danny Ryan:                       I hear you talking command line, and I don’t think PowerShell because that’s much too new for me. I think back to bat batch files. I can create a bat file …

Chris Edwards:                  Yeah, you can use batch files or you can use PowerShell. Oh.

Danny Ryan:                       All right, so-

Chris Edwards:                  I like batch files, too.

Danny Ryan:                       I do. It’s just something about them. I don’t know what it is. Maybe it’s just fond memories of them.

Let’s talk a little bit about … I know for folks who are listening, the two day workshop that we do. Just for people who might not be aware of this. We have a two day Jive migration workshop where we got through a lot of … We teach people about sort of this whole process. We do that initial inventory. We help them with sort of map, beginning steps towards mapping out the data, and then giving them an estimate and a project plan out of those two days. We end up doing it for a fixed price. Right now the fixed is $7500. That could change.

Tell me a little bit … I know you’ve been involved in those, and tell me a little bit more about that two day workshop.

Chris Edwards:                  The original workshop was like a one day, all day deal. We’ve actually just recently chunked it into two separate sessions, basically three hours at a time. Because we can actually go through and … We try to establish initial goals of the workshop itself. Let’s learn a little about your environment, or learn as much about your environment as we possibly can. What content types are important to you in Jive? Get a flavor of how much data we’re talking about moving. Are there any customizations?

It’s almost an interview process with the customer to say, “Show us what you’ve got in Jive-

Danny Ryan:                       Mm-hmm (affirmative).

Chris Edwards:                  -and we’ll give you an idea of how the tool works.” We give an overview in session one. The first three hours are how the tool works, and show some resulting content and some sample migration details. Then … originally we had it all in one day. Well, that doesn’t enable us to kind of go gather and get more information from the customers. We broke it up in two sessions. Now we can actually provide the tool to the customer. Let them run it in their environment and get that first set of things that we talked about, I mentioned earlier, people and places.

Danny Ryan:                       Mm-hmm (affirmative).

Chris Edwards:                  By allowing the customer to run the tool, one, we prove that we can actually run it in their environment and we get past that hurdle. Make sure there’s no issues and everything’s happy there. Then we also get an initial set of data that we can query load and present in a more formatted spreadsheet for session two. Right? Jumping into session two. Say, “Okay, let’s talk through what did we find? What seems to be jumping out as … We talked about this in session one. What are things that jump out in this data now that we need to kind of dig into? What are all the content types that maybe we missed in session one? Things like video, things like tasks. Do we need to worry about that?”

We go through the workshop and we have a list of content types that we found are pretty important. We prioritize them.

Danny Ryan:                       Mm-hmm (affirmative).

Chris Edwards:                  Which ones are the most important. With ones, where does this content type in Jive, maybe it’s a message, maybe it’s a Jive document, where does it ultimately end up in SharePoint? Where do you ultimately want to see it? Maybe you don’t want to move it at all. We kind of flag each piece of content and indicate which ones the tool already supports and which ones the tool requires some customization.

These are usually a work in progress. It usually indicates where we need to put our spikes in our backlog. What do we need to actually focus on the tool to do some more specific things?

That’s what having the tool with a command line tool is nice because it’s very easy to make changes to it to adapt to the customer’s situation. We identify those deltas, identify the customizations, and we go through and … Part of the worksheet is to try and set maybe what the overall base structure for the milestones and things look like. You know?

When do we plan on … When do we have to start migration? How long do we think it’s going to take? How many weeks are we going to need to do this? Or how many days? It just depends on how large things are. We set up the risks. We talk through risks. Just generally get all the information needed at the end of session two to put together our product backlog. Then that’s where we go off the ThreeWill perspective, and we produce the backlog based on all the stuff we learnt from session one, session two, and then the data we’ve gotten from running the tool, and we produce a backlog and just a high level plan for how we would proceed with migration.

Danny Ryan:                      That comes with an estimate as well, and … Yup. They can then make a decision about … It’s typically for … From what I’ve seen so far, perhaps you’ve seen the same thing? These are pretty much driven off of, they have a deadline that is what they’re Jive is going to get turned off. Or they have to make it by some certain date, and you try to sort of work backwards and make sure that we’ve got appropriate schedule that we can do everything we need to do.

I know it’s just … It’s nice in general just having the tool to go ahead and … Earlier on in the project just go ahead and get as much data as you can. At least into SQL Server and file storage just so you have it there, and it helps everybody sleep at night that when they do turn it off, at least you’ve got an archive.

Chris Edwards:                  That’s actually … You can actually look at this tool or just the process itself as a … You can look at it as we’re trying to get a migration done, but maybe you’re not really trying to get a migration done. Maybe you’re just trying to do data salvage, right? You don’t really intend to move this content anywhere, other than get it out of Jive and get it into a form that you could work with. We support that as well.

Danny Ryan:                       Yeah.

Chris Edwards:                  It doesn’t necessarily have to go from Jive into some other system and be consumed there. It could actually be pulled, organized, and then we actually sit down with the customer and show them how to access that content that’s been pulled. They can choose to stop there if they wanted to.

Danny Ryan:                       Yeah.

Chris Edwards:                  It’s designed so that we could sort of flex in that different way.

Danny Ryan:                       That’s awesome. Well, I appreciate all the hard work you put into this little, what has grown up to be a big boy tool. I know it sort of just started out out of necessity, but some of the best products in the world are started out because you need to scratch your own back. It just happens.

Thank you, Chris, for taking the time to do this.

For folks who are interested in more and maybe learning more about the tool, or even better more about the workshop that we do, definitely reach out to us. The obvious way is just through the “contact us” page on the ThreeWill.com website. Let’s just have a, maybe an initial meeting where we talk about what you get in that workshop because it’s great. It gives you all the data you need to make a decision whether to move or not to move, and all done at a fixed price, which is kind of nice.

Thank you, Chris, for taking the time to do this. I’ll let you get back to the real world, the real work that you do. Thanks, everybody for listening to the podcast. Have a wonderful day. Bye bye.

Chris EdwardsJive to SharePoint Migration Tool

2 comments

Join the conversation

Join the conversation

This site uses Akismet to reduce spam. Learn how your comment data is processed.