"Platform Migration is a team sport."
In this episode, we talk through migrating to Shopify: things to watch out for, and experienced advice on how to make the most of migrating to Shopify.
Our expert guest contributors are:
The Unofficial Shopify Podcast
3/8/2022
Paul Reda: I have a confession to make.
Kurt Elster: Yes?
Paul Reda: My website, it’s not on Shopify.
Kurt Elster: Oh, no!
Paul Reda: Would it be hard to get it on to Shopify?
Kurt Elster: If you were to say… migrate that website?
Paul Reda: If I wanted to migrate it? I mean, I’ve been running a Magento store for 10 years. I don’t know if you know about this.
Kurt Elster: This is not true. You’re full of lies.
Paul Reda: My Magento store does very well. This is like a side gig for me. I just come in because I need someone to talk to.
Kurt Elster: This… What?
Paul Reda: My Magento store, it’s very successful. I don’t need any of this bullshit in here. But if I wanted to migrate my very successful Magento store to Shopify, do you know anyone I could talk to about that?
Kurt Elster: Ooh! I know several people. Wait a second, I think this is a setup. I don’t think you even have a Magento store. I think this is all lies. I do know a few folks, not just us. I know some Shopifolk who can guide us on that. I know some apps, some partners, some Shopify partners, some team members. I know a cast of characters who would be happy to jump on the opportunity to migrate your fictitious Magento store to Shopify, Paul Reda.
Paul Reda: All right. That’s too many people. Do I need an entire cast of characters? I think I just need like one guy who knows how to do spreadsheets. That’s all it is, right, is just doing spreadsheets?
Kurt Elster: Yes, actually. Yes. That really… A data migration between platforms, regardless of what those platforms are, you’re going to be a spreadsheet jockey. We’re talking about my spreadsheets, my favorite topic. Oh, I have goosebumps thinking about spreadsheets. This is gonna be a good episode.
This is The Unofficial Shopify Podcast. I’m your host, Kurt Elster, A.K.A.:
Ezra Firestone Soundboard Clip: Tech Nasty!
Kurt Elster: I’m joined by show producer, Paul Reda, Producer Paul.
Paul Reda: Ew!
Sound Board: Ew!
Kurt Elster: And today we are going to learn how to migrate to Shopify. So, if you’re already on Shopify, you could probably quit listening. No!
Paul Reda: Just to be clear, when we talk about spreadsheets we’re not talking about like .XLS files, or .XLS files like Excel files, or whatever the Libreoffice one is, or whatever the other ones. We’re talking pure, raw, uncut .CSVs.
Kurt Elster: Comma-separated value. CSV.
Paul Reda: The real. The hard stuff.
Kurt Elster: Yeah. I’m a big dork so I like to do everything I can in plain text. For me, any formatting applied by an application, I’m like, “Please. You are just an additional layer between me and the computer. Just bring me closer to that cyberspace, all right? I want to get in the Metaverse with Zuckerberg.” And plain text lets us do that. A .CSV, comma-separated value file, is a plain text spreadsheet. Ooh, very exciting.
And so, when we talk about data portability, when we import or export stuff in Shopify, it’s all .CSV.
Paul Reda: I mean, that’s why your nickname was Zero Cool, right? When you were a ‘90s hacker?
Kurt Elster: In Hackers? Yes. Yeah. I went by Zero Cool. The movie Hackers was actually inspired by my real-life events, even though I think I was like 12 when that movie came out.
So, we’ve done store migrations in the past, and the reality is it’s always you’re gonna move data from a store that isn’t Shopify to a store that is Shopify.
Paul Reda: Because if you’re already on Shopify, the stuff’s already there.
Kurt Elster: Well, we could address, hey, if for whatever reason you had to migrate or clone a store in Shopify, there are several apps that will do it for you. I know Rewind makes one. I think it’s called Replay. I’d have to look it up. Or you export everything as spreadsheet with a tool like Matrixify, formerly Excelify. And spoiler alert, Matrixify becomes the star of this episode.
Paul Reda: What if I have a store that needs to be English and French because of the damn Canadians?
Kurt Elster: All right, so that is the one really hard part, and I learned talking to someone who has a Canadian store that there are French language cops who will track you down if you do not have your stuff in French or have not at least made a tremendous and obvious effort to getting it into French. Then it gets tougher, and so actually I talked to a Shopify launch engineer and asked them their opinion, asked some other people their opinion, and the answer really was fairly consistent, was like you probably want to use this app called Langify, and what’s really cool about it, up front getting the other languages into Shopify, and there’s multiple ways to do that, there’s several ways to do it. But then how do you maintain it going forward?
And the really cool thing that people love about Langify I have learned here is it gives you a point-and-click interface once it’s done. So, you could load up your own website in this Langify editor and be like, “Oh, this is mistranslated.” Click, edit, update, done. This sounds very good. What we did learn, though, is moving multiple languages across from a platform that may run multilingual store, that’s your most complex scenario.
So, I’m gonna say right now-
Paul Reda: More complex than like a giant crazy product catalog?
Kurt Elster: Oh, 100%.
Paul Reda: Oh, really? Okay.
Kurt Elster: Yeah. What’s funny about the giant crazy product catalog-
Paul Reda: Yeah. Because, I mean, I have like a thousand SKUs in my Magento store, so I was worried.
Kurt Elster: That’s not even that crazy.
Paul Reda: I meant 10,000.
Kurt Elster: 10,000. All right, now we’re getting somewhere. The one we’re working on now is 200,000 SKUs.
Paul Reda: Oh my God.
Kurt Elster: Yeah. The biggest one we did, the biggest catalog we’ve messed with is about half a million SKUs.
Paul Reda: That’s too many products.
Kurt Elster: The problem you get in there is your spreadsheet software starts to disagree with you about the sanity of opening this file. And so, that’s where in those scenarios, like if you’ve got that much data to deal with, you’re better off going even nerdier than where I’m at, and you need like a super nerd who can put that stuff in a SQL database and then do scripting on it. You can see my level of confidence on SQL databases really dropped off a cliff there.
Paul Reda: So, what you’re saying, instead of Zero Cool, we gotta find Acid Burn?
Kurt Elster: You have like IMDB open for the movie Hackers? Just going through the cast? Had I known, I would have downloaded some Hackers sound effects.
All right, so when we’re talking about what we can import into Shopify, it’s like what are we discussing here?
Paul Reda: The whole shebang. My store needs to be on Shopify and every piece of data I have is absolutely necessary to the working of my store, Kurt.
Kurt Elster: All right, so the stuff we could get into Shopify… We can get the catalog in, so that’s really… It’s like products with images, our collections, which pretty much every platform is gonna call those categories, our pages, our blog posts, which I’ve discovered blog posts could be a pain because they often have images in them and now how are… The chances are those image files are gonna break, so that one’s a tough one.
Paul Reda: Yeah, because the URL is like probably hard coded in the blog post as pointing to something that’s hosted on the old store.
Kurt Elster: Yeah.
Paul Reda: Yeah.
Kurt Elster: That’s where often an automation isn’t gonna work and you’re gonna need your spreadsheet jockey or your database nerd to do some find replace.
Paul Reda: Those are two different species of people that will-
Kurt Elster: Yeah. I’m spreadsheet jockey.
Paul Reda: Oh, okay.
Kurt Elster: Yeah. And then we’ll be joined later by our team member, Karl Meisterheim.
Paul Reda: Who’s a database nerd?
Kurt Elster: Yeah. By the way, his name really is Meisterheim.
Paul Reda: It’s like a normal name.
Kurt Elster: Yeah. Karl Meisterheim. It’s fun to say, though.
Paul Reda: It’s not like a weird name that you gotta call it out like that.
Kurt Elster: Well, now I feel like a jerk.
Paul Reda: Yeah.
Kurt Elster: All right, you could move that stuff across, and then also customs, orders, and then even other platforms will do a metafield equivalent thing called custom fields. You can now migrate that over into metafields, as well. And then, aha, all this stuff is gonna have different URL structures. Well, it turns out doing… that’s like the least scary thing to fix using URL redirects. So, I can get the data into Shopify as long as I can format it in a sane fashion. Shopify is really good with being able to import spreadsheets.
Paul Reda: You know, you’re okay smart, but-
Kurt Elster: He’s like moderately intelligent.
Paul Reda: Yeah. You’re like-
Kurt Elster: I could string a sentence together.
Paul Reda: You’re fine. You’re like replacement level smart. Is there someone smarter than you I could talk to about this.
Kurt Elster: Yeah. Absolutely. 100%. This episode, we’re gonna need some backup. And so, I talked to Stewart Knapman. He is a Merchant Success Manager at Shopify and formerly a longtime Shopify partner at a successful New Zealand agency, Lucid. And he gave us a lot of advice and I have sound clips for this episode.
Paul Reda: Yeah. I mean, that’s what I was setting up. I know that.
Kurt Elster: So, knowing that in most scenarios we can import most everything, we need to figure out what we actually want to import. Because earlier, you said like, “Well, I want 100% of my store.”
Paul Reda: Yeah.
Kurt Elster: It turns out you may… Maybe, maybe not. Let’s hear what Mr. Knapman has to say here.
Stewart Knapman: Building yourself a checklist, and going through everything, and just doing a bit of a mental order to say, “Okay, I know where everything is,” and using the analogy of moving a house, right? You kind of need to take stock of what you’ve got, and it gives you a really, really good opportunity at that point to look at certain things and go, “Well, I know I need to do this. I want to keep this. But I don’t actually need to keep that. And I can get rid of that. Or I’m gonna need this new thing and I can replace it.”
Kurt Elster: So, his advice for step one really was… He said, “All right, make a checklist,” and the analogy to use here is, which I loved, was this is like moving. Anyone who’s moved realizes-
Paul Reda: Oh yeah. You throw all your shit out.
Kurt Elster: Yeah.
Paul Reda: You’re like, “Obviously, I’m gonna move all that shit,” but then when you actually have to think about it, you just want to throw half that shit out.
Kurt Elster: Yeah. As soon as you’re moving, quickly your threshold for what does and doesn’t spark joy really changes, and suddenly you’re donating a lot more stuff than you otherwise would have. And so, the same is true here, where it’s like, “All right, make a list of this is the stuff we want to migrate over.” But I was talking about how… Oh, one of the weird call outs or catch outs may be in my blog I’ve embedded these images in a weird way and so when I bring them over, either they’re going to be broken, or it’s gonna be like an extra 20 hours of hassle to get those images to work, or even if I get them over, and I’ve run into all these scenarios. Even if I get them over, because they were published seven years ago, it’s got really tiny, postage stamp images.
Paul Reda: Oh, it was like 320 x 320?
Kurt Elster: Yeah. It just looks weird in my new layout.
Paul Reda: Yeah.
Kurt Elster: So, maybe you’re like, “All right, hold on. Is this worth it?” And then you check Google Analytics and go, “Oh, it’s not.” The less stuff you have to migrate over, the better. It makes your life easier. So, if you have scenarios like that, where you’re like, “All right, I’m willing to let this go or this isn’t worth the hassle to move,” okay, great. That’s gonna make the entire process easier, faster, cheaper. You’ll lose less hair pulling it out in the process.
So, all right, you make your list, and you know you could import it into Shopify. All right, so we’re good there. The other problem is I gotta get the data out of the old platform. And I think at this point, pretty much every platform is gonna give you some affordance for just export a .CSV of what I want either native in it or in-
Paul Reda: There’ll be like an app or something that’ll help you get out.
Kurt Elster: Yeah. Like I remember years ago, Prestashop, happy to let you import stuff via .CSV, but if you wanted to export your stuff, that was like you had to go pay 50 bucks for an app. I’m sure it’s changed now. I’m dumping on a platform I haven’t used in years. But in Shopify, honestly, we’re spoiled by the ease of use with .CSVs. Because just about everything in Shopify has import exports in the corner, and you could say like, “I want to import everything,” or filter, or filter and export only this stuff. That I have discovered is a fairly unique thing to Shopify.
In Magento, it’s, “All right, you gotta go into settings, and then pick your catalog,” and it just gives you the whole thing. So, again, here’s where I might need… My worst-case scenario is I may need a database guru who can go and query the database directly and export everything. But ideally, if I could just get it out as a .CSV, life gets easier.
But there’s another call out here. Some of your data may not necessarily be in the platform. Like customers, orders, products, it’s all in that platform. What about subscriptions? What about… That’s just a nightmare in itself. What about reviews? Reviews are rarely in the platform. It’s usually in a third-party app.
Paul Reda: Well, those would be in an app. Those would be in like Stamped, or Yotpo, or something like that, right?
Kurt Elster: Potentially. So, we’ve got… This came up with Stewart. I got another clip. Here we go. Let’s see. What are we on here? 6-2.
Stewart Knapman: Doing some research and sort of knowing, okay, well, I need reviews. How am I gonna get reviews? What options are there for me? And so, looking at the different apps that will allow you to do that, whether that’s Yotpo, or Okendo, or whatever that is, if you can align yourself with one of those review apps, they’re gonna have options for you to try and make your life easier, as well.
Migration is a team sport. You’re not necessarily gonna be doing this just by yourself. You might be in a position where you’re gonna do this by yourself, but I think if we’re thinking about you’re an established business, you’re a bigger business, you probably already have a team. There are lots of other people you can draw on, as well.
Kurt Elster: So, the line I love from Stewart is migration is a team sport. And his advice was in a data migration, especially for like a larger, more established business, that just by virtue of being older and bigger has more data and pieces, feel free to just lean on whoever you need to lean on. And his advice was in the case of reviews, it’s probably in an app. And you’re probably going to Shopify, you’re going to be using an app, so hit up their support and tell them. They want the customer, and you are absolutely not the first person to come to these apps and say, “Hey, I’ve got this data that I’m trying to get onto your platform so that I could please use your platform as a new customer.”
Or maybe sometimes in a lot of these apps they’re already cross-platform, so you can go to them and go, “Hey, we’re migrating. We’re keeping you as long as we can make this work. And we’re going from X platform to Shopify. Help.” And so, his advice was don’t be afraid to ask for help here when the data is not in platform. And that’s been my experience with reviews apps, is you’re not the first person to ask, so just ask them. How did other people do this? And they’re happy to tell you and happy to help.
Paul Reda: So, we know what we’re taking out. We know what we’re keeping. We know how to get it out. Now we got a giant, messy pile of .CSVs that a human can’t even read. Now what do I do?
Kurt Elster: Once… All right, so yeah, you’re right. We’ve indexed it. We now have just a big folder of .CSVs potentially. It’s like, “This is-“
Paul Reda: Hopefully.
Kurt Elster: Like, “All right, here’s all our data. It’s several megabytes of just text.”
Paul Reda: Yeah. I got 10 .CSVs. One of them has 100,000 products in it. It barfs if I try to open it on my laptop. Now what do I do?
Kurt Elster: All right, so at its core, if you wanted to, you could, and you know, on your fictitious Magento store that I don’t believe in?
Paul Reda: Not fictitious.
Kurt Elster: Very, very not real.
Paul Reda: Very successful.
Kurt Elster: You’re only here because… like a charity case? You’re just here to be my friend?
Paul Reda: You know, it’s like I just… You need something to do. I can’t spend every day on my yacht.
Kurt Elster: I know. You can only buy so many Lambos.
Paul Reda: Well, you know, and it’s like I come in here and you’re just like, “I gotta pay my mortgage,” and I’m like, “Wow, what’s that like?” It’s just like going to the zoo, kind of. I get to see what the plebs look like.
Kurt Elster: All right, so we have a folder with a bunch of .CSVs in it, and we want to get those into Shopify, and you could… There’s a number of tools here. If the data’s not terribly complex, you could just remap it in your spreadsheet it and just import it in the format that Shopify wants. That’s like your easiest case scenario and probably not gonna happen. Like I wouldn’t even bother with attempting this unless it was like, “Well, I got 30 products and I don’t feel like copying and pasting.”
The other tools are Shopify itself has a couple tools. There’s store importer, which works pretty well but doesn’t support Magento, so in this use case, didn’t help us. And they have another tool specific to Magento, but it has some limitations. It does not grab… I forget which specifically, but it couldn’t do every possible entity and use case that we wanted. And so, I asked Stewart about this, because I don’t want to spin my wheels and have to try every single thing, and plus when you’re dealing with these huge sums of data, it could take hours to days. I could imagine a universe in which it takes a week to move all the data over, so I don’t want to be like, “Well, we spent a week on one tool, and it didn’t work, and then we tried and spent another…” It’s just not worthwhile. And so, let’s ask someone who’s done it before what to do.
Stewart Knapman: I would definitely recommend, if you can, hire an agency that has done this before. They’re going to know all of the steps. They’re gonna know all of the pitfalls. They’re gonna be able to look at your specific case and say, “This is what we need to do when we get to this point.” Now, they’re not necessarily gonna be able to do everything for you. There will be homework. They will guide you and they will help you to do that, so my recommendation is they will use a bunch of tools. There’s a Shopify import tool, which is an app on the Shopify app store, but there’s also a couple of other third-party ones. Matrixify for many years is… When I get to a point and I’m like, “Oh, I need to go and double check the Matrixify documentation on this piece,” it’s super, super helpful. And even reaching out to their team when we’ve been stuck in the past, and their support is amazing.
Kurt Elster: So, when someone who has worked both as a Shopify partner and as a Shopify Merchant Success Manager tells you X app for this scenario is their secret weapon cheat code, you just skip to using that one, right? He said Matrixify works, it does the job, and it has great support and great documentation. Ooh, I love those two things. I love all of that.
Paul Reda: You love reading documentation.
Kurt Elster: Oh! I do. Give me some good documents to read. And truly, that’s one of the things that attracted us to the Shopify platform 10 years ago.
Paul Reda: That’s true.
Kurt Elster: We were like, “Man, this documentation is good and up to date.”
Paul Reda: Yeah. Prestashop 10 years ago, ooooh buddy.
Kurt Elster: Yeah.
Paul Reda: Not good.
Kurt Elster: And it was written in French. Poor Prestashop. We’re really dumping on Prestashop this episode.
Paul Reda: Prestashop was awful 10 years ago. That’s it. That’s the only thing we have to say is 10 years ago.
Kurt Elster: We had one client on it, and they are now on Shopify Plus and thriving. So, really I took Stewart’s to heart, which was like, “Just skip straight to using Matrixify.” And so, on this most recent data migration job where we go from Magento to Shopify Plus, that’s what we did. We’re gonna start with Matrixify and the advantage to it is it just works. I mean, it literally, because what Matrixify is meant for, its primary use case is I want to be able to manipulate any and all of the data in my Shopify store with a sane spreadsheet. And it works phenomenally well at that. It has gotten me out of many jams that I would have otherwise had to just do a lot of copying and pasting for many hours.
But it’ll also do data imports from other platforms, and it does it in an opinionated way, where there’s not a lot of settings, just good documentation, and they’re like, “All right, here’s what you do. Just follow these directions and then it just works.” Capital I-J-W. It just works. But I’m not the guy doing the work here.
Paul Reda: I was gonna say, I don’t feel like you can hack the planet all by yourself.
Kurt Elster: Yes. I had to have support from a team member, so I talked to our colleague, our friend, our delightful, delightful buddy, Karl Meisterheim, about the reality of the migration, and the hoops to fix it, and what you gotta do. So, let’s get him in here and have him talk us through his experience using Matrixify in this fashion.
Kurt Elster: We heard moments ago from Stewart Knapman, who described Matrixify as his secret weapon or his cheat code. He’s like, “This is the real deal tool that makes life easy on doing these data migrations into, on getting the data into Shopify.” And you are using Matrixify in a real-life project going from Magento to Shopify, right?
Karl Meisterheim: That’s correct.
Kurt Elster: And have you used Matrixify before or is this the first time? What’s the deal here?
Karl Meisterheim: This is the first time. I’ve heard of Matrixify. People talk about it in hushed tones of reverence, but this is the first opportunity I have had to work with it, so I’m pretty excited to try it out.
Kurt Elster: And all right, initial impression. Is it this secret weapon cheat code, at least on data import?
Karl Meisterheim: Yes. My initial impression is it is very much that it takes these horrendous .CSV files that we throw at it and somehow makes sanity out of it, and it gives you just enough options to do the things you need to do in most cases.
Kurt Elster: I interpret just enough options and it being able to transliterate spreadsheet insanity into usable Shopify data that’s translating from developer, it’s easy to use?
Karl Meisterheim: Yes, it’s easy to use. The thing I like about it is there’s a ton of things that you could potentially want to do with the data you’re bringing in depending on where you got it from, and what the format of the data looks like, and there’s no way that Matrixify could anticipate all of those things or even provide an interface that would make sense to do the various one-off things that different stores are going to need to do. But because it’s all .CSV files, they take the data that you send it, give you the handful of the most common options that you would need to implement, but then they’ll give you the file back that they’ve sanitized and you can make some more tweaks yourself, do all the custom things you need to do, and then it just works.
Kurt Elster: The phrase, “It just works,” is what I like. So, all right, in this particular instance we want to migrate products, collections, customers, orders, and create redirects where necessary, and get all of that into Shopify from Magento .CSVs, and so we export our Magento .CSV. No nonsense there. It’s just like system settings, export. A little frustrated that Magento exporting .CSVs, not as robust as in Shopify, like your only option is you’re getting the whole .CSV, and then you’re getting everything, and then you manipulate it.
Did you have to do anything to clean up this data? This was advice I got when I asked about this on Twitter, is like, “Make sure that data from Magento is good.”
Karl Meisterheim: Well, I’m so glad you asked, Kurt, so there are two things I would encourage people to consider. The first is depending on the size of the data that you’re getting out of Magento, it’s a potential common pitfall that the PHP installation Magento is running on does not have enough execution time, or potentially even memory limits set in the PHP any file, and so those exports will just fail from the get-go, which is something we actually ran into if you recall.
Kurt Elster: Yeah, I remember. I read it was like, “Well, here’s a common issue, like try to get the .CSV out of Magento. May not be as easy as you think.” And then sure enough, as soon as we tried, immediately it’s like, “All right, well, it’s running,” and this must be so common that there’s like a message in Magento that pops up that’s like, “Hey, here's some things to make sure are happening or it won’t work.”
Karl Meisterheim: Well, that’s the funny thing too is the error message that came through from Magento very much did not indicate that this was the problem, but a little Googling quickly… People chimed in with what that actually meant. And then if you have some server fu in your back pocket, it’s not a big deal to increase those limits in PHP run time and get that to work. It’s just not something everyone probably feels comfortable with or even has access to, because you need access typically to do that.
But once you get the file out, then yes, there’s oftentimes lots of issues with the data in the file, so I suggest two things. One, either be really good at Excel, which thankfully a lot of people are, and there’s all kinds of resources for working with Excel, because you’re gonna want to do things like look for duplicate rows based on the SKUs and the titles, and things like that, and get those out of there. Find blanks or other issues with the data that you don’t want in there.
But for other folks who don’t have Excel fu like myself, I’m more of a database guy because my background is in database-backed web apps for 20 years. There are some cheat codes that you can apply by basically setting up your own local instances of say like a MySQL database and you could bring that .CSV data right out of Magento into your local database and then you have the full power of the SQL language to do whatever it is you want to do to that data before you export it again, which is what I ended up doing here.
Kurt Elster: We went from like, “Matrixify is cheat code, really easy,” very quickly to Stewart’s other advice was, “Hire an agency or someone who’s done this before,” and as soon as I heard… You’re like, “Yeah, if you’re good with Excel,” I’m like, “Okay, yeah. I’m a real spreadsheet jockey. I’m with you.” And you’re like, “But it’s probably easier just to fire it up in a SQL database and then work with it,” and then that’s when I got terrified.
Karl Meisterheim: Well, as a caveat to the listeners out there, part of the reason is this particular project we’re working on, there’s over 210,000 rows of data. And so, the typical spreadsheet programs that I use are barfing. They’re very slow. They’re hard to work with.
Kurt Elster: That’s the technical term.
Karl Meisterheim: I actually don’t even have Excel on my computer because I don’t have a Microsoft Office subscription or whatever, so I’m using Apple’s Numbers and trying to use Google Sheets and whatever, editing the .CSVs directly, so for me it was much more performant to use the local database, because it’s just a lot faster and I know SQL. But I don’t think it’s a requirement, per se, and especially if you have Excel and it’s not gonna choke on the size of the data you’re throwing at it.
Kurt Elster: Yeah. I think in our case it is like that Excel spreadsheet, to give people an idea of how big this thing… The Excel file is what… I think it was 400 megs?
Karl Meisterheim: 411 megabytes.
Kurt Elster: 411 megabytes. And that was just the catalog. And it’s like also not inclusive of the images. Those you pull in separately. And so, I am so blessed to have recently purchased a MacBook Pro with the M1 processor, and you know, since buying that thing I’ve not seen… I’ve not had to wait for anything it feels like. And so, it was a surprise to see like, “Oh, I loaded up this Excel file,” and then try and do a find and replace action or try and sort it and you’re gonna see that beach ball. Like, “Oh, beach ball. I missed you. Welcome back.”
So, any other watch outs, call outs, learnings from doing this migration? If you had to do it again?
Karl Meisterheim: Yeah. Well, I think a couple things. One is just recognizing there is an order this has to go in, so if you’re doing a complete store migration you have to go in the order of moving the products over from Magento. Do that first, then customers, and then finally the orders themselves, because those other pieces have to be in place, obviously, for the orders to make sense. The other thing is I found Matrixify has some really good documentation and tutorials, and so all of this stuff is laid out, they talk about all the sort of edge cases you might want to consider, things to think through, so it’s worth it to take the time up front to really read through each of those, make sure you’ve got the big picture idea of what has to happen.
Kurt Elster: Yeah. That goes along with Stewart’s initial advice, which was, “Hey, plan ahead.” Take an audit of this is the data that we want to get out. This is the data we can get out. And so, there’s just stuff you want to leave behind. All right, I don’t have to think about it. Life gets much easier. So, limit inventory of just the data we want to move over, and then you have to figure out, “All right, can we get that data out?” And that’s really like platform and configuration dependent, where it’s like does this data live on the platform or does it live in an app? All right, can I get it out? And then once I get it out, can I associate it to other things? And then can I import it?
And so, I think you’re right. Reviewing the tool that you’re going to use to do the import, which in this case sounds like especially in the case of Magento is gonna be Matrixify, review that documentation. And then you start to get a much better sense of like, “All right, in ideal scenarios, this is how this should go.”
Karl Meisterheim: Oh, and it’s nice because Matrixify has been around long enough now that there’s not only their own documentation, which is excellent, but a ton of resources that you can easily find through Google where people talk about their own experiences. I would also toss out two other quick gotchas that you want to pay attention to. The first one is that there’s an option when you import the information on the screen for bringing the file from Magento over where you can specify the old site URL, and that’s useful for making sure the images will carry over correctly.
Oftentimes in Magento, those default locations end up getting adjusted, which was the case for our project, so you really want to make sure you understand what that path is and that you verify that the new URLs for those images are actually going to work. Otherwise, you’re gonna end up importing all these products and the images are going to be broken, and that’s no good for anyone.
The second thing is that it’s slow, and this is not Matrixify’s fault. This is the fact that Shopify has rate limits on APIs, and so if you have a large number of records, just plan on it taking time. It’s not bounded by the CPU. It’s bounded by API call, so we’re talking hours and days, not minutes and hours.
Kurt Elster: So, for our 411-meg .CSV, how long do we think that’s gonna take to import?
Karl Meisterheim: Well, it’s funny you should ask, so I was just looking at this this morning, and there are different aspects of what it’s importing, but for just the products themselves, there’s talking four days.
Kurt Elster: Yeah. I don’t think that’s crazy.
Karl Meisterheim: Well, when you’re… I suppose not, but you know, again, coming from a database background or whatever, it’s like-
Kurt Elster: You feel it’s like an arbitrary limit.
Karl Meisterheim: Yeah. It’s simply recognizing that it’s not as easy as dumping the data and reading it right off disk into Shopify. It’s there’s other things happening in the background through the ways that the APIs work that just make it naturally slow because of the rate limiting.
Kurt Elster: Fortunately, it’s something we only have to do once, ideally.
Karl Meisterheim: Yes.
Kurt Elster: And then later, we also have to deal with recent data migration. Well, talk me through that concept of recent data migration.
Karl Meisterheim: So, I’m assuming you’re referring to the idea that once we do this import, there’s going to have been additional things that happened in the Magento store since we first pulled out those files that we’re going to have to account for.
Kurt Elster: Yeah. You migrate your catalog over and it’s like all right, in the migration it’s going to migrate everything as of today, and then in a store setup, a store migration project like this, we start with the data migration, because I want to have all my content present, and then we’re gonna set up the theme, and our apps, and our integrations, and middleware, and all that stuff. And so, it could be weeks to months before a site launches after that initial migration.
Well, in that time, presumably the other store stayed up, and it was getting orders. Well, shoot. What do I do with I’ve got all these orders and customers, a fraction of what I initially moved over, but they still… I don’t want to lose those.
Karl Meisterheim: Exactly. So, in our case, in the perfect world you would at least put a pause potentially on adding new products to the store during that timeframe if possible. That would just avoid having to redo that. But there’s really two ways to account for. One is you can re-export the data from Magento and reupload it using Matrixify and it will find existing records and it will add new ones, so it’ll catch the new ones and bring them over. That’s not such a great solution for us with the products because it would reprocess all 210,000 products, take four days again, which by that time things could very well have changed.
But for smaller data sets like customers and orders, where we might be talking only hours, that could make sense to do it that way. Alternatively, you could re-export that data and then go into the .CSV file and just grab the most recent rows and chop those out and bring just those in. Which is something we’ll probably have to do with the products if they keep adding.
Kurt Elster: And the wonderful advantage is here is I look forward to you having to worry about this for me.
Karl Meisterheim: That’s what I’m here for, Kurt.
Kurt Elster: On this future project. The other thing I wanted to talk to you… Well, all right, so wrapping up our Matrixify, our glowing Matrixify review, I say we’re thumbs up on this as a utility, but you need to be… Go into it eyes open and determine whether or not you’re technically comfortable with it.
Karl Meisterheim: Definitely.
Kurt Elster: Yeah, but it’s not like you’re writing code. It’s hey, can you work a spreadsheet?
Karl Meisterheim: Yeah. And again, it’s can you work a spreadsheet and how many rows of data are we talking. If it’s a small to moderate size store, probably not a problem. When you get into the bigger ones, that’s where just the performance becomes an issue. You might have more issues.
Kurt Elster: Yeah. Moving a few, moving 500 entities, items over, is just a lot easier and a lot less fraught than I have to move 10,000, 100,000. Just the more you add to it, the harder it gets.
So, the other thing I want to discuss with you is you started a newsletter not long ago on topic called Liquid Weekly, Liquid being Shopify’s templating language, and I love it.
Karl Meisterheim: Oh, good.
Kurt Elster: I’m declaring myself subscriber number one. And tell me about Liquid Weekly. What’s in it?
Karl Meisterheim: Yeah. Well, thanks for asking about it. So, as a full stack developer, I’ve specialized in Shopify for a number of years, but I couldn’t find any regular newsletters that had content that I was interested in, and so I decided to start one myself, and it primarily serves three purposes. The first is I look for content that’s interesting and helpful to people who do Shopify development who just are living this space, and love this stuff, and want to keep up to date with it, and so that’s the goal for each weekly issue is to find those articles, those tutorials, those postings that might be of interest.
Second thing I try to do with Liquid Weekly is really use it as a platform to promote and encourage developers in the community. I do that by helping to profile them. We’ve had issues where I have developers kind of give us… I call it a slice of life, where they talk about their workflow, and what they do, and really help get them in front of potential clients, or they can showcase their articles that they’ve written, or apps, or other projects they’re doing. And the final thing with Liquid Weekly is really using it as a way of bringing together those legitimate apps, and services, and job opportunities for Shopify developers in front of that audience, because it’s really hard to find a place where you can go and see all that information, and I think it’s a win for both sides.
Kurt Elster: Yeah. I’ve greatly enjoyed it. I’ve found it to be a valuable and interesting resource as an agency owner and Shopify partner. So, where do I go to subscribe?
Karl Meisterheim: I’m glad you asked, so we go to liquidweekly.com and there’s a subscription right there, a form you can enter your email, and there’s also an archive page you can link to and see all of our past issues, look at all our past content. It’s as easy as that. We’re also on Twitter @LiquidWeekly and either way will get you on the list.
Kurt Elster: LiquidWeekly.com and Matrixify is good. That’s our final verdict. Matrixify is good. Excellent. Thank you, Karl. I appreciate it.
Karl Meisterheim: It’s my pleasure, Kurt.
Kurt Elster: Our verdict from Karl there on Matrixify is thumbs up. It works. But, and the but here is I think with these data migrations, even with a tool as easy to use as it sounds like Matrixify is, you still gotta be real comfortable working with spreadsheets and handling this data and being able to get it. And getting it out of Shopify wasn’t hard, but as we’re hearing from Karl, it’s like all right, A, he had to manipulate it a little bit to get it to work right, and that was easier because Matrixify is like, “All right, here’s all the ways this failed,” and it details it for you with sane descriptive errors.
But the file was so big, he had to put it in a database to mess with it. I straight up don’t know how to do that. I’d be watching YouTube videos for hours before I could even start doing that. So, I think Stewart’s advice had been, “Hey, hire someone who’s done this before and make your life easy if you can.” I’m still gonna lean on that. It’s self-serving, right? But I don’t know, it’s a complex thing to do these data migrations.
Paul Reda: Yeah, it sounds like it. Like there’s many different parts that can go wrong.
Kurt Elster: Like 100%, it can be done. 100%, people have done it before countless times. And without a doubt, it’s complicated and hard. There’s just a lot of places to go wrong.
Paul Reda: There’s just a lot of places for it to break.
Kurt Elster: Yeah.
Paul Reda: Yeah.
Kurt Elster: And you have a lot of interdependencies, like even just importing stuff in the wrong order, you’ll mess it up.
Paul Reda: Okay. Yeah.
Kurt Elster: My products have to go in first, and then my customers have to go in, and then my orders. Because orders references both things.
Paul Reda: Yeah. Orders are made up of products, but if there’s no products in there where you upload the orders, the orders just have nothing in them, or the inputted order data is useless.
Kurt Elster: I want it to look good, work right, and just be as close to a native Shopify piece of data as possible. And it would let you import it in the wrong order, and it would still work, but like then you wouldn’t be able to just click through from the order to the product, and so there’s just like a lot of little things. You really want someone who has the experience, who’s done it before.
Paul Reda: So, that’s it? I don’t need to know anything else?
Kurt Elster: Well, Stewart did have one more pro tip and this one is valuable because I learned in the past the hard way that sometimes you gotta go back to that old store, get a piece of data that was forgotten about. And if you’ve already shut it down and transferred the domain name, uh oh. Now what? So, Stewart had some sage wisdom.
Stewart Knapman: When going through that process, one of the things that we really recommend doing is to set up another subdomain for that old site so you can still get to it. Because I guarantee, like two weeks down the track, there’ll be something and you want to reference that, or something you missed that you wanted to double check, and you want to have access to that site. And so, leave it up for a little bit so that you can get to it.
And then once you realize that yeah, when you’re comfortable with that, then you’re cool. You can turn it off.
Kurt Elster: Stewart’s advice was hey, keep the old store accessible. It gets its own little mini migration. You move it to a subdomain. So, we’ve got… Well, what’s the name of your store again?
Paul Reda: Paul’sStuff.com?
Kurt Elster: Oh! Okay. Is there a hyphen in it? So, at Paul’s Stuff you would then… When the new store, the Shopify store goes live at Paul’s Stuff dot com-
Paul Reda: Yeah. Transfer. I’m keeping the domain, obviously.
Kurt Elster: Yeah. Yeah. Well, why would you give up on that success? The old store, for at least a couple weeks, until you know all right, we have not had to touch this thing once, you put it on a subdomain like dev.paul’sstuff.com, old.paul’sstuff.com, nobodytouchthis.paul’sstuff.com, like whatever. Just so it’s there and available to you. And then you could put it on a password or whatever so people don’t see it and accidentally place orders from the old store.
Paul Reda: You know, this all sounds great. It sounds like you really know what you’re doing. There’s a real good roadmap here. But I think I’m gonna stick with my current setup because it’s kind of like why mess with what works. I don’t want to derail the money train that I’m currently riding.
Kurt Elster: Really messing with me.
Paul Reda: I don’t understand what you’re talking about.
Kurt Elster: You don’t have a store, let alone a highly successful Magento store that has made you financially independent. All of these are lies. Paul’sStuff.com isn’t real.
Paul Reda: Wow.
Kurt Elster: And the movie Hackers had nothing to do with me.
Paul Reda: I’m calling Julie. We’re worried about you.
Kurt Elster: You can still call me Zero Cool.
Paul Reda: You just won’t accept reality.
Kurt Elster: Oh, now we’re gaslighting me. I see what’s going on here. All right, before I lose my mind with you, let’s recap a few things. So, when considering your migration, you gotta make a checklist. You gotta index the data you need and then decide what data you don’t, especially if it’s not worth the hassle to migrate it over. Really, a house cleaning. Clean house. Figure out if and how that data can be exported. And I think at this point, for the most part your platform will have some option for dump this out in a spreadsheet, and your worst-case scenario is like either I have to get someone to pull it from the database directly for me or I need to… It’s living in an app.
I recommend and Stewart at Shopify recommends hire an agency with prior experience to do this. It’s gonna make life a lot easier if you’ve never done this before. Or if you’re gonna DIY it, definitely choose to use one of these automation tools, like Shopify Store Importer, or Matrixify, and you could do DIY it with that.
Paul Reda: The agency you hire is gonna be using Matrixify too. From what I’ve heard of Matrixify, it’s a very powerful and complex product, so-
Kurt Elster: Beloved by Shopify partners.
Paul Reda: Yeah, but like you don’t want to roll with it yourself, I don’t think.
Kurt Elster: Probably not. I think it really comes down to like if you have to ask, just hire someone to do it. You’ll know if you have the confidence level to mess with dealing with a platform migration like this. And then finally, I like this easy bit of advice. Just temporarily, for two weeks, a month, whatever, leave the old site up on a subdomain in case you need to access it. It’s nice to have that.
And my final, final bit of advice, make backups. When you migrate to Shopify, make backups.
Paul Reda: Oh, is there a company, perhaps, that provides backups?
Kurt Elster: Oh, Rewind. Oh my gosh. Those guys.
Paul Reda: Are they good? I don’t know.
Kurt Elster: Well, I believe Rewind was the first backup solution.
Paul Reda: I’m kidding. They sponsor the show. They’re the best. 10 out of 10.
Kurt Elster: 10 of 10. A-plus. All right, I have nothing else to add on platform migrations other than just get someone smarter than you to do it. Make your life easy. The time has come for our outro. I would love to hear your thoughts on this episode, so please join our Facebook group, Unofficial Shopify Podcast Insiders, and talk to us. We would love to hear from you. Tech Nasty, out.
Ezra Firestone Sound Board Clip: Tech Nasty!
Paul Reda: Good.