Blog

Magento Data Importing with REST API

Magento Data Importing with REST API

Dealing with large product catalogs can be a pain. Creating products, categories, and attributes manually is often not an option. For most of our large-catalog stores, it would take years to enter manually. Bulk updates of existing products can also take huge amounts of time and effort. For reliable and efficient product data import, it’s essential to use an smooth and accurate tool capable of automating Magento data import.

Fortunately Magento 2 has a large selection of import tools. Some are extensions that require you to update CSVs, but these can be just as time consuming as doing it manually for you to prep all the data yourself. Other tools are automated 3rd party tools and can be pricey.

We usually recommend a different path to success. Magento 2 Core comes with a native tool that does all the work for you: the Magento 2 REST API (an acronym that stands for: Representational State Transfer Application Programming Interface. Whew!).

We can take any data source (CSV, database, another website, another Magento website) and create a script to format and send Magento 2 data in a format that handles creating new and updated products for you.

With this method, you can schedule regular data updates without you having to lift a finger: Magento can request your data source, generate the correct formatting for your products, and initiate the product import and updates automatically, as often as you would like.

Problem: Import a large product catalog to Magento all at once

Managing an e-commerce catalog is generally a lot of work, but what if your catalog has thousands of products? Anything manual is completely out the window.

Magento 2 has a CSV import system, but with thousands of products, it will take so much time for the server to import and iterate over your CSV rows that it will ultimately just run out of resources, failing to complete. Even splitting the CSV into chunks will take way too much time and effort to deal with in a way that won’t completely occupy all of your free time.

Enter the REST API: Magento 2’s way of communicating with the outside world. You can leverage the power of the REST API to handle all kinds of data import challenges.

What is the REST API?

The Magento 2 REST API framework provides developers services that can communicate with Magento 2. Some features include:

  • You can safely authenticate services to send/receive data from Magento 2
  • You can ensure proper permissions are honored depending on the data you are working with
  • 3rd Party Services can be integrated with just a few lines of code in a configuration file
  • You can easily work with database models like customers, addresses, orders, etc
  • Magento provides APIs to deal with multiple services at once (for example, uploading products and categories at the same time)

We can develop scripting to automate any data source to work with these Magento APIs. Say your data source is a CMS, like WordPress or Sharepoint. We can create a script to connect to this data source and pipe it to Magento on a regular schedule, ensuring your changes will always migrate to Magento as frequently as possible.

What’s a typical product import look like?

We will take your data and format it in a JSON (JavaScript Object Notation) format, which is the most common way to send REST API’s data. So an example of a product import in this format would be:

{
  "product": {
        "sku": "A19458-SKU",
        "name": "Your Killer New Product",
        "price": 30.00,
        "status": 1,
        "type_id": "simple",
        "attribute_set_id":4,
        "weight": 3
    }
}

Magento can loop through thousands of products this way, matching data by SKU, or if no SKU exists, generating the product for you.

CRON Scheduling

The scheduling of this automated import process will be handled by CRON (stands for: Command Run On). All web servers can handle scheduled requests using CRON. We would typically schedule your import as frequently as possible, based on the amount of products in each request, and how long we expect Magento 2 to work on importing the data.

With the REST API, you can import hundreds of products per second.

If you find that you are in over your head dealing with your catalog updates, reach out to us. We feel your pain, and we can examine the best way to automate your data so you can spend your valuable time in more meaningful ways instead of being bogged down by your spreadsheets.

Share this post!

Leave a reply