Using GraphQL to Unlock eCommerce Data for Frontend Teams: Part 1
Enterprise database technology trends.
“While the data layer space is still in constant movement, one thing seems clear: GraphQL and its related technologies are here to stay.” State of JS 2020 Survey
This is the first of two blog posts in which we’ll explore how GraphQL is an important enabling technology for eCommerce. The eCommerce ecosystem stands to gain speed and productivity by the adoption of GraphQL at a time when the mandate for businesses to provide engaging customer experiences was never more profound.
The eCommerce Landscape in 2021
eCommerce sales have been on the rise. In the US alone, consumer online spend in 2020 was up an incredible 44.0% year over year, according to Digital Commerce 360 estimates. It was the highest annual U.S. eCommerce growth in at least two decades, and nearly triple the 15.1% jump in 2019.
Online merchants seek higher levels of flexibility and control, and speed is more important than ever - speed of delivering new experiences and speed of customer conversion. Headless commerce is growing in popularity, and for brands of all sizes, it can provide more ecommerce platform options and enable them to decouple the front-end applications from backend data sources.
A dizzying array of backends power modern businesses — CRMs, CMSs, databases, 1st and 3rd party APIs and services, inventory and product management systems, recommendation engines — to name a few. And backend data sources only grow in number and complexity as more businesses up their digital experience game. For example, typical eCommerce websites pull in data from at least half a dozen backends:
- A core eCommerce system like Shopify, BigCommerce, or popular Jamstack backends like Snipcart, which are typically operated by the eCommerce team
- A core CMS like AgilityCMS, Contentful etc., typically managed by the marketing team
- A set of logistics endpoints such as FedEx, UPS, or a myriad shipping and delivery services
- An inventory management system
- Email, CRM and other marketing automation systems like Hubspot, Mailchimp, Klaviyo email marketing and so on
- A recommendation system
- The list goes on...
What is GraphQL?
As you can probably guess from its name GraphQL is a query language - that’s what the “QL” stands for. Simply put, it is used to request data - or send queries, to a database.
GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. It is used to load data (send queries) from a database.
And it does this in a much more efficient manner than traditional methods and services. GraphQL is designed to make APIs flexible, fast, precise and developer-friendly.
What Is API?
API is an abbreviation for an Application Programming Interface. As the name suggests, it is a request-based, programmable interface that relays data between the server and the client based on calls/requests. But, the primary task of an API in all essentiality is to deliver content from your CMS service/platform (server) responsively to 3rd party applications and device platforms (client).
Learn more about API on our blog.
Examples of using GraphQL:
Example 1:
Consider this simple example: Today, you need just a list of users. Tomorrow, your list must include only the users who bought a certain product. Then, you want the users matched with their addresses and sorted by ZIP code, but only for U.S. addresses.
When the queries must work with data structures that are multi-layered and sometimes interconnected, GraphQL simplifies how we ask a question to produce just the right answers.
Example 2:
In an example from air travel, a schedule may be filled with flights, and each flight is filled with passengers. Each passenger also has their own physical characteristics, like height or weight, as well as preferences and medical needs. Finding a list of all flights with passengers who need an extra-large wheelchair takes only a few words with GraphQL.
Why is GraphQL gaining popularity?
Created by Facebook, implemented in their mobile app in 2012 and open-sourced in 2015, GraphQL is rapidly becoming the standard for API-based data access.
Why? GraphQL is a simplified mechanism for presenting queries to the database. The real secret is that GraphQL ensures that the developer and application only loads the relevant and absolutely necessary data, even if it's from multiple sources.
Traditional databases are based on a language called SQL, short for Structured Query Language. Simple requests are pretty easy to write in SQL, but problems arise when tabular data has too much structure.
Trying to find a good query to retrieve exactly the right subset that matches requires deep thought, experimentation, and iteration. Developers spend time thinking through complex chains of operations that match and merge tables — called JOINS — instead of working on the application.
GraphQL is a new architectural element that eliminates the need for frontend teams to repeatedly ask backend teams for new endpoints or changes to exisitng endpoints.
Benefits of GraphQL APIs: Simplify data access and accelerate app development
GraphQL APIs help abstract the complexities of multiple backends behind a unified endpoint. Front-end developers benefit because they can fetch data from the multiple backends with a single query to the GraphQL endpoint.
1 | GraphQL opens doors to powerful developer tools.
It lets the client specify exactly what data it needs. It makes it easier to aggregate data from multiple sources.
GraphQL creates a uniform API across your entire application without being limited by a specific storage engine. You provide functions for each field in the type system, and GraphQL calls them with optimal concurrency. GraphQL APIs give apps continuous access to new features and encourage cleaner, more maintainable server code.
2 | GraphQL is incredibly effective for building modern mobile and web.
It makes it easier to evolve APIs over time, and enables powerful developer tools. GraphQL APIs are organized in terms of types and fields, not endpoints. Thus, GraphQL uses types to ensure Apps only ask for what’s possible and provide clear and helpful errors.
This query language has also become tightly aligned with the modern Jamstack style of developing web applications.
Some of the major libraries, like Gatsby, use GraphQL as their lingua franca for extracting information from the database. Programmers who take up this development style naturally choose the language.
3| Multichannel and Omnichannel Marketing is easier with GraphQL
The Flexibility of GraphQL Makes Supporting New Channels Easy in your Multichannel and Omnichannel marketing digital strategy.
4 | Customer Experience and UX are better with GraphQL
Apps using GraphQL are fast and stable. Websites using GraphQL can be quick even on slow mobile network connections.
5| GraphQL can save money
It saves time for your developers and can encourage them to write more precise queries that return only the essential data, thereby saving bandwidth - and thus saving your digital marketing budgets.
GraqhQL vs REST API
Many open-source GraphQL extensions offer certain features not available with REST APIs. While typical REST APIs require loading from multiple URLs, GraphQL APIs get all the data your app needs in a single request.
GraphQL does not dictate a specific application architecture. It can be introduced on top of an existing REST API and can work with existing API management tools.
GraphQL CMS APIs gather all the data your app needs in a single request, unlike REST APIs that require loading from multiple URLs.
In this blog, we describe how you can build a GraphQL API that easily connects two important systems in an eCommerce stack. The systems we are connecting through our GraphQL endpoint are the Shopify eCommerce platform and the Agility CMS content management system.
Connecting the Agility CMS with Shopify
Thanks to Joel Varty and team at Agility CMS for challenging us to crack this important scenario. Here we connect just two systems and in two different ways. It’s not difficult to see how complexity is compounded when additional integrations are needed between any two components and then new connections are required for each additional component of your eCommerce ecosystem. We believe that this can serve as a scalable foundation for your eCommerce ecosystem.
We use the Agility CMS for all our product “copy”: titles, descriptions, and images. We also use Agility to associate products with things like categories and target audiences. We’ll keep track of prices and product variants in the Shopify eCommerce platform.
The connections
We’ll look at just two integrations to connect our core systems. The first is to retrieve product information from both systems in one API. The second is a synchronization process — when a new product is added in Shopify, a record is created in Agility, so that the marketing team can get to work with descriptions, categorizations, and so on.
Now, you could build some of your integration logic into your front-end application(s). But that’s a lot of logic to write and maintain, is brittle and fraught with risk, and raises more than a few security questions. You could also build this integration into a back-end service.
But then, not only are you maintaining your eCommerce site, but you’re the proud new owner of a backend service — essentially an IT product. You’re on the hook for all the care and feeding that goes into building and maintaining an IT product. It’s not a small task.
GraphQL as the solution
The GraphQL API you build in StepZen is an API of APIs - different backends, authorizations, response protocols, syntax and more are made uniform in the context of your GraphQL API.
So GraphQL is a good solution because it lets you abstract the complexities of backends and power your apps with the data they need from multiple backends with a single query.
StepZen is a fully managed service that makes it easy to build GraphQL APIs and connect data from REST, GraphQL, databases, or any backend. Furthermore, we secure the API, host and manage the service so that you can speed app development while maintaining security and control of your data. You have no servers to manage, no code to write to parallelize execution, ensure keys and queries are secure, and the cache is handled by StepZen.
Let’s look at some code: combining Shopify API with the Agility API
Let’s look at what it takes to combine the results from the Shopify API with the Agility API when we want to retrieve a product listing.
StepZen’s configuration-driven approach means that we don’t need to program the integration logic. Instead, we configure our GraphQL schema and therein specify how StepZen talks to the various backend systems and links data together.
Retrieving and linking data from multiple systems
In our example scenario, with the product data coming from both Shopify and Agility, we define a product type in our GraphQL schema that looks like this:
type Product {
title: String
description: String
image: String
createdAt: String
agilityId: ID!
admin_graphql_api_id: String
hero: Boolean
type: String
use: String
price: String
shopifyId: ID!
audience: String
variants: [ShopifyProductVariants]
@materializer(
query: "shopifyProductVariants"
arguments: [{ name: "product_id", field: "shopifyId" }]
)
images: [ShopifyProductImages]
@materializer(
query: "shopifyProductImages"
arguments: [{ name: "product_id", field: "shopifyId" }]
)
options: [ShopifyProductOptions]
@materializer(
query: "shopifyProductOptions"
arguments: [{ name: "product_id", field: "shopifyId" }]
)
}
This is mostly straightforward GraphQL Schema Definition Language (SDL), but those familiar with SDL will notice a custom directive, @materializer. This tells StepZen how to retrieve elements for a type using queries defined elsewhere in the schema. (For more on @materializer, see the docs.)
And one of those queries looks like this:
type Query {
shopifyProductVariants(shopifyId: ID!): [ShopifyProductVariants]
@rest(
endpoint: "https://$store_name.myshopify.com/admin/api/2021-04/products/$shopifyId;.json"
resultroot: "product.variants[]"
configuration: "shopify_config"
)
}
Here we see another StepZen custom directive, @rest, which tells StepZen how to use Shopify’s REST API to retrieve variants given a product ID.
We can define a similar query to retrieve data from Agility, given a product ID. Notice the list of directive setters that rename fields from Agility to match the Product Type fields that we specified earlier.
agilityProduct(agilityId: ID!): AgilityProduct
@rest(
endpoint: "https://$instance;-api.agilitycms.cloud/preview/en-us/item/$agilityId"
configuration: "agility_config"
setters: [
{ field: "audience", path: "fields.audience" }
{ field: "description", path: "fields.description" }
{ field: "createdAt", path: "properties.modified" }
{ field: "agilityId", path: "contentID" }
{ field: "image", path: "fields.image" }
{ field: "hero", path: "fields.hero" }
{ field: "type", path: "fields.type" }
{ field: "use", path: "fields.use" }
]
)
So with a simple GraphQL schema, we’ve configured a single GraphQL API that we can use to retrieve product data from both Agility and Shopify. Now our front-end developers don’t need to be concerned with how to retrieve data from any backend system we work with. They can run a single query for a product and pull some data from Shopify and some from Agility.
Better still, if we need to change from Shopify to another eCommerce provider, we simply change the StepZen configuration. The frontend site or app doesn’t need to change or know about the specific backend systems that provide the data for the Product Type.
Synchronizing data between systems
Now what about synchronizing data between systems? When our product team creates a new product in Shopify, we want our marketing team to see the correct information in Agility. We can address this with the same GraphQL API we build to support our eCommerce site, ensuring that we keep everything in sync for our backend and frontend developers. To do this we take advantage of Shopify’s web hook capability, and instruct our Shopify store to call a StepZen API whenever a new product is created.
This Shopify web hook executes a ‘Mutation’ on the StepZen GraphQL endpoint like so:
shopifyToAgilitySequence(
title: String!
description: String!
shopifyId: ID!
image: String!
): [Map]
@sequence(
steps: [
{ mutation: "shopifyToAgility" }
{ mutation: "shopifyToAgilityMap" }
]
)
A custom StepZen directive called @sequence executes two mutations in sequence.
- When Shopify calls the webhook endpoint, the shopifyToAgilitySequence executes the initial mutation, shopifyToAgility, to provide all the product “copy” to the Agility CMS. We will later query this data to power our NextJS frontend application.
- The data written to AgilityCMS is accessible for our content team to create and manage content from either platform. All the import and synchronization tasks are automated for us.
- The second mutation, shopifyToAgilityMap is executed to map the product shopifyId generated from the initial web hook to the new Agility product that was created in the mutation, shopifyToAgility.
- shopifyToAgilityMap writes to a custom content model in AgilityCMS to ensure all Agility product data is synced with the Shopify product data via id mapping.
- @sequence executes in real time to allow fields of the previous mutation (shopifyToAgility) to immediately be used as arguments to the subsequent mutation (shopifyToAgilityMap) in the sequence.
- After shopifyToAgilitySequence finishes running the steps of the sequence, there is one product created in two platforms, Shopify and AgilityCMS, tied together by id mapping, ready for staging on the frontend application.
Recap
We’ve looked at how to build a GraphQL API that can retrieve and link data from multiple eCommerce systems and also synchronize data between systems – just two ways of combining data from core eCommerce systems.
Whatever systems you start with, the list is bound to grow, as you continuously evolve your customers’ experiences. As eCommerce companies look for ways to bring systems (and data) together and future proof their infrastructure, GraphQL offers an important solution. Using a unified, managed GraphQL API, we can eliminate the need for developers to build and maintain a costly and likely complex set of integrations as your systems scale. It enables backends to be abstracted behind a single easy-to-query endpoint, unlocking the data your front-end teams need to deliver experiences at speed. Backend teams can maintain control of the data, and add more queries and types as market and customer needs evolve, but the core foundation remains constant.
We’ve shown you how you can easily connect and link any backend data through configuring your GraphQL SDL, and a few custom GraphQL directives.
You can easily expand to connect additional backends or configure new backends without rework of the front-end app. At runtime, you don’t build or manage a GraphQL server, worry if backends are down or the response from one has changed. You don’t have to make tradeoffs between adding a cache or changing logic or myriad other concerns to run and maintain your code effectively.
Where to go from here:
I am looking forward to joining Agility’s Joel Varty in this Developer Workshop on July 8 2021 > Power up your Jamstack with GraphQL and Headless CMS. We’ll explore this solution in detail. Please join us - it’s going to be fun and we look forward to your feedback and questions.
StepZen is a fully managed service that enables developers to easily build a GraphQL API for REST, GraphQL, databases, or any backend. Try it Free > stepzen.com
About the Author
Anant Jhingran is an engineering and product expert in databases, ML and APIs. He has shipped deeply technical products that have been deployed in 1000's of enterprises. He helped take Apigee public, as well as its acquisition by Google. He has a PhD in database systems from UC Berkeley, and is accomplished in his professional career (IBM Fellow; CTO of IBM's Information Management Division; Distinguished Alumnus, IIT Delhi).
He is the author of over a dozen patents and over 20 technical papers, and is frequently giving keynotes in industry and academic conferences.
twitter: @jhingran