Big Data

Why Retailers and CPG's should harness the power of the cloud

By Stacy Klimkowski • June 21, 2016

“You have got your head up in the clouds,” used to seem like an insult, however, these days, whether you are a supplier or a retailer, if you are not thinking about how the cloud can help your business take analytics to the next level, then your head may truly be up in the clouds.  We tend to hear a lot about how digital companies are using cloud computing, but if you work in retail analytics, you may be relatively unfamiliar with what the cloud may actually represent for high scale data analysis, business agility, and data security. This article will outline what exactly the cloud is, how it is ideal for large scale data processing and machine learning from high velocity transaction data, as well as how the security compares to inefficient on premise based systems.

What is the cloud?

In simple terms, the cloud is a remote database that is generally rented from a third party. Prior to cloud infrastructures, data was typically stored and processed off of remote servers that were independently owned by businesses. As a result, each business was responsible for the cost of the computer servers, warehousing/storage, and maintenance. Now, third party cloud providers like Google and Amazon have invested billions of dollars to house and manage these servers in efficient data centers.  By doing so, they are creating enormous economies of scale and can rent out their space for a fraction of what it would cost to build a data center. Despite the efficiency and growing popularity of the cloud for data storage, many legacy syndicated data systems used for retail analytics today are still run through servers housed in their own independent warehouses. However, cloud infrastructures have created an increasing amount of opportunities for companies that are adept at solving challenges that stem from provisioning high volume retail data. Such analyses require years of purchase data, across entire stores of products and categories (previously limited to one to three categories with legacy systems), which can be used to make merchandising, promotion, and operational decisions.

How secure is it?

Any time there is a transaction of data (between the cloud and data processors) data is encrypted in transit. Data stored at rest is also encrypted.  The security of the cloud is similar in a sense if you were to find a passcode locked iphone 6; it is nearly impossible to crack. The physical security on these premises is second to probably only the government with many providers using best in class physical, electronic and biometric protocols. Furthermore, it is in the best interest of the third party cloud providers to keep the data safe. Billions of dollars have been invested into cloud infrastructures. The largest cloud infrastructure, Amazon Web Services, is estimated to be worth $128 billion to $168 billion, or about half the value of Amazon’s enterprise value according to a recent article.   

What are the major benefits of cloud computing?

       1) Cost and economics:

One of the major benefits of using a cloud infrastructure for data storage is the ability to scale up computing power with very little investment. Companies like Google and Amazon have built incredible economies of scale by building and renting out the space, and in turn, have alleviated the burdens of managing data such as storage, security, new hardware, personnel etc. Their operations require data centers larger than any third party data center network, and since they already have strong revenue sources, they are able to license out server capacity at competitive prices.  In addition, they have made dozens of cost cuts in the last five years as their marginal cost continues to decline.  Customers receive pass through benefits and are awarded with a strong ROI proposition.  This means retailers can store multiple years worth of transaction data which in the past was uncommon due to the high cost of storage.

2) Scale and Speed:

Cloud computing can not only retrieve more data, but also do so faster than if it were to be processed on a single server. Since data is stored within an entire warehouse of servers, large data requests can be computed across unlimited number of servers and reduce processing times. Not only that, but the speed in which new projects can be set up is a matter of hours vs. waiting weeks to go through on-site servers to load data. 

Now, with the right software and cloud infrastructure a retailer can explore and perform data discovery to extract richer insights on larger sets of data. This convergence of increasing amounts of data, less cost per terabyte, and greater speed is enabling new analyses, like anomaly detection, instantaneous 1:1 personalization, and affinity modeling to be performed on-demand. For more on that check out the blog FMCG Retailers Analytics Improve with Supplier’s 1st Party Data.

3) Data agility:

With the ability to store and transfer terabytes of data, a multitude of data sources can be stored and downloaded quickly, leading to enhanced collaboration across departments, partners, dealers, and customers ( Application programming interfaces (APIs) allow both small and large scale data sets to be accessed on-demand. Algorithms can now be embedded into devices to make them intelligent, and behavioral context such as consumer preferences can be embedded into digital touchpoints like mobile, in-store digital displays, beacons, emails, etc. 

Why is it important for retail analytics?

With the rise in ecommerce, social, mobile, and loyalty programs, consumers are leaving behind more behavioral data “bread crumbs” during their path to path to purchase than ever before. The main challenges facing retailers and brands are being able to collect all of these crumbs to fully understand the shopper’s journey and needs states, and being able to act on them swiftly. An Accenture research paper, A new era for retail: cloud computing changes the game, summarizes the challenges; “Many retailers have been floundering in floods of internal and external data, looking for insights to help them make merchandising decisions. Such data may be months old before it is analyzed. The questions of what to stock, how much to stock and what promotions to use grow more complex as retailers expand into new markets, integrate online and bricks-and-mortar storefronts, and offer new services.” Cloud computing can help solve these issues and facilitate the processing of near real-time transaction level data that can help solve merchandising and marketing challenges that brick-and-mortar retailers are facing.

Customers have come to expect a seamless shopping experience, and the only way to fully understand the way in which they shop is to utilize integrated data sources and solutions. Loyalty, transactions, and financial data can all be blended for more informed decision making.  A article, Sunnier days ahead for retailers that use cloud computing, states “traditional brick-and-mortar retailers must understand and harness the benefits of cloud computing to optimize the in-store experience, market to the individual and maximize every sale. If they don’t, they risk falling behind their competition.”


Undoubtedly, cloud computing will change the way that transaction data is consumed. As more and more data is collected, the need to fully integrate and analyze large amounts of data will be made possible with cloud computing and will enable marketers, suppliers, and retailers to better tailor product assortment, promotions, and merchandising to match the needs of their shoppers. 


About SwiftIQ:

SwiftIQ uses high scale data processing and machine learning to deliver contextually relevant insights and digital experiences for retailers and brands. Its platform unifies and analyzes data primarily from in-store transactions as well as online behavior and third party sources to predict and inform category captains, shopper marketing, assortment, supply chain and content delivery decisions. SwiftIQ’s unique ability to process billions of basket-level transaction records in near real-time and convert that into on-demand mobile visualizations, dayparts, affinities and attribution fosters a new level of retailer/supplier collaboration and innovation.

Since launching its category captain and consumer behavioral analytics platforms in late 2014, SwiftIQ now analyzes over $60 billion in offline, receipt-level point of sale data. The company serves 5 of the global leading category suppliers and several billion-dollar retailers. SwiftIQ, named a Top Innovator twice by DataWeek, has also been recognized by Forrester, Forbes, NACS, ProgressiveGrocer and ComputerWorld for its achievements. 

Leave a Comment