Algorithms for prediction based on existing data have been around since the dawn of computing systems. For example, using algorithms for calculation of credit risk scores and predict the weather have been around since the 1950’s. In the modern computing era, we have had tools such as SQL Server, Excel, SAS, R, and other frameworks for developing, testing and executing algorithms for taking your existing data and using statistics to make predictions about the future. Customers are using these platforms to analyze their customer, sales, or product data and make predictions. Common scenarios for predictive analytics include fraud scoring, classification of customers into categories, pricing optimization and prediction of customer churn.
Historically, an organization needed the following in order to leverage any ability to make predictions:
- Access to data analysts who work with you to develop the algorithms from scratch to provide the predictive analytics engine for your data
- Access to a large volume of historical data for testing and optimization algorithms
- Large volumes of computing power to run the analytics algorithms
- Servers for running your algorithms against your data on a regular basis
For example, an insurance company would use a set of proprietary algorithms to analyze their insurance customer data to continually optimize and calculate the price for premiums they should charge based on the analyzed risk of groups of customers. A typical insurance company would have built a large server cluster to run these calculations on a monthly basis. These can be very expensive investments up front to harness the power of these algorithms.
For small to medium sized businesses, harnessing these algorithms has historically been out of reach because of the complexity, investment and maturity required. There was no way to easily experiment with predictive analytics algorithms to see if they could be of value to these organizations.
Across the industry, this is changing – instead of needing to build your own predictive analytics program from scratch, you can rent one!
In addition, the tools are becoming more user friendly and business centric so that instead of needing data scientists and statisticians, you’ll be able to harness prebuilt models and adapt them to your business needs. For example, instead of inventing your own customer churn algorithms, what if you could pull one off the shelf and use it immediately? This is the huge potential for cloud based analytics and why we’re excited about using these new tools with our customers.
Using cloud based services such as Microsoft Azure Machine Learning, we can help you develop algorithms that work with your organization’s data to make predictions quickly and easily. Instead of starting from scratch, Azure Machine Learning allows you to leverage already established and industry tested algorithms for analyzing your data to make predictions against your data. The key advantage to such a cloud based platform is you only pay for use and you can start running experiments and trying out these approaches for free. Instead of investing up front you can develop a machine learning process in the cloud and execute it on demand.
Imagine you have a database of your customers that includes a number of “features” such as their age, their location, the products they have bought previously, how many times they have called into the call center, etc. and a historical record of how much each customer spent with your organization. Using this data, we could create a model to predict how much we think the customer is going to spend with us.
The “Machine Learning” aspect of this is that our algorithms are going to use the data from our customer database to “learn” which of these features has an impact on the average spend of our customer. As we gain more data over time, we can continue to feed this data into the algorithm and the ability to predict will improve as new data is fed into it.
Once we have an algorithm that is optimized, we use this to predict a specific customer’s spending habits based on their particular features.
If the predictive power of our model is strong, based on these characteristics of our customer, we can predict the amount of money they will spend with us. We can also continually test the accuracy of our model and its ability to make good predictions based on our historical data.
The business value of such an algorithm is that if we can predict the value of our customers based on their characteristics, we can make better decisions on where to invest in them. For example, we might send customers with a predicted higher spend more incentives to continue to do business with us or we might provide more proactive marketing communication to ensure we don’t lose them. If they call into the call center, we might place them in higher priority so they get the best service. With the ability to predict the future, we can start to optimize our business processes to make better investments of our limited time, capital, and staff.
With a cloud based platform such as Azure Machine Learning, we can take our data and create an “experiment” which analyzes the existing data to see if there are features that provide strong predictors. The experiment allows us to use our existing data to both optimize the algorithm automatically and to test the predictive power of the algorithm. The cost of running the experiments is $1 US per hour, so instead of investing expensive platforms, you can rent one at a very low cost.
At such a low cost, you can easily try before you buy – instead of investing in expensive platforms and complex proprietary systems, why not run your data through Azure Machine Learning’s pipeline and see if you can create value for a very small investment?
Once you have an algorithm that provides your organization some powerful predictive capabilities, you can publish it as a production service with just a click of a button. Using this service, you can submit data ongoing and it will return the predicted values. For example, if you developed an algorithm for predicting whether your customer is a potential fraud risk, you could then send in customer records as they register with you on demand to score them as fraud risks. The cost of processing the transactions is also low at $0.50 per 1,000 transactions. Imagine if you have a 1,000,000 customers in your database – you could have them all scored for only $500!
Microsoft has also created a new Azure Marketplace for data algorithms. This means that if you were to develop a very good algorithm for prediction, you can now sell it to your peers as a market. For example, Dun and Bradstreet offers their address correction algorithms through this service for matching and cleaning up addresses from incoming data files. Microsoft has also published several algorithms in the market place for customer churn prediction, text analytics, product recommendations and anomaly detection. For organizations that have large volumes of historical data and expertise in analytics, this is an easy way to leverage the cloud to sell their services to other customers.
Our business intelligence consultants work with organizations to help implement these algorithms and can work with you to analyze your data, clean it and optimize the use of data within your organization.