Algorithms for shopping basket and sequence analytics extract statistically significant item sets (i.e. shopping baskets) or sequences from transactions and then extract general rules from these. The greatest challenge here is the amount of data to be handled given that transaction data can be huge, often reaching hundreds of millions of data sets. Other challenges concern content: Which rules are statistically significant? How do we find rules for all items? How do rules work in chains?
For many years prudsys AG has been developing a high performance package of shopping basket and sequence analytics algorithms. These not only increase the speed of transactions and the amount of transaction data handled but they also cover a wide range of applications: Shopping basket analysis, sequence analysis, link analysis, sequential shopping basket analysis, taxonomy processing, combination with reinforcement learning algorithms for chain optimisation, automated parameter tuning and ... much more.
Advantages
Very high speed, analysis of millions of transaction in just a few seconds.
Processing of virtually unlimited amounts of data: All algorithms are available in decomposition processing versions. This means that data is read and processed a block at a time so that there is no need to hold all the transaction data in memory at the same time.
Automation of parameter selection: The setting of parameters such as the minimum number of rules is automatically set by the algorithm which varies algorithm parameters such as minimal support or confidence levels until it finds the target setting.
Sequence analysis can be optimised with RL post-processing so that the rules or their sequential use are optimised in Markov chains.
Worthy of note here are the algorithms used for sequential analytics. The majority of the current implementations of sequence analysis are much slower than the less-complex algorithmic shopping basket analysis. In order to overcome this problem, the most important part of shopping basket analytics, i.e. the recursive assembly of longer, large item sets from shorter ones, has been successfully transferred to sequence analysis. The result is a family of patented sequence analysis methods whose speed approaches that of shopping basket analytics. This solution is unique, world-wide.
Integration
A comprehensive package of shopping basket and sequence analysis algorithms is available for the XELOPES library and is also implemented in prudsys RDE. Like all the models in the XELOPES library, the algorithms can be serialised in PMML, a format that is easy to store and enables the standardised exchange of models.
