1. Introduction

A Gaussian Process is a statistcal method that uses a probabilistic approach to model the relationship between variables. Unlike other methods, a Gaussian Process does not assume a specific functional form for the relationship between the variables. Instead, it models the relationship as a distribution over functions. This allows Gaussian Processes to capture complex and non-linear relationships between variables. GPs provide a relatively intuitive way of encoding domain knowledge through prior specification and kernel design, where the kernel determines the shape of the covariance matrix and therefore the shape of the predicted function. Most importantly, and unlike most machine learning algorithms, GPs give the principled measure of the uncertainty associated with each estimate. This uncertainty can be used to guide decision-making and to identify areas where additional data may be needed to improve the accuracy of the predictions. GPs are also more interpretable than deep learning methods. They typically have a small number of parameters that directly capture information about the data, like the length-scales of variation or time-scales of periodicity, unlike the weights of a neural network which have no direct physical correspondence.

GP, often referred to as Kriging, has been used within the geostatistics community for well over half a century (Krige, 1952) but the the application of GPs remains far from straightforward. In its simplest form, an exact GP is computationally expensive for large datasets, scaling cubically with the number of observation datapoints. Furthermore, there are comparatively few examples of GP applications suitable for non-experts. Of these worked through examples, most are GP applications to toy dataset rather than complex real-world problems. In practice, many decisions relating to model design and implementation are developed through years of practice, reading academic literature, re-implementing experiments in code repositories and detailed knowledge of software packages run by small communities.

This paper aims to formalise the intuition of GP practitioners. We present a framework to help modellers use and confidently apply GP regression to their own datasets. The review is not an exhaustive list of possible GP applications but offers a starting point to using ‘application grade’ methods straightforwardly with current open-source software. For this reason, many recommendations follow a simple but principled approach to dealing with the challenges of applying GPs to real world data. First we define GP regression and classification in Section 2. Second, we relate the this work to previous research in Section 3. We then present a step-by-step framework in Section 4 and discuss commonly used software packages in Section 5. The framework is also applied to four case studies in Section 6. Finally, we discuss the appropriateness of using GP regression in the current machine learning landscape and non-GP alternatives to solving these tasks in Section 7.