The Test to Use Before All Other Tests

Figure 9: A more appropriate representation of the Example 1 process

Figure 5: Histogram for Example 2

This test will quantify the chances that you can successfully fit any probability model to your data. By using this simple test to examine the assumptions behind all probability models, you can avoid making serious mistakes. This column will illustrate this test and explain why it works.

Example 1

Figure 3: 199 successive differences for Example 1

While a large p-value doesn’t guarantee that a probability model exists, or tell you which model might work, a small p-value provides a red light to save you from wasted effort.

When presented with a collection of data from operations or production, many will start their analysis by computing descriptive statistics and fitting a probability model to the data. But before you do this, there’s an easy test that you need to perform.

If no probability model exists that will fit your data, what do you do next? If the data have been collected in such a way that they should be homogeneous, then the lack of homogeneity indicated by the Ramirez-Runger test is a signal that some unknown cause is changing your process outcomes. So the question becomes, “How can we identify this unknown cause?”

One caveat is needed. Since the Ramirez-Runger test depends upon the time-order sequence of the data, it should always be used with data in their native ordering. Specifically, it can’t be used on data that have been rearranged into a ranking where the values are placed in ascending or descending order.

So what happens next?

Figure 7: X chart for Example 1

Figure 6: 199 successive differences for Example 2

The Ramirez-Runger test statistic is:

Thus, with a p-value of 0.238, it’s not unreasonable to think that you might find some probability model that would fit these data and describe the process outcomes.

How this test works

These data are not sufficiently well-behaved to be represented by a single probability model. However, that doesn’t keep your software from drawing a bell-shaped curve over the histogram as in Figure 8.

Figure 4: Example 2 data

This statistic will be approximately distributed according to an F-distribution having (n–1) numerator degrees of freedom and (0.62*(n–1)) denominator degrees of freedom. The probability of exceedance (the p-value) for this test statistic may be obtained in most spreadsheet programs. In Excel we use the formula:
= FDIST(test stat., num. d.f., denom. d.f.)
= FDIST(2.60, 199, 123)

We might be tempted to immediately test these data to see if they’re consistent with a normal probability model. However, before we try to test for a lack of fit with a specific probability model, it’s instructive to test the data to see if they’ll fit any probability model.

When the p-value for the Ramirez-Runger test is small, you’ll know that you can’t fit a probability model to your data. Neither can you estimate process parameters nor compute confidence intervals, test hypotheses, or use any other statistical analysis techniques. Rather, you’ll need to use a more fine-grained approach, looking for the assignable causes of exceptional variation within the data themselves. And, of course, this will lead to the use of process behavior charts.

Ramirez-Runger tells us that, regardless of how pretty Figure 9 may look, there are only 13 chances in a billion that it’s correct. A more realistic representation of the process that produced the data for Example 1 would look something like Figure 9.

When they’re not equivalent, it’s unreasonable to assume that the data came from a sequence of independent and identically distributed random variables. And when the data show evidence that they didn’t come from a sequence of independent and identically distributed random variables, the notion of a probability model vanishes. At this point, you should abandon all hope of ever fitting a reasonable probability model to your data.

Figure 2: Histogram of the 200 values of Example 1

Figure 8: Histogram of Example 1 done wrong

This value may be interpreted as the probability that any probability model can be found that will actually describe the process outcomes. With 13 parts per billion as your chance for success, you can save yourself a lot of time and trouble by simply not trying to fit a probability model to these data. So, what do you do instead? This will be discussed below.

Example 2

To do this, we use a test developed and published by Brenda Ramirez and George Runger in 2006. The test statistic compares two different measures of dispersion. The first of these is the standard deviation statistic computed using all of the data. The second of these is the average of the differences between successive values. Here, the time order is given by reading the data in rows. Figure 3 lists the 199 successive differences.

So, while the process behavior chart remains the final arbiter of when a process is operated unpredictably, the Ramirez-Runger test provides a computation-based alternative that can keep you from making serious mistakes. If you’re not already starting your analysis with a process behavior chart, then the Ramirez-Runger test is the test to use before all other tests.


The data in Figure 1 are 200 sequential values obtained over time from a single process. The average is 12.86, and the standard deviation statistic is s = 3.462. The histogram is shown in Figure 2.

Aristotle told us that to identify the causes that affect our system, we’ll need to look at those points in time where the system changes. And this simply can’t be done with summary statistics that ignore the time-order sequence within the data. All techniques built on these symmetric statistics implicitly assume that the system isn’t changing in any manner.

A probability model is a limiting property for an infinite sequence of random variables that are independent and identically distributed. And a sequence of independent and identically distributed random variables will display the same amount of variation regardless of whether the computation is carried out globally or sequentially.

The Ramirez-Runger test provides a simple numerical test to quantify the lack of homogeneity contained in a data set. Since homogeneity is implicitly assumed by virtually every statistical analysis technique, the Ramirez-Runger test should be a starting point for your analysis.

Thus, the Ramirez-Runger test statistic is

So the Ramirez-Runger test compares a global estimate of dispersion with a sequential estimate. The global estimate is the usual standard deviation statistic. The sequential estimate is the average of the successive differences divided by its bias correction factor, 1.128. The ratio of these two estimates, when squared, will be approximately distributed according to an F-distribution with (n–1) and (0.62*(n–1)) d.f. The probability of exceedance, or p-value, for this test statistic quantifies how reasonable it is to consider the two estimates as being equivalent.

The average of these 199 successive differences is 2.42.
Average successive difference = 2.42

Figure 10 shows the X chart for Example 2. There, we see no evidence of a lack of homogeneity, which is consistent with the Ramirez-Runger test p-value of 0.238. This process shows no evidence of changes during the period covered. The average of 10.1 and standard deviation of 1.8 properly characterize the process outcomes.

Figure 10: X chart for Example 2


And Excel will return the probability of exceedance for this statistic. The p-value for our test statistic of 2.60 is:
FDIST(2.60, 199, 123) = 0.000 000 013

So, what are the chances that we can fit a probability model to these data? These data were written in rows, and the 199 successive differences listed in Figure 6 have an average of 1.905.

As soon as we plot the XmR chart for Example 1, we see why no probability model exists for these data. The process is meandering around, with frequent changes in location.

Unlike all other tests of fit, the Ramirez-Runger test takes the time-order sequence of the data into account. Why is this important? It goes back to the definition of a probability model.

With 199 and 123 degrees of freedom, our test statistic of 1.126 has a p-value of:
= FDIST(test stat., num. d.f., denom. d.f.)
= FDIST(1.126, 199, 123) = 0.238

Once we know that some unknown cause is changing our system, we must shift from using complex analysis techniques to a more fundamental approach. Rather than trying to compute our way around the lack of homogeneity, we need to find out when and why the process is changing. The primary technique for doing this is a process behavior chart.

When the p-value for the Ramirez-Runger test gets larger than 5% or 10%, you’ll have an inconclusive result. The Ramirez-Runger test uses both global and local measures of dispersion, but the fact that these are both based on summary statistics means that they can miss some signals of a lack of homogeneity within the data. Just because Example 2 had a Ramirez-Runger p-value of 0.238 didn’t mean that the X-chart in Figure 11 was going to show a predictable process.

Figure 1: Example 1 data

Once again, the data in Figure 4 are 200 sequential values obtained over time from a single process. The average is 10.11, and the standard deviation statistic is s = 1.792. The histogram is shown in Figure 5.