matplotlib：繪圖時忽略異常值

Question

我正在繪製來自各種測試的一些數據。有時在一次測試中，我碰巧有一個異常值（比如說0.1），而其他所有值都小三個數量級。

隨着matplotlib，我暗算範圍[0, max_data_value]

我如何才能放大到我的數據並沒有顯示異常，這將弄亂在我的劇情x軸？

我應該簡單地把95百分位數和範圍[0, 95_percentile]在X軸上？

Answer 1

對異常值沒有單一的「最佳」測試。理想情況下，你應該包含先驗信息（例如「這個參數不應該超過x因爲......」）。

大多數異常值測試使用中位數絕對偏差，而不是第95百分位或其他基於方差的測量。否則，計算的方差/ stddev將嚴重偏離異常值。

這裏有一個函數實現了一個更常見的異常值測試。

def is_outlier(points, thresh=3.5): 
    """ 
    Returns a boolean array with True if points are outliers and False 
    otherwise. 

    Parameters: 
    ----------- 
     points : An numobservations by numdimensions array of observations 
     thresh : The modified z-score to use as a threshold. Observations with 
      a modified z-score (based on the median absolute deviation) greater 
      than this value will be classified as outliers. 

    Returns: 
    -------- 
     mask : A numobservations-length boolean array. 

    References: 
    ---------- 
     Boris Iglewicz and David Hoaglin (1993), "Volume 16: How to Detect and 
     Handle Outliers", The ASQC Basic References in Quality Control: 
     Statistical Techniques, Edward F. Mykytka, Ph.D., Editor. 
    """ 
    if len(points.shape) == 1: 
     points = points[:,None] 
    median = np.median(points, axis=0) 
    diff = np.sum((points - median)**2, axis=-1) 
    diff = np.sqrt(diff) 
    med_abs_deviation = np.median(diff) 

    modified_z_score = 0.6745 * diff/med_abs_deviation 

    return modified_z_score > thresh 

由於使用它的一個例子，你會做類似如下：

import numpy as np 
import matplotlib.pyplot as plt 

# The function above... In my case it's in a local utilities module 
from sci_utilities import is_outlier 

# Generate some data 
x = np.random.random(100) 

# Append a few "bad" points 
x = np.r_[x, -3, -10, 100] 

# Keep only the "good" points 
# "~" operates as a logical not operator on boolean numpy arrays 
filtered = x[~is_outlier(x)] 

# Plot the results 
fig, (ax1, ax2) = plt.subplots(nrows=2) 

ax1.hist(x) 
ax1.set_title('Original') 

ax2.hist(filtered) 
ax2.set_title('Without Outliers') 

plt.show() 

Answer 2

如果你不大驚小怪，通過喬，它提到的拒絕離羣純粹是這樣做的美學原因，你可以設置你的陰謀的x軸限制：

plt.xlim(min_x_data_value,max_x_data_value) 

這些值是您希望顯示的限制值。

plt.ylim(min,max)也適用於在y軸上設置限制。