No packages match

HRTnomaly - Historical, Relational, and Tail Anomaly-Detection Algorithms

The presence of outliers in a dataset can substantially bias the results of statistical analyses. To correct for outliers, micro edits are manually performed on all records. A set of constraints and decision rules is typically used to aid the editing process. However, straightforward decision rules might overlook anomalies arising from disruption of linear relationships. Computationally efficient methods are provided to identify historical, tail, and relational anomalies at the data-entry level (Sartore et al., 2024; <doi:10.6339/24-JDS1136>). A score statistic is developed for each anomaly type, using a distribution-free approach motivated by the Bienaymé-Chebyshev's inequality, and fuzzy logic is used to detect cellwise outliers resulting from different types of anomalies. Each data entry is individually scored and individual scores are combined into a final score to determine anomalous entries. In contrast to fuzzy logic, Bayesian bootstrap and a Bayesian test based on empirical likelihoods are also provided as studied by Sartore et al. (2024; <doi:10.3390/stats7040073>). These algorithms allow for a more nuanced approach to outlier detection, as it can identify outliers at data-entry level which are not obviously distinct from the rest of the data. --- This research was supported in part by the U.S. Department of Agriculture, National Agriculture Statistics Service. The findings and conclusions in this publication are those of the authors and should not be construed to represent any official USDA, or US Government determination or policy.

Last updated

anomaly-detectioncellwise-outliersmachine-learningunsupervised-learningopenmp

3.92 score 1 stars 24 scripts 207 downloads

spMC - Continuous-Lag Spatial Markov Chains

A set of functions is provided for 1) the stratum lengths analysis along a chosen direction, 2) fast estimation of continuous lag spatial Markov chains model parameters and probability computing (also for large data sets), 3) transition probability maps and transiograms drawing, 4) simulation methods for categorical random fields. More details on the methodology are discussed in Sartore (2013) <doi:10.32614/RJ-2013-022> and Sartore et al. (2016) <doi:10.1016/j.cageo.2016.06.001>.

Last updated

openblasopenmp

3.44 score 55 scripts 233 downloads

plpoisson - Prediction Limits for Poisson Distribution

Prediction limits for the Poisson distribution are produced from both frequentist and Bayesian viewpoints. Limiting results are provided in a Bayesian setting with uniform, Jeffreys and gamma as prior distributions. More details on the methodology are discussed in Bejleri and Nandram (2018) <doi:10.1080/03610926.2017.1373814> and Bejleri, Sartore and Nandram (2021) <doi:10.1007/s42952-021-00157-x>.

Last updated

2.70 score 8 scripts 243 downloads

inca - Integer Calibration

Specific functions are provided for rounding real weights to integers and performing an integer programming algorithm for calibration problems. These functions are useful for census-weights adjustments, survey calibration, or for performing linear regression with integer parameters <https://www.nass.usda.gov/Education_and_Outreach/Reports,_Presentations_and_Conferences/reports/New_Integer_Calibration_%20Procedure_2016.pdf>. This research was supported in part by the U.S. Department of Agriculture, National Agriculture Statistics Service. The findings and conclusions in this publication are those of the authors and should not be construed to represent any official USDA, or US Government determination or policy.

Last updated

openblascpp

2.40 score 2 stars 25 scripts 657 downloads

dual - Automatic Differentiation with Dual Numbers

Automatic differentiation is achieved by using dual numbers without providing hand-coded gradient functions. The output value of a mathematical function is returned with the values of its exact first derivative (or gradient). For more details see Baydin, Pearlmutter, Radul, and Siskind (2018) <https://jmlr.org/papers/volume18/17-468/17-468.pdf>.

Last updated

2.18 score 1 stars 6 scripts 643 downloads