5

I have transaction details for credit data (bank transfers, peer to peer transfers, etc). Currently, I have one year worth of data which I cannot properly classify.

I'm looking for input and suggestions about these two questions:

  1. How do I detect anomalies in the last hour with previous credit transaction data that I have?

  2. How can I detect anomalies in real-time for the current transaction?

Ethan
  • 1,625
  • 8
  • 23
  • 39
Kira
  • 51
  • 1
  • 3
  • 1
    Hi. Welcome to the site :) – Dawny33 Feb 13 '17 at 12:22
  • Maybe this paper's methods can help you: https://ieeexplore.ieee.org/document/6406617 they are designed for anomaly detection in transactions when you don't know which ones are the bad ones (i.e. unsupervised learning) – Rodrigo Zepeda Jun 03 '21 at 14:28

1 Answers1

3

There are a few factors to consider in anomaly detection. A simple method would be to plot a boxplot of the data and calculate outliers this way (boxplot description).

If the time series is seasonal, you could take an approach similar to Twitter's anomaly detection algorithm (Twitter Anomaly Detection). This uses an outlier detection method (ESD) applied to the seasonal decomposition of the time series.

You could use clustering and SVMs as well.

There is also change-point detection which I understand less but can be very effective.

Really, your answer depends on how complex your data is. Sometimes, a simple heuristic algorithm works the best.

Here are some similar questions:

Anomaly detection in Time Series Data - Help Required

How to classify and cluster this time series data

Community
  • 1
Hobbes
  • 1,429
  • 9
  • 15
  • There is no seasonal data. Transactions are uniform. There is no training data. Would time-series work fine? – Kira Feb 14 '17 at 07:21
  • I would avoid any time series approach that specifically addresses seasonality. Start simple and work your way up. Some quick boxplots to visualize your data, and see if this captures your outliers. ESD (http://www.itl.nist.gov/div898/handbook/eda/section3/eda35h3.htm) could be a useful method. Clustering could also work, and it is unsupervised, however, there are some pitfalls to k-means. Without knowing the data, it is hard to say what time series approaches will be appropriate but it's definitely worth a look. – Hobbes Feb 14 '17 at 16:03