Data Enrichment

Learn to extract value from AIS data through Yapily’s data enrichment products

Introduction

Yapily uses sophisticated data processing & machine learning techniques developed specifically for Open Banking to empower our customers to unlock the true value of the data, easily and scalably. Fuelling our enrichment algorithms using Open Banking API data means better accuracy, less exceptions and accelerated improvement as our algorithms are constantly fed new data points to drive learning.

Enrichment is currently available on the Get Account Transactions endpoint offers the following categorisation content:

  • Transaction categorisation
  • Merchant identification

Categorisation Engine

The Yapily Categorisation Engine parses Open Banking transactions, identifying the merchants and different Income and Expense categories, which can serve as the foundation of Personal Financial Management or Lending Decision applications.

The Yapily Categorisation Engine combines the following techniques to uncover extensive value from vast amounts of raw data:

  • Highly precise string operations on transaction descriptions
  • Mapping bank codes
  • State of the art machine learning based on the transaction descriptions & dates.

The engine itself does not store any data, passing transactions straight through it and returning categories associated with each transaction. Hosted s ecurely in the cloud, it can scale independently - ensuring no adverse impact on performance.

Prerequisites

Before retrieving data from the Get Account Transactions endpoint, you will first need to follow these steps:

  1. Obtain a Consent from the user
  2. Get Accounts for the user
  3. Get Account Transactions adding the optional parameter with=merchant

Categorisation Schema

When the with=merchant parameter is added, the account transactions endpoint supplements transactions with the enrichment. An example of the merchant dentification can be seen below:

Request:

curl https://api.yapily.com/accounts/{account-id}/transactions?with=merchant \
  -H 'Accept: application/json' \
  -H 'consent: string' \
  -u API_KEY:API_SECRET

Response (with Categorisation enrichment):


{
    "data": [{
        "transactionInformation": "TESCO PFS BASINGSTOKE 2020/07/30 3803",
        "bookingDateTime": "2020-08-01 00:00:00",
        "transactionAmount": {
            "amount": 10,
            "currency": "GBP"
        }
        ,
        "proprietaryBankTransactionCode": "DEB",
            "isoBankFamilyCode": "CCRD",
            "isoBankSubFamilyCode": "POSD"
        },
        "enrichment": {
            "categorisation": {
                 "categories": [“TRANSPORT”],
                 "source": "MERCHANT"
            },
            "transactionHash": {
                "hash": "b0781fd71caa48c75039ec01c0ffb011.1"
            },
            "merchant": {
                 "merchantName": "TESCO PETROL",
                 "parentGroup": "TESCO"
            },
            "location": "BASINGSTOKE",
            "correctedDate": "2020-07-30T00:00:00.000Z"
        }
    }]
}

Elements

Where possible we return the following:

  • merchantName
  • correctedDate This is extracted from the transaction string, which is likely to be the date the transaction took place on. The bookingDateTime can refer to the post-clearing value
  • location
  • paymentProcessor e.g. Paypal

Where we find a merchant, we return it with its associated category. For organisations selling a diverse range of products that don't fit within a single category, we assign multiple merchants with different categories and add a parentGroup to gather those merchants together. So Sainsbury's and Sainsbury's Bank are merchants with appropriate categories that also have Sainsbury's as their parentGroup.

As with categories, the merchant and category of a transaction may change as we improve the accuracy of our merchant service and add new merchants.

Fees and Cash

Yapily’s Categorisation engine will identify whether certain transactions are cash withdrawals or bank fees, based on the transaction codes or description. In the case of bank fees, we set the merchantName to the institution that manages the account. For cash withdrawals, we also parse merchantName from the description to gain more insights.

Please note: transactions marked with subcategory ATM Withdrawal, indicate ATM withdrawals or cashback at merchant, not a transaction with that merchant.

Transaction Categories

To see the latest list of categories, use the Get Categories endpoint

Please note: where the model cannot identify a merchant we omit the merchant field entirely.

Transaction Hash

Some institutions do not return a unique identifier (i.e. transaction id) for transactions. The transaction hash was introduced to provide an identifier for transactions retrieved via the Yapily API, which may be helpful in cases where a transaction id is not present.

Where you can find it

By default, the Yapily Transaction Hash is returned for every transaction and is located within Enrichment in the transaction response. You can find an example transaction below:


{
    "date": "2020-07-31T00:00:00Z",
    "bookingDateTime": "2020-07-31T00:00:00Z",
    "status": "PENDING",
    "amount": -57.87,
    "currency": "GBP",
    "transactionAmount": {
        "amount": -57.87,
        "currency": "GBP"
    },
    "isoBankTransactionCode": {
        "domainCode": {
            "code": "PMNT",
            "name": "Payments"
        },
        "familyCode": {
            "code": "ICDT",
            "name": "Issued Credit Transfers"
        },
        "subFamilyCode": {
            "code": "DMCT",
            "name": "Domestic Credit Transfer"
        }
    },
    "proprietaryBankTransactionCode": {
        "code": "DEB",
        "issuer": "LBG"
    },
    "enrichment": {
        "transactionHash": {
            "hash": "3c327ac9d43ef541c4dc6d163cf042bc.1"
        }
}

How it works

As the name suggests, the transaction hash is generated by hashing a number of fields together from the transaction itself. These fields can be found below:

  • Account ID
  • Institution ID
  • Credit/Debit indicator
  • Date
  • Amount
  • Description

In addition to this, once the hash is generated each transaction is ranked in the context of the API Call. The reason for this is to account for the example where a person may have a transaction with the same description, for the same amount, on the same day, paid for from the same account.

Limitations

While the ambition of the transaction hash is to consistently provide a unique transaction identifier, there will be instances where the transaction hash may change or the same transaction hash could be assigned to different transactions. This is due to the hash being reliant on live institution data, which could change at any time. Yapily will continue to monitor the performance of the transaction hash and mitigate edge cases where duplications or changes may occur, however it is not possible to guarantee that a transaction hash will be completely unique or immutable.