Payments Synchronization Guidance

Synchronizing Payments using the Pushpay API

Overview

As payments are made in Pushpay the information about those payments is made available via the Pushpay public API - a common need is to query for recent payments and then record them in a 3rd party system. This document serves as a guide to our API users who are looking to build a new integration using the API.

Glossary

  • Organization - the organization using Pushpay to provide a payment solution - they will have 1 or more merchant listings.
  • Merchant listing - a merchant listing is a single giving page or listing in the mobile app.  Payments are always linked to a single listing.  Sometimes this is just referred to as "Merchant" or "Listing".
  • Reference Field - Reference fields are custom fields belonging to each listing - payments made for a listing can include these reference fields.  
    • There are 3 types of custom field - Text box, Number field and Drop down list (single select).  A merchant may only have one drop down field.
    • A drop down reference field has both a label and a value - the value can be used to represent the internal value for a Fund in a 3rd party system if necessary.
    • Fields are always owned by the listing - a single field can not be shared across multiple listings.
  • Fund - the fund a payment is associated with. Funds have a label and value similar to drop down reference fields.
  • Public API - the API we make available to 3rd party integrations - it's documented here: https://pushpay.io/docs/operations
  • Community Member - When a payment is made to a merchant listing there is an associated community member linked to that payment.  Community members have a name, email address and a type (Pending or Registered) and an Export Key (Your ID).
  • Your ID - The your ID is the "External system" identifier for a community member - many churches would call this an "envelope number" or "person id" or "community member id" etc. depending on the Church management system they are making use of.   At current we have 2 common names in use for this field - in the UI and file exports we call this "Your ID", in the public API we call this "ExportKey".
  • Settlement - The terms "Settlement", "Deposit" and "Batch/Settlement Batch" are all interchangeable here and represent a batch of transactions in Pushpay (credit card or ACH) which will be paid into the merchants bank account as one or more deposits. Settlements are used to aid in reconciliation of deposits to payments, so the deposited amount can split across the chart of accounts based on the Fund the payment was attributed to.

Authentication

  • Authentication requires using OAuth2 code flow - as documented in our API docs - https://pushpay.io/docs/security
  • Client ID and Client Secret in your integrations should be configurable, as should the URL for the API.
    • Pushpay will issue different credentials for sandbox and production.
    • The sandbox environment's API has a different URL to production.
  • Access token lifetime is 1 hour - refresh token lifetime is unlimited.  The refresh token must be stored in a secure manner.
  • When new merchant listings are added to an organisation,  you will need to re-authorize.
    • If you don't re-authorize - any new listings will not be returned in the in-scope merchants list and you won't be able to query for payments made to the new listing.

Rate Limiting

For guidance on how to handle rate limiting please see here.

Process - Payment Import

A step-by-step process of how a payment synchronization generally works.

  1. Authentication
    https://pushpay.io/docs/security#oauth_code_flow

    This involves redirecting to the authorize OAuth2 endpoint, requesting the scopes required by including a "scope" query string parameter with the following values (space separated):

    • read
    • merchant:view_payments
    • merchant:view_recurring_payments
    • list_my_merchants
    • merchant:view_community_member
    • merchant:view_community_members
  2. Get an Access Token
    https://pushpay.io/docs/security#oauth_refresh_token

  3. Retrieve the list of in-scope merchants
    https://pushpay.io/docs/operations#get__v1_merchants_in-scope
    GET /v1/merchants/in-scope

    • Return the list of merchant listings your code can retrieve payments for.

    • For each merchant listing you will be able to:

      • Find the "key" of the merchant listing, used to retrieve the list of payments as well as the list of community members.

      • See the list of additional fields configured for this listing.

        {
  "page": 0,
  "pageSize": 25,
  "total": 1,
  "totalPages": 1,
  "items": [
    {
      "homeCountry": "US",
      "version": 123,
      "key": "MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ",
      "handle": "demochurch",
      "name": "Demo Church",
      "address": "123 Summer grove",
      "location": {
        "latitude": -36.8567852,
        "longitude": 174.7583516
      },
      "paymentParameters": {
        "currency": "USD",
        "payButtonLabel": "Pay",
        "limits": {
          "min": 10.0,
          "max": 15000.0
        },
        "paymentPlaceholder": "e.g. 50.00",
        "mustBePaidInSafari": false
      },
      "referenceDefinitions": [
        {
          "id": 3,
          "order": 0,
          "valueType": "Text",
          "hasChoices": false,
          "label": "Full Name",
          "placeholder": "Full Name",
          "isRequired": true,
          "maxLength": 100
        },
        {
          "id": 4,
          "order": 0,
          "valueType": "Email",
          "hasChoices": false,
          "label": "Email",
          "placeholder": "Email",
          "isRequired": true,
          "maxLength": 100
        },
        {
          "id": 123,
          "order": 0,
          "valueType": "SingleSelect",
          "hasChoices": false,
          "label": "Giving Type",
          "isRequired": true,
          "choices": [
            {
              "label": "General / Tithes",
              "value": "1000",
              "order": 0,
              "isDefault": true
            },
            {
              "label": "New Buildings",
              "value": "1001",
              "order": 1,
              "isDefault": false
            },
            {
              "label": "Staff Support",
              "value": "1002",
              "order": 2,
              "isDefault": false
            }
          ]
        }
      ],
      "_links": {
        "self": {
          "href": "https://api.pushpay.com/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ"
        },
        "merchantsettlements": {
          "href": "https://api.pushpay.com/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ/settlements"
        }
      }
    }
  ],
  "_links": {
    "self": {
      "href": "https://api.pushpay.com/v1/merchants/in-scope"
    }
  }
}

  4. Loop through the in-scope merchants.
    https://pushpay.io/docs/operations/payments#get__v1_merchant_merchantKey_payments

    GET /v1/merchant/{merchantKey}/payments?updatedFrom=2015-12-11T00:00:00Z&page=0
    GET /v1/merchant/{merchantKey}/payments?updatedFrom=2015-12-11T00:00:00Z&page=1
    GET /v1/merchant/{merchantKey}/payments?updatedFrom=2015-12-11T00:00:00Z&page=2

    "_links": {
                "self": {
                    "href": "https://api.pushpay.com/v1/merchant/MTpZc2M4M3hOM05KMmdxOHpDQklvYkxqQWpfY2M/payments?updatedFrom=2015-12-11T00:00:00Z&page=1"
                },
                "next": {
                    "href": "https://api.pushpay.com/v1/merchant/MTpZc2M4M3hOM05KMmdxOHpDQklvYkxqQWpfY2M/payments?updatedFrom=2015-12-11T00:00:00Z&page=2"
                }
            }

  5. Donor Identification
    https://pushpay.io/docs/operations/payments#get__v1_merchant_merchantKey_payments

    In the payment, find the payer representation - which will have a key - this is the "pushpay account key":

    "payer": {
                "key": "MDoxWWpVN2dpTjNzeDdfMTdCcXk1bnZjOUJ5Qzg",
                "emailAddress": "joe.bloggs@test.com",
                "mobileNumber": "+15555555555",
                "fullName": "Joe Bloggs"
                ...
              },

    This key can then be used to retrieve the pushpay account's details:

    GET /v1/merchant/{merchantKey}/community/{memberKey}

    And you can also set the exportKey using PATCH

    PATCH /v1/merchant/{merchantKey}/community/{memberKey}

        {
            exportKey": "12343"
            }
    Note! Depending on your integration's goals, you may also want to synchronize recurring payments - the API is documented here: https://pushpay.io/docs/operations/payments#get__v1_merchant_merchantKey_payments. For a financial system integration we also make the settlements for a merchant available via the API: https://pushpay.io/docs/operations#settlements

Process - Settlement Batch Import

A step-by-step process of how a settlement batch import generally works.

  1. Authentication
    https://pushpay.io/docs/security#oauthcodeflow

    This involves redirecting to the authorize OAuth2 endpoint, and requesting the scopes required for the payments integration by including a "scope" query string parameter with the following values (space separated):

    • read
    • merchant:view_payments
    • list_my_merchants
    • merchant:view_community_member
    • merchant:view_community_members

    Once the user consents to give your integration access you can retrieve the access token and refresh token. The refresh token will need to be stored somewhere secure. You can request new access tokens any time - so a good way to design your synchronization process is just to get a new access token prior to retrieving a processing new payments.

    This will:

    • Return the list of merchant listings your code can retrieve settlements for.

    • For each merchant listing you will be able to:

      • Find the "key" of the merchant listing, used to retrieve the list of payments as well as the list of community members.

      • See the list of additional fields configured for this listing - this will include the field which holds the fund information.

        {
    "page": 0,
    "pageSize": 25,
    "total": 1,
    "totalPages": 1,
    "items": [
    {
      "homeCountry": "US",
      "version": 123,
      "key": "MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ",
      "handle": "demochurch",
      "name": "Demo Church",
      "address": "123 Summer grove",
      "location": {
      "latitude": -36.8567852,
      "longitude": 174.7583516
      },
      "paymentParameters": {
      "currency": "USD",
      "payButtonLabel": "Pay",
      "limits": {
        "min": 10.0,
        "max": 15000.0
      },
      "paymentPlaceholder": "e.g. 50.00",
      "mustBePaidInSafari": false
      },
      "referenceDefinitions": [
      {
        "id": 3,
        "order": 0,
        "valueType": "Text",
        "hasChoices": false,
        "label": "Full Name",
        "placeholder": "Full Name",
        "isRequired": true,
        "maxLength": 100
      },
      {
        "id": 4,
        "order": 0,
        "valueType": "Email",
        "hasChoices": false,
        "label": "Email",
        "placeholder": "Email",
        "isRequired": true,
        "maxLength": 100
      },
      {
        "id": 123,
        "order": 0,
        "valueType": "SingleSelect",
        "hasChoices": false,
        "label": "Giving Type",
        "isRequired": true,
        "choices": [
        {
          "label": "General / Tithes",
          "value": "1000",
          "order": 0,
          "isDefault": true
        },
        {
          "label": "New Buildings",
          "value": "1001",
          "order": 1,
          "isDefault": false
        },
        {
          "label": "Staff Support",
          "value": "1002",
          "order": 2,
          "isDefault": false
        }
        ]
      }
      ],
      "_links": {
      "self": {
        "href": "https://api.pushpay.com/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ"
      },
      "merchantsettlements": {
        "href": "https://api.pushpay.com/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ/settlements"
      }
      }
    }
    ],
    "_links": {
    "self": {
      "href": "https://api.pushpay.com/v1/merchants/in-scope"
    }
    }
  }

  2. Fetch recent settlements
    https://pushpay.io/docs/operations/settlements#get__v1_merchant_merchantKey_settlements

    Fetch the settlements updated since last time your code checked.

    Note! There is no merchant Key in the URL - this is because a single settlement can be for payments made to multiple listings, depending on how the individual merchant listings are configured e.g. if they remit funds into the same bank account or not.

    GET /v1/settlements?updatedFrom=2015-12-11T00:00:00Z&page=0

    The results are paged so you will need to keep fetching pages of settlements until you've read them all.

    GET /v1/settlements?updatedFrom=2015-12-11T00:00:00Z&page=1
    GET /v1/settlements?updatedFrom=2015-12-11T00:00:00Z&page=2

    Each page of settlements details returned will include some links in the response for the next page, if there is one - using these links can simplify the process of paging in our API.

        "_links": {
            "self": {
            "href": "https://api.pushpay.com/v1/merchant/MTpZc2M4M3hOM05KMmdxOHpDQklvYkxqQWpfY2M/settlements?updatedFrom=2015-12-11T00:00:00Z&page=1"
            },
            "next": {
            "href": "https://api.pushpay.com/v1/merchant/MTpZc2M4M3hOM05KMmdxOHpDQklvYkxqQWpfY2M/settlements?updatedFrom=2015-12-11T00:00:00Z&page=2"
                }
            }

    As an alternative, if it makes sense you can loop through individual merchant listings, retrieving the settlements per merchant listing (this is similar the payment import process described earlier in this document).

  3. Loop through the in-scope merchants.
    https://pushpay.io/docs/operations/settlements#get__v1_merchant_merchantKey_settlements

    For each merchant you can retrieve the list of batches updated since the last time synchronization was performed (in UTC)

    GET /v1/merchant/{merchantKey}/settlements?updatedFrom=2015-12-11T00:00:00Z&page=0

    The results are paged so you will need to keep fetching pages of settlements until you've read them all.

    GET /v1/merchant/{merchantKey}/settlements?updatedFrom=2015-12-11T00:00:00Z&page=1
    GET /v1/merchant/{merchantKey}/settlements?updatedFrom=2015-12-11T00:00:00Z&page=2

  4. Fetch the payments for a settlement.

    In the details of each settlement is a link to the payments within a settlement ("settlementpayments") - fetching this will retrieve a paged list of payments.

        {
            "key": "MDpkQUFOQ1FzdE1BLVZfVWZFdEZkQ3dvb3YyTDg",
            "name": "Settlement #1",
            "totalAmount": {
            "amount": "202.00",
            "currency": "USD"
            },
            "type": "ACH",
            "totalPayments": 2,
            "estimatedDepositDate": "2016-01-03T07:00:00Z",
            "isReconciled": true,
            "_links": {
            "self": {
                "href": "https://api.pushpay.com/v1/settlement/MDpkQUFOQ1FzdE1BLVZfVWZFdEZkQ3dvb3YyTDg"
            },
            "settlementpayments": {
                "href": "https://api.pushpay.com/v1/settlement/MDpkQUFOQ1FzdE1BLVZfVWZFdEZkQ3dvb3YyTDg/payments"
            }
            }
    }
    Note! There is an "isReconciled" flag on settlements, but currently this can only be set via the UI, not via the API.

    The payments are paged - you will need to retrieve all pages of payments to be able to see the entire list of payments.

    GET /v1/settlement/{settlementKey}/payments?page=0
    GET /v1/settlement/{settlementKey}/payments?page=1
    GET /v1/settlement/{settlementKey}/payments?page=2

    Note! You can not explicitly order the payments in a settlement, the API will retrieve them in createdOn ascending order (order in which the payments were made).

  5. Synchronize Community
    GET: https://pushpay.io/docs/operations/community#get__v1_merchant_merchantKey_community_memberKey
    PATCH: https://pushpay.io/docs/operations/community#patch__v1_merchant_merchantKey_community_memberKey

    If the system you are synchronizing with manages a list of people, the identifier for these people (sometimes referred to as person ID or envelope number) may be stored in Pushpay as the export Key (Your ID) of the community member. So in the payment, find the payer representation - which will have a key - this is the "pushpay account key"

        "payer": {
        "key": "MDoxWWpVN2dpTjNzeDdfMTdCcXk1bnZjOUJ5Qzg",
        "emailAddress": "joe.bloggs@test.com",
        "mobileNumber": "+15555555555",
        "fullName": "Joe Bloggs"
        ...
      },

    This key can then be used to retrieve the pushpay account's details:

    GET /v1/merchant/{merchantKey}/community/{memberKey}

    And you can also set the exportKey using PATCH

    PATCH /v1/merchant/{merchantKey}/community/{memberKey}

        {
      "exportKey": "12343"
    }

Community Management

If you are leveraging the "exportKey" (Your ID) field to store the ID of the individual in your church management system / CRM system - then you will need to update these in pushpay to reflect changes in the external system e.g. when a merge occurs. To assist with this we have an API which allows making changes to the community in response to external merges etc.

  1. Authenticating
    https://pushpay.io/docs/security#oauth_code_flow

    • read
    • merchant:view_payments
    • merchant:view_recurring_payments
    • list_my_merchants
    • merchant:view_community_members
    • merchant:manage_community_members

    Once the user consents to give your integration access you can retrieve the access token and refresh token. The refresh token will need to be stored somewhere secure. You can request new access tokens any time - so a good way to design your synchronization process is just to get a new access token prior to retrieving a processing new payments.

  2. Get an access token using the refresh token
    https://pushpay.io/docs/security#oauth_refresh_token

  3. Retrieve the list of in-scope merchants
    https://pushpay.io/docs/operations/merchants#get__v1_merchants_in-scope

    GET /v1/merchants/in-scope

    This will return the list of merchant listings your code can retrieve/manage the community for

        {
      "page": 0,
      "pageSize": 25,
      "total": 1,
      "totalPages": 1,
      "items": [
        {
          "homeCountry": "US",
          "version": 123,
          "key": "MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ",
          "handle": "demochurch",
          "name": "Demo Church",
          "address": "123 Summer grove",
          "location": {
            "latitude": -36.8567852,
            "longitude": 174.7583516
          },
          "paymentParameters": {
            "currency": "USD",
            "payButtonLabel": "Pay",
            "limits": {
              "min": 10.0,
              "max": 15000.0
            },
            "paymentPlaceholder": "e.g. 50.00",
            "mustBePaidInSafari": false
          },
          "referenceDefinitions": [
            {
              "id": 3,
              "order": 0,
              "valueType": "Text",
              "hasChoices": false,
              "label": "Full Name",
              "placeholder": "Full Name",
              "isRequired": true,
              "maxLength": 100
            },
            {
              "id": 4,
              "order": 0,
              "valueType": "Email",
              "hasChoices": false,5
              "label": "Email",
              "placeholder": "Email",
              "isRequired": true,
              "maxLength": 100
            },
            {
              "id": 123,
              "order": 0,
              "valueType": "SingleSelect",
              "hasChoices": false,
              "label": "Giving Type",
              "isRequired": true,
              "choices": [
                {
                  "label": "General / Tithes",
                  "value": "1000",
                  "order": 0,
                  "isDefault": true
                },
                {
                  "label": "New Buildings",
                  "value": "1001",
                  "order": 1,
                  "isDefault": false
                },
                {
                  "label": "Staff Support",
                  "value": "1002",
                  "order": 2,
                  "isDefault": false
                }
              ]
            }
          ],
          "_links": {
            "self": {
              "href": "https://api.pushpay.com/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ"
            },
            "merchantsettlements": {
              "href": "https://api.pushpay.com/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ/settlements"
            }
          }
        }
      ],
      "_links": {
        "self": {
          "href": "https://api.pushpay.com/v1/merchants/in-scope"
        }
      }
    }

  4. Update community with merges
    https://pushpay.io/docs/operations/community#patch__v1_merchant_merchantKey_community_memberKey

    You can do this by making a PATCH request to the community URL for each merchant listing (identified by it's merchant Key). The request allows you to make an "update" from an old "exportKey" (Your ID) to a new "exportKey" value. This is used when a individual in the 3rd party system has been merged, resulting in either a "winner" existing individual ID, or a brand new "merged" individual ID.

    PATCH /v1/merchant/{merchantKey}/community

    {
    "operations": [
    {
        "update": {
        "exportKey": {
            "from": "1234",
            "to": "4561"
        }
        }
    },
    {
        "update": {
        "exportKey": {
            "from": "2312",
            "to": "4561"
        }
        }
    },
    ...
    Info! You can only submit up to 25 operations at a time - so you may need to make multiple requests to apply community updates that have happened since the last time you synchronized.

  5. Bulk assign export Keys
    https://pushpay.io/docs/operations/community#patch__v1_merchant_merchantKey_community_memberKey

    Optional If building an integration against a merchant which already has an existing community (e.g. already using Pushpay before you built your integration) then you may want to assign an "exportKey" value to each member of the community. This can be performed individually using the PATCH operation on an individual community member ID, but depending on community size, this could be both slow and consume the allowable requests for a day. We recommend reading the community 1 page at a time (page size is currently always 25) and making a single PATCH request to update all community members on that page. Note the key refers to the key of the community member:
        {
      "operations": [
        {
          "set": {
            "key": "MDoxWWpVN2dpTjNzeDdfMTdCcXk1bnZjOUJ5Qzg",
            "exportKey": "1234"
          }
        },
        {
          "set": {
            "key": "ABIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlT9",
            "exportKey": "1234"
          }
        },
        ...

Webhooks

If your integration needs to synchronize payments constantly, rather then in batches, you can do so by configuring a webhook (either via the Merchant Admin portal, or by using the webhooks API) https://pushpay.io/docs/operations#post__v1_merchant_merchantKey_webhooks. Once the webhook is configured, every time a payment is created or updated, a POST request will be sent to the URI you configured in the webhook. The body of the events are JSON:

{
    "subscription": "http://api.pushpay.com/v1/webhook/token",
    "events": [
    {
        "date": "2015-01-02T03:04:05Z",
        "eventType": "payment_created",
        "entityType": "Payment",
        "links": {
        "merchant": "http://api.pushpay.com/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ",
        "payment: "/v1/merchant/MTIzOkRUclhHb1Jtc24tX3NKMGxjZzJ3cUJqb1ZlTQ/payment/q235azs3KMGxjZzJ3cUJqb1349s0909"
        }
    }
    ]
}

No sensitive information is included in the webhook event - it will only contain links to the affected items which can be retrieved to find more details.

Warning! Webhook delivery is never guaranteed - and Pushpay currently will not retry sending a webhook event if its delivery fails. As such we always recommend using a combination of webhooks and a periodic check so your integration can eventually recover if it missed a webhook message.

Compensation

A common problem for systems synchronizing payments is how to deal with bank payments which may take up to 7 days to process. Within Pushpay these payments will be immediately visible, and have a status of "Processing". At the point they succeed (or return) the status will then change to "Success" or "Failed".  This could take up to 7 days to happen. In some systems you may already have the concept of a "Processing" payment, in which case the 2 systems will align well. Otherwise, there are 2 options: * Not displaying the payment until it's status has changed to Success. However, this can lead to a lot of user confusion. * Show the payment immediately, but then "handle" the uncommon case of the payment failing, rather then succeeding. We refer to this as compensation - as soon as the payment has a status of processing, treat it as a successful payment in your system. If it later fails, either void the payment in your system- OR - create a second "compensating" payment that has all the same details but a negative amount (so it negates the original payment).  By doing so you provide an experience which is much less confusing to merchant administrators.

Info! If fetching recently updated payments (ORDER by updatedOn ASC/DESC), you will find that this not only includes new payments, but also payments which have changed status - including processing payments which have transitioned from "Processing" to "Success" or "Failed". You may find a payment in a status of "Processing" is updated several times without having it's status change, this is because in Pushpay we are updating the payment as it tracks through the stages of processing in the ACH network.

Sandbox Access

Development is always done against the sandbox environment - Pushpay does not issue production API credentials until it has seen evidence of an integration working against the sandbox environment. Sandbox access can be requested (as well as general support for the API) by sending an email to api@pushpay.com To configure sandbox access we need to know:

  • The name of your organization, or the merchant you are developing the integration for.
  • At least one email address that can be invited as a merchant administrator for the merchant in our sandbox environment.
  • The purpose of your integration (so we can determine what scopes should be allowable for your client).