General API guide

Last updated on October 4, 2023

Overview

The General API Guide provides a set of API endpoints to perform various actions related to the Analytics Connector service. These endpoints allow you to manage connectors, retrieve information about available topics, check connector statuses, and more. This section provides step-by-step instructions on using each API endpoint effectively.

Prerequisites

• Access to the AGS Admin Portal.

Get a list of topics or analytics data

Retrieve a list of all available namespaces and topics via the API endpoint [GET] /analytics-connector/v1/admin/tools/kafka/topics/{eventType}. Replace {eventType} with the appropriate event type. The API will return a response containing namespaces and their associated topics:

{
  "namespace1" : [
    "topic-analytics-event-1",
    "topic-analytics-event-2"
  ] 
}

Get Connector status

Check the status of a specific connector using the API endpoint [GET] /analytics-connector/v1/admin/connectors/{id}/status. Replace {id} with the connector's ID obtained from the API response after successful creation. The API will return the connector's status and related information:

{
  "time": "2023-06-08T10:22:01Z",
  "connector": "<connector-name>",
  "status": "ACTIVE",
  "error": null
}

Get Connector list

Retrieve a list of all connector configurations via the API endpoint [GET] /analytics-connector/v1/admin/connectors. The API will return a list of connector configurations.

Pause S3 Connector

Pause a specific connector using the API endpoint [PUT] /analytics-connector/v1/admin/connectors/{id}/deactivate. Replace {id} with the connector's ID. The API will return a response confirming the pause.

{
  "id": 1,
  "active": false
}

Resume Connector

Resume or reactivate a specific connector using the API endpoint [PUT] /analytics-connector/v1/admin/connectors/{id}/activate. Replace {id} with the connector's ID. The API will return a response confirming the reactivation.

{
  "id": 1,
  "active": true
}

Edit Connector

Edit the configuration of a specific connector using the API endpoint [PUT] /analytics-connector/v1/admin/connectors{id}. Replace {id} with the connector's ID. Modify the connector's configuration in the provided JSON structure.

Example S3 connector:

{
    "storageType": "KAFKA_S3",
    "config": {
        "bucketName": "<s3-bucket-name>", 
        "region": "<s3-bucket-region>",
        "customPath": "<s3-path>",
        "customPartitionDuration": "<s3-partition-duration>",
        "partSize": "5242880",
        "flushInterval": "<flush-sending-interval>",
        "eventType": "<event-type>", 
        "flushSize": "<flush-size>"
    },
    "filter": {
        "<namespace>": [
            "<topic>"
        ]
    }
}

note

The event type, storage type, and connector name cannot be changed.

Delete Connector

Delete a specific connector using the API endpoint [DELETE] /analytics-connector/v1/admin/connectors/{id}. Replace {id} with the connector's ID. The API will return a response confirming the deletion.

{
  "deleted": true,
  "id": 1
}

Get the Access Token Using IAM Client ID and Client Secret

1. Go to Admin Portal and sign in
2. Authorizations Menu > IAM Clients
3. Create New
4. Pick Confidential on Client Type
5. Generate secret, save id and secret to be used later
6. Fill in the Client Name e.g. analytics-connector
7. Redirect URl can be filled with http://127.0.0.1
8. Click Create button
9. In the newly IAM Clients created, Go to the tab Permissions
10. Click Add button
11. Add permission resource
    • ADMIN:ROLE with Action Read
    • ADMIN:NAMESPACE:*:CLIENT with Action Read
    • ADMIN:ANALYTICS:CONNECTOR with Action Create, Read, Update, Delete
12. Go to IAM apidocs
13. Authorize > Basic Authorization
14. Fill in the IAM Client Id in the username
15. Fill in the Client's secret in the password, then click Authorize in the Basic Authorization section
16. Go to this endpoint /iam/v3/oauth/token
17. Select the client_credentials as the grant_type
18. Click Execute button
19. Copy the access_token and paste it into the Connector API Authorization HTTP Bearer section