Skip to main content

Check out Port for yourselfย 

OpenCost

Port's Opencost integration allows you to model Opencost resources in your software catalog and ingest data into them.

Overviewโ€‹

This integration allows you to:

  • Map and organize your desired Opencost resources and their metadata in Port (see supported resources below).
  • Watch for Opencost object changes (create/update/delete) in real-time, and automatically apply the changes to your entities in Port.

Supported Resourcesโ€‹

The resources that can be ingested from Opencost into Port are listed below. It is possible to reference any field that appears in the API responses linked below in the mapping configuration.

Setupโ€‹

Choose one of the following installation methods:

Using this installation option means that the integration will be able to update Port in real time using webhooks.

Prerequisites

To install the integration, you need a Kubernetes cluster that the integration's container chart will be deployed to.

Please make sure that you have kubectl and helm installed on your machine, and that your kubectl CLI is connected to the Kubernetes cluster where you plan to install the integration.

Troubleshooting

If you are having trouble installing this integration, please refer to these troubleshooting steps.

For details about the available parameters for the installation, see the table below.

To install the integration using Helm:

  1. Go to the Opencost data source page in your portal.

  2. Select the Real-time and always on method:

  3. A helm command will be displayed, with default values already filled out (e.g. your Port client ID, client secret, etc).
    Copy the command, replace the placeholders with your values, then run it in your terminal to install the integration.

BaseUrl & webhook configuration

AppHost deprecation

integration.config.appHost is deprecated: Please use liveEvents.baseUrl for webhook URL settings instead.

In order for the Opencost integration to update the data in Port on real-time changes in Opencost, you need to specify the liveEvents.baseUrl parameter. The liveEvents.baseUrl parameter should be set to the url of your Opencost integration instance. Your integration instance needs to have the option to setup webhooks via http requests/recieve http requests , so please configure your network accordingly.

Debugging local integrations

To test webhooks or live event delivery to your local environment, expose your local pod or service to the internet using ngrok:

ngrok http http://localhost:8000

This will provide a public URL you can use for webhook configuration and external callbacks during development.

If liveEvents.baseUrl is not provided, the integration will continue to function correctly. In such a configuration, to retrieve the latest information from the target system, the scheduledResyncInterval parameter has to be set, or a manual resync will need to be triggered through Port's UI.

Scalable Mode for Large Integrations

If you are deploying the integration at scale and want to decouple the resync process from the live events process (recommended for large or high-throughput environments), you can enable scalable mode by adding the following flags to your Helm install command:

  --set workload.kind="CronJob"  \
--set workload.cron.resyncTimeoutMinutes=60 \
--set scheduledResyncInterval="'*/60 * * * *'" \
--set liveEvents.worker.enabled=true
Selecting a Port API URL by account region

The port_region, port.baseUrl, portBaseUrl, port_base_url and OCEAN__PORT__BASE_URL parameters are used to select which instance or Port API will be used.

Port exposes two API instances, one for the EU region of Port, and one for the US region of Port.

This table summarizes the available parameters for the installation.

ParameterDescriptionRequired
port.clientIdYour port client idโœ…
port.clientSecretYour port client secretโœ…
port.baseUrlYour Port API URL - https://api.getport.io for EU, https://api.us.getport.io for USโœ…
integration.identifierChange the identifier to describe your integrationโœ…
integration.typeThe integration typeโœ…
integration.eventListener.typeThe event listener typeโœ…
integration.config.opencostHostThe Opencost server URLโœ…
scheduledResyncIntervalThe number of minutes between each resyncโŒ
initializePortResourcesDefault true, When set to true the integration will create default blueprints and the port App config MappingโŒ
sendRawDataExamplesEnable sending raw data examples from the third party API to port for testing and managing the integration mapping. Default is trueโŒ

Advanced integration configuration

For advanced configuration such as proxies or self-signed certificates, click here.

Configurationโ€‹

Port integrations use a YAML mapping block to ingest data from the third-party api into Port.

The mapping makes use of the JQ JSON processor to select, modify, concatenate, transform and perform other operations on existing fields and values from the integration API.

Default mapping configurationโ€‹

This is the default mapping configuration for this integration:

Default mapping configuration (Click to expand)
createMissingRelatedEntities: true
deleteDependentEntities: true
resources:
- kind: cost
selector:
query: 'true'
port:
entity:
mappings:
blueprint: '"openCostResourceAllocation"'
identifier: .name
title: .name
properties:
cluster: .properties.cluster
namespace: .properties.namespace
startDate: .start
endDate: .end
cpuCoreHours: .cpuCoreHours
cpuCost: .cpuCost
cpuEfficiency: .cpuEfficiency
gpuHours: .gpuHours
gpuCost: .gpuCost
networkCost: .networkCost
loadBalancerCost: .loadBalancerCost
pvCost: .pvCost
ramBytes: .ramBytes
ramCost: .ramCost
ramEfficiency: .ramEfficiency
sharedCost: .sharedCost
externalCost: .externalCost
totalCost: .totalCost
totalEfficiency: .totalEfficiency
- kind: cloudcost
selector:
query: 'true'
cloudcostAggregate: provider
port:
entity:
mappings:
blueprint: '"openCostCloudcost"'
identifier: .properties.provider + "-" + .window.start + "-" + .window.end
title: .properties.provider + "-" + .window.start + "-" + .window.end
properties:
startDate: .window.start
endDate: .window.end
listCost: .listCost.cost
netCost: .netCost.cost
amortizedNetCost: .amortizedNetCost.cost
invoicedCost: .invoicedCost.cost
amortizedCost: .amortizedCost.cost

Monitoring and sync statusโ€‹

To learn more about how to monitor and check the sync status of your integration, see the relevant documentation.

Examplesโ€‹

Examples of blueprints and the relevant integration configurations:

Costโ€‹

Cost blueprint
{
"identifier": "openCostResourceAllocation",
"description": "This blueprint represents an OpenCost resource allocation in our software catalog",
"title": "OpenCost Resource Allocation",
"icon": "Cluster",
"schema": {
"properties": {
"cluster": {
"type": "string",
"title": "Cluster"
},
"namespace": {
"type": "string",
"title": "Namespace"
},
"startDate": {
"title": "Start Date",
"type": "string",
"format": "date-time"
},
"endDate": {
"title": "End Date",
"type": "string",
"format": "date-time"
},
"cpuCoreHours": {
"title": "CPU Core Hours",
"type": "number"
},
"cpuCost": {
"title": "CPU Cost",
"type": "number"
},
"cpuEfficiency": {
"title": "CPU Efficiency",
"type": "number"
},
"gpuHours": {
"title": "GPU Hours",
"type": "number"
},
"gpuCost": {
"title": "GPU Cost",
"type": "number"
},
"networkCost": {
"title": "Network Cost",
"type": "number"
},
"loadBalancerCost": {
"title": "Load Balancer Cost",
"type": "number"
},
"pvCost": {
"title": "PV Cost",
"type": "number"
},
"ramBytes": {
"title": "RAM Bytes",
"type": "number"
},
"ramCost": {
"title": "RAM Cost",
"type": "number"
},
"ramEfficiency": {
"title": "RAM Efficiency",
"type": "number"
},
"sharedCost": {
"title": "Shared Cost",
"type": "number"
},
"externalCost": {
"title": "External Cost",
"type": "number"
},
"totalCost": {
"title": "Total Cost",
"type": "number"
},
"totalEfficiency": {
"title": "Total Efficiency",
"type": "number"
}
},
"required": []
},
"mirrorProperties": {},
"calculationProperties": {},
"relations": {}
}
Integration configuration
createMissingRelatedEntities: true
deleteDependentEntities: true
resources:
- kind: cost
selector:
query: "true"
window: "month"
aggregate: "pod"
step: "window"
resolution: "1m"
filter: 'labels:"app:internal-service","app:service-2"+service:"notification","account","functions"'
port:
entity:
mappings:
blueprint: '"openCostResourceAllocation"'
identifier: .name
title: .name
properties:
cluster: .properties.cluster
namespace: .properties.namespace
startDate: .start
endDate: .end
cpuCoreHours: .cpuCoreHours
cpuCost: .cpuCost
cpuEfficiency: .cpuEfficiency
gpuHours: .gpuHours
gpuCost: .gpuCost
networkCost: .networkCost
loadBalancerCost: .loadBalancerCost
pvCost: .pvCost
ramBytes: .ramBytes
ramCost: .ramCost
ramEfficiency: .ramEfficiency
sharedCost: .sharedCost
externalCost: .externalCost
totalCost: .totalCost
totalEfficiency: .totalEfficiency

Cloudcostโ€‹

Cloudcost blueprint
{
"identifier": "openCostCloudcost",
"description": "This blueprint represents cloud cost allocations from your OpenCost instance",
"title": "OpenCost CloudCost",
"icon": "Opencost",
"schema": {
"properties": {
"startDate": {
"title": "Start Date",
"type": "string",
"format": "date-time"
},
"endDate": {
"title": "End Date",
"type": "string",
"format": "date-time"
},
"listCost": {
"title": "List Cost",
"type": "number"
},
"netCost": {
"title": "Net Cost",
"type": "number"
},
"amortizedNetCost": {
"title": "Amortized Net Cost",
"type": "number"
},
"invoicedCost": {
"title": "Invoiced Cost",
"type": "number"
},
"amortizedCost": {
"title": "Amortized Cost",
"type": "number"
}
},
"required": []
},
"mirrorProperties": {},
"calculationProperties": {},
"aggregationProperties": {},
"relations": {}
}
Integration configuration
- kind: cloudcost
selector:
query: 'true'
port:
entity:
mappings:
identifier: .properties.provider + "-" + .window.start + "-" + .window.end
title: .properties.provider + "-" + .window.start + "-" + .window.end
blueprint: '"openCostCloudcost"'
properties:
startDate: .window.start
endDate: .window.end
listCost: .listCost.cost
netCost: .netCost.cost
amortizedNetCost: .amortizedNetCost.cost
invoicedCost: .invoicedCost.cost
amortizedCost: .amortizedCost.cost

Let's Test Itโ€‹

This section includes a sample response data from OpenCost. In addition, it includes the entity created from the resync event based on the Ocean configuration provided in the previous section.

Payloadโ€‹

Here is an example of the payload structure from OpenCost

Cost response data
{
"name": "ingress-nginx",
"properties": {
"cluster": "cluster-one",
"node": "minikube",
"container": "controller",
"controller": "ingress-nginx-controller",
"controllerKind": "deployment",
"namespace": "ingress-nginx",
"pod": "ingress-nginx-controller-7799c6795f-29n7j",
"services": [
"ingress-nginx-controller-admission",
"ingress-nginx-controller"
],
"labels": {
"app_kubernetes_io_component": "controller",
"app_kubernetes_io_instance": "ingress-nginx",
"app_kubernetes_io_name": "ingress-nginx",
"gcp_auth_skip_secret": "true",
"kubernetes_io_metadata_name": "ingress-nginx",
"pod_template_hash": "7799c6795f"
},
"namespaceLabels": {
"app_kubernetes_io_instance": "ingress-nginx",
"app_kubernetes_io_name": "ingress-nginx",
"kubernetes_io_metadata_name": "ingress-nginx"
}
},
"window": {
"start": "2023-10-30T00:00:00Z",
"end": "2023-10-31T00:00:00Z"
},
"start": "2023-10-30T09:05:00Z",
"end": "2023-10-30T11:50:00Z",
"minutes": 165,
"cpuCores": 0.1,
"cpuCoreRequestAverage": 0.1,
"cpuCoreUsageAverage": 0,
"cpuCoreHours": 0.275,
"cpuCost": 0.00869,
"cpuCostAdjustment": 0,
"cpuEfficiency": 0,
"gpuCount": 0,
"gpuHours": 0,
"gpuCost": 0,
"gpuCostAdjustment": 0,
"networkTransferBytes": 0,
"networkReceiveBytes": 0,
"networkCost": 0,
"networkCrossZoneCost": 0,
"networkCrossRegionCost": 0,
"networkInternetCost": 0,
"networkCostAdjustment": 0,
"loadBalancerCost": 0,
"loadBalancerCostAdjustment": 0,
"pvBytes": 0,
"pvByteHours": 0,
"pvCost": 0,
"pvs": "None",
"pvCostAdjustment": 0,
"ramBytes": 94371840,
"ramByteRequestAverage": 94371840,
"ramByteUsageAverage": 0,
"ramByteHours": 259522560,
"ramCost": 0.00102,
"ramCostAdjustment": 0,
"ramEfficiency": 0,
"externalCost": 0,
"sharedCost": 0,
"totalCost": 0.00972,
"totalEfficiency": 0,
"lbAllocations": "None"
}
Cloudcost response data
{
"properties": {
"provider": "AWS",
"accountID": "123456789012",
"invoiceEntityID": "AWS-123456789012",
"service": "Amazon Elastic Compute Cloud",
"category": "Compute",
"region": "us-east-1",
"labels": {
"environment": "production",
"team": "platform"
}
},
"window": {
"start": "2024-03-01T00:00:00Z",
"end": "2024-03-31T23:59:59Z"
},
"listCost": {
"cost": 1250.75,
"kubernetesPercent": 0.85
},
"netCost": {
"cost": 1100.50,
"kubernetesPercent": 0.85
},
"amortizedNetCost": {
"cost": 1050.25,
"kubernetesPercent": 0.85
},
"invoicedCost": {
"cost": 1250.75,
"kubernetesPercent": 0.85
},
"amortizedCost": {
"cost": 1200.30,
"kubernetesPercent": 0.85
},
"items": [
{
"resourceId": "i-0abc123def456789",
"name": "eks-node-1",
"properties": {
"instanceType": "m5.xlarge",
"operatingSystem": "Linux"
},
"tags": {
"Name": "eks-node-1",
"kubernetes.io/cluster/my-cluster": "owned"
},
"cost": 625.37,
"start": "2024-03-01T00:00:00Z",
"end": "2024-03-31T23:59:59Z"
},
{
"resourceId": "i-0def456789abc1234",
"name": "eks-node-2",
"properties": {
"instanceType": "m5.xlarge",
"operatingSystem": "Linux"
},
"tags": {
"Name": "eks-node-2",
"kubernetes.io/cluster/my-cluster": "owned"
},
"cost": 625.38,
"start": "2024-03-01T00:00:00Z",
"end": "2024-03-31T23:59:59Z"
}
]
}

Mapping Resultโ€‹

The combination of the sample payload and the Ocean configuration generates the following Port entity:

Cost entity in Port
{
"identifier": "ingress-nginx",
"title": "ingress-nginx",
"blueprint": "openCostResourceAllocation",
"team": [],
"properties": {
"cluster": "cluster-one",
"namespace": "ingress-nginx",
"startDate": "2023-10-30T09:05:00.000Z",
"endDate": "2023-10-30T11:50:00.000Z",
"cpuCoreHours": 0.275,
"cpuCost": 0.00869,
"cpuEfficiency": 0,
"gpuHours": 0,
"gpuCost": 0,
"networkCost": 0,
"loadBalancerCost": 0,
"pvCost": 0,
"ramBytes": 94371840,
"ramCost": 0.00102,
"ramEfficiency": 0,
"sharedCost": 0,
"externalCost": 0,
"totalCost": 0.00972,
"totalEfficiency": 0
},
"relations": {},
"createdAt": "2023-10-15T09:30:57.924Z",
"createdBy": "hBx3VFZjqgLPEoQLp7POx5XaoB0cgsxW",
"updatedAt": "2023-10-30T11:49:20.881Z",
"updatedBy": "hBx3VFZjqgLPEoQLp7POx5XaoB0cgsxW"
}
Cloudcost entity in Port
{
"identifier": "aws_2024_03_01_t_00_00_00_z_2024_03_31_t_23_59_59_z",
"title": "AWS-2024-03-01T00:00:00Z-2024-03-31T23:59:59Z",
"properties": {
"startDate": "2024-03-01T00:00:00Z",
"endDate": "2024-03-31T23:59:59Z",
"listCost": 1250.75,
"netCost": 1100.5,
"amortizedNetCost": 1050.25,
"invoicedCost": 1250.75,
"amortizedCost": 1200.3
},
"relations": {},
"icon": "Opencost"
}