Skip to content

ETL pipeline: export company data from Openapi to PostgreSQL #25

@francescobianco

Description

@francescobianco

Use case

A common ETL scenario: pull company data from the Openapi IT company API and load it into a local PostgreSQL database for reporting or enrichment pipelines.

Example flow

use Openapi\Client;
use Openapi\OauthClient;

$oauth = new OauthClient();
$token = json_decode($oauth->createToken(['GET:company.openapi.com/IT-advanced']), true)['token'];

$client = new Client($token);
$companies = json_decode($client->get('https://company.openapi.com/IT-search', [
    'provincia' => 'MI',
    'codice_ateco' => '6201',
]), true);

foreach ($companies['items'] as $company) {
    $pdo->execute(
        'INSERT INTO companies (cf, name, address) VALUES (?, ?, ?) ON CONFLICT DO NOTHING',
        [$company['cf'], $company['name'], $company['address']]
    );
}

Open questions

  • Should the SDK expose a pagination helper or leave that to the consumer?
  • How do we handle partial failures mid-batch — retry, skip, or fail fast?
  • Is there interest in an official example script for this use case in examples/?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions