Skip to content

[feature request] Use new LFX API for affiliation versus using gitdm data #116

Open
Open
[feature request] Use new LFX API for affiliation versus using gitdm data#116
Assignees
lukaszgryglicki
Labels
enhancementNew feature or request

Description

@caniszczyk
caniszczyk
opened on Nov 13, 2025

LFX has a new API they are working on for developer affiliation. We should use that as a source of truth and deprecate the gitdm approach some time next year.

Activity

  1. caniszczyk
    added
    enhancementNew feature or request
    on Nov 13, 2025
  2. caniszczyk
    assigned
    lukaszgryglicki
    on Nov 13, 2025

  3. lukaszgryglicki commented on Nov 13, 2025

    @lukaszgryglicki
    lukaszgryglicki
    on Nov 13, 2025
    Member

    Hi, I can start working on this after ongoing huge migration to another infrastructure which should be finished by end of January 2026 - is this OK @caniszczyk ?

  4. caniszczyk commented on Nov 13, 2025

    @caniszczyk
    caniszczyk
    on Nov 13, 2025 via email
    Author

  5. lukaszgryglicki commented on Nov 13, 2025

    @lukaszgryglicki
    lukaszgryglicki
    on Nov 13, 2025
    Member

    Great, thank you.

  6. lukaszgryglicki commented on Feb 2, 2026

    @lukaszgryglicki
    lukaszgryglicki
    on Feb 2, 2026
    Member

    I need to fix missing commits data after GitHub removed them from their API responses, so GHA no longer have it, so I will work on this after I "fix" missing commits data issue on both DevStats and velocity.

  7. jonathimer commented on Mar 9, 2026

    @jonathimer
    jonathimer
    on Mar 9, 2026

    The LFX team is ready to start build out an API to provide affiliations data to DevStats.

    What we’re proposing:
    The basic flow we have in mind is a lookup endpoint where you send us one or moe identity — a GitHub username, a git email address, or both — and we return the affiliations data we have for that contributor.

    What we need from you to design this well:

    • Identity inputs: We’re planning to support GitHub username and git email as lookup keys. Are there other identity types you’d need (e.g., GitLab username, Gerrit account)?
    • Response shape: What affiliation fields does DevStats consume today from gitdm? At minimum we’d return the organization name and date ranges for each affiliation, but we want to make sure we match what you need. For example, do you need independent/volunteer status, or multiple concurrent affiliations?
    • Bulk vs. single lookups: Do you need to resolve affiliations one contributor at a time, or would a bulk endpoint (send N identities, get N results) be more practical for your sync workflow?
    • Freshness & sync cadence: How often does DevStats currently refresh affiliation data? We want to understand if you’d need real-time lookups, or if a periodic sync (e.g., daily/weekly) works.
    • Authentication: We’re planning API key-based auth for the API. Does that work for your infrastructure, or do you have preferences (OAuth, etc.)?
    • Rate limits / volume: Roughly how many unique contributors does DevStats need to resolve? This helps us plan capacity on our side.

    We see this as a win-win — DevStats gets access to a much richer, continuously maintained affiliation dataset, and we get a high-profile consumer that helps us validate and improve the API. Happy to set up a call to hash out the details if that’s easier than async on this thread.

  8. caniszczyk commented on Mar 9, 2026

    @caniszczyk
    caniszczyk
    on Mar 9, 2026
    Author

    Regarding identity inputs, an email or github id is good enough to start, devstats doesn't support Gerrit etc

    I'd envision us doing a cron style job every 24 hours to update affiliations on our end

    API key based auth to start is fine, no need for an OAuth dance imho

    @lukaszgryglicki for more commentary

  9. lukaszgryglicki commented on Mar 10, 2026

    @lukaszgryglicki
    lukaszgryglicki
    on Mar 10, 2026
    Member

    OK, so input would be GitHub username/login, response shape should return list of companies with date ranges, eventually also list of emails configured for this GitHub account. Please return Independent for date ranges where given person worked on his/her own. Bulk lookups would be a perfect option - I can then do one request for all currently unknown contributors (with paging support) and get everything I need from single request by browsing pages. We can do this weekly, DevStats is not using real time affiliations data as all dashboards are pre-calculated periodically. API based authentication is OK. DevStats have about 400000 contributors in total, we can assume that more than 50% of them are unknown.

  10. joanagmaia commented on Mar 12, 2026

    @joanagmaia
    joanagmaia
    on Mar 12, 2026

    Hey @lukaszgryglicki, thanks for the detailed breakdown of the requirements.

    We should be able to start working on this in the coming weeks. For the bulk lookups, would it work for you if we exposed an endpoint where you can submit up to 1,000 GitHub usernames/logins in the request body, and we return the affiliations for all of those profiles in a single response?
    In that approach, the endpoint itself wouldn’t use pagination, the limit of 1,000 logins per request would effectively act as the batching mechanism.

    Alternatively, we could support pagination in the response and allow larger input lists of GitHub profiles.

    Let me know which option you’d prefer.

  11. lukaszgryglicki commented on Mar 12, 2026

    @lukaszgryglicki
    lukaszgryglicki
    on Mar 12, 2026
    Member

    Bulk lookup with 1000 limit is OK. The key thing is to be able to do bulk lookup.

  12. joanagmaia commented on Apr 2, 2026

    @joanagmaia
    joanagmaia
    on Apr 2, 2026

    Hey @lukaszgryglicki @caniszczyk

    The LFX Affiliations API is now ready to be integrated.
    As previously discussed (with some adjustments), the API supports:

    One thing to note: LFX Serve, the self-service product where maintainers will be able to manage their own affiliations, is not yet ready. We expect it to be available by end of April. So we should only release the integration once the self serve affiliations is live.

    For this reason, we don't expect active usage of this API before end of April. However, integration work can start now, the API is stable and ready to be tested against.

    To get started, you'll need an API key, which I'll share in a different channel.

    cc @jonathimer

  13. lukaszgryglicki commented on Apr 3, 2026

    @lukaszgryglicki
    lukaszgryglicki
    on Apr 3, 2026
    Member

    Hi, thanks, I will take a look next Friday or eventually late today (time permitting).

  14. lukaszgryglicki commented on Apr 10, 2026

    @lukaszgryglicki
    lukaszgryglicki
    on Apr 10, 2026
    Member

    Let me know once this service is running, I'll start integrating then.
    Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions